IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO.

9, SEPTEMBER 2010

2307

Switching Bilateral Filter With a Texture/Noise Detector for Universal Noise Removal
Chih-Hsing Lin, Jia-Shiuan Tsai, and Ching-Te Chiu
AbstractIn this paper, we propose a switching bilateral lter (SBF) with a texture and noise detector for universal noise removal. Operation was carried out in two stages: detection followed by ltering. For detection, we propose the sorted quadrant median vector (SQMV) scheme, which includes important features such as edge or texture information. This information is utilized to allocate a reference median from SQMV, which is in turn compared with a current pixel to classify it as impulse noise, Gaussian noise, or noise-free. The SBF removes both Gaussian and impulse noise without adding another weighting function. The range lter inside the bilateral lter switches between the Gaussian and impulse modes depending upon the noise classication result. Simulation results show that our noise detector has a high noise detection rate as well as a high classication rate for salt-and-pepper, uniform impulse noise and mixed impulse noise. Unlike most other impulse noise lters, the proposed SBF achieves high peak signal-to-noise ratio and great image quality by efciently removing both types of mixed noise, salt-and-pepper with uniform noise and salt-andpepper with Gaussian noise. In addition, the computational complexity of SBF is signicantly less than that of other mixed noise lters. Index TermsGaussian noise, image restoration, impulse noise, mixed noise, nonlinear lters, switch bilateral lter, switching scheme.

I. INTRODUCTION

OISE is introduced into images during acquisition, signal amplication and transmission [6], [27][31]. An important problem of image processing is to effectively remove noise from an image while keeping its features. Noise removal is a difcult task because images may be corrupted by different types of noise, such as additive, impulse or signal dependent noise [27][29]. The solution depends upon the type of noise added to the image [28][30]. Linear ltering possesses mathematical simplicity and offers satisfactory performance on images with additive Gaussian noise [29][31]. However, linear techniques blur edges and fail for non-Gaussian and/or impulse noise. This

Manuscript received September 02, 2009; revised March 02, 2010; accepted March 20, 2010. First published April 08, 2010; current version published August 18, 2010. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Kiyoharu Aizawa. C.-H. Lin is with the Institute of Communications Engineering, National Tsing-Hua University, Hsinchu, Taiwan. R.O.C. (e-mail: chihhsinglin@gmail. com). J.-S. Tsai is with the Department of Computer Science, National Tsing-Hua University, Hsinchu, Taiwan, R.O.C. (e-mail: thymine666@gmail.com). C.-T. Chiu is with the Department of Computer Science and Institute of Communications Engineering, National Tsing-Hua University, Hsinchu, Taiwan, R.O.C. (e-mail: ctchiu@cs.nthu.edu.tw). Color versions of one or more of the gures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identier 10.1109/TIP.2010.2047906

disadvantage leads to the use of nonlinear ltering in image processing [29][31]. In this paper, we propose a ltering scheme that can remove both the additive Gaussian noise and the impulse noise. Additive Gaussian noise is characterized by adding to each image pixel a value with a zero-mean Gaussian distribution [6], [27]. Such noise is usually introduced during image acquisition [27]. The zero-mean distribution property allows such noise to be removed by averaging pixel values locally [27]. Traditional linear lters can remove noise effectively but with the side effect of blurring edges and details signicantly [6], [31], [32]. The more advanced methods for noise removal aim at preserving edges and details in images while removing the noise [25], [32]. Tomasi and Manduchi propose a bilateral lter that uses weights based upon spatial and radiometric similarity [1]. The bilateral lter has good results in removing noise while preserving edges in images [1], [32]. In addition, this method is noniterative, local and simple [1], [32]. Impulse noise is characterized by replacing a portion of an image pixels with noise values, leaving the remainder unchanged [29]. Such noise is introduced due to acquisition or transmission errors [27], [29], [30]. Nonlinear lters have been developed for removing impulse noise such as the traditional median lter [29]. Extensions of the median lter [2], [5], [14][21], [23], [24], [26] are proposed to meet various criteria, e.g., robustness, preservation of edge, or preservation of details. The learning algorithm [7] and the switching noise lters [14], [16], [17], [19], [22][24], [26] are also proposed. The genetic programming (GP) lter [7] is based upon the learning algorithm that is used to build two detectors, and this method requires a training procedure to arrive at an optimal classication based upon the measure of pixels and their neighbors. Methods that require training are less easily controlled and more unpredictable than traditional methods. The switching scheme detects impulse noise pixels before ltering and replaces them with estimated values while leaving the remaining pixels unchanged [14], [16], [17], [19], [22][24], [26]. Filters that can remove Gaussian or impulse noise, or any mixture thereof, have also been proposed [3], [4], [6], [8][11], [13]. The median-based signal-dependent rank ordered mean (SDROM) lter can remove impulse noise rather effectively, but when applied to images with Gaussian or mixed noise, it often produces a visually disappointing output [8]. This is because the rank-ordered mean gets corrupted in a high noise intensity window. Another median-based lter, the adaptive centerweighted median lter (ACWMF), uses a comparison of the center weighted medians and adaptive thresholds for detection [9]. When applied to Gaussian or mixed noise images, it creates blur and removes the details. The directional weighted me-

1057-7149/$26.00 2010 IEEE

2308

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 9, SEPTEMBER 2010

Fig. 2. Four quadrant blocks in a single 5

2 5 window.

Fig. 1. (a) Bright region contaminated with pepper-like impulse noises. (b) Dark region corrupted by salt-like impulses. (c) Lower window is noise-free and the upper window is corrupted by 20% uniform noise.

dian (DWM) lter uses an iterative ltering approach, and the detector is based upon absolute differences within the ltering window [10]. The estimation is done using an adaptive weighted median lter. For ensuring high accuracy of detection, iterative ltering is applied, which takes a longer total processing time but removes more details with each iteration. The fuzzy impulse noise detection and reduction method (FIDRM) lter is a fuzzy lter that consists of two consecutively applied lters [13]. The FIDRM lter effectively removes salt-and-pepper noise, but its performance in the case of uniform impulse noise is not satisfactory, as some of the uniform impulse noise may not produce large gradient values. Tomasi and Manduchis bilateral lter cannot adequately remove impulse noise because the difference between the noise pixel and surrounding pixels are too large. Therefore, local weighting is too small to change the noise value. The trilateral lter is a modication of the bilateral lter with incorporated rank-order absolute difference (ROAD) statistics for impulse noise detection [1], [6]. It has been especially designed for uniform impulse and Gaussian noise removal. The ROAD value could be false under the case that half of the pixels in the processing window are corrupted. In this paper, we propose a universal noise removal lter based upon the detect and replace methodology. To detect noise, the absolute difference between a current pixel value and the reference median is computed. If the absolute difference is large, then the current pixel is considered as impulse or Gaussian noise. On the other hand, when the absolute difference is small, the current pixel may be considered as noise-free pixel. It is very important to nd a proper reference median in the window, because a nonproper one would lead to false detection. In order to nd the proper reference median, we have to understand properties such as edge or texture in the current window. We dene

a sorted quadrant median vector (SQMV) and present an approach to recognize edge and texture. Based upon the edge/texture content in the window, a proper reference median can be selected. To achieve good visual image quality, it is important that the noise lter not only replaces noisy pixels but also preserves the edge. The bilateral lter can achieve this, but it cannot remove impulse noise. Therefore, we propose a switching bilateral lter (SBF) for removing both Gaussian and impulse noise. From the noise classication result in the noise detector, SBF switches between the Gaussian and impulse modes. SBF shows very good results in removing salt-and-pepper noise, uniform impulse noise and Gaussian noise. It efciently removes both types of mixed noise, salt-and-pepper with uniform impulse noise and salt-and-pepper with Gaussian noise. This paper is arranged as follows. In Section II, we propose the SQMV scheme, and the features of SQMV and the approach for detecting noise are discussed. Section III presents the twostage detection method and the switching bilateral lter for universal noise removal. Section IV presents the simulations on noise detection and noise removal, with visual examples and numerical results. Finally, a brief conclusion is given in Section V. II. SORTED QUADRANT MEDIAN VECTOR (SQMV) FOR NOISE DETECTION A. Noise Models When an image is corrupted by impulse noise, a portion of the and pixel values are replaced with random values. Let denote the intensity values of a noise-free image and the cor. Then the noisy image rupted image at the pixel location can be described as follows: (1) The value of indicates the probability that a noise-free is the intensity image is corrupted by impulse noise, and . The value value of the impulse noise at the location is in the range of maximum luminance value of and minimum luminance value . When only takes or , the noise model is called values of either takes random values salt-and-pepper noise. And when with a uniform distribution, from the interval

LIN et al.: SWITCHING BILATERAL FILTER WITH A TEXTURE/NOISE DETECTOR FOR UNIVERSAL NOISE REMOVAL

2309

Fig. 3. (a) Uniform region. (b) Diagonal edge in dark side. (c) Diagonal edge in bright side. (d) Vertical edge. (e) Horizontal edge. (f) Diagonal line. (g) Gradual change edge. (h) Texture.

the model is called uniform impulse noise. For the case of is produced from additive Gaussian noise, each noise value is a zero-mean Gaussian distribution and the noisy image related to the original image by (2) In this paper, mixed impulse noise containing either salt-andpepper noise or uniform impulse noise or both of them is considered. In addition, impulse noise mixed with Gaussian noise is also considered. Gaussian noise is independent of impulse noise; therefore, it is processed in a separate path. B. Motivation of the Noise Detection Scheme Existing two-state noise detectors fail in several conditions when a portion of the pixels in an image is contaminated with impulse noise. For example, when the number of pixels with impulse noise is equal to or larger than half of the size of a processing window, the median of the processing window falls to the value of the impulse noise. The central pixel and its half neighboring pixels exhibit all salt-like or all pepper-like impulse noise, as shown in Fig. 1, where Fig. 1(a) shows a bright region contaminated with pepper-like impulse noise and Fig. 1(b) shows a dark region corrupted by salt-like impulses noise. The absolute difference between the median and the central pixel is zeros, and the central pixel is, thus, identied as a noise-free pixel. Another case occurs when adding uniform impulse noise into images with ne texture or details. For nely textured or detailed images, the difference between pixels is larger than usual. Since the uniform impulse noise value could be any value between maximum and minimum, it is hard to tell the difference between a texture and a uniform impulse noise. Fig. 1(c) shows

a portion of Lenas hair which is corrupted by 20% uniform impulse noise. There are two 3 3 windows in the gure where the lower one is not corrupted and the upper one is contaminated. The medians of the lower and upper windows are 83 and 118, respectively. The absolute differences between the median and the central pixel are 34 and 32 for the lower and upper window. Since the absolute differences are so close, it is difcult to distinguish between the ne texture and the noise corrupted window. One of the reasons that previous approaches fail to detect these cases is because the processing window size is too small to process information to distinguish texture or noise. For images with ne details, a processing window of size 3 3 may fail to distinguish between noise and detail. For a larger window size, such as 5 5, the resulting median value drifts from the median value of a small-size window because new textures are added into the larger window. Using a median value from a larger window may cause false noise detection and blur the image during ltering. C. Denition of Sorted Quadrant Median Vector (SQMV) To overcome the previous problems, we propose a sorted quadrant median vector (SQMV). We start from a larger window but observe medians from subwindows in the larger one. The medians in the subwindows are sorted and the result vector is called SQMV, and it reveals edge and texture information in the larger window. With the median values in the subwindow and edge/texture information in the larger window, the median drifting problem in the larger window or lack of texture inforwith size mation problems can be avoided. For a window , we divide the window into four subwin, with the central pixel as the dows of size ). Let be corner pixel in the four subwindows (Fig. 2; in the luminance of the central pixel located at the position

2310

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 9, SEPTEMBER 2010

Fig. 4. Example of SQMD

and SQMD

for the three cases: (a) without edge, (b) weak edge, and (c) strong edge or texture.

Fig. 5. (a) Original image: part of an airplane. (b) Noise-free image edge drawn by the edge detector. (c) Edge drawn by the edge detector for an image with Gaussian noise ( = 10). (d) Edge drawn by the edge detector for an image with salt-and-pepper noise (p = 20%).

a window. Then, the set of points in a window can be expressed as (3) The set of pixels in the four subwindow blocks of size are dened as (4) (5) (6) (7) and In the following discussion, we discuss the case the window size is 5 5, with its four quadrant blocks of size 3 3. Each quadrant block has a median value expressed as (8) , and denote the medians of the top where right, top left, bottom left, and bottom right of the four quadrant are sorted in an increasing blocks. These four median values order and the SQMV is dened as (9) Here, , and D. Features of SQMV After sorting the medians of the four subwindows, some of the medians value are very similar while others are different, leading to the formation of clusters. The clusters of , and and the order of the four , and , which are mapped into the medians clusters, include important features of the window such as detail and edges. From the SQMV clusters and geometric shape in the window, we obtain seven different patterns, listed in the following. , and are the medians sorted in an ascending order such that . 1) Uniform Region: When the pixel values in window are similar, the values of the four medians are close to each other. The SQMV falls into a single cluster as shown in Fig. 3(a). 2) Diagonal Edge: When there is a diagonal edge in window , the values of the four sorted medians are divided into two unequal clusters such as in Fig. 3(b) or in Fig. 3(c), depending upon whether the majority of pixels in the window is dark or light. Four types of patterns (I, II, III, and IV) exist depending upon the direction of the edge for both cases. The to the clusters are also listed mapping of the four medians in Fig. 3(b) and (c). For example, case I in Fig. 3(b) shows is higher than the others, and therefore, that the value of is associated with . and are mapped into
Fig. 6. Direction average: the pixels in the window that are needed in each case. The four pixels of (a) vertical, (b) horizontal, and (c) diagonal directions are located in the black box.

Fig. 7. Switching scheme detection with two detectors.

LIN et al.: SWITCHING BILATERAL FILTER WITH A TEXTURE/NOISE DETECTOR FOR UNIVERSAL NOISE REMOVAL

2311

TABLE I IMPULSE DETECTION RATIO

Fig. 8. Pseudo-code of noise detector. Fig. 9. Optimal model.

 values and  are linearly related in the Gaussian noise

cluster , and the order of , does not matter. The mapping of in other cases and in Fig. 3(b) and (c) are obtained in a similar way. 3) Vertical Edge: When there is a vertical edge in window , the values of the four sorted medians are divided into . two equal clusters Two patterns exists depending on whether the right hand side or left hand side is darker. When the left hand side is is associated darker than the right hand side, and is associated with with , as shown in Fig. 3(d), case II. Fig. 3(d), case I shows the other pattern.

4) Horizontal Edge: Similar to the case of a vertical edge, a horizontal edge divides the SQMV into two equal clusters and . The associations of SQMV with the corresponding unsorted medians and are shown in Fig. 3(e) for the two patterns. 5) Diagonal Line: When there is a diagonal line in window , the values of the four sorted medians are divided into two and . The equal clusters diagonal lines in the middle of the window have similar medians and the two triangles on the side of the window have similar

2312

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 9, SEPTEMBER 2010

TABLE II CLASSIFICATION ACCURACY

medians. The unsorted medians and are associated with the two clusters as shown in Fig. 3(f). 6) Gradual Change Edge: Gradual change edge is similar to the diagonal line mentioned in case (5) except that the two triangles on the side of the window have different median values. The values of the four sorted medians are divided into three clusters and and their associated medians as shown in Fig. 3(g). , 7) Texture: When there is a complex texture in window the values of the four sorted medians are quite different from each other, as shown in Fig. 3(h). The SQMV is divided into have this kind four clusters. In a natural image, very few of property. E. Edge/Texture Identication With the Clusters of SQMV The clusters of SQMV provides edge and texture information within a window for an image. Although there are seven patterns for classifying the geometric shape and clusters from SQMV, we can condense the previously mentioned seven patterns into three edge/texture cases based upon the cluster distribution. To as the determine the cluster distribution, we dene difference between the two boundary values (or the maximum and minimum) of the sorted quadrant medians (10)

is dened as the difference between the In addition, two center values of the sorted quadrant medians (11) and provide a measure of the similarity between the four quadrant blocks. Based upon the information and , three edge/texture cases without of edge, weak edge, and strong edge or texture are obtained as shown in Fig. 4. The without edge case occurs when is small (i.e., the difference between the maximum and minimum median is small). It means that the pixel values in the window are similar so there is no edge in it, such as the case is large but is small, of Fig. 3(a). When such as the patterns in Fig. 3(b), (c), and (g), there is a weak edge in the window. The case of strong edge or texture happens and are large, as shown in when both Fig. 3(d), (e), (f), and (h). and , we Based upon the information of propose the edge/texture detector summarized in the following: Edge/Texture Detector Without edge weak edge strong edge or texture

(12)

LIN et al.: SWITCHING BILATERAL FILTER WITH A TEXTURE/NOISE DETECTOR FOR UNIVERSAL NOISE REMOVAL

2313

Fig. 10. (a) Part of the original Lena image. (b) Image corrupted with mixed-noise (Salt-and-Pepper and Uniform, p (e) ACWMF lter. (f) Median 3 3 lter. (g) SDROM lter. (h) DWM lter. (i) Proposed lter.

= 30%). (c) Trilateral lter. (d) GP lter.

Using the edge/texture detector to classify SQMV into three simple cases, we can analyze the texture information in the window. SQMV is an efcient and robust edge detection scheme for a noisy image. We found through experimentation that a good value of lies in the interval [2540]. Fig. 5(a) shows a part of an airplane and Fig. 5(b) contains the edge drawn by the proposed edge detector for weak edge detection with and . Here, is set to 40. Then a Gaussian noise with a variance of 10 is added to the noise-free image; the same edge detection is applied to this noisy image and the detected edges are plotted in Fig. 5(c). The same procedure is applied for the case of Fig. 5(d) with 20% salt-and-pepper noise. For noisy images with Gaussian and impulse noise [Fig. 5(c) and (d)], the proposed edge detector works as well as in the noise-free image. F. Reference Median When the number of medians inside a cluster is more than that inside other clusters, e.g., cases in Fig. 3(a)-(c), and (g) (without edge or weak edge cases), the cluster with the most number of medians represents the majority feature in this window. This cluster is dened as a major cluster and the average of and is used as the reference median value for comparison. When the difference between the current pixel value and

the reference median in the major cluster is large, then the current pixel is not similar to the majority of the block and is very likely to be a noise pixel. For the cases in Fig. 3(d), (e), (f), and (h)(strong edge or texture), there is no major cluster in the SQMV. Under this situation, it is necessary to decide which cluster the current pixel values, the pattern in this falls into. From the order of four window can be classied into the three cases: vertical edge, horizontal edge, diagonal line or texture. A direction average approach is adopted to determine which cluster is more similar to the current pixel. Depending upon the case, the four pixels in the major pattern are averaged, represented by (13) As shown in Fig. 6, the four pixel values of the major pattern within the block are averaged. For example, in the case of is close to , then a vertical edge, if is chosen as the reference median value in the window. On the other hand, if is close to , then we choose as the reference median value. Even if there is a complex texture in the window, as shown in Fig. 3(h), the ltering result would be less articial when the value of a current pixel

2314

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 9, SEPTEMBER 2010

Fig. 11. (a) Part of the original Lena image. (b) Image corrupted with mixed-noise (Salt-and-Pepper and Gaussian noise; p (d) GP lter. (e) ACWMF lter. (f) Median 3 3 lter. (g) SDROM lter. (h) DWM lter. (i) Proposed lter.

= 20% = 10). (c) Trilateral lter.

;

is similar to is dened as

or

. The reference median (SQMR)

(14)

edges. Each pixel is replaced by a weighted average of the intensities in the window. The weighting function gives high weighting to those pixels that are both near the central pixel and similar to the central pixel. be the current pixel, and let be the pixels in Let window that surrounds and a are the location of and . The output is dened as follows: of bilateral lter (15)

The comparison between the reference median and a current pixel is adopted for detecting whether a current pixel is noisy or not. III. SWITCHING BILATERAL FILTER A. Bilateral Filter The bilateral lter proposed by Tomasi and Manduchi is a nonlinear lter which removes Gaussian noise while preserving

where (16) and (17)

LIN et al.: SWITCHING BILATERAL FILTER WITH A TEXTURE/NOISE DETECTOR FOR UNIVERSAL NOISE REMOVAL

2315

TABLE III COMPARATIVE RESTORATION RESULTS IN PSNR (dB) FOR SALT-AND-PEPPER NOISE

The Gaussian lter and range lter are dened in (16) and (17), respectively. In the Gaussian lter, the Euclidean distance beand is calculated and the difference of lutween minance is computed in the range lter. The bilateral lter shows great results in removing Gaussian noise while keeping the edge, but it is difcult to remove impulse noise. Because the noisy pixel is very different from its neighbors, the surrounding weights are too small to change the noisy pixel in the range lter. In order to remove impulse noise and Gaussian noise using a bilateral lter, we propose a switching bilateral lter, discussed in the following. B. Switching Scheme In the switching scheme, the noise detector searches for noisy pixels in a corrupted image and tries to distinguish them from uncorrupted ones. The lter is applied to the noisy samples only, thus, preventing blurred edges or removal of ne details. The most difcult part in the process is to accurately discriminate the noise from the image details. Two critical situations may occur; the image detail may be falsely treated as the noise and, therefore, ltered from the image; and the noisy pixel can be interpreted as the image detail and, thus, left unchanged. If the noise detector is good enough, the rst type of error is not critical, because only a small part of the image would be unnecessarily ltered. On the other hand, the second type of error can produce a negative effect on the output image quality, because only a small percentage of noisy pixels are sufcient to decrease the image quality.

We propose an extension to the standard switching-scheme, which uses an additional detector to identify edge or detail in a current window. The noise detection is done in two stages, and therefore, the lter is called a two-stage lter (Fig. 7). In this scheme, the edge detector shown in (12) identies the edge existence in the window. This information is used in the noise detector to decide the reference median for noise identication. The noise detector also decides whether a current pixel should be ltered by using an SBF or whether it should bypass the SBF. and . The former is There are two kinds of SBF, a bilateral lter for impulse noise and the latter is for Gaussian and denote the noisy pixel and the ltered noise. Let and denote binary control pixel, respectively. Also, let signals generated by the noise detector. The ltered image is dened as follows: (18) The ltered image contains the original pixels only if both detectors identify the pixels as noise-free. If the noise detector as an impulse noise, then the pixel is proidenties a pixel cessed by the SBF . When the pixel is not identied as an impulse noise, but the difference between the current pixel and the reference median is still large than a threshold, then pixel is identied as Gaussian noise and is ltered by . The switching scheme is implemented in a recursive manner, similar to [7] and [8]. Thus, half of the samples in the current ltering window are composed of the results from the previous steps.

2316

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 9, SEPTEMBER 2010

TABLE IV COMPARATIVE RESTORATION RESULTS IN PSNR (dB) FOR UNIFORM IMPULSE NOISE

Noise in the pixels processed in the previous step are likely to be moved and, thus, recursive implementation produces better results than nonrecursive implementation.

excellent results were obtained using thresholds selected from the following set of values: (19)

C. Noise Detector Design A noise detector is used in the proposed lter to determine whether or not the current pixel is corrupted. This decision is made using the features of SQMV, which can show the property of the background and is more reliable than only one median value. We obtain the reference median for noise identication from SQMV. If the reference median is improper, it can lead to lost detection or over detection. The lost noisy pixels have a great negative effect on the results and the undesired ltering removes the details. When a current pixel is very different from the reference median, it is identied as an impulse noise. When the difference between the current pixel and reference median is not too much, it may be a Gaussian noise or a noise-free pixel. Because the image background can give four different SQM values, the reference median can be selected from SQMV as described in Section II. The decision making mechanism is realized by employing a reference median and two thresholds ( and ) and the noise detection algorithm is shown Fig. 8. and are thresholds for identifying impulse noise or Gaussian noise. From simulations on a large variety of images,

The salt-and-pepper impulse noise contains either the maximum or minimum value so it is easy to detect. The selection yields satisfactory results in lof tering salt-and-pepper impulse noise, while the setting of consistently performs well in removing uniform impulse and Gaussian noise. D. Switching Bilateral Filter In this section, we propose a new universal noise removal be the algorithm: the switching bilateral lter (SBF). Let be the pixels in a current pixel, and let window surrounding . Finally, the SBF is dened as follows:

(20) where (21)

LIN et al.: SWITCHING BILATERAL FILTER WITH A TEXTURE/NOISE DETECTOR FOR UNIVERSAL NOISE REMOVAL

2317

TABLE V COMPARATIVE RESTORATION RESULTS IN PSNR (dB) FOR GAUSSIAN NOISE

and (22) It is difcult for a bilateral lter to remove impulse noise because the difference between the noisy pixel and its neighbors is huge. This makes the radiometric weighting function too small to change the noisy pixel. The trilateral lter adds a new weighting function which is based upon ROAD statistic to remove the impulse noise. However, the trilateral lter has to be implemented iteratively. For an image with impulse noise, the trilateral lter processes each pixel individually and it takes too much processing time and creates a blurred result. with SQMR of the window in a bilateral By replacing lter, we can remove impulse noise without adding another weighting function. The difference between neighbors and median would not be too large and, thus, the edges and details can be preserved while removing the noise. Our proposed SBF provides a sharper image than a median lter. The experiments function gives better noise removal results showed that than the median function alone. The noise detector deals with a large number of Gaussian noise pixels and noise-free pixels, which cannot be always precisely identied. Therefore, has to be capable of keeping the details and edges in an image when noise is detected. Our proposed SBF removes not only Gaussian noise but also impulse noise while keeping the details and the edges.

E. Parameter Selection for the Switching Bilateral Filter and that control the bilateral There are two parameters and that is optimal for all lter. There is no single set of images. In the switching scheme, we take different values for the two parameters in the different noise models. Through our experiment, when an edge is detected by the , and otherwise. In the edge detector, we take Gaussian noise model, the relationship between and the standard deviation of Gaussian noise has been presented [11], [12]. for each standard deviation For the proposed lter, better value is shown in Fig. 9. For the impulse noise model, we also found through experimentation that a good initial value of is 40 and any value in the interval [30, 50] should work well to remove both impulse noise and mixed impulse noise.

IV. RESULTS Simulations are carried out to verify the noise removing capability of the SBF and the results are compared with several exiting lters. Our method produced results superior to other methods in both visual image quality and quantitative measures. Simulations were made on several 512 512 8-bit grayscale test images corrupted with salt-and-pepper noise, uniform impulse noise, mixed impulse noise, and Gaussian noise.

2318

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 9, SEPTEMBER 2010

TABLE VI COMPARATIVE RESTORATION RESULTS IN PSNR (dB) FOR MIXED-NOISE

A. Implementations and Testing Procedures We implement the proposed SBF based upon the dataow shown in Fig. 7. Testing is divided into two parts, one for noise detection and the other for noise removal. In the noise detection testing, only impulse noise is considered. A corrupted image is passed through the edge and noise detector. Pixels are processed row by row from top to bottom and, in each row, from left to right. The processed pixel is the central pixel of its associated 5 5 window, which is adopted for the edge detection, as shown in Fig. 2. The edge detection is based upon (12). Here, the parameter is set to 40. Equation (14) is used for nding the reference median in this window. The dataow in Fig. 8 is used to

detect noise. The selection of yields satisfactory results for salt-and-pepper impulse noise, while the setconsistently performs well for uniting of form and mixed impulse noise. Various kinds of impulse noise, generated randomly, are used to corrupt images with different noise ratio. For each noise ratio in the corrupted image, simulations are repeated a number of times and the average results are reported. The detection rate and classication accuracy of each image are compared with the results of the SDROM and GP. In the noise ltering testing, impulse, Gaussian and mixed noise are considered. Images are corrupted with salt-and-pepper, uniform impulse, Gaussian, and mixed noise with different noise ratio. Pixels are processed in the same

LIN et al.: SWITCHING BILATERAL FILTER WITH A TEXTURE/NOISE DETECTOR FOR UNIVERSAL NOISE REMOVAL

2319

order and the same window size as those in the noise detection testing. Pixels are processed by the edge and noise detector rst and then ltered by SBF if the current pixels are detected as noise pixels. When an edge is detected by the edge detector, , and otherwise in (16). For the impulse we set noise model, we also nd through experimentation that a good is 40 and any value in the interval [30, 50] initial value of should work well to remove both impulse noise and mixed impulse noise in (21). The noniteration SBF is adopted and the results are compared with other lters. For those lters used in our comparison, their parameters are set as suggested by authors to obtain optimal results. For example, we adopt a 3 3 window for the traditional median lters since the PSNR and visual image quality are better than those of a 5 5 median lter. The detail results, observation and discussion are described in the following. B. Comparison of Noise Detection In Table I, the ratios of the noise impulse detection for two images (Lena and Boats) are shown for three different lters, SDROM, GP, and our proposed SBF. This ratio presents the number of detected impulses divided by the total number of impulses. Three noise models, salt-and-pepper, uniform impulse, and mixed impulse noise, are simulated. Our proposed SBF lter has the highest detection rate for all three cases. Compared with other methods for salt-and-pepper noise, the detection ratios are slightly higher when the noise level ranges from 10% to 40% and performs better with a noise level of 50%. For uniform impulse noise, our SBF outperforms the other methods for noise levels from 10% to 50%. For the same test sets used in Table I, Table II shows the comparison for classication rate, which presents the number of correctly classied pixels (correctly detected noise plus correctly detected noise-free pixel) divided by the total number of pixels. For salt-and-pepper noise, our proposed lter gives a better classication rate than GP. Although the classication rate is lower than SDROM, our detection rate is higher (Table I). As for uniform impulse noise, the classication rate of our proposed lter is lower than SDROM and GP. The reason is that due to the high detection rate compared with the other two methods, some pixels are wrongly marked as noise. In the high ratio of impulse noise, the classication ratio is as good as the others. C. Image Quality The following simulations show that our approach provides a visually appealing output. The SBF can restore an image corrupted with mixed noise, as demonstrated by two cases. One is the Lena image corrupted by mixed salt-and-pepper and uniform impulse noise with %. The other is the boat image contaminated by mixed salt-and-pepper and Gaussian % and . The original, noisy image and noise with the one ltered by our proposed SBF are compared with the trilateral lter, GP lter, ACWMF lter, Median lter, SDROM lter, and DWM lter (Figs. 10 and 11). Compared with the original image, it is clear that the SBF lter can eliminate noise while preserving the edge and ne details. The visual quality of images restored by the SBF lter has been demonstrated in the previously shown gures. Quantitative

measures of signal restoration are discussed in the following. The peak-to-noise ratio (PSNR) is used as a quantitative meaimage and is sure for comparison. If is an original a restored image of , the PSNR of is given by (23) The proposed SBF lter is compared with implementations of the standard median lter whose lter window is 3 3 and the ACWMF [9], SDROM [8], bilateral [1], trilateral [6], DWM [10], and two-stage GP [7] lters. Table III compares the PSNR % and in values on impulse noise (salt-and-pepper) for %. The SBF shows better PSNR than other lters except the SDROM. Table IV shows the PSNR value for uniform %, the SBF and SDROM lter show impulse noise; for the best results, but SBF and the trilateral are better than the %. SDROM lter for Table V shows the PSNR values corrupted with Gaussian and ); and the SBF produces nearly the noise ( same results with the bilateral and trilateral lters. Finally, the three kinds of mixed noise: salt-and-pepper and Gaussian with % and ; uniform impulse and Gaussian with % and ; and salt-and-pepper and uniform impulse with % are compared with the other lters and the results are listed in Table VI. In Table VI (a), the SBF shows the best values followed by the trilateral lter. In Table VI (b), the trilateral shows better results in most of the images and the SBF is very close to it. In Table VI (c), the SBF lter consistently yields the highest PSNR for each image. V. CONCLUSION The major contribution of this paper is to propose SQMV for edge/texture detection, noise detection and the switching bilateral lter. The edge detector, with a simple structure, obtains a the reference median value for noise detection, which detects impulse and Gaussian noise. Both detectors are based upon robust estimators of SQMV. Many noise removal algorithms, such as bilateral ltering, tend to treat impulse noise as edge pixels, and end up with unsatisfactory results. In order to process impulse pixels and edge pixels differently, we introduce two detectors based upon SQMV in a neighborhood of a pixel. We incorporate SQMV into switching bilateral ltering by replacing the current pixel with a proper median value in the range lter function. The new nonlinear lter is called the switching bilateral lter and the switching control signal is from the noise detector. With regard to the impulse detection rate and classication rate, the noise detector shows a good performance in identifying noise even in mixed noise models. In most of the noise model cases, the proposed lter outperforms other lters, both in PSNR and visually. Moreover, it shows excellent performance in the simultaneous removal of both impulse and Gaussian noise and without adding another weighting function. A mathematical model based upon this work can be developed to better understand the SQMV and SBF. A theoretical development also helps to gain an insight for the connection between the SBF and other related lters. A mathematical model and theoretical development will be pursued in our future work.

2320

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 9, SEPTEMBER 2010

REFERENCES
[1] C. Tomasi and R. Manduchi, Bilateral ltering for gray and color images, in Proc. IEEE Int. Conf. Computer Vision, 1998, pp. 839846. [2] H. Lin and A. N. Willson Jr., Median lters with adaptive length, IEEE Trans. Circuits Syst., vol. 35, no. 6, pp. 675690, Jun. 1988. [3] S. J. Ko and Y. H. Lee, Center weighted median lters and their applications to image enhancement, IEEE Trans. Circuits Syst., vol. 38, no. 9, pp. 984993, Sep. 1991. [4] R. C. Hardie and K. E. Barner, Rank conditioned rank selection lters for signal restoration, IEEE Trans. Image Process., vol. 3, no. 3, pp. 192206, Mar. 1994. [5] G. Pok, J. C. Liu, and A. S. Nair, Selective removal of impulse noise based on homogeneity level information, IEEE Trans. Image Process., vol. 12, no. 1, pp. 8591, Jan. 2003. [6] R. Garnett, T. Huegerich, C. Chui, and W. He, A universal noise removal algorithm with an impulse detector, IEEE Trans. Image Process., vol. 14, no. 11, pp. 17471754, Nov. 2005. [7] N. I. Petrovic and V. Crnojevic, Universal impulse noise lter based on genetic programming, IEEE Trans. Image Process., vol. 17, no. 7, pp. 11091120, Jul. 2008. [8] E. Abreu, M. Lightstone, S. K. Mitra, and K. Arakawa, A new efcient approach for the removal of impulse noise from highly corrupted images, IEEE Trans. Image Process., vol. 5, no. 6, pp. 10121025, Jun. 1996. [9] T. Chen and H. R. Wu, Adaptive impulse detection using center-weighted median lters, IEEE Signal Process. Lett., vol. 8, no. 1, pp. 13, Jun. 2001. [10] Y. Dong and S. Xu, A new directional weighted median lter for removal of random-valued impulse noise, IEEE Signal Process. Lett., vol. 14, no. 3, pp. 193196, Mar. 2007. [11] S. Peng and L. Lucke, Multi-level adaptive fuzzy lter for mixed noise removal, in Proc. IEEE Int. Symp. Circuits Systems, Seattle, WA, Apr. 1995, vol. 2, pp. 15241527. [12] M. Zhang and B. K. Gunturk, Multiresolution bilateral ltering for imgage denoiseing, IEEE Trans. Image Process., vol. 17, no. 12, pp. 23242333, Dec. 2008. [13] S. Schulte, M. Nachtegael, V. De Witte, D. Van der Weken, and E. E. Kerre, A fuzzy impulse noise detection and reduction method, IEEE Trans. Image Process., vol. 15, no. 5, pp. 11531162, May 2006. [14] T. Chen, K. K. Ma, and L. H. Chen, Tri-state median lter for image denoising, IEEE Trans. Image Process., vol. 8, no. 12, pp. 18341838, Dec. 1999. [15] T. Sun and Y. Neuvo, Detail-preserving median based lters in image processing, Pattern Recognit. Lett., vol. 15, pp. 341347, Apr. 1994. [16] Z. Wang and D. Zhang, Progressive switching median lter for the removal of impulse noise from highly corrupted images, IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol. 46, no. 1, pp. 7880, Jan. 1999. [17] P. E. Ng and K. K. Ma, A switching median lter with boundary discriminative noise detection for extremely corrupted images, IEEE Trans. Image Process., vol. 15, no. 6, pp. 15061516, Jun. 2006. [18] K. S. Srinivasan and D. Ebenezer, A new fast and efcient decisionbased algorithm for removal of high-density impulse noises, IEEE Signal Process. Lett., vol. 14, no. 3, pp. 189192, Mar. 2007. [19] V. Crnojevic, Impulse noise lter with adaptive MAD-based threshold, in Proc. Int. Conf. Image Processing, Mar. 2005, pp. 337340. [20] R. H. Chan, C. W. Ho, and M. Nikolova, Salt-and-pepper noise removal by median-type noise detectors and detail preserving regularization, IEEE Trans. Image Process., vol. 14, no. 10, pp. 14791485, Oct. 2005. [21] H. Hwang and R. A. Haddad, Adaptive median lters: New algorithms and results, IEEE Trans. Image Process., vol. 4, no. 4, pp. 499502, Apr. 1995. [22] H. Xu, G. Zhu, F. Peng, and D. Wang, Adaptive fuzzy switching lter for images corrupted by impulse noise, Pattern Recognit. Lett., vol. 25, pp. 16571663, 2004. [23] N. Alajlan, M. Kamel, and E. Jernigan, Detail preserving impulsive noise removal, Signal Process.: Image Commun., vol. 19, pp. 9931003, 2004.

[24] P. S. Windyga, Fast impulsive noise removal, IEEE Trans. Image Process., vol. 10, no. 1, pp. 173179, Jan. 2001. [25] D. Van De Ville, M. Nachtegael, D. Van der Weken, E. E. Kerre, W. Philips, and I. Lemahieu, Noise reduction by fuzzy image ltering, IEEE Trans. Fuzzy Syst., vol. 11, no. 4, pp. 429436, Aug. 2003. [26] T. Chen and H. R. Wu, Space variant median lters for the restoration of impulse noise corrupted images, IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol. 48, no. 8, pp. 784789, Aug. 2001. [27] W. K. Pratt, Digital Image Processing. New York: Wiley, 1978. [28] I. Pitas and A. N. Venetsanopoulos, Nonlinear mean lters in image processing, IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-34, no. 3, pp. 573584, Jun. 1986. [29] I. Pitas and A. N. Venetsanopoulos, Nonlinear Digital Filters: Principles and Applications. Norwell, MA: Kluwer, 1990. [30] I. Pitas and A. N. Venetsanopoulos, Order statistics in digital image processing, Proc. IEEE, vol. 80, no. 12, pp. 18931921, Dec. 1992. [31] A. K. Jain, Fundamentals of Digital Image Processing. Upper Saddle River, NJ: Prentice-Hall, 1989, Information and System Sciences Series. [32] M. Elad, On the origin of the bilateral lter and ways to improve it, IEEE Trans. Image Process., vol. 11, no. 10, pp. 11411151, Oct. 2002. Chih-Hsing Lin was born in I-Lan, Taiwan, R.O.C., in 1977. He received the B.S. degree in electric engineering from Ming Hsin of Science and Technology University, Hsinchu, Taiwan, in 2000, and the M.S. degree in electric engineering from Chung Yuan Christian University, Chung-Li, Taiwan, R.O.C., in 2002, and is currently working toward the Ph.D. degree in the Institute of Communications Engineering, National Tsing Hua University, Hsinchu. His research interests are delay-locked loops, frequency synthesizers, low power, and image lter design.

Jia-Shiuan Tsai was born in I-Lan, Taiwan, R.O.C., in 1983. He received the B.S. degree in technology application from National Taiwan Normal University, Taipei, Taiwan, in 2006, and the M.S. degree in computer science from National Tsing Hua University, Hsinchu, Taiwan, in 2009. His research interests are image lter, HDR image synthesis.

Ching-Te Chiu received the B.S. and M.S. degree from National Taiwan University, Taipei, Taiwan, R.O.C., in 1986 and 1988, respectively, and the Ph.D. degree from University of Maryland, College Park, Maryland, in 1992, all in electrical engineering. She was an Associate Professor with National Chung Cheng University, Chia-Yi, Taiwan from 1993 to 1994. From 1994 to 1996, she was member of technical staff with AT&T, Murray Hill, NJ, and at Lucent Technologies, Murray Hill, from 1996 to 2000, and with Agere Systems, Santa Clara, CA, from 2000 to 2003. Since 2004, she has joined Department of Computer Science and Institute of Communications Engineering, National Tsing Hua University, Hsinchu, Taiwan. Her research interests are video and communication integrated circuit design. She has been working on high dynamic range tone mapping processor chip design, high speed switch fabric IC design and SERDES interface design. Her previous chip designs include high denition television video decoder, the standard television demodulation, the SONET/SDH mapper and framer, the ATM core/edge switch, 10 Gbps I/P router trafc management, and FEC decoder.