Documente Academic
Documente Profesional
Documente Cultură
Research Article
Abstract: This study presents an image denoising technique using multiscale non-local means (NLM) filtering combined with
hard thresholding in curvelet domain. The inevitable ringing artefacts in the reconstructed image – due to thresholding – is
further processed using a guided image filter for better preservation of local structures like edges, textures and small details.
The authors decomposed the image into three different curvelet scales including the approximation and the fine scale. The low-
frequency noise in the approximation sub-band and the edges with small textural details in the fine scale are processed
independently using a multiscale NLM filter. On the other hand, the hard thresholding in the remaining coarser scale is applied
to separate the signal and the noise subspace. Experimental results on both greyscale and colour images indicate that the
proposed approach is competitive at lower noise strength with respect to peak signal to noise ratio and structural similarity index
measure and excels in performance at higher noise strength compared with several state-of-the-art algorithms.
1 Introduction key signal components of the image by thresholding out the noise.
The universal VisuShrink [5], the unbiased estimator-based
The restoration of the original image from its noisy observation – SureShrink [6], the Bayesian prior-based BayesShrink [7], the
contaminated during acquisition, reception or transmission – is a statistical co-dependent bivariate shrinkage-based BiShrink [8], the
stepping stone for many image processing or computer vision probabilistic shrinkage function-based ProbShrink [9], the linear
tasks. Often the characteristics of noise depend on the sensor type, weight estimation-based Stein's unbiased risk estimate-linear
pixel dimension, exposure time, ISO speed, temperature, and expansion of thresholds (SURE-LET) [10] and the recently
ambient illumination level. However, in most of the applications, proposed NeighShrink [11] are few thresholding techniques (λγ in
the distribution of noise can be characterised as additive white
(2b)) that were applied in wavelet domain to preserve the high
Gaussian noise (AWGN)
magnitude signal coefficients. However, the inability of wavelets in
y = z + η, (1) representing the curved edges (C2 singularity) degrades the
performance of wavelet-based image denoising approaches. The
where y is the observed (noisy) image, z is the latent image and implementation of anisotropic scaling in curvelet transform aides
η ∈ ℵ(0, σ 2) is the Gaussian noise of zero mean and σ 2 variance. in representing image edges more sparsely compared with
wavelets. Starck et al. [12] employed hard thresholding on the
Available denoising methods can be broadly categorised into
magnitude of complex curvelet coefficients for denoising.
three classes: spatial, transform and dual domain approaches. The
However, these transform domain-based thresholding (shrinkage)
underlying idea behind spatial filtering techniques differs only to
techniques suffer from inevitable ringing artefacts due to a sudden
the extent of the weights (kernels) that are calculated – either
jump in coefficient magnitudes.
locally or non-locally – to estimate different data points in an
Dual domain or hybrid approaches combine the advantages of
image [1]. Bilateral filter (BF) [2], non-local means filter (NLM)
both spatial and transform domain techniques to improve the
[3] and recently proposed guided image filter (GIF) [4] are few
overall image denoising quality. The multi-resolution BF (MBF)
modern edge preserving filters that exploit either local, non-local
incorporated both BF and wavelet thresholding in the
or both self-similarity among the image patches for its restoration
approximation and the detail scales to suppress simultaneously the
in the spatial domain. In contrast to spatial domain approaches,
coarser grain (or low-frequency) and fine grain (or high-frequency)
transform domain techniques represent signals with fewer non-zero
noise [13]. Knaus et al. [14] proposed a dual domain image
coefficients. A threshold, as given in (2b), is employed in this
denoising method by integrating joint BF and short time Fourier
approach to separate the signal and the noise subspace for image
transform-based wavelet shrinkage technique. The inability of
restoration
wavelets in representing edges [15] and the limitation of BF in
preserving gradient direction of edges [4] may degrade the overall
Y γ = Tγ y , (2a)
performance of such techniques. Recently, NLM filter – instead of
BF – has been considered in several kinds of literature to improve
^ Y γ, if | Y γ | > lγ, the performance of hybrid domain approaches [16–18]. However,
Yγ = (2b)
0, otherwise, the block matching 3D collaborative filter (BM3D) excelled in
denoising by grouping the similar (non-local) patches and
^
z^ = Tγ−1 Y γ . (2c) collaboratively filtering the 3D blocks using 1D wavelet
thresholding [19]. Though BM3D is considered the state-of-the-art
technique, it is still unable to denoise few homogeneous regions
Here, the transformed coefficients Y is thresholded at each scale γ
that manifest as low-frequency noise [20]. Moreover, the patch-
and reconstructed using inverse transform T −1 to generate the based methods that search for more number of patches with similar
denoised image z^. local spatial structures may reduce at a higher noise level and thus
The energy-compaction property of several multi-resolution restricts the performance of denoising. Motivated by the fact that
signal transformation T is proved to be suitable for preserving the the residual sparsity among the non-local similar patches can be
Fig. 2 Performance of CT technique – with NLM filtering in the approximation & the fine scale – in terms of
(a) PSNR, (b) SSIM measure for a number of decomposition levels Nγ
To determine the linear coefficients ap′, bp′ a constraint (as Therefore, the filtered image z^ in (11) is obtained as the local
given in (6)) is added to minimise the cost function E ap′, bp′ , combination of some percentage of the input image with its local
which is defined as mean. In addition, the following observations may be noted:
into RGB space to obtain the final denoised image using the colour 4.1 Parameter optimisation
transform, S (inverse transform, S−1, See (15)) matrix as
All the parameters in the proposed image denoising algorithm are
T T uncorrelated to each other. They act independently to improve the
yY, yU, yV = S yR, yG, yB (14) overall performance of the restored image. Therefore, we
separately tuned each parameter at maximum PSNR and SSIM
where yRGB and yYUV is the noisy image in RGB and YUV colour measure for several noise strengths σ on the greyscale images
space, respectively (see (15)) Since the U and V channels are more shown in Fig. 8 (assuming the other parameters are constant). The
corrupted compared with the Y channel, we have selected different average variations in PSNR and SSIM measure for the smoothing
scale dependent constant, k for threshold (as given in Table 1). parameters of NLM filter in the approximation and the fine scale,
However, at higher noise strength, σ ≥ 30 the U and V channels k1 & k2 (Eq.4), the scale-dependent threshold variable k (5) and the
become severely degraded by noise. It requires further processing regularisation parameter of GIF k3 (7) are shown in Figs. 9 & 10
to preserve the finest details of the latent image. Thus, we applied separately for four different noise strengths σ = 10, 25, 50, 75. The
GIF as an additional post-processing step (in the reconstructed optimum value of the scale-dependent constant k as shown in
RGB image) to improve the visual quality of the denoised image at Figs. 9a & 10a ranges between 1.5 and 1.8. Similarly, Figs. 9b, c,
higher noise strength. The same may be ineffective at the lower 10b & c reveal that the NLM filter smoothing parameters must be
noise level. tuned between 0.4 and 0.7, in an approximation sub-band and 0.3–
0.8 in the fine scale, respectively. On the other hand, the
4 Experimental results & discussion regularisation parameter of GIF, k3 in Figs. 9d & 10d provides the
maximum PSNR and SSIM measure at k3 = [2.5, 5.5]. Table 1
The overall performance of the proposed algorithm depends on outlines the final selected values used in our algorithm for both
different regularisation parameters of the filters and the scale- single-channel and colour image denoising.
dependent constant of the curvelet threshold. We investigated the
influence of each parameter to obtain the optimum value for the
highest denoising quality. PSNR and SSIM [28] are adopted in this 4.2 Performance evaluation & discussion
study for quantitative evaluation of image denoising quality. The denoising quality of the proposed algorithm was evaluated
However, for multichannel image denoising, the single channel qualitatively as well as quantitatively for both greyscale and RGB
SSIM measure is extended as the mean of all the indices calculated images. The experiments were conducted on a few standard
for each colour (RGB) channel [29]. In addition, the accuracy of greyscale images as shown in Fig. 8 and on the 24 reference
the proposed algorithm both in terms of denoising quality and run- (colour) images of the TID2008 database [30]. To perpetuate
time complexity is evaluated to measure its suitability in time uniformity in comparison, all the images were resized to 512 × 512
constrained real-time applications. and contaminated with simulated AWGN of standard deviations,
σ = [10, 20, 30, 40, 50, 75]. It may also be noted that the strength of
noise in each RGB channel is considered to be independent and
Fig. 10 Plot of average SSIM measure for varying parameter values
Fig. 9 Plot of average PSNR (in dB) measure for varying parameter
(a) Scale-dependant constant for thresholding (k), (b) NLM filter smoothing parameter
values
in approximation sub-band k1 , (c) Fine scale k2 at different noise strengths σ , (d)
(a) Scale-dependant constant for thresholding (k), (b) NLM filter smoothing parameter
in approximation sub-band k1 , (c) Fine scale k2 at different noise strengths σ , (d) GIF regularisation parameter k3 at different noise strengths σ
Fig. 12 Quantitative assessments for the respective denoised images are
(a) Corrupted image σ = 40 : PSNR: 16.093, SSIM: 0.359, (b) CT [12]: PSNR: 29.116, SSIM: 0.830, (c) NLM [34]: PSNR: 27.507, SSIM: 0.758, (d) MBF [13]: PSNR: 26.413,
SSIM: 0.816, (e) MC-SURE-LET [26]: PSNR: 30.498, SSIM: 0.846, (f) LPG-PCA [29]: PSNR: 30.686, SSIM: 0.819, (g) CBM3D [35]: PSNR: 31.113, SSIM: 0.865, (h) Proposed
method: PSNR: 31.135, SSIM: 0.870. A patch of each image is magnified for better visual interpretation
• The spatial edge-preserving filtering in curvelet domain suppressing visible colour artefacts near smooth regions while
combined with thresholding enhances the performance of image preserving the sharp image edges and texture details. Moreover, it
denoising at all noise strengths σ compared with the individual is also known that the denoising quality of GIF depends on another
methods of NLM filter [3] and CT [12]. external input image called the guidance image. Here, the denoised
• Compared with other combined approaches in [13, 17], the image in Fig. 13c is restored by utilising the local features of the
proposed technique improves the denoising performance at a flash image in Fig. 13a (which acts as the guidance image, G).
higher noise strengths and provides better and comparable However, the proposed method denoise the image by considering
results – with BM3D [19], GSRC [21] & CBM3D [35] – at the content of the input noisy image, exclusively.
lower σ for both grey and colour image. The anisotropic scaling of curvelet to represent edges (C2
smooth) with optimal sparsity and the high intra-scale similarity
For completeness, the competitiveness of the proposed among the coefficients motivated the authors to consider NLM as a
algorithm is compared with the GIF [4], as shown in Fig. 13. The multiscale filter to suppress noise in different scales. Unlike CT
results indicate the superiority of the proposed method in [12], the proposed multiscale filtering is able to distinguish
Table 2 Mean PSNR (dB) and SSIM measure between original and restored images for different denoising techniques on test
image in Fig. 8
PSNR, dB SSIM [28]
Denoised methods σ = 10 σ = 20 σ = 30 σ = 40 σ = 50 σ = 75 σ = 10 σ = 20 σ = 30 σ = 40 σ = 50 σ = 75
CT [12] 32.592 29.594 28.043 26.960 26.134 24.631 0.938 0.880 0.834 0.797 0.766 0.706
NLM [3] 35.103 31.452 29.035 27.252 25.836 23.167 0.956 0.901 0.839 0.776 0.716 0.585
NeighSURE [11] 34.732 31.022 28.978 27.621 26.609 24.910 0.956 0.905 0.858 0.816 0.780 0.707
NSST [31] 35.284 32.027 30.173 28.885 27.874 26.134 0.959 0.919 0.884 0.852 0.822 0.758
PL-PCA [32] 35.608 32.238 30.026 29.069 27.691 26.002 0.964 0.920 0.887 0.857 0.822 0.754
K-SVD [33] 36.129 32.641 30.484 28.930 27.684 25.487 0.963 0.921 0.879 0.838 0.799 0.711
MBF [13] 33.386 30.107 28.322 27.142 26.291 24.875 0.934 0.882 0.841 0.805 0.773 0.700
NLMNT [17] 34.744 31.690 29.597 27.917 26.501 23.699 0.958 0.907 0.853 0.798 0.746 0.635
BM3D [19] 36.152 32.703 30.574 28.766 27.868 25.629 0.965 0.928 0.889 0.850 0.819 0.737
GSRC [21] 35.373 32.324 30.652 29.024 28.153 26.129 0.958 0.918 0.887 0.858 0.823 0.752
Proposed 35.443 32.081 30.526 29.089 27.927 26.261 0.962 0.919 0.896 0.864 0.826 0.759
Table 3 Mean PSNR (dB) and SSIM measure between original and restored images for different denoising techniques on
TID2008 database [30]
PSNR, dB SSIM [28, 29]
Denoising methods σ = 10 σ = 20 σ = 30 σ = 40 σ = 50 σ = 75 σ = 10 σ = 20 σ = 30 σ = 40 σ = 50 σ = 75
CT [12] 32.118 29.117 27.523 26.474 25.698 24.322 0.883 0.809 0.755 0.715 0.684 0.629
NLM [34] 33.577 30.089 27.864 26.151 24.743 22.015 0.9044 0.821 0.741 0.691 0.603 0.566
MBF [13] 32.634 29.050 26.763 24.973 23.459 20.466 0.883 0.801 0.741 0.694 0.655 0.583
SURE-LET [26] 35.723 31.897 29.820 28.429 27.402 25.654 0.935 0.870 0.817 0.772 0.735 0.662
LPG-PCA [29] 34.974 30.894 28.742 27.279 26.215 24.287 0.935 0.862 0.799 0.743 0.694 0.589
CBM3D [35] 36.546 32.719 30.558 29.046 28.052 26.048 0.943 0.882 0.822 0.766 0.733 0.627
Proposed 35.682 32.142 30.154 28.953 28.143 26.204 0.9318 0.874 0.829 0.777 0.745 0.673
between fine edges and noises both in the approximation and the The run-time complexity is compared with the CT technique
fine scale. On the other hand, the spatial domain NLM filter search [12], BM3D [19], MBF [13] and GSRC [21] techniques. The
for similar patches in a larger neighbouring window (which is experiment was conducted on a greyscale image of dimensions
again local [23]) and the similarity among patches further 1024 × 1024, 512 × 512, 256 × 256 and 128 × 128, respectively.
decreases with higher noise strength. Similarly, the inability of BF Each algorithm was executed at least 50 times for four different
in preserving gradient direction and the limitations of the wavelet noise levels, σ = 10, 25, 50, 75. The runtime was computed on a
in representing image edges degrade the performance of MBF [13]. 4.00 GB RAM, 64-bit Intel(R) Core™, i7, 4770 CPU @ 3.40 GHz
The patch-based methods like PL-PCA, LPG-PCA, BM3D and PC using MATLAB R2015a. Fig. 14 illustrates the mean–standard
recently proposed GSRC search for more number of patches with deviation plot of execution time (in seconds) for the corrupted
similar local spatial structures that decrease at a higher noise level Lena image denoised using the above methods. It can be observed
and thus restricts the performance of denoising. However, the that the use of fast filtering algorithms and the limited
competitiveness of the proposed algorithm is comparable with decomposition level of the proposed method accelerates the
several state-of-the-art techniques at lower noise strength but denoising process and it is comparable with [12]. Unlike, BM3D
improves at a higher value of σ for both grey and colour images. [19] and GSRC [21], the complexity of the proposed technique is
independent of noise standard deviation. Moreover, the iterative
4.3 Computational complexity GSRC technique achieves a modest increase in denoising quality
(at higher noise strength) with higher implementation complexity.
The use of fast NLM [34] and GIF O(N) [4] algorithms in our
CT-based hybrid approach aides in minimising the overall
complexity of image denoising. Moreover, the speed of the
5 Conclusion
operation of the proposed method is accelerated by limiting the Most image denoising methods assume that the signal is smooth or
number of decomposition levels to Nγ = 3. piecewise smooth and the noise is oscillatory. However, many fine
structures and small details of an image are as oscillatory as noise,
which poses challenges in image modelling. The proposed signal [12] Starck, J.L., Candès, E.J., Donoho, D.L.: ‘The curvelet transform for image
denoising’, IEEE Trans. Image Process., 2002, 11, (6), pp. 670–684
modelling analysed the image in three curvelet scales to denoise [13] Zhang, M., Gunturk, B.K.: ‘Multiresolution bilateral filtering for image
both the smooth (low-frequency) and oscillatory (high-frequency) denoising’, IEEE Trans. Image Process., 2008, 17, (12), pp. 2324–2333
noise. Unlike spatial domain implementation, curvelet domain [14] Knaus, C., Zwicker, M.: ‘Dual-domain image denoising’. 2013 20th IEEE Int.
implementation of multiscale NLM filter extracts more adequately Conf. on Image Processing (ICIP), 2013, pp. 440–444
[15] Ma, J., Plonka, G.: ‘The curvelet transform’, IEEE Signal Process. Mag.,
the self-similarity among the high correlated coefficients within 2010, 27, (2), pp. 118–133
individual scales. The visual and quantitative results yield the [16] Wu, K., Zhang, X., Ding, M.: ‘Curvelet based non-local means algorithm for
potential of multiscale NLM in exploiting self-similarity among the image denoising’, AEU-Int. J. Electron. Commun., 2014, 68, (1), pp. 37–43
patches (in curvelet domain) even at higher noise strength [17] Kumar, B.S.: ‘Image denoising based on non-local means filter and its method
noise thresholding’, Signal Image Video Process., 2013, 7, (6), pp. 1211–1227
σ ≥ 40 . It is also observed that the implementation of edge [18] Coupé, P., Manjón, J.V., Robles, M., et al.: ‘Adaptive multiresolution non-
preserving (NLM) filter – instead of hard thresholding – in the local means filter for three-dimensional magnetic resonance image
finer scale aides in retaining well-connected edges with small denoising’, IET Image Process., 2012, 6, (5), pp. 558–568
image details. On the other hand, the application of hard [19] Dabov, K., Foi, A., Katkovnik, V., et al.: ‘Image denoising by sparse 3-D
transform-domain collaborative filtering’, IEEE Trans. Image Process., 2007,
thresholding in the coarser scale removed the insignificant noise 16, (8), pp. 2080–2095
coefficients from the high magnitude signals. The inevitable [20] Knaus, C., Zwicker, M.: ‘Progressive image denoising’, IEEE Trans. Image
ringing artefacts in the reconstructed image (which is mostly Process., 2014, 23, (7), pp. 3114–3125
ignored in many transform domain techniques) is further processed [21] Zha, Z., Liu, X., Zhou, Z., et al.: ‘Image denoising via group sparsity residual
constraint’ IEEE Int. Conf. on Acoustics, Speech and Signal Processing
using GIF to keep the sharp image edges with a minimum loss of (ICASSP), New Orleans, LA, 2017, pp. 1787–1791, doi: 10.1109/
fine details. The results on both greyscale and colour images have ICASSP.2017.7952464
shown encouraging quantitative and qualitative improvement [22] Karami, A., Tafakori, L.: ‘Image denoising using generalised Cauchy filter’,
compared with several state-of-the-art techniques at higher noise IET Image Process., 2017, 11, (9), pp. 767–776
[23] Buades, A., Coll, B., Morel, J.M.: ‘Non-local means denoising’, Image
strength. However, the proposed technique unable to preserve the Process. Online, 2011, 1, pp. 208–212
fine-scale details at lower noise strength requires further study and [24] Deledalle, C.A., Duval, V., Salmon, S.: ‘Non-local methods with shape-
it is left as the future scope of the work. adaptive patches (NLM-SAP)’, J. Math. Imag. Vis., 2012, 43, (2), pp. 103–
120
[25] Alecu, A., Munteanu, A., Pizurica, A., et al.: ‘Information-theoretic analysis
6 References of dependencies between curvelet coefficients’. IEEE Int. Conf. on Image
Processing, 2006, pp. 1617–1620
[1] Milanfar, P.: ‘A tour of modern image filtering: new insights and methods, [26] Luisier, F., Blu, T.: ‘SURE-LET multichannel image denoising: interscale
both practical and theoretical’, IEEE Signal Process. Mag., 2013, 30, (1), pp. orthonormal wavelet thresholding’, IEEE Trans. Image Process., 2008, 17,
106–128 (4), pp. 482–492
[2] Tomasi, C., Manduchi, R.: ‘Bilateral filtering for gray and color images’. [27] Podpora, M., Korbas, G.P., Kawala-Janik, A.: ‘YUV vs RGB-choosing a
Sixth Int. Conf. on Computer Vision, 1998, pp. 839–846 color space for human-machine interaction’. Position papers of the 2014
[3] Buades, A., Coll, B., Morel, J.M.: ‘A review of image denoising algorithms, Federated Conf. on Computer Science and Information Systems, 2014, pp.
with a new one’, Multiscale Model. Simul., 2005, 4, (2), pp. 490–530 29–34
[4] He, K., Sun, J., Tang, X..: ‘Guided image filtering’, IEEE Trans. Pattern [28] Wang, Z., Bovik, A.C., Sheikh, H.R., et al.: ‘Image quality assessment: from
Anal. Mach. Intell., 2013, 35, (6), pp. 1397–1409 error visibility to structural similarity’, IEEE Trans. Image Process., 2004, 13,
[5] Donoho, D.L., Johnstone, J.M.: ‘Ideal spatial adaptation by wavelet (4), pp. 600–612
shrinkage’, Biometrika, 1994, 81, (3), pp. 425–455 [29] Zhang, L., Dong, W., Zhang, D., et al.: ‘Two-stage image denoising by
[6] Donoho, D.L., Johnstone, I.M., Kerkyacharian, G., et al.: ‘Wavelet shrinkage: principal component analysis with local pixel grouping’, Pattern Recognit.,
asymptopia?’, J. R. Stat. Soc. B, Methodol., 1995, 57, (2), pp. 301–369 2010, 43, (4), pp. 1531–1549
[7] Chang, S.G., Yu, B., Vetterli, M.: ‘Adaptive wavelet thresholding for image [30] Ponomarenko, N., Lukin, V., Zelensky, A., et al.: ‘Tid2008-a database for
denoising and compression’, IEEE Trans. Image Process., 2000, 9, (9), pp. evaluation of full-reference visual quality assessment metrics’, Adv. Mod.
1532–1546 Radioelectron., 2009, 10, (4), pp. 30–45
[8] Sendur, L., Selesnick, I.W.: ‘Bivariate shrinkage with local variance [31] Easley, G., Labate, D., Lim, W.Q.: ‘Sparse directional image representations
estimation’, IEEE Signal Process. Lett., 2002, 9, (12), pp. 438–441 using the discrete shearlet transform’, Appl. Comput. Harmon. Anal., 2008,
[9] Pizurica, A., Philips, W.: ‘Estimating the probability of the presence of a 21, (1), pp. 25–46
signal of interest in multiresolution single- and multiband image denoising’, [32] Deledalle, C.A., Salmon, J., Dalalyan, A.S.: ‘Image denoising with patch
IEEE Trans. Image Process., 2006, 15, (3), pp. 654–665 based PCA: local versus global’. British Machine Vision Conf. (BMVC),
[10] Blu, T., Luisier, F.: ‘The SURE-LET approach to image denoising’, IEEE 2011, vol. 81, pp. 425–455
Trans. Image Process., 2007, 16, (11), pp. 2778–2786
[11] Dengwen, Z., Wengang, C.: ‘Image denoising with an optimal threshold and
neighbouring window’, Pattern Recognit. Lett., 2008, 29, (11), pp. 1694–1697