Skip to main content
SpringerLink
Log in

SW/SE-CNN: semi-wavelet and specific image edge extractor CNN for Gaussian image denoising

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

A Correction to this article was published on 05 February 2024

This article has been updated

Abstract

Several state-of-the-art convolutional neural networks (CNNs)-based methods are available for image denoising tasks. CNNs are typically trained using the backpropagation algorithm, which requires all operations in the network to be differentiable. Most CNN operations satisfy this requirement and can be applied to backpropagation-based training algorithms. However, some transforms, including wavelet transform, which is useful for speeding up CNN computations as well as performing multi-resolution analysis, are not strictly differentiable. This paper addresses this challenge by proposing a wavelet-like transform that is differentiable. This new design is, in fact, a new CNN architecture named semi-wavelet, specific edge convolutional neural network (SW/SE-CNN), consisting of three newly designed layers. The first layer is a Semi-Wavelet (SW)-based layer which is a differential down-sampling operator for wavelet approximation. That is, the SW layer converts the input image into four channels. Three of these channels are estimations of the vertical, horizontal, and diagonal edges of the original image; and the fourth channel is a down-sampled version of it. The second proposed layer, called Semi-Wavelet Inverse (SWI), is to restore the original image by using the four SW output channels. Additionally, a specific edge extractor (SE), as another new layer, is designed on the basis of the well-known Sobel operator to extract specific edges of the image. The reason behind proposing the SE layer is to provide more edge information for the network; and the motive for including the SW layer is to speed the network up as well as multi-resolution analysis. Then, the new SW/SE-CNN architecture is implemented for Gaussian image denoising. The experimental results show that the new SW/SE-CNN outperformed the state-of-the-art methods for Gaussian image denoising based on the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) measurements for grayscale as well as color images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Change history

References

  1. Gonzalez RC, Woods RE (2007) Image processing. Digital Image Processing 2:1

    Google Scholar 

  2. Dabov K, Foi A, Katkovnik V, Egiazarian K (2007) Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans Image Process 16(8):2080–2095

    ADS  MathSciNet  PubMed  Google Scholar 

  3. Gu S, Zhang L, Zuo W and Feng X (2014) Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp 2862–2869

  4. Liu P, Zhang H, Zhang K, Lin L and Zuo W (2018) Multi-level wavelet-CNN for image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2018, pp 773–782

  5. Tian C, Xu Y, Zuo W (2020) Image denoising using deep CNN with batch renormalization. Neural Netw 121:461–473

    PubMed  Google Scholar 

  6. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans Image Process 26(7):3142–3155

    ADS  MathSciNet  PubMed  Google Scholar 

  7. Zhang K, Zuo W, Zhang L (2018) FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans Image Process 27(9):4608–4622

    ADS  MathSciNet  Google Scholar 

  8. Goyal B, Dogra A, Agrawal S, Sohi B, Sharma A (2020) Image denoising review: From classical to state-of-the-art approaches. Inf Fusion 55:220–244

    Google Scholar 

  9. Ilesanmi AE, Ilesanmi TO (2021) Methods for image denoising using convolutional neural network: a review. Complex Intell Syst 7(5):2179–2198

    Google Scholar 

  10. Rekha H, Samundiswary P (2023) Image denoising using fast non-local means filter and multi-thresholding with harmony search algorithm for WSN. Int J Adv Intell Paradig 24(1–2):92–109

    Google Scholar 

  11. Shao L, Yan R, Li X, Liu Y (2013) From heuristic optimization to dictionary learning: A review and comprehensive comparison of image denoising algorithms. IEEE Trans Cybern 44(7):1001–1013

    PubMed  Google Scholar 

  12. Benesty J, Chen J and Huang Y (2010) Study of the widely linear Wiener filter for noise reduction. In: 2010 IEEE international conference on acoustics, speech and signal processing, 2010: IEEE, pp. 205–208

  13. Teng Y, Zhang Y, Chen Y, Ti C (2015) Adaptive morphological filtering method for structural fusion restoration of hyperspectral images. IEEE J Sel Top Appl Earth Obs Remote Sens 9(2):655–667

    ADS  Google Scholar 

  14. Hardie RC, Barner KE (1994) Rank conditioned rank selection filters for signal restoration. IEEE Trans Image Process 3(2):192–206

    ADS  CAS  PubMed  Google Scholar 

  15. Buades A, Coll B and Morel J-M (2005) A non-local algorithm for image denoising. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), 2005, 2:60–65

  16. Xiong Z, Ramchandran K, Orchard MT, Zhang Y-Q (1999) A comparative study of DCT-and wavelet-based image coding. IEEE Trans Circuits Syst Video Technol 9(5):692–695

    Google Scholar 

  17. Fathi A, Naghsh-Nilchi AR (2012) Efficient image denoising method based on a new adaptive wavelet packet thresholding function. IEEE Trans Image Process 21(9):3981–3990

    ADS  MathSciNet  PubMed  Google Scholar 

  18. Starck J-L, Candès EJ, Donoho DL (2002) The curvelet transform for image denoising. IEEE Trans Image Process 11(6):670–684

    ADS  MathSciNet  PubMed  Google Scholar 

  19. Huang Q, Hao B, Chang S (2016) Adaptive digital ridgelet transform and its application in image denoising. Digit Signal Process 52:45–54

    Google Scholar 

  20. Xu J, Yang L, Wu D (2010) Ripplet: a new transform for image processing. J Vis Commun Image Represent 21(7):627–639

    Google Scholar 

  21. Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15(12):3736–3745

    ADS  MathSciNet  PubMed  Google Scholar 

  22. Elad M and Aharon M (2006) Image denoising via learned dictionaries and sparse representation. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06) 1:895–900

  23. Chatterjee P, Milanfar P (2009) Clustering-based denoising with locally learned dictionaries. IEEE Trans Image Process 18(7):1438–1451

    ADS  MathSciNet  PubMed  Google Scholar 

  24. Mairal J, Bach F, Ponce J, Sapiro G and Zisserman A (2009) Non-local sparse models for image restoration. In 2009 IEEE 12th international conference on computer vision, 2009: IEEE, pp. 2272–2279

  25. Yang H-Y, Wang X-Y, Niu P-P, Liu Y-C (2014) Image denoising using nonsubsampled shearlet transform and twin support vector machines. Neural Netw 57:152–165

    PubMed  Google Scholar 

  26. Zhu X, Milanfar P (2010) Automatic parameter selection for denoising algorithms using a no-reference measure of image content. IEEE Trans Image Process 19(12):3116–3132

    ADS  MathSciNet  PubMed  Google Scholar 

  27. Cao M, Li S, Wang R, Li N (2015) Interferometric phase denoising by median patch-based locally optimal wiener filter. IEEE Geosci Remote Sens Lett 12(8):1730–1734

    ADS  Google Scholar 

  28. Wei H, Zheng W (2021) Image denoising based on improved gaussian mixture model. Sci Program 2021:1–8

    Google Scholar 

  29. Hua T, Li Q, Dai K, Zhang X, Zhang H (2023) Image denoising via neighborhood-based multidimensional Gaussian process regression. Signal Image Video Process 17(2):389–397

    Google Scholar 

  30. Buades A, Coll B, Morel J-M (2008) Nonlocal image and movie denoising. Int J Comput Vision 76(2):123–139

    Google Scholar 

  31. Dong W, Zhang L, Shi G, Li X (2012) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630

    ADS  MathSciNet  PubMed  Google Scholar 

  32. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D 60(1–4):259–268

    ADS  MathSciNet  Google Scholar 

  33. Osher S, Burger M, Goldfarb D, Xu J, Yin W (2005) An iterative regularization method for total variation-based image restoration. Multiscale Model Simul 4(2):460–489

    MathSciNet  Google Scholar 

  34. Weiss Y and Freeman WT (2007) What makes a good model of natural images? In: 2007 IEEE conference on computer vision and pattern recognition, 2007: IEEE, pp. 1–8

  35. Roth S, Black MJ (2009) Fields of experts. Int J Comput Vis 82(2):205

    Google Scholar 

  36. Li SZ (2009) Markov random field modeling in image analysis. Springer

    Google Scholar 

  37. Lan X, Roth S, Huttenlocher D and Black MJ (2006) Efficient belief propagation with learned higher-order Markov random fields. In: European conference on computer vision, Springer, pp. 269–282

  38. Monma Y, Aro K, Yasuda M (2022) Hierarchical Gaussian Markov random field for image denoising. IEICE Trans Inf Syst 105(3):689–699

    Google Scholar 

  39. Li C, Yin W, Jiang H, Zhang Y (2013) An efficient augmented Lagrangian method with applications to total variation minimization. Comput Optim Appl 56(3):507–530

    MathSciNet  Google Scholar 

  40. Schmidt U and Roth S (2014) Shrinkage fields for effective image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 2774–2781

  41. Chen Y, Pock T (2016) Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans Pattern Anal Mach Intell 39(6):1256–1272

    PubMed  Google Scholar 

  42. Chen Y, Yu W and Pock T (2015) On learning optimized reaction diffusion processes for effective image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 5261–5269

  43. Burger HC, Schuler CJ, Harmeling S (2012) Image denoising: Can plain neural networks compete with BM3D?. In: 2012 IEEE conference on computer vision and pattern recognition, IEEE, pp. 2392–2399

  44. Chen J, Chen J, Chao H and Yang M (2018) Image blind denoising with generative adversarial network based noise modeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3155–3164

  45. Mao X, Shen C and Yang Y-B (2016) Image restoration using very deep convolutional encoder–decoder networks with symmetric skip connections. In: Advances in neural information processing systems, pp. 2802–2810

  46. Xie J, Xu L and Chen E (2012) Image denoising and inpainting with deep neural networks. In: Advances in neural information processing systems, pp. 341–349

  47. Zhang L, Zuo W (2017) Image restoration: From sparse and low-rank priors to deep priors [lecture notes]. IEEE Signal Process Mag 34(5):172–179

    ADS  Google Scholar 

  48. Dong W, Wang P, Yin W, Shi G, Wu F, Lu X (2018) Denoising prior driven deep neural network for image restoration. IEEE Trans Pattern Anal Mach Intell 41(10):2305–2318

    PubMed  Google Scholar 

  49. Cruz C, Foi A, Katkovnik V, Egiazarian K (2018) Nonlocality-reinforced convolutional neural networks for image denoising. IEEE Signal Process Lett 25(8):1216–1220

    ADS  Google Scholar 

  50. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Google Scholar 

  51. Ioffe S and Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint: arXiv:1502.03167

  52. He K, Zhang X, Ren S and Sun J (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  53. Mallat S (1999) A wavelet tour of signal processing. Elsevier

    Google Scholar 

  54. Santhanam V, Morariu VI and Davis LS (2017) Generalized deep image to image regression. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5609–5619

  55. Zhang K, Zuo W, Gu S and Zhang L (2017) Learning deep CNN denoiser prior for image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3929–3938

  56. Mao X-J, Shen C and Yang Y-B (2016) Image restoration using convolutional auto-encoders with symmetric skip connections. arXiv preprint: arXiv:1606.08921

  57. Bae W, Yoo J and Chul Ye J (2017) Beyond deep residual learning for image restoration: Persistent homology-guided manifold simplification. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 145–153

  58. Ronneberger O, Fischer P and Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, pp. 234–241

  59. Zhang M, Yang C, Yuan Y, Guan Y, Wang S, Liu Q (2021) Multi-wavelet guided deep mean-shift prior for image restoration. Signal Process Image Commun 99:116449

    Google Scholar 

  60. Tian C, Zheng M, Zuo W, Zhang B, Zhang Y, Zhang D (2023) Multi-stage image denoising with the wavelet transform. Pattern Recogn 134:109050

    Google Scholar 

  61. Tai Y, Yang J, Liu X and Xu C (2017) Memnet: a persistent memory network for image restoration. In: Proceedings of the IEEE international conference on computer vision, pp. 4539–4547

  62. Gou Y, Hu P, Lv J, Zhou JT, Peng X (2022) Multi-scale adaptive network for single image denoising. Adv Neural Inf Process Syst 35:14099–14112

    Google Scholar 

  63. Ren C, He X, Wang C and Zhao Z (2021) Adaptive consistency prior based deep network for image denoising. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8596–8606

  64. Zhang Y, Li K, Li K, Zhong B and Fu Y (2019) Residual non-local attention networks for image restoration. arXiv preprint: arXiv:1903.10082

  65. Jia X, Liu S, Feng X and Zhang L (2019) Focnet: a fractional optimal control network for image denoising. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6054–6063.

  66. Xia Z and Chakrabarti A (2020) Identifying recurring patterns with deep neural networks for natural image denoising. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 2426–2434

  67. Peng Y, Zhang L, Liu S, Wu X, Zhang Y, Wang X (2019) Dilated residual networks with symmetric skip connection for image denoising. Neurocomputing 345:67–76

    Google Scholar 

  68. Quan Y, Chen Y, Shao Y, Teng H, Xu Y, Ji H (2021) Image denoising using complex-valued deep CNN. Pattern Recogn 111:107639

    Google Scholar 

  69. Tian C, Xu Y, Zuo W, Du B, Lin C-W, Zhang D (2021) Designing and training of a dual CNN for image denoising. Knowl-Based Syst 226:106949

    Google Scholar 

  70. Wang P et al. (2018) Understanding convolution for semantic segmentation. In: 2018 IEEE winter conference on applications of computer vision (WACV), IEEE, pp. 1451–1460

  71. Rosenfeld A (2013) Multiresolution image processing and analysis. Springer

    Google Scholar 

  72. Heaton J, Goodfellow I, Bengio Y and Courville A (2016) Deep learning. The MIT Press, 2016, 800 pp, ISBN: 0262035618. Genetic programming and evolvable machines, 19(1–2):pp. 305–307, 2018

  73. Araujo A, Norris W, Sim J (2019) Computing receptive fields of convolutional neural networks. Distill 4(11):e21

    Google Scholar 

  74. Bao H (2019) Investigations of the influences of a CNN's receptive field on segmentation of subnuclei of bilateral amygdalae. arXiv preprint: arXiv:1911.02761

  75. Yu F and Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint: arXiv:1511.07122

  76. Ma K et al (2016) Waterloo exploration database: new challenges for image quality assessment models. IEEE Trans Image Process 26(2):1004–1016

    ADS  MathSciNet  Google Scholar 

  77. Agustsson E and Timofte R (2017) Ntire 2017 challenge on single image super-resolution: dataset and study. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 126–135

  78. Martin D, Fowlkes C, Tal D and Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings eighth IEEE international conference on computer vision. ICCV 2001, 2001, vol. 2:416–423

  79. Huang J-B, Singh A and Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5197–5206

  80. Moorthy AK, Bovik AC (2009) Visual importance pooling for image quality assessment. IEEE J Sel Top Signal Process 3(2):193–201

    ADS  Google Scholar 

  81. Lei Ba J, Kiros JR and Hinton GE (2016) Layer normalization. ArXiv e-prints: arXiv:1607.06450

  82. Zhang L, Wu X, Buades A, Li X (2011) Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J Electron Imaging 20(2):023016

    ADS  Google Scholar 

  83. Kingma DP and Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint: arXiv:1412.6980

  84. Vedaldi A and Lenc K (2015) Matconvnet: convolutional neural networks for matlab. In: Proceedings of the 23rd ACM international conference on Multimedia, pp. 689–692

  85. Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Process Lett 9(3):81–84

    ADS  CAS  Google Scholar 

  86. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    ADS  PubMed  Google Scholar 

  87. Vaswani A et al. (2017) Attention is all you need. Advances in neural information processing systems, vol. 30

Download references

Author information

Authors and Affiliations

  1. University of Isfahan, Isfahan, Iran

    Shahram Esteki & Ahmad R. Naghsh-Nilchi

Authors
  1. Shahram Esteki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ahmad R. Naghsh-Nilchi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmad R. Naghsh-Nilchi.

Ethics declarations

Conflict of interest

Please note that there is no conflict of interests over this article, to the best of our knowledge.

Data availability

Data generated during this study are available from the corresponding author on request.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised to correct the note of Table 3, 4 & 5.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Esteki, S., Naghsh-Nilchi, A.R. SW/SE-CNN: semi-wavelet and specific image edge extractor CNN for Gaussian image denoising. Neural Comput & Applic 36, 5447–5469 (2024). https://doi.org/10.1007/s00521-023-09314-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09314-1

Keywords

Search

Navigation