Skip to main content
SpringerLink
Log in

Boosting image denoising effect via low-level noise injection

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In the past decade, supervised denoising models trained on large datasets have demonstrated impressive performance in image denoising due to their superior denoising effect. However, these models lack flexibility and exhibit varying degrees of degradation in denoising performance in practical applications, particularly when the noise distribution of the given noisy images does not match the training images. Our preliminary experiments suggest that even under ideal conditions, the denoised images obtained using these supervised denoising models (also known as preprocessed images) are already very similar to the ground truth images in terms of pixel intensities. Adding low-level noise to a preprocessed image can approximate the intensities of some pixels to their original values, but not all pixels. Based on this observation, we propose a novel two-stage approach to enhance the denoising effect of existing supervised denoisers using a low-noise injection strategy. In the first stage, we use a state-of-the-art supervised denoiser to denoise the given noisy image and obtain a preprocessed image. Then, we repeatedly inject different random low-level Gaussian noises to further improve certain pixels of the preprocessed image. The generated images are used as target images, and we obtain corresponding fine-tuned images within the framework of the unsupervised deep image prior (DIP) method by fully utilizing its flexibility. As a result, we obtain several denoised fine-tuned images that, respectively, approximate the ground truth image at specific pixels and complement each other. In the second stage, these fine-tuned images are fed to an unsupervised fusion network, which fully leverages the complementarity among the sample images to generate a fused image as the final denoised result. Experimental results demonstrate that the proposed method significantly improves the denoising effectiveness of synthetic noisy images, especially far surpassed the state-of-the-art methods in dealing with real noisy images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The data and materials are available from the corresponding author on reasonable request.

References

  1. Chen, H., Gu, J., Liu, Y., Magid, S.A., Dong, C., Wang, Q., Pfister, H., Zhu, L.: Masked image training for generalizable deep image denoising. arXiv preprint arXiv:2303.13132

  2. Zou, Z., Chen, K., Shi, Z., Guo, Y., Ye, J.: Object detection in 20 years: a survey. In: Proceedings of the IEEE

  3. Neshatavar, R., Yavartanoo, M., Son, S., Lee, K.M.: CVF-SID: Cyclic multi-variate function for self-supervised image denoising by disentangling noise from image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17583–17591 (2022)

  4. Zhang, Y., Li, D., Law, K.L., Wang, X., Qin, H., Li, H.: IDR: self-supervised image denoising via iterative data refinement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2098–2107 (2022)

  5. Zhang, Y., Li, K., Li, K., Sun, G., Kong, Y., Fu, Y.: Accurate and fast image denoising via attention guided scaling. IEEE Trans. Image Process. 30, 6255–6265 (2021)

    Article  PubMed  ADS  Google Scholar 

  6. Byun, J., Cha, S., Moon, T.: FBI-denoiser: fast blind image denoiser for poisson-gaussian noise. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5768–5777 (2021)

  7. Pang, T., Zheng, H., Quan, Y., Ji, H.: Recorrupted-to-recorrupted: unsupervised deep learning for image denoising. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2043–2052 (2021)

  8. Cheng, S., Wang, Y., Huang, H., Liu, D., Fan, H., Liu, S.: NBNet: noise basis learning for image denoising with subspace projection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4896–4906 (2021)

  9. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Transactions on Image Processing 16(8), 2080–2095 (2007). https://doi.org/10.1109/TIP.2007.901238

    Article  MathSciNet  PubMed  ADS  Google Scholar 

  10. Buades, A., Coll, B., Morel, J.-M., A non-local algorithm for image denoising. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, pp. 60–65. IEEE (2005)

  11. Gu, S., Zhang, L., Zuo, W., Feng, X.: Weighted nuclear norm minimization with application to image denoising. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2862–2869 (2014). https://doi.org/10.1109/CVPR.2014.366

  12. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017). https://doi.org/10.1109/TIP.2017.2662206

    Article  MathSciNet  PubMed  ADS  Google Scholar 

  13. Ronneberger, O., Fischer, P., Brox, T: U-net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241. Springer (2015)

  14. Liu, P., Zhang, H., Zhang, K., Lin, L., Zuo, W.: Multi-level wavelet-CNN for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 773–782 (2018)

  15. Liu, D., Wen, B., Fan, Y., Loy, C.C., Huang, T.S.: Non-local recurrent network for image restoration. In: Advances in Neural Information Processing Systems, vol. 31

  16. Zhang, K., Zuo, W., Zhang, L.: Ffdnet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Process. 27(9), 4608–4622 (2018)

    Article  MathSciNet  ADS  Google Scholar 

  17. Anwar, S., Barnes, N.: Real image denoising with feature attention. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3155–3164 (2019). https://doi.org/10.1109/ICCV.2019.00325

  18. Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., Timofte, R.: Swinir: image restoration using Swin transformer. In: IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pp. 1833–1844 (2021). https://doi.org/10.1109/ICCVW54120.2021.00210

  19. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.-H.: Restormer: efficient transformer for high-resolution image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5728–5739 (2022)

  20. Guo, S., Yan, Z., Zhang, K., Zuo, W., Zhang, L.: Toward convolutional blind denoising of real photographs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1712–1722 (2019)

  21. Wang, Z., Liu, J., Li, G., Han, H.: Blind2unblind: self-supervised image denoising with visible blind spots In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2027–2036 (2022)

  22. Wang, Z., Fu, Y., Liu, J., Zhang, Y.: LG-BPN: local and global blind-patch network for self-supervised real-world denoising. arXiv preprint arXiv:2304.00534

  23. Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M., Aila, T.: Noise2noise: learning image restoration without clean data. arXiv preprint arXiv:1803.04189

  24. Krull, A., Buchholz, Jug, F.: Noise2void-learning denoising from single noisy images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2129–2137 (2019)

  25. Batson, J., Royer, L.: Noise2self: blind denoising by self-supervision. In: International Conference on Machine Learning, PMLR, pp. 524–533 (2019)

  26. Huang, T., Li, S., Jia, X., Lu, H., Liu, J.: Neighbor2neighbor: self-supervised denoising from single noisy images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14781–14790 (2021)

  27. Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep image prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9446–9454 (2018)

  28. Hendrycks, D., Gimpel, K.: Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415

  29. Mou, C., Zhang, J., Wu, Z.: Dynamic attentive graph learning for image restoration. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4328–4337 (2021)

  30. Ren, C., He, X., Wang, C., Zhao, Z.: Adaptive consistency prior based deep network for image denoising. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8596–8606 (2021)

  31. Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., Timofte, R.: Plug-and-play image restoration with deep denoiser prior. IEEE Trans. Pattern Anal. Mach. Intell. 44(10), 6360–6376 (2022). https://doi.org/10.1109/TPAMI.2021.3088914

    Article  PubMed  Google Scholar 

  32. Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3929–3938 (2017)

  33. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)

    Article  MathSciNet  PubMed  ADS  Google Scholar 

  34. Martin, D., Fowlkes, C., Tal, D., Malik, J: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings 8th IEEE International Conference on Computer Vision. ICCV 2001, vol. 2, pp. 416–423. IEEE (2001)

  35. Plotz, T., Roth, S.: Benchmarking denoising algorithms with real photographs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1586–1595 (2017)

  36. Xu, J., Li, H., Liang, Z., Zhang, D., Zhang, L.: Real-world noisy image denoising: a new benchmark. arXiv preprint arXiv:1804.02603

Download references

Funding

This research was funded by Natural Science Foundation of China grant no. (62162043), Jiangxi Postgraduate Innovation Special Fund Project, grant no. (YC2022-s033).

Author information

Authors and Affiliations

  1. School of Mathematics and Computer Sciences, Nanchang University, Nanchang, 330031, China

    Jian Xiao, Xiaohui Cheng, Shaoping Xu, Wuyong Tao & Yanyang Xiao

Authors
  1. Jian Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaohui Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shaoping Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wuyong Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yanyang Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SX contributed to the conception of the study, and JX wrote the main manuscript text, and XC and WT contributed significantly to analysis and manuscript preparation, and YX conducted experiments. All authors reviewed the manuscript.

Corresponding author

Correspondence to Shaoping Xu.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiao, J., Cheng, X., Xu, S. et al. Boosting image denoising effect via low-level noise injection. SIViP 18, 1053–1067 (2024). https://doi.org/10.1007/s11760-023-02785-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02785-8

Keywords

Search

Navigation