Advertisement

Adversarial Training for Dual-Stage Image Denoising Enhanced with Feature Matching

  • Xinyao Sun
  • Navaneeth Kamballur Kottayil
  • Subhayan Mukherjee
  • Irene Cheng
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11010)

Abstract

We propose a dual-stage convolutional neural network, augmented with adversarial training, to address the shortcoming of current convolutional neural networks in image denoising. Our dual-stage approach, coupled with feature matching, is especially effective in recovering fine detail under high noise level. First, we use residual learning denoising to output a preliminary denoised reference image. Then, an image reconstruction denoiser uses a multi-scale feature selection layer, which deploys skip-connections and ResNet blocks to recover the image detail based on the noisy image and the reference image. This dual-stage denoising is augmented with the feedback from a discriminator, which forms an adversarial training framework and guides the denoising towards a clean image construction. The feature matching process embedded in the discriminator ensures that the framework can be generalized to a diverse collection of image content. Experimental results show better denoising performance in public benchmark datasets compared with the state-of-the-art approaches.

Keywords

Image denoising Adversarial training Generative Residual learning Feature matching 

References

  1. 1.
    Abadi, M., et al.: TensorFlow: a system for large-scale machine learning. OSDI 16, 265–283 (2016)Google Scholar
  2. 2.
    Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: International Conference on Machine Learning, pp. 214–223 (2017)Google Scholar
  3. 3.
    Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 60–65. IEEE (2005)Google Scholar
  4. 4.
    Burger, H.C., Schuler, C.J., Harmeling, S.: Image denoising: can plain neural networks compete with BM3D? In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2392–2399. IEEE (2012)Google Scholar
  5. 5.
    Chatterjee, P., Milanfar, P.: Clustering-based denoising with locally learned dictionaries. IEEE Trans. Image Process. 18(7), 1438–1451 (2009)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Chen, F., Zhang, L., Yu, H.: External patch prior guided internal clustering for image denoising. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 603–611 (2015)Google Scholar
  7. 7.
    Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Dong, W., Zhang, L., Shi, G., Li, X.: Nonlocally centralized sparse representation for image restoration. IEEE Trans. Image Process. 22(4), 1620–1630 (2013)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)Google Scholar
  11. 11.
    Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  12. 12.
    Gu, S., Zhang, L., Zuo, W., Feng, X.: Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2862–2869 (2014)Google Scholar
  13. 13.
    Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of Wasserstein GANs. In: Advances in Neural Information Processing Systems, pp. 5769–5779 (2017)Google Scholar
  14. 14.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  15. 15.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)Google Scholar
  16. 16.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  17. 17.
    Levin, A., Nadler, B.: Natural image denoising: optimality and inherent bounds. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2833–2840. IEEE (2011)Google Scholar
  18. 18.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online dictionary learning for sparse coding. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 689–696. ACM (2009)Google Scholar
  19. 19.
    Mao, X., Shen, C., Yang, Y.B.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Advances in Neural Information Processing Systems, pp. 2802–2810 (2016)Google Scholar
  20. 20.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena 60(1–4), 259–268 (1992)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Szegedy, C., et al.: Going deeper with convolutions. In: CVPR (2015)Google Scholar
  22. 22.
    Warde-Farley, D., Bengio, Y.: Improving generative adversarial networks with denoising feature matching (2016)Google Scholar
  23. 23.
    Weiss, Y., Freeman, W.T.: What makes a good model of natural images? In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8. IEEE (2007)Google Scholar
  24. 24.
    Xu, J., Zhang, L., Zuo, W., Zhang, D., Feng, X.: Patch group based nonlocal self-similarity prior learning for image denoising. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 244–252 (2015)Google Scholar
  25. 25.
    Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 479–486. IEEE (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Xinyao Sun
    • 1
  • Navaneeth Kamballur Kottayil
    • 1
  • Subhayan Mukherjee
    • 1
  • Irene Cheng
    • 1
  1. 1.Multimedia Research CenterUniversity of AlbertaEdmontonCanada

Personalised recommendations