Advertisement

Multi-scale Convolutional Neural Networks for Non-blind Image Deconvolution

  • Xuehui Wang
  • Feng Dai
  • Jinli Suo
  • Yongdong Zhang
  • Qionghai Dai
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10736)

Abstract

Image deconvolution appears in many image-related problems. Previous works tried to train neural networks directly on blurry/clean pairs to restore clean images but failed. In this work, we propose a novel neural network, trained end-to-end, pixels-to-pixels, to deblur images from blurry ones. Our key insight is to build multi-scale convolutional neural networks that extract various scale feature maps which is essential for recovering sharp images and removing artifacts. The networks take input image of arbitrary size and produce output within efficient time. We demonstrate that our approach yields better result than the state-of-the-art deconvolution algorithms on a large dataset.

Keywords

Non-blind deconvolution Multi-scale CNN 

Notes

Acknowledgments

This work is supported by National Nature Science Foundation of China (61327013, 61379084, 61402440) and the Key Research Program of the Chinese Academy of Sciences, Grant No. KFZD-SW-407.

References

  1. 1.
    Burger, H.C., Schuler, C.J., Harmeling, S.: Image denoising: can plain neural networks compete with BM3D? In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2392–2399. IEEE (2012)Google Scholar
  2. 2.
    Cho, S., Lee, S.: Fast motion deblurring. ACM Trans. Graph. 28, 145 (2009)CrossRefGoogle Scholar
  3. 3.
    Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image restoration by sparse 3d transform-domain collaborative filtering. In: Electronic Imaging, p. 681207 (2008)Google Scholar
  4. 4.
    Danielyan, A., Katkovnik, V., Egiazarian, K.: BM3D frames and variational image deblurring. IEEE Trans. Image Process. 21(4), 1715–1728 (2012)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Eigen, D., Krishnan, D., Fergus, R.: Restoring an image taken through a window covered with dirt or rain. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 633–640 (2013)Google Scholar
  6. 6.
    Fergus, R., Singh, B., Hertzmann, A., Roweis, S.T., Freeman, W.T.: Removing camera shake from a single photograph. ACM Trans. Graph. 25, 787–794 (2006)CrossRefGoogle Scholar
  7. 7.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: AISTATS, vol. 9, pp. 249–256 (2010)Google Scholar
  8. 8.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  9. 9.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)
  10. 10.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the ACM International Conference on Multimedia, pp. 675–678. ACM (2014)Google Scholar
  11. 11.
    Krishnan, D., Fergus, R.: Fast image deconvolution using hyper-Laplacian priors. In: Advances in Neural Information Processing Systems, pp. 1033–1041 (2009)Google Scholar
  12. 12.
    Levin, A., Fergus, R., Durand, F., Freeman, W.T.: Image and depth from a conventional camera with a coded aperture. ACM Trans. Graph. 26(3), 70 (2007)CrossRefGoogle Scholar
  13. 13.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Google Scholar
  14. 14.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the International Conference on Machine Learning, pp. 807–814 (2010)Google Scholar
  15. 15.
    Roth, S., Black, M.J.: Fields of experts: a framework for learning image priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 860–867. IEEE (2005)Google Scholar
  16. 16.
    Schmidt, U., Rother, C., Nowozin, S., Jancsary, J., Roth, S.: Discriminative non-blind deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 604–611 (2013)Google Scholar
  17. 17.
    Schuler, C.J., Christopher Burger, H., Harmeling, S., Scholkopf, B.: A machine learning approach for non-blind image deconvolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1067–1074 (2013)Google Scholar
  18. 18.
    Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Xie, J., Xu, L., Chen, E.: Image denoising and inpainting with deep neural networks. In: Advances in Neural Information Processing Systems, pp. 341–349 (2012)Google Scholar
  20. 20.
    Xu, L., Jia, J.: Two-phase kernel estimation for robust motion deblurring. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 157–170. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15549-9_12CrossRefGoogle Scholar
  21. 21.
    Xu, L., Ren, J.S., Liu, C., Jia, J.: Deep convolutional neural network for image deconvolution. In: Advances in Neural Information Processing Systems, pp. 1790–1798 (2014)Google Scholar
  22. 22.
    Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 479–486. IEEE (2011)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Xuehui Wang
    • 1
    • 2
  • Feng Dai
    • 1
  • Jinli Suo
    • 3
  • Yongdong Zhang
    • 1
  • Qionghai Dai
    • 3
  1. 1.Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS)Institute of Computing Technology, CASBeijingChina
  2. 2.Graduate University of Chinese Academy of SciencesBeijingChina
  3. 3.Department of AutomationTsinghua UniversityBeijingChina

Personalised recommendations