Advertisement

Blind Image Deblurring via Deep Discriminative Priors

  • Lerenhan Li
  • Jinshan Pan
  • Wei-Sheng Lai
  • Changxin Gao
  • Nong SangEmail author
  • Ming-Hsuan Yang
Article
  • 255 Downloads

Abstract

We present an effective blind image deblurring method based on a data-driven discriminative prior. Our work is motivated by the fact that a good image prior should favor sharp images over blurred ones. In this work, we formulate the image prior as a binary classifier using a deep convolutional neural network. The learned prior is able to distinguish whether an input image is sharp or not. Embedded into the maximum a posterior framework, it helps blind deblurring in various scenarios, including natural, face, text, and low-illumination images, as well as non-uniform deblurring. However, it is difficult to optimize the deblurring method with the learned image prior as it involves a non-linear neural network. In this work, we develop an efficient numerical approach based on the half-quadratic splitting method and gradient descent algorithm to optimize the proposed model. Furthermore, we extend the proposed model to handle image dehazing. Both qualitative and quantitative experimental results show that our method performs favorably against the state-of-the-art algorithms as well as domain-specific image deblurring approaches.

Keywords

Blind mage deblurring Deep learning Discriminative prior 

Notes

Acknowledgements

This work is partially supported by the National Natural Science Foundation of China (Nos. 61433007 and 61872421), the National Science Foundation CAREER (No. 1149783), the Natural Science Foundation of Jiangsu Province (No. BK20180471), and gifts from Adobe and Nvidia. Lerenhan Li is supported by a scholarship from China Scholarship Council.

References

  1. Berman, D., Treibitz, T., & Avidan, S. (2016). Non-local image dehazing. In IEEE conference on computer vision and pattern recognition.Google Scholar
  2. Bigdeli, S. A., Zwicker, M., Favaro, P., & Jin, M. (2017). Deep mean-shift priors for image restoration. In Neural information processing systems.Google Scholar
  3. Boracchi, G., & Foi, A. (2012). Modeling the performance of image restoration from motion blur. IEEE Transactions on Image Processing, 21(8), 3502–3517.MathSciNetCrossRefzbMATHGoogle Scholar
  4. Chakrabarti, A. (2016). A neural approach to blind motion deblurring. In European conference on computer vision.Google Scholar
  5. Cho, S., & Lee, S. (2009). Fast motion deblurring. ACM Transactions on Graphics, 28(5), 145.CrossRefGoogle Scholar
  6. Dong, C., Loy, C. C., He, K., & Tang, X. (2014). Learning a deep convolutional network for image super-resolution. In European conference on computer vision.Google Scholar
  7. Dong, C., Deng, Y., Change Loy, C., & Tang, X. (2015). Compression artifacts reduction by a deep convolutional network. In IEEE international conference on computer vision.Google Scholar
  8. Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., & Freeman, W. T. (2006). Removing camera shake from a single photograph. ACM Transactions on Graphics, 25(3), 787–794.CrossRefzbMATHGoogle Scholar
  9. Glorot, X., & Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics (pp. 249–256).Google Scholar
  10. He, K., Sun, J., & Tang, X. (2011). Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12), 2341–2353.CrossRefGoogle Scholar
  11. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In IEEE Conference on computer vision and pattern recognition.Google Scholar
  12. Hirsch, M., Schuler, C. J., Harmeling, S., & Schölkopf, B. (2011). Fast removal of non-uniform camera shake. In IEEE International conference on computer vision.Google Scholar
  13. Hradiš, M., Kotera, J., Zemcík, P., & Šroubek, F. (2015). Convolutional neural networks for direct text deblurring. In British machine vision conference.Google Scholar
  14. Hu, Z., & Yang, M. H. (2012). Good regions to deblur. In European conference on computer vision.Google Scholar
  15. Hu, Z., Cho, S., Wang, J., & Yang, M. H. (2014). Deblurring low-light images with light streaks. In IEEE conference on computer vision and pattern recognition.Google Scholar
  16. Huiskes, M. J., & Lew, M. S. (2008). The MIR flickr retrieval evaluation. In ACM international conference on multimedia information retrieval.Google Scholar
  17. Jin, M., Hirsch, M., & Favaro, P. (2018). Learning face deblurring fast and wide. In IEEE conference on computer vision and pattern recognition workshops.Google Scholar
  18. Kim, J., Lee, J. K., & Lee, K. M. (2016). Accurate image super-resolution using very deep convolutional networks. In IEEE conference on computer vision and pattern recognition.Google Scholar
  19. Köhler, R., Hirsch, M., Mohler, B., Schölkopf, B., & Harmeling, S. (2012). Recording and playback of camera shake: Benchmarking blind deconvolution with a real-world database. In European conference on computer vision.Google Scholar
  20. Krishnan, D., & Fergus, R. (2009). Fast image deconvolution using hyper-laplacian priors. In Neural information processing systems.Google Scholar
  21. Krishnan, D., Tay, T., & Fergus, R. (2011). Blind deconvolution using a normalized sparsity measure. In IEEE conference on computer vision and pattern recognition.Google Scholar
  22. Kupyn, O., Budzan, V., Mykhailych, M., Mishkin, D., & Matas, J. (2018). Deblurgan: Blind motion deblurring using conditional adversarial networks. In IEEE conference on computer vision and pattern recognition.Google Scholar
  23. Lai, W. S., Ding, J. J., Lin, Y. Y., & Chuang, Y. Y. (2015). Blur kernel estimation using normalized color-line prior. In IEEE conference on computer vision and pattern recognition.Google Scholar
  24. Lai, W. S., Huang, J. B., Ahuja, N., & Yang, M. H. (2017). Deep laplacian pyramid networks for fast and accurate super-resolution. In IEEE conference on computer vision and pattern recognition.Google Scholar
  25. Lai, W. S., Huang, J. B., Hu, Z., Ahuja, N., & Yang, M. H. (2016). A comparative study for single image blind deblurring. In IEEE conference on computer vision and pattern recognition.Google Scholar
  26. Leclaire, A., & Moisan, L. (2013). Blind deblurring using a simplified sharpness index. In International conference on scale space and variational methods in computer vision.Google Scholar
  27. Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2009). Understanding and evaluating blind deconvolution algorithms. In IEEE conference on computer vision and pattern recognition.Google Scholar
  28. Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2011). Efficient marginal likelihood optimization in blind deconvolution. In IEEE conference on computer vision and pattern recognition.Google Scholar
  29. Li, B., Peng, X., Wang, Z., Xu, J., & Feng, D. (2017a). Aod-net: All-in-one dehazing network. In IEEE international conference on computer vision.Google Scholar
  30. Li, B., Peng, X., Wang, Z., Xu, J. Z., & Feng, D. (2017b). Aod-net: All-in-one dehazing network. In IEEE international conference on computer vision.Google Scholar
  31. Li, B., Ren, W., Fu, D., Tao, D., Feng, D., Zeng, W., et al. (2019). Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing, 28(1), 492–505.MathSciNetCrossRefGoogle Scholar
  32. Lin, M., Chen, Q., & Yan, S. (2013). Network in network. arXiv:1312.4400.
  33. Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In IEEE conference on computer vision and pattern recognition.Google Scholar
  34. Mao, X., Shen, C., & Yang, Y. B. (2016). Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In Neural information processing systems.Google Scholar
  35. Michaeli, T., & Irani, M. (2014). Blind deblurring using internal patch recurrence. In European conference on computer vision.Google Scholar
  36. Nah, S., Kim, T. H., & Lee, K. M. (2017). Deep multi-scale convolutional neural network for dynamic scene deblurring. In IEEE conference on computer vision and pattern recognition.  https://doi.org/10.1109/CVPR.2017.35.
  37. Noroozi, M., Chandramouli, P., & Favaro, P. (2017). Motion deblurring in the wild. In German conference on pattern recognition.Google Scholar
  38. Pan, J., Hu, Z., Su, Z., Yang, M. H. (2014a). Deblurring face images with exemplars. In European conference on computer vision.Google Scholar
  39. Pan, J., Hu, Z., Su, Z., Yang, M. H. (2014b). Deblurring text images via l0-regularized intensity and gradient prior. In IEEE conference on computer vision and pattern recognition.Google Scholar
  40. Pan, J., Sun, D., Pfister, H., & Yang, M. H. (2016). Blind image deblurring using dark channel prior. In IEEE conference on computer vision and pattern recognition.Google Scholar
  41. Pan, J., Sun, D., Pfister, H., & Yang, M.-H. (2018). Deblurring images via dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(10), 2315–2328.CrossRefGoogle Scholar
  42. Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., & Yang, M. H. (2016). Single image dehazing via multi-scale convolutional neural networks. In European conference on computer vision.Google Scholar
  43. Schuler, C. J., Hirsch, M., Harmeling, S., & Schölkopf, B. (2016). Learning to deblur. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(7), 1439–1451.CrossRefGoogle Scholar
  44. Shen, Z., Lai, W. S., Xu, T., Kautz, J., & Yang, M. H. (2018). Deep semantic face deblurring. In IEEE conference on computer vision and pattern recognition.Google Scholar
  45. Sreehari, S., Venkatakrishnan, S. V., Wohlberg, B., Buzzard, G. T., Drummy, L. F., Simmons, J. P., et al. (2016). Plug-and-play priors for bright field electron tomography and sparse interpolation. IEEE Transactions on Computational Imaging, 2(4), 408–423.MathSciNetGoogle Scholar
  46. Sun, J., Cao, W., Xu, Z., & Ponce, J. (2015). Learning a convolutional neural network for non-uniform motion blur removal. In IEEE conference on computer vision and pattern recognition.Google Scholar
  47. Sun, L., Cho, S., Wang, J., & Hays, J. (2013). Edge-based blur kernel estimation using patch priors. In IEEE international conference on computational photography.Google Scholar
  48. Tai, Y. W., Tan, P., & Brown, M. S. (2011). Richardson-lucy deblurring for scenes under a projective motion path. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(8), 1603–1618.CrossRefGoogle Scholar
  49. Tao, X., Gao, H., Shen, X., Wang, J., & Jia, J. (2018). Scale-recurrent network for deep image deblurring. In IEEE conference on computer vision and pattern recognition.Google Scholar
  50. Vedaldi, A., & Lenc, K. (2015). Matconvnet: Convolutional neural networks for matlab. In ACM international conference on multimedia.Google Scholar
  51. Whyte, O., Sivic, J., Zisserman, A., & Ponce, J. (2012). Non-uniform deblurring for shaken images. International Journal of Computer Vision, 98(2), 168–186.MathSciNetCrossRefzbMATHGoogle Scholar
  52. Xu, L., & Jia, J. (2010). Two-phase kernel estimation for robust motion deblurring. In European conference on computer vision.Google Scholar
  53. Xu, L., Lu, C., Xu, Y., & Jia, J. (2011). Image smoothing via \(l_0\) gradient minimization. ACM Transactions on Graphics, 30(6), 174.Google Scholar
  54. Xu, L., Zheng, S., & Jia, J. (2013). Unnatural l0 sparse representation for natural image deblurring. In IEEE conference on computer vision and pattern recognition.Google Scholar
  55. Yan, R., & Shao, L. (2016). Blind image blur estimation via deep learning. IEEE Transactions on Image Processing, 25(4), 1910–1921.MathSciNetGoogle Scholar
  56. Yan, Y., Ren, W., Guo, Y., Wang, R., & Cao, X. (2017). Image deblurring via extreme channels prior. In IEEE conference on computer vision and pattern recognition.Google Scholar
  57. Zhang, J., Pan, J., Lai, W. S., Lau, R., & Yang, M. H. (2017a). Learning fully convolutional networks for iterative non-blind deconvolution. In IEEE conference on computer vision and pattern recognition.Google Scholar
  58. Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017b). Learning deep CNN denoiser prior for image restoration. In IEEE conference on computer vision and pattern recognition.Google Scholar
  59. Zhao, P., Cheng, Y., & Pedersen, M. (2015). Objective assessment of perceived sharpness of projection displays with a calibrated camera. In Colour and visual computing symposium.Google Scholar
  60. Zoran, D., & Weiss, Y. (2011). From learning models of natural image patches to whole image restoration. In IEEE international conference on computer vision.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.National Key Laboratory of Science and Technology on Multispectral Information Processing, School of AutomationHuazhong University of Science and TechnologyWuhanChina
  2. 2.Electrical Engineering and Computer ScienceUniversity of CaliforniaMercedUSA
  3. 3.School of Computer Science and EngineeringNanjing University of Science and TechnologyNanjingChina

Personalised recommendations