Advertisement

Deblurring Natural Image Using Super-Gaussian Fields

  • Yuhang Liu
  • Wenyong DongEmail author
  • Dong Gong
  • Lei Zhang
  • Qinfeng Shi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11205)

Abstract

Blind image deblurring is a challenging problem due to its ill-posed nature, of which the success is closely related to a proper image prior. Although a large number of sparsity-based priors, such as the sparse gradient prior, have been successfully applied for blind image deblurring, they inherently suffer from several drawbacks, limiting their applications. Existing sparsity-based priors are usually rooted in modeling the response of images to some specific filters (e.g., image gradients), which are insufficient to capture the complicated image structures. Moreover, the traditional sparse priors or regularizations model the filter response (e.g., image gradients) independently and thus fail to depict the long-range correlation among them. To address the above issues, we present a novel image prior for image deblurring based on a Super-Gaussian field model with adaptive structures. Instead of modeling the response of the fixed short-term filters, the proposed Super-Gaussian fields capture the complicated structures in natural images by integrating potentials on all cliques (e.g., centring at each pixel) into a joint probabilistic distribution. Considering that the fixed filters in different scales are impractical for the coarse-to-fine framework of image deblurring, we define each potential function as a super-Gaussian distribution. Through this definition, the partition function, the curse for traditional MRFs, can be theoretically ignored, and all model parameters of the proposed Super-Gaussian fields can be data-adaptively learned and inferred from the blurred observation with a variational framework. Extensive experiments on both blind deblurring and non-blind deblurring demonstrate the effectiveness of the proposed method.

Notes

Acknowledgements

This work is in part supported by National Natural Science Foundation of China (No. 61672024, 61170305 and 60873114) and Australian Research Council grants (DP140102270 and DP160100703). Yuhang has been supported by a scholarship from the China Scholarship Council.

Supplementary material

474172_1_En_28_MOESM1_ESM.pdf (7.6 mb)
Supplementary material 1 (pdf 7747 KB)

References

  1. 1.
    Köhler, R., Hirsch, M., Mohler, B., Schölkopf, B., Harmeling, S.: Recording and playback of camera shake: benchmarking blind deconvolution with a real-world database. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7578, pp. 27–40. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33786-4_3CrossRefGoogle Scholar
  2. 2.
    Cho, S., Lee, S.: Fast motion deblurring. In: ACM SIGGRAPH Asia 2009 Papers, pp. 145:1–145:8 (2009)Google Scholar
  3. 3.
    Xu, L., Jia, J.: Two-phase kernel estimation for robust motion deblurring. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 157–170. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15549-9_12CrossRefGoogle Scholar
  4. 4.
    Pan, J., Sun, D., Pfister, H., Yang, M.H.: Blind image deblurring using dark channel prior. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 1628–1636 (2016)Google Scholar
  5. 5.
    Fergus, R., Singh, B., Hertzmann, A., Roweis, S.T., Freeman, W.T.: Removing camera shake from a single photograph. ACM Trans. Graph. 25(25), 787–794 (2006)CrossRefGoogle Scholar
  6. 6.
    Levin, A., Weiss, Y., Durand, F., Freeman, W.T.: Efficient marginal likelihood optimization in blind deconvolution. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 2657–2664 (2011)Google Scholar
  7. 7.
    Babacan, S.D., Molina, R., Katsaggelos, A.K.: Variational Bayesian blind deconvolution using a total variation prior. IEEE Trans. Image Process. 18(1), 12–26 (2009)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Perrone, D., Favaro, P.: Total variation blind deconvolution: the devil is in the details. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 2909–2916 (2014)Google Scholar
  9. 9.
    Tzikas, D., Likas, A., Galatsanos, N.: Variational Bayesian blind image deconvolution with student-T priors. In: IEEE International Conference on Image Processing, pp. 109–112 (2007)Google Scholar
  10. 10.
    Babacan, S.D., Molina, R., Do, M.N., Katsaggelos, A.K.: Bayesian blind deconvolution with general sparse image priors. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 341–355. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33783-3_25CrossRefGoogle Scholar
  11. 11.
    Krishnan, D., Tay, T., Fergus, R.: Blind deconvolution using a normalized sparsity measure. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 233–240. IEEE (2011)Google Scholar
  12. 12.
    Xu, L., Zheng, S., Jia, J.: Unnatural l0 sparse representation for natural image deblurring. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 1107–1114 (2013)Google Scholar
  13. 13.
    Ge, D., Idier, J., Carpentier, E.L.: Enhanced sampling schemes for MCMC based blind Bernoulli Gaussian deconvolution. Signal Process. 91(4), 759–772 (2009)CrossRefGoogle Scholar
  14. 14.
    Roth, S., Black, M.J.: Fields of experts. Int. J. Comput. Vis. 82(2), 205 (2009)CrossRefGoogle Scholar
  15. 15.
    Cho, T.S., Paris, S., Horn, B.K.P., Freeman, W.T.: Blur kernel estimation using the radon transform. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 241–248 (2011)Google Scholar
  16. 16.
    Joshi, N., Szeliski, R., Kriegman, D.J.: PSF estimation using sharp edge prediction. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  17. 17.
    Sun, L., Cho, S., Wang, J., Hays, J.: Edge-based blur kernel estimation using patch priors. In: IEEE International Conference on Computational Photography, pp. 1–8 (2013)Google Scholar
  18. 18.
    Lai, W.S., Ding, J.J., Lin, Y.Y., Chuang, Y.Y.: Blur kernel estimation using normalized color-line priors. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 64–72 (2015)Google Scholar
  19. 19.
    Zhou, Y., Komodakis, N.: A MAP-estimation framework for blind deblurring using high-level edge priors. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 142–157. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10605-2_10CrossRefGoogle Scholar
  20. 20.
    Komodakis, N., Paragios, N.: MRF-based blind image deconvolution. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7726, pp. 361–374. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-37431-9_28CrossRefGoogle Scholar
  21. 21.
    Michaeli, T., Irani, M.: Blind deblurring using internal patch recurrence. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 783–798. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10578-9_51CrossRefGoogle Scholar
  22. 22.
    Gong, D., Tan, M., Zhang, Y., Hengel, A.V.D., Shi, Q.: Blind image deconvolution by automatic gradient activation. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 1827–1836 (2016)Google Scholar
  23. 23.
    Gong, D., Tan, M., Zhang, Y., van den Hengel, A., Shi, Q.: Self-paced kernel estimation for robust blind image deblurring. In: International Conference on Computer Vision, pp. 1661–1670 (2017)Google Scholar
  24. 24.
    Yan, Y., Ren, W., Guo, Y., Wang, R., Cao, X.: Image deblurring via extreme channels prior. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 6978–6986 (2017)Google Scholar
  25. 25.
    Nimisha, T., Singh, A.K., Rajagopalan, A.: Blur-invariant deep learning for blind-deblurring. In: The IEEE International Conference on Computer Vision, vol. 2 (2017)Google Scholar
  26. 26.
    Xu, X., Pan, J., Zhang, Y.J., Yang, M.H.: Motion blur kernel estimation via deep learning. IEEE Trans. Image Process. 27(1), 194–205 (2018)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Zhang, L., Wei, W., Zhang, Y., Shen, C., van den Hengel, A., Shi, Q.: Cluster sparsity field: an internal hyperspectral imagery prior for reconstruction. Int. J. Comput. Vis. 126(8), 797–821 (2018)CrossRefGoogle Scholar
  28. 28.
    Besag, J.: Spatial interaction and the statistical analysis of lattice systems. J. R. Stat. Soc. Ser. B (Methodological) 36(2), 192–236 (1974)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Schmidt, U., Gao, Q., Roth, S.: A generative perspective on MRFs in low-level vision. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 1751–1758 (2010)Google Scholar
  30. 30.
    Weiss, Y., Freeman, W.T.: What makes a good model of natural images? In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)Google Scholar
  31. 31.
    Schmidt, U., Schelten, K., Roth, S.: Bayesian deblurring with integrated noise estimation. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 2625–2632 (2011)Google Scholar
  32. 32.
    Zhang, H., Zhang, Y., Li, H., Huang, T.S.: Generative Bayesian image super resolution with natural image prior. IEEE Trans. Image Process. 21(9), 4054–4067 (2012)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Palmer, J.A., Wipf, D.P., Kreutz-Delgado, K., Rao, B.D.: Variational EM algorithms for non-Gaussian latent variable models. In: Advances in Neural Information Processing Systems, pp. 1059–1066 (2005)Google Scholar
  34. 34.
    Levin, A., Weiss, Y., Durand, F., Freeman, W.T.: Understanding blind deconvolution algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2354 (2011)CrossRefGoogle Scholar
  35. 35.
    Murphy, K.P.: Machine Learning: A Probabilistic Perspective. MIT Press, Cambridge (2012)zbMATHGoogle Scholar
  36. 36.
    Wipf, D., Zhang, H.: Revisiting Bayesian blind deconvolution. J. Mach. Learn. Res. 15(1), 3595–3634 (2014)MathSciNetzbMATHGoogle Scholar
  37. 37.
    Whyte, O., Sivic, J., Zisserman, A., Ponce, J.: Non-uniform deblurring for shaken images. Int. J. Comput. Vis. 98(2), 168–186 (2012)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Hirsch, M., Schuler, C.J., Harmeling, S., Scholkopf, B.: Fast removal of non-uniform camera shake. In: International Conference on Computer Vision, pp. 463–470 (2011)Google Scholar
  39. 39.
    Hirsch, M., Sra, S., Scholkopf, B., Harmeling, S.: Efficient filter flow for space-variant multiframe blind deconvolution. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 607–614 (2010)Google Scholar
  40. 40.
    Martin, D.R., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the International Conference on Computer Vision, vol. 2, no. 11, pp. 416–423 (2002)Google Scholar
  41. 41.
    Levin, A., Weiss, Y., Durand, F., Freeman, W.T.: Understanding and evaluating blind deconvolution algorithms. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 1964–1971 (2009)Google Scholar
  42. 42.
    Levin, A., Fergus, R., Durand, F., Freeman, W.T.: Image and depth from a conventional camera with a coded aperture. ACM Trans. Graph. 26(3), 70 (2007)CrossRefGoogle Scholar
  43. 43.
    Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In: IEEE International Conference on Computer Vision, pp. 479–486 (2011)Google Scholar
  44. 44.
    Shan, Q., Jia, J., Agarwala, A.: High-quality motion deblurring from a single image. ACM Trans. Graph. 27(3), 15–19 (2008)CrossRefGoogle Scholar
  45. 45.
    Whyte, O., Sivic, J., Zisserman, A.: Deblurring shaken and partially saturated images. Int. J. Comput. Vis. 110(2), 185–201 (2014)CrossRefGoogle Scholar
  46. 46.
    He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011)CrossRefGoogle Scholar
  47. 47.
    Krishnan, D., Fergus, R.: Fast image deconvolution using hyper-Laplacian priors. In: Advances in Neural Information Processing Systems, pp. 1033–1041 (2009)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Computer SchoolWuhan UniversityHubeiChina
  2. 2.School of Computer ScienceThe University of AdelaideAdelaideAustralia

Personalised recommendations