Skip to main content
SpringerLink
Log in

Pyramidical Based Image Deblurring via Kernel Continuity Prior

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

Blind image deblurring aims to reconstruct a sharp version of a given blurred image by estimating the blur kernel. This is generally a highly ill-posed problem because both the blur kernel and the sharp image are unknown. To solve this problem, different methods use sophisticated image priors to construct an intermediate sharp image and then estimate the blur kernel. In this paper, we utilize the advantage of the pyramid scheme that is usually used in blind deblurring literature, and instead of using complicated constraints to reconstruct the intermediate latent image, we use a super-resolution approach in a scale-by-scale manner. This scheme is performed quickly, although we realized that multi-scale prior leads to noisy kernels, and the estimated blur kernels are not sufficiently continuous. We focus on kernel regularization to encourage sparsity and continuity, thus we add a kernel continuity constraint to the optimization function. Finally, by conducting experiments and providing quantitative and visual results, we demonstrate that the proposed deblurring method is fast and provides acceptable results for blurred cases where state-of-the-art methods fail.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data Availability

The datasets generated during and/or analysed during the current study are available in the [Lai] repository, [http://vllab.ucmerced.edu/wlai24/cvpr16_deblur_study/].

References

  1. Y. Bai, G. Cheung, X. Liu, W. Gao, Graph-based blind image deblurring from a single photograph. IEEE Trans. Image Process. 28(3), 1404–1418 (2019). https://doi.org/10.1109/TIP.2018.2874290

    Article  MathSciNet  Google Scholar 

  2. Y. Bai, H. Jia, M. Jiang, X. Liu, X. Xie, W. Gao, Single-image blind deblurring using multi-scale latent structure prior. IEEE Trans. Circuits Syst. Video Technol. 30(7), 2033–2045 (2020). https://doi.org/10.1109/TCSVT.2019.2919159

    Article  Google Scholar 

  3. S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011). https://doi.org/10.1561/2200000016

    Article  MATH  Google Scholar 

  4. T.F. Chan, C.K. Wong, Total variation blind deconvolution. IEEE Trans. Image Process. 7(3), 370–375 (1998). https://doi.org/10.1109/83.661187

    Article  Google Scholar 

  5. M. Chang, H. Feng, Z. Xu, Q. Li, Low-light image restoration with short- and long-exposure raw pairs. IEEE Trans. Multimed. 24, 702–714 (2022). https://doi.org/10.1109/TMM.2021.3058586

    Article  Google Scholar 

  6. L. Chen, F. Fang, S. Lei, F. Li, G. Zhang, “Enhanced sparse model for blind deblurring,” European Conference on Computer Vision, pp. 631–646, 2020. https://doi.org/10.1007/978-3-030-58595-2_38

  7. X. Chen, Y. Zhu, J. Sun, Y. Zhang, Robust motion blur kernel estimation by kernel continuity prior. IEEE Access 8, 46162–46175 (2020). https://doi.org/10.1109/ACCESS.2019.2961597

    Article  Google Scholar 

  8. S. Cho, S. Lee, Fast motion deblurring. ACM Trans. Graph. 28(5), 145 (2009). https://doi.org/10.1145/1618452.1618491

    Article  Google Scholar 

  9. A. Eqtedaei, A. Ahmadyfard, Coarse-to-fine blind image deblurring based on K-means clustering. Vis. Comput. (2023). https://doi.org/10.1007/s00371-023-02785-2

    Article  Google Scholar 

  10. R. Fergus, B. Singh, A. Hertzmann, S.T. Roweis, W.T. Freeman, Removing camera shake from a single photograph. ACM Trans. Graph. 25(3), 787–794 (2006). https://doi.org/10.1145/1141911.1141956

    Article  MATH  Google Scholar 

  11. G. Freedman, R. Fattal, Image and video upscaling from local self-examples. ACM Trans. Graph. (2011). https://doi.org/10.1145/1944846.1944852

    Article  Google Scholar 

  12. T. Goldstein, S. Osher, The split Bregman method for L1-regularized problems. SIAM J. Imag. Sci. 2(2), 323–343 (2009). https://doi.org/10.1137/080725891

    Article  MathSciNet  MATH  Google Scholar 

  13. S. Gu, L. Zhang, W. Zuo, and X. Feng, “Weighted nuclear norm minimization with application to image denoising,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 2862–2869, 2014. https://doi.org/10.1109/CVPR.2014.366

  14. K. He, J. Sun, and X. Tang, “Guided Image Filtering,” European Conference on Computer Vision, pp. 1–14, 2010. https://doi.org/10.1007/978-3-642-15549-9_1

  15. Z. Hu and M.H. Yang, “Good regions to deblur,” European Conference on Computer Vision, pp. 59–72, 2012. https://doi.org/10.1007/978-3-642-33715-4_5

  16. N. Joshi, R. Szeliski, and D.J. Kriegman, “PSF estimation using sharp edge prediction,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, 2008. https://doi.org/10.1109/CVPR.2008.4587834

  17. J. Kim, J.K. Lee, and K.M. Lee, “Accurate image super-resolution using very deep convolutional networks,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654, 2016. https://doi.org/10.1109/CVPR.2016.182

  18. J. Kotera, F. Šroubek, P. Milanfar, Blind deconvolution using alternating maximum a posteriori estimation with heavy-tailed priors. Int. Conf. Comput. Anal. Images Patterns 2(8048), 59–66 (2013). https://doi.org/10.1007/978-3-642-40246-3_8

    Article  MathSciNet  Google Scholar 

  19. D. Krishnan and R. Fergus, “Fast image deconvolution using hyper-laplacian priors,” Proceedings of Neural Information Processing Systems, pp. 1033–1041, 2009

  20. D. Krishnan, T. Tay, and R. Fergus, “Blind deconvolution using a normalized sparsity measure,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 233–240, 2011. https://doi.org/10.1109/CVPR.2011.5995521

  21. W.S. Lai, J. Bin Huang, N. Ahuja, and M.H. Yang, “Deep laplacian pyramid networks for fast and accurate super-resolution,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 5835–5843, 2017. https://doi.org/10.1109/CVPR.2017.618

  22. W.S. Lai, J. Bin Huang, Z. Hu, N. Ahuja, and M.H. Yang, “A comparative study for single image blind deblurring,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1709, 2016. https://doi.org/10.1109/CVPR.2016.188

  23. A. Levin, Y. Weiss, F. Durand, and W.T. Freeman, “Understanding and evaluating blind deconvolution algorithms,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 1964–1971, 2009. https://doi.org/10.1109/CVPR.2009.5206815

  24. A. Levin, Y. Weiss, F. Durand, W.T. Freeman, Understanding blind deconvolution algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2354–2367 (2011). https://doi.org/10.1109/TPAMI.2011.148

    Article  Google Scholar 

  25. X. Li, Y. Jiang, J. Zhang, M. Li, H. Luo, S. Yin, Lesion-attention pyramid network for diabetic retinopathy grading. Artif. Intell. Med. 126, 102259 (2022). https://doi.org/10.1016/j.artmed.2022.102259

    Article  Google Scholar 

  26. X. Li, Y. Jiang, Y. Liu, J. Zhang, S. Yin, H. Luo, RAGCN: region aggregation graph convolutional network for bone age assessment from x-ray images. IEEE Trans. Instrum. Meas. 71, 1–12 (2022). https://doi.org/10.1109/TIM.2022.3190025

    Article  Google Scholar 

  27. X.J. Mao, C. Shen, and Y. Bin Yang, “Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections,” Advances in neural information processing systems, pp. 2810–2818, 2016. https://doi.org/10.48550/arXiv.1603.09056

  28. T. Michaeli and M. Irani, “Blind deblurring using internal patch recurrence,” European Conference on Computer Vision, pp. 783–798, 2014. https://doi.org/10.1007/978-3-319-10578-9_51

  29. S. Nah, T.H. Kim, and K.M. Lee, “Deep multi-scale convolutional neural network for dynamic scene deblurring,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 257–265, 2017. https://doi.org/10.1109/CVPR.2017.35

  30. S. Osher, L.I. Rudin, Feature-oriented image enhancement using shock filters. SIAM J. Numer. Anal. 27(4), 919–940 (1990). https://doi.org/10.1109/TPAMI.2012.213

    Article  MATH  Google Scholar 

  31. J. Pan, Z. Hu, Z. Su, M.H. Yang, L0-regularized intensity and gradient prior for deblurring text images and beyond. IEEE Trans. Pattern Anal. Mach. Intell. 39(2), 342–355 (2017). https://doi.org/10.1109/TPAMI.2016.2551244

    Article  Google Scholar 

  32. J. Pan, Z. Lin, Z. Su, and M.H. Yang, “Robust Kernel Estimation with Outliers Handling for Image Deblurring,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 2800–2808, 2016. https://doi.org/10.1109/CVPR.2016.306

  33. J. Pan, R. Liu, Z. Su, X. Gu, Kernel estimation from salient structure for robust motion deblurring. Signal Process. Image Commun. 28(9), 1156–1170 (2013). https://doi.org/10.1016/j.image.2013.05.001

    Article  Google Scholar 

  34. J. Pan, R. Liu, Z. Su and G. Liu, “Motion blur kernel estimation via salient edges and low rank prior,” IEEE International Conference on Multimedia and Expo, pp. 1–6, 2014. https://doi.org/10.1109/ICME.2014.6890182

  35. J. Pan, W. Ren, Z. Hu, M.H. Yang, Learning to deblur images with exemplars. IEEE Trans. Pattern Anal. Mach. Intell. 41(6), 1412–1425 (2019). https://doi.org/10.1109/TPAMI.2018.2832125

    Article  Google Scholar 

  36. J. Pan, Z. Su, Fast ℓ0-regularized Kernel estimation for robust motion deblurring. IEEE Signal Process. Lett. 20(9), 841–844 (2013). https://doi.org/10.1109/LSP.2013.2261986

    Article  Google Scholar 

  37. J. Pan, D. Sun, H. Pfister, and M. H. Yang, “Blind image deblurring using dark channel prior,” IEEE Conference on Computer Vision and Pattern Recognition, vol. 2016-Decem, pp. 1628–1636, 2016. https://doi.org/10.1109/TPAMI.2017.2753804

  38. N. Parikh, S. Boyd, Proximal algorithms. Found. Trends Optim. 1(3), 127–239 (2014). https://doi.org/10.1561/2400000003

    Article  Google Scholar 

  39. W. Ren, X. Cao, J. Pan, X. Guo, W. Zuo, M.H. Yang, Image deblurring via enhanced low-rank prior. IEEE Trans. Image Process. 25(7), 3426–3437 (2016). https://doi.org/10.1109/TIP.2016.2571062

    Article  MathSciNet  MATH  Google Scholar 

  40. C.J. Schuler, M. Hirsch, S. Harmeling, B. Scholkopf, Learning to deblur. IEEE Trans. Pattern Anal. Mach. Intell. 38(7), 1439–1451 (2016). https://doi.org/10.1109/TPAMI.2015.2481418

    Article  Google Scholar 

  41. Q. Shan, J. Jia, A. Agarwala, High-quality motion deblurring from a single image. ACM Trans. Graph. (2008). https://doi.org/10.1145/1360612.1360672

    Article  Google Scholar 

  42. L. Sun, S. Cho, J. Wang, and J. Hays, “Edge-based blur kernel estimation using patch priors,” IEEE International Conference on Computational Photography, pp. 1–8, 2013. https://doi.org/10.1109/ICCPhot.2013.6528301

  43. F. Wen, R. Ying, Y. Liu, P. Liu, T.-K. Truong, A simple local minimal intensity prior and an improved algorithm for blind image deblurring. IEEE Trans. Circuits Syst. Video Technol. 31(8), 2923–2937 (2021). https://doi.org/10.1109/TCSVT.2020.3034137

    Article  Google Scholar 

  44. L. Xu and J. Jia, “Two-phase kernel estimation for robust motion deblurring,” European Conference on Computer Vision, pp. 157–170, 2010. https://doi.org/10.1007/978-3-642-15549-9_12

  45. L. Xu, C. Lu, Y. Xu, J. Jia, Image smoothing via L0 gradient minimization. ACM Trans. Graph. 30(6), 174 (2011). https://doi.org/10.1145/2070781.2024208

    Article  Google Scholar 

  46. L. Xu, S. Zheng, and J. Jia, “Unnatural L0 sparse representation for natural image deblurring,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 1107–1114, 2013. https://doi.org/10.1109/CVPR.2013.147

Download references

Acknowledgements

The authors would like to thank Dr. Yuanchao Bai for providing the source code for the method [2].

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering, Shahrood University of Technology, Shahrood, Iran

    Amir Eqtedaei & Alireza Ahmadyfard

Authors
  1. Amir Eqtedaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alireza Ahmadyfard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amir Eqtedaei.

Ethics declarations

Conflict of interest

The authors declare that they have no confict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Eqtedaei, A., Ahmadyfard, A. Pyramidical Based Image Deblurring via Kernel Continuity Prior. Circuits Syst Signal Process 42, 4362–4389 (2023). https://doi.org/10.1007/s00034-023-02327-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-023-02327-0

Keywords

Search

Navigation