Skip to main content
SpringerLink
Log in

Defocus blur detection via adaptive cross-level feature fusion and refinement

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Convolutional neural networks have achieved competitive performance in defocus blur detection (DBD). However, due to the different receptive fields of different convolutional layers, there are distinct differences in the features generated by these layers, and the complementary information between cross-level features cannot be fully utilized. Besides, there are still challenges to be solved in homogeneous regions. To tackle the above issues, we focus on both homogeneous region dataset augmentation and model design to propose a novel DBD model based on adaptive cross-level feature fusion and refinement. Specifically, in terms of homogeneous region dataset enhancement, a Laplace filter is used to extract the homogeneous region image patch of the training image to realize homogeneous region image augmentation, which improves the robustness of the model for DBD in the homogeneous region; in terms of model design, we propose an adaptive fusion mechanism with self-learning weights and design the adaptive cross-level feature fusion module, which adaptively discriminates between different levels of features and fuses them step-by-step. In addition, we design the cross-level feature refinement module and embed it into the network, which captures the complementary information of the cross-level features, and refines cross-level feature information from coarse to fine in the decoder stage. Experimental results on two commonly used datasets show that the proposed method outperforms 13 state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

All data generated or analyzed during this paper can be obtained from the corresponding author on reasonable request.

References

  1. Cai, R., Fang, M.: Blind image quality assessment by simulating the visual cortex. Vis. Comput. (2022). https://doi.org/10.1007/s00371-022-02614-y

    Article  Google Scholar 

  2. Liu, Z., Xiang, Q., Tang, J., Wang, Y., Zhao, P.: Robust salient object detection for RGB images. Vis. Comput. 36, 1823–1835 (2020). https://doi.org/10.1007/s00371-019-01778-4

    Article  Google Scholar 

  3. Zhang, X., Wang, R., Jiang, X., Wang, W., Gao, W.: Spatially variant defocus blur map estimation and deblurring from a single image. J. Vis. Commun. Image Represent. 35(Feb.), 257–264 (2016). https://doi.org/10.1016/j.jvcir.2016.01.002

    Article  Google Scholar 

  4. Jiang, Z., Zhang, Z., Yu, Y., Liu, R.: Publisher Correction: Bilevel modeling investigated generative adversarial framework for image restoration. Vis. Comput. (2022). https://doi.org/10.1007/s00371-022-02713-w

    Article  Google Scholar 

  5. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. (2017). https://doi.org/10.1109/TIP.2017.2662206

    Article  MathSciNet  Google Scholar 

  6. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016). https://doi.org/10.1109/TPAMI.2015.2439281

    Article  Google Scholar 

  7. Liu, Z., Liu, J.: Hypergraph attentional convolutional neural network for salient object detection. Vis. Comput. (2022). https://doi.org/10.1007/s00371-022-02499-x

    Article  Google Scholar 

  8. Li, P., Wang, D., Wang, L., Lu, H.: Deep visual tracking: review and experimental comparison. Pattern Recognit. 76, 323–338 (2018)

    Article  Google Scholar 

  9. Liu, R., Li, Z., Jia, J.: Image partial blur detection and classification. IEEE Conf. Comput. Vis. Pattern Recognit. (2008). https://doi.org/10.1109/CVPR.2008.4587465

    Article  Google Scholar 

  10. Pang, Y., Zhu, H., Li, X., Li, X., Pang, Y.: Classifying discriminative features for blur detection. IEEE Trans. Cybern. 46(10), 2220–2227 (2017). https://doi.org/10.1109/TCYB.2015.2472478

    Article  Google Scholar 

  11. Saad, E., Hirakawa, K.: Defocus blur-invariant scale-space feature extractions. IEEE Trans. Image Process. 25(7), 3141–3156 (2016). https://doi.org/10.1109/TIP.2016.2555702

    Article  MathSciNet  Google Scholar 

  12. Su, B., Lu, S., Tan, C.L.: Blurred image region detection and classifification. ACM International Conference on Multimedia, pp 1397–1400 (2011)

  13. Zhao, J., Feng, H., Xu, Z., Li, Q., Tao, X.: Automatic blur region segmen tation approach using image matting. Signal Image Video Process. 7(6), 1173–1181 (2013). https://doi.org/10.1007/s11760-012-0381-6

    Article  Google Scholar 

  14. Zhu, X., Cohen, S., Schiller, S., Milanfar, P.: Estimating spatially varying defocus blur from a single image. IEEE Trans. Image Process. 22(12), 4879–4891 (2013). https://doi.org/10.1109/TIP.2013.2279316

    Article  MathSciNet  Google Scholar 

  15. Tang, C., Wu, J., Hou, Y., Wang, P., Li, W.: A spectral and spatial approach of coarse-to-fine blurred image region detection. IEEE Signal Process. Lett. (2016). https://doi.org/10.1109/LSP.2016.2611608

    Article  Google Scholar 

  16. Shi, J., Xu, L., Jia, J.: Discriminative blur detection features. IEEE Conf. Comput. Vis. Pattern Recognit. (2014). https://doi.org/10.1109/CVPR.2014.379

    Article  Google Scholar 

  17. Zhang, Y., Hirakawa, K.: Blur processing using double discrete wavelet transform. IEEE Conf. Comput. Vis. Pattern Recognit. (2013). https://doi.org/10.1109/CVPR.2013.145

    Article  Google Scholar 

  18. Vu, C.T., Phan, T.D., Chandler, D.M.: \({ S}_{3}\): a spectral and spatial measure of local perceived sharpness in natural images. IEEE Trans. Image Process. 21(3), 934–945 (2012). https://doi.org/10.1109/TIP.2011.2169974

    Article  MathSciNet  Google Scholar 

  19. Zhu, X., Cohen, S., Schiller, S., et al.: Estimating spatially varying defocus blur from a single image. IEEE Trans. Image Process. 22(12), 4879–4891 (2013). https://doi.org/10.1109/TIP.2013.2279316

    Article  MathSciNet  Google Scholar 

  20. Golestaneh, S.A., Karam, L.J.: Spatially-varying blur detection based on multiscale fused and sorted transform coeffificients of gradient magnitudes. IEEE Conf. Comput. Vis. Pattern Recognit. (2017). https://doi.org/10.1109/CVPR.2017.71

    Article  Google Scholar 

  21. Tang, C., Wu, J., Hou, Y., Wang, P., Li, W.: A spectral and spatial approach of coarse-to-fifine blurred image region detection. IEEE Signal Process. Lett. 23(11), 1652–1656 (2016). https://doi.org/10.1109/LSP.2016.2611608

    Article  Google Scholar 

  22. Xin, Y., Eramian, M.: LBP-based segmentation of defocus blur. IEEE Trans. Image Process. 25(4), 1–1 (2016). https://doi.org/10.1109/TIP.2016.2528042

    Article  MathSciNet  Google Scholar 

  23. Park, J., Tai, Y.W., Cho, D., Kweon, I.S.: A unified approach of multi-scale deep and hand-crafted features for defocus estimation. IEEE Comput. Soc. (2017). https://doi.org/10.1109/CVPR.2017.295

    Article  Google Scholar 

  24. Zhao, W., Zhao, F., Wang, D., Lu, H.: Defocus blur detection via multi-stream bottom-top-bottom fully convolutional network. IEEE Conf. Comput. Vis. Pattern Recognit. (2018). https://doi.org/10.1109/CVPR.2018.00325

    Article  Google Scholar 

  25. Karaali, A., Harte, N., Jung, C.R.: Deep multi-scale feature learning for defocus blur estimation. IEEE Trans. Image Process. 31, 1097–1106 (2022)

    Article  Google Scholar 

  26. Zhao, W., Zheng, B., Lin, Q., Lu, H.: Enhancing diversity of defocus blur detectors via cross-ensemble network. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) (2019). https://doi.org/10.1109/CVPR.2019.00911

    Article  Google Scholar 

  27. Ma, K., Fu, H., Liu, T., Wang, Z., Tao, D.: Deep blur mapping: exploiting high-level semantics by deep neural networks. IEEE Trans. Image Process. 27, 5155–5166 (2018). https://doi.org/10.1109/TIP.2018.2847421

    Article  MathSciNet  Google Scholar 

  28. Lee, J., Lee, S., Cho, S., Lee, S.: Deep defocus map estimation using domain adaptation. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) (2019). https://doi.org/10.1109/CVPR.2019.01250

    Article  Google Scholar 

  29. Tang, C., Zhu, X., Liu, X., Wang, L., Zomaya, A.: DeFusionNET: defocus blur detection via recurrently fusing and refining multi-scale deep features. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) (2019). https://doi.org/10.1109/CVPR.2019.00281

    Article  Google Scholar 

  30. Tang, C., Liu, X., Zhu, X., Zhu, E., Sun, K., Wang, P., Wang, L., Zomaya, A.: R\(^{2}\)MRF: defocus blur detection via recurrently refining multi-scale residual features. Proc. AAAI Conf. Artif. Intell. 34(7), 12063–12070 (2020). https://doi.org/10.1609/aaai.v34i07.6884

    Article  Google Scholar 

  31. Zhao, W., Hou, X., He, Y., Lu, H.: Defocus blur detection via boosting diversity of deep ensemble networks. IEEE Trans. Image Process. (2021). https://doi.org/10.1109/TIP.2021.3084101

    Article  Google Scholar 

  32. Li, J., Fan, D., Yang, L., Gu, S., Lu, G., Xu, Y., Zhang, D.: Layer-output guided complementary attention learning for image defocus blur detection. IEEE Trans. Image Process. (2021). https://doi.org/10.1109/TIP.2021.3065171

    Article  Google Scholar 

  33. Zhao, W., Shang, C., Lu, H.: Self-generated defocus blur detection via dual adversarial discriminators. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) (2021). https://doi.org/10.1109/CVPR46437.2021.00686

    Article  Google Scholar 

  34. Jiang, Z., Xu, X., Zhang, L., Zhang, C., Foo, S., Zhu, C.: MA-GANet: a multi-attention generative adversarial network for defocus blur detection. IEEE Trans. Image Process. 31, 3494–3508 (2022). https://doi.org/10.1109/TIP.2022.3171424

    Article  Google Scholar 

  35. Zhao, Z., Yang, H., Luo, H.: Hierarchical edge-aware network for defocus blur detection. Complex Intell. Syst. 8, 4265–4276 (2022). https://doi.org/10.1007/s40747-022-00711-y

  36. Lin, X., Li, H., Cai, Q.: Hierarchical complementary residual attention learning for defocus blur detection. Neurocomputing 501, 88–101 (2022). https://doi.org/10.1016/j.neucom.2022.06.023

    Article  Google Scholar 

  37. Li, J., Liang, B., Lu, X., Li, M., Lu, G., Xu, Y.: From global to local: multi-patch and multi-scale contrastive similarity learning for unsupervised defocus blur detection. IEEE Trans. Image Process. 32, 1158–1169 (2023). https://doi.org/10.1109/TIP.2023.3240856

    Article  Google Scholar 

  38. Abuolaim, A., Brown, M.S.: Defocus Deblurring Using Dual-Pixel Data. European Conference on Computer Vision, Springer, pp 111–126 (2020)

  39. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention, Springer, pp 234–241 (2015)

  40. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (2016). https://doi.org/10.1109/CVPR.2016.90

    Article  Google Scholar 

  41. Wei, J., Wang, S., Huang, Q.: F3net: fusion, feedback and focus for salient object detection. AAAI Conf. Artif. Intell. 34, 12321–12328 (2020)

    Google Scholar 

  42. Deng, J., Dong, W., Socher, R., Li, L.J., Kai, L., Li, F.F.: Imagenet: a large-scale hierarchical image database. IEEE Conference on Computer Vision and Pattern Recognition, pp 248–255 (2009)

  43. Fan, D., Cheng, M., Liu, Y., Li, T., Borji, A.: Structure-measure: a new way to evaluate foreground maps. IEEE Int. Conf. Comput. Vis. (ICCV) (2017). https://doi.org/10.1109/ICCV.2017.487

    Article  Google Scholar 

  44. Zhang, K., Wang, T., Luo, W., et al.: MC-Blur: a comprehensive benchmark for image deblurring. IEEE Trans. Circuits Syst. Video Technol. (2023). https://doi.org/10.1109/TCSVT.2023.3319330

    Article  Google Scholar 

  45. Zhang, K., Luo, W., Zhong, Y., et al.: Deblurring by realistic blurring. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2737–2746 (2020)

Download references

Acknowledgements

This work is supported by the Chinese Academy of Sciences-Youth Innovation Promotion Association, Grant Number 2020220, recipient Hang Yang; Key-Area Research and Development Program of Guangdong Province (Grant No. 2019B010155003); Science & Technology Development Project of Jilin Province, Key R &D Programs No. 20210201078GX.

Author information

Author notes

  1. Peiyu Liu, Haitao Nie, Zhongbo Zhang, and Chunyu Li have contributed equally to this work.

Authors and Affiliations

  1. Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Science, Changchun, 130033, China

    Zijian Zhao, Hang Yang, Haitao Nie & Chunyu Li

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Zijian Zhao

  3. AVIC Shenyang Aircraft Design & Research Institute, Shenyang, 110087, China

    Peiyu Liu

  4. Jilin University, Changchun, 130012, China

    Zhongbo Zhang

Authors
  1. Zijian Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peiyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haitao Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhongbo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chunyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hang Yang.

Ethics declarations

Conflict of interest

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Z., Yang, H., Liu, P. et al. Defocus blur detection via adaptive cross-level feature fusion and refinement. Vis Comput (2024). https://doi.org/10.1007/s00371-023-03229-7

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00371-023-03229-7

Keywords

Search

Navigation