Skip to main content
SpringerLink
Log in

DDABNet: a dense Do-conv residual network with multisupervision and mixed attention for image deblurring

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Not all pixels are appropriate for image deblurring by traditional image deblurring methods. Similarly, not all feature information is appropriate for network model learning. To extract feature information effectively and ensure that the network pays attention to the feature information that is conducive to image deblurring, a dense Do-conv residual network with multisupervision and mixed attention is proposed, which is called DDABNet. First, we construct the Do-CA-Conv residual block by focusing on the important channels at the convolution level and densely connect the Do-CA-Conv residual blocks by summation in the first half and concatenation in the second half to construct DoDense(S &C) block. Second, at the third-scale stage of the encoder and decoder, to build interdimensional dependencies in the DoDense(S &C) block, we rotate the feature cube along certain dimensions to obtain an attention map and rotate it back before the first and second Do-CA-Conv residual blocks. Third, after the DoDense(S &C) block, followed by adaptive mixed-channel attention and spatial attention, which enhance attention to important information, the DoDense(S &C) attention block(DDAB) is constructed. Finally, the encoder and decoder outputs are constrained with multisupervision in the frequency domain. A visual inspection and the objective evaluation index of the benchmark datasets show the effectiveness of our proposed network.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

The datasets generated or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Ma L, Zhao F, Hong* H, Wang L, Zhu Y (2023) Complementary parts contrastive learning for fine-grained weakly supervised object co-localization. IEEE Trans Circuits Syst Video Technol 33(11): 6635–6648

  2. Lei M, Xinyi L, Hanyu H, Yaozhong Z, Lei W, Jinmeng W (2022) Scribbleattention hierarchical network for weakly supervised salient object detection in optical remote sensing images. Appl Intell 53: 12999–13017

  3. Ma L, Xu Q, Hong H, Shi Y, Zhu Y, Wang L (2023) Joint ordinal regression and multiclass classification for diabetic retinopathy grading with transformers and CNNs fusion network. Appl Intell 53:27505–27518

  4. Abdullah A, Brown MS (2020) Defocus deblurring using dual-pixel data. European Conference on Computer Vision, pp 1–25

  5. Cao J, Li Y, Sun M et al (2020) Do-conv: Depthwise over-parameterized convolutional layer. IEEE Trans Image Process 31:3726–3736

    Article  Google Scholar 

  6. Cho SJ, Ji SW, Hong JP, et al (2021) Rethinking coarse-to-fine approach in single image deblurring. In: IEEE Conference on computer vision and pattern recognition, pp 4641–4650

  7. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: IEEE Conference on computer vision and pattern recognition, pp 1251–1258

  8. Chu X, Chen L, Chen C, et al (2021) Revisiting global statistics aggregation for improving image restoration. European Conference on Computer Vision, pp 53–71

  9. Fergus R, Singh B, Hertzmann A et al (2006) Removing camera shake from a single photograph. ACM Trans Graph 25(3):787–794

    Article  Google Scholar 

  10. Fu X, Huang J, Ding X et al (2017) Clearing the skies: A deep network architecture for single-image rain removal. IEEE Trans Image Process 26(6):2944–2956

    Article  MathSciNet  Google Scholar 

  11. Fu X, Huang J, Zeng D, et al (2017) Removing rain from single images via a deep detail network. In: IEEE Conference on computer vision pattern recognition, pp 1715–1723

  12. Gao H, Tao X, Shen X, et al (2019) Dynamic scene deblurring with parameter selective sharing and nested skip connections. In: IEEE Conference on computer vision and pattern recognition, pp 3843–3851

  13. He K, Zhang X, Ren S, et al (2016) Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778

  14. He Z, Patel VM (2018) Density-aware single image de-raining using a multi-stream dense network. IEEE Conference on Computer Vision and Pattern Recognition, pp 695–704

  15. Jiang K, Wang Z, Yi P, et al (2020) Multi-scale progressive fusion network for single image deraining. IEEE Conference on Computer Vision and Pattern Recognition, pp 8343–8352

  16. Karaali A, Jung CR (2018) Edge-based defocus blur estimation with adaptive scale selection. IEEE Trans Image Process 27(3):1126–1137

    Article  MathSciNet  Google Scholar 

  17. Krishnan D, Tay T, Fergus R (2011) Blind deconvolution using a normalized sparsity measure. In: IEEE Conference on computer vision and pattern recognition, pp 2657–2664

  18. Kupyn O, Budzan V, Mykhailych M, et al (2018) Deblurgan: Blind motion deblurring using conditional adversarial networks. IEEE Conference on Computer Vision and Pattern Recognition, pp 8183–8192

  19. Kupyn O, Martyniuk T, Wu J, et al (2019) Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. In: IEEE Conference on computer vision and pattern recognition, pp 8878–8887

  20. Lee J, Lee S, Cho S, et al (2019) Deep defocus map estimation using domain adaptation. In: IEEE Conference on computer vision and pattern recognition, pp 12214–12222

  21. Lee J, Son H, Rim J, et al (2021) Iterative filter adaptive network for single image defocus deblurring. In: IEEE Conference on computer vision and pattern recognition, pp 2034–2042

  22. Levin A, Weiss Y, Durand F, et al (2011) Efficient marginal likelihood optimization in blind deconvolution. In: IEEE Conference on computer vision and pattern recognition, pp 2657–2664

  23. Li X, Zheng S, Jia J (2013) Unnatural l0 sparse representation for natural image deblurring. In: IEEE Conference on computer vision and pattern recognition, pp 1107–1114

  24. Li X, Wu J, Lin Z, et al (2018) Recurrent squeeze-and-excitation context aggregation net for single image deraining. European Conference on Computer Vision, pp 262–277

  25. Li X, Wang W, Hu X, et al (2020) Selective kernel networks. IEEE Conference on Computer Vision and Pattern Recognition, pp 510–519

  26. Liang J, Cao J, Fan Y, et al (2022) Vrt: A video restoration transformer

  27. Loshchilov I, Hutter F (2016) Sgdr: Stochastic gradient descent with warm restarts. In: International conference on learning representations, pp 1–9

  28. Misra D, Nalamada T, Arasanipalai AU, et al (2020) Rotate to attend: Convolutional triplet attention module. IEEE Winter Conference on Applications of Computer Vision, pp 3138–3147

  29. Nah S, Kim TH, Lee KM (2016) Deep multi-scale convolutional neural network for dynamic scene deblurring. IEEE Conference on Computer Vision and Pattern Recognition pp 3883–3891

  30. Pan J, Sun D, Pfister H et al (2017) Deblurring images via dark channel prior. IEEE Trans Pattern Anal Mach Intell 40:2315–2328

    Article  Google Scholar 

  31. Park D, Dong UK, Kim J, et al (2019) Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training. European Conference on Computer Vision, pp 372–343

  32. Qi S, Jia J, Agarwala A (2008) High-quality motion deblurring from a single image. ACM Trans Graph 27(3):1–10

    Google Scholar 

  33. Ren D, Zuo W, Hu Q, et al (2019) Progressive image deraining networks: A better and simpler baseline. IEEE Conference on Computer Vision and Pattern Recognition, pp 3932–3941

  34. Shi J, Li X, Jia J (2015) Just noticeable defocus blur detection and estimation. In: IEEE Conference on computer vision and pattern recognition, pp 657–665

  35. Sun L, Sakaridis C, Liang J, et al (2021) Event-based fusion for motion deblurring with cross-modal attention. In: European conference on computer vision, pp 412–428

  36. Tao X, Gao H, Wang Y, et al (2018) Scale-recurrent network for deep image deblurring. In: IEEE Conference on computer vision and pattern recognition, pp 8174–8182

  37. Wang Q, Wu B, Zhu P, et al (2020) Eca-net: Efficient channel attention for deep convolutional neural networks. In: IEEE Conference on computer vision and pattern recognition, pp 11531–11539

  38. Wei W, Meng D, Zhao Q, et al (2020) Semi-supervised transfer learning for image rain removal. In: IEEE Conference on computer vision and pattern recognition, pp 3872–3881

  39. Woo S, Park J, Lee YJ, et al (2018) Cbam:convolutional block attention. In: European conference on computer vision, pp 1–17

  40. Xin Y, Feng X, Zhang S et al (2014) Efficient patch-wise non-uniform deblurring for a single image. IEEE Trans Multimed 16(6):1510–1524

    Article  Google Scholar 

  41. Yang W, Tan RT, Feng J, et al (2017) Deep joint rain detection and removal from a single image. In: IEEE Conference on computer vision and pattern recognition, pp 1685–1694

  42. Yasarla R, Patel VM (2019) Uncertainty guided multi-scale residual learning-using a cycle spinning cnn for single image de-raining. IEEE Conference on computer vision and pattern recognition, pp 8397–8406

  43. Zamir SW, Arora A, Khan S, et al (2021) Multi-stage progressive image restoration. In: IEEE Conference on computer vision and pattern recognition, pp 14816–14826

  44. Zamir SW, Arora A, Khan S et al (2022) Restormer: efficient transformer for high-resolution image restoration. IEEE conference on computer vision and pattern recognition, pp 5718–5729

  45. Zhang H, Cisse M, Dauphin YN, et al (2017) mixup: Beyond empirical risk minimization. International Conference on Learning Representations, pp 1–13

  46. Zhang H, Sindagi V, Patel VM (2017) Image de-raining using a conditional generative adversarial network. IEEE Trans Circ Syst Vid Technol 30(11):3943–3956

    Article  Google Scholar 

  47. Zhang H, Dai Y, Li H, et al (2019) Deep stacked hierarchical multi-patch network for image deblurring. IEEE Conference on Computer Vision and Pattern Recognition, pp 5971–5979

  48. Zhang J, Pan J, Ren J, et al (2018) Dynamic scene deblurring using spatially variant recurrent neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2521–2529

  49. Zhang Y, Tian Y, Kong Y, et al (2021) Residual dense network for image restoration. In: IEEE Transactions on pattern analysis and machine intelligence, pp 2480–2495

  50. Ma L, Hong H, Meng F, Wu Q, Wu J (2023) Deep progressive asymmetric quantization based on causal intervention for fine-grained image retrieval. IEEE Transactions on Multimedia, pp 1–13

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grants No.62201406 and No.62171329, and in part by the Guiding Scientific Research Project of Hubei Provincial Department of Education on No.B2021086.

Author information

Authors and Affiliations

  1. Wuhan Institute of Technology, School of Electrical Information, Wuhan, China, 430025

    Yu Shi, Zhigao Huang, Jisong Chen, Lei Ma, Lei Wang, Xia Hua & Hanyu Hong

  2. Hubei Key Laboratory of Optical Information and Pattern Recognition, Wuhan, 430025, China

    Yu Shi, Zhigao Huang, Jisong Chen, Lei Ma, Lei Wang, Xia Hua & Hanyu Hong

Authors
  1. Yu Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhigao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jisong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lei Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xia Hua
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hanyu Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Ma.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, Y., Huang, Z., Chen, J. et al. DDABNet: a dense Do-conv residual network with multisupervision and mixed attention for image deblurring. Appl Intell 53, 30911–30926 (2023). https://doi.org/10.1007/s10489-023-05122-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05122-1

Keywords

Search

Navigation