Abstract
Image deblurring is an important problem encountered in many image restoration tasks. To remove the motion blur of images captured from dynamic scenes, various Convolutional Neural Networks (CNNs) based methods are developed to restore the latent sharp image via an end-to-end trainable. However, these CNNs-based methods cannot restore enough structure details as no significant information is provided by the blurry input only. In this work, we propose a reference-guided deep deblurring method by incorporating the high-quality reference image into the deep network for better deblurring effect. Concretely, the correlation between the blurry input and the reference image is computed in the high-level feature space, and further represented by the similarity maps. To pick up the most relevant similarity maps to the input, the selective attention module is designed, and the selected similarity maps are decoded to reconstruct the deblurred sharp image. Extensive experimental results on two public datasets demonstrate that the proposed method achieves superior performance to existing methods quantitatively and qualitatively. More ablation experiments also validate that the favourable deblurred results can still be obtained even if the reference image is not similar with the input image.
Similar content being viewed by others
References
Bai Y, et al. (2018) Graph-based blind image deblurring from a single photograph. IEEE Trans Image Proc 28.3:1404–1418
Chakrabarti A (2016) A neural approach to blind motion deblurring. In: ECCV
Chen X, Yang J, Wu Q (2010) Image deblur in gradient domain. Opt Eng 49 :117003
Cho S, Lee S (2009) Fast motion deblurring. In: SIG- GRAPH 2009
Gao H et al (2019) Dynamic scene deblurring with parameter selective sharing and nested skip connections. In: 2019 IEEE/CVF Con- ference on computer vision and pattern recognition (CVPR), pp 3843–3851
He K et al (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), pp 770–778
Jia J (2014) Mathematical models and practical solvers for uniform motion deblurring. In: Motion Deblurring
Jia J (2007) Single image motion deblurring using transparency. In: 2007 IEEE Conference on computer vision and pattern recognition, pp 1–8
Jolicoeur-Martineau A (2019) The relativistic discriminator: a key element missing from standard GAN. arXiv:1807.00734
Kim T, Mu Lee K (2015) Generalized video deblurring for dynamic scenes. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR), pp 5426–5434
Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980
Krishnan D, Fergus R (2009) Fast Image Deconvolution using Hyper-Laplacian Priors. In: NIPS
Kupyn O, et al. (2019) DeblurGAN-v2: deblurring (orders-of-magnitude) faster and better. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 8877–8886
Kupyn O et al (2018) DeblurGAN: Blind motion deblurring using conditional adversarial networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 8183–8192
Lai W-S et al (2016) A comparative study for single image blind deblurring. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1701–1709
Lai W-S et al (2016) A comparative study for single image blind deblurring. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1701–1709
Levin A (2006) Blind motion deblurring using image statistics. In: NIPS
Xinwang L et al (2019) Multiple kernel k k-means with incomplete kernels. IEEE Trans Pattern Anal Mach Intell 42.5:1191–1204
Michaeli T, Irani M (2014) Blind deblurring using internal patch recurrence. In: European conference on computer vision. Springer, pp 783–798
Nah S, Kim T, Mu Lee K (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 257–265
Pan J, et al. (2016) Blind image deblurring using dark channel prior. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1628–1636
Paszke A et al (2017) Automatic differentiation in pytorch
Rajagopalan A, Chellappa R (2014) Motion Deblurring: algorithms and systems
Ren D et al (2020) Neural blind deconvolution using deep priors. In: 2020 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 3338–3347
Sellent A, Rother C, Roth S (2016) Stereo video deblurring. arXiv:1607.08421
Shan Q, Jia J, Agarwala A (2008) High-quality motion deblurring from a single image. ACM Trans Graph 27:73
Shim G, Park J, Kweon IS (2020) Robust reference-based super-resolution with similarity-aware deformable convolution. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 8422–8431
Sun J et al (2015) Learning a convolutional neural network for non-uniform motion blur removal. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR), pp 769– 777
Tao X et al (2018) Scale-recurrent network for deep image deblurring. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 8174–8182
Wang Y et al (2017) The light field attachment: Turning a DSLR into a light field camera using a low budget camera ring. IEEE Tran Vis Comput Graph 23(10):2357–2364
Xu L, Jia J (2010) Two-phase kernel estimation for robust motion deblurring. In: ECCV
Xu L et al (2014) Deep convolutional neural network for image deconvolution. In: NIPS
Yan Y et al (2017) Image deblurring via extreme channels prior. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 6978–6986
Yang F et al (2020) Learning texture transformer network for image super-resolution. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 5790– 5799
Yang F et al (2020) Learning texture transformer network for image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5791–5800
Yu X, Ye X, Gao Q (2020) Infrared handprint image restoration algorithm based on apoptotic mechanism. IEEE Access 8:47334–47343
Zamir SW et al (2021) Multi-stage progressive image restoration. arXiv:2102.02808
Zhang J et al (2018) Dynamic scene deblurring using spatially variant recurrent neural networks. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 2521– 2529
Zhang Z et al (2019) Image super-resolution by neural texture transfer. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 7974–7983
Zhang Z et al (2019) Image super-resolution by neural texture transfer. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7982–7991
Zheng H et al (2017) Combining exemplar-based approach and learning- based approach for light field super-resolution using a hybrid imaging system. In: 2017 IEEE International conference on computer vision workshops (ICCVW), pp 2481–2486
Zheng H et al (2018) CrossNet: An end-to-end reference-based super resolution network using cross-scale warping. In: ECCV
Zheng H et al (2018) Crossnet: An end-to-end reference-based super resolution network using cross-scale warping. In: Proceedings of the european conference on computer vision (ECCV), pp 88–104
Zheng H et al (2017) Learning cross-scale correspondence and patchbased synthesis for reference-based super-resolution. In: BMVC
Zhou S et al (2020) Cross-scale internal graph neural network for image super-resolution. In: Advances in neural information processing systems 33
Acknowledgements
This work was supported by the General Program of National Natural Science Foundation of China (NSFC) (Grant No. 61806147), and Shanghai Natural Science Foundation of China (Grant No. 18ZR1441200).
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, Y., Luo, Y. & Lu, J. Reference-guided deep deblurring via a selective attention network. Appl Intell 52, 3867–3879 (2022). https://doi.org/10.1007/s10489-021-02585-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-021-02585-y