Skip to main content

Single image super-resolution based on multi-scale dense attention network

Soft Computing volume 27, pages 2981–2992 (2023)Cite this article

Abstract

Deep convolutional neural networks (CNNs) have enabled significant progress in the single image super-resolution (SISR) field in recent years. However, most super-resolution methods based on deep neural networks obtain spatial information using a single-size convolution kernel, which leads to insufficient local feature extraction. In addition, since convolution serves for local operations, it fails to capture the image’s inherent attributes. This paper tackles these problems by proposing a novel multi-scale dense attention network (MSDAN) based on the attention mechanism and multi-scale feature extraction. The network employs multi-scale dense blocks (MSDB) that utilize convolution kernels of different sizes to extract feature information of distinct scales. Additionally, the attention is used to model global features effectively and strengthen the network’s expressiveness. Hierarchical feature fusion is followed by multi-level feature extraction, which fuses each unit’s feature map output. Finally, the image is reconstructed using up-sampling. The experimental results show that the proposed network outperforms other several state-of-the-art super-resolution reconstruction algorithms regarding objective evaluation indicators and visual effects on datasets Set5, Set14, BSD100 and Urban100.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Data availability

Enquiries about data availability should be directed to the authors.

References

  • Ahn N, Kang B, Sohn K-A (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European conference on computer vision (ECCV), pp 252–268

  • Arbelaez P, Maire M, Fowlkes C, Malik J (2010) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  • Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473

  • Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: British machine vision conference. BMVA Press, Guildford, pp 1–10

  • Castro EB, Nakano M, Perez GS, Meana HP (2017) Improvement of image super-resolution algorithms using iterative back projection. IEEE Lat Am Trans 15(11):2214–2219

    Article  Google Scholar 

  • Chu J, Li X, Zhang J, Lu W (2020) Super-resolution using multi-channel merged convolutional network. Neurocomputing 394:136–145

    Article  Google Scholar 

  • Ding L, Xu W, Chen Y (2020) Density peaks clustering by zero-pointed samples of regional group borders. Comput Intell Neurosci 2020(2):1–15

    Article  Google Scholar 

  • Dong C, Loy CC, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  • Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision, pp 391–407

  • Emami H, Aliabadi MM, Dong M, Chinnam RB (2019) SPA-GAN: spatial attention GAN for image-to-image translation. arXiv preprint arXiv:1908.06616

  • Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  • Huang J-B, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5197–5206

  • Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017a) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  • Huang Y, Shao L, Frangi AF (2017b) Simultaneous super-resolution and cross-modality synthesis of 3D medical images using weakly-supervised joint convolutional sparse coding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6070–6079

  • Jiang K, Wang Z, Yi P, Jiang J, Xiao J, Yao Y (2018) Deep distillation recursive network for remote sensing imagery super-resolution. Remote Sens 10(11):1700

    Article  Google Scholar 

  • Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  • Kuen J, Wang Z, Wang G (2016) Recurrent attentional networks for saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3668–3677

  • Lan R, Sun L, Liu Z, Lu H, Su Z, Pang C et al (2020) Cascading and enhanced residual networks for accurate single-image super-resolution. IEEE Trans Cybern 51(1):115–125

    Article  Google Scholar 

  • Li J, Fang F, Mei K, Zhang G (2018) Multi-scale residual network for image super-resolution. In: Proceedings of the European conference on computer vision (ECCV), pp 517–532

  • Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W (2019) Feedback network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3867–3876

  • Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 136–144

  • Luo Y, Qin J, Xiang X, Tan Y, Liu Q, Xiang L (2020) Coverless real-time image information hiding based on image block matching and dense convolutional network. J Real-Time Image Proc 17(1):125–135

    Article  Google Scholar 

  • Qin D, Gu X (2020) Single-image super-resolution with multilevel residual attention network. Neural Comput Appl 32(19):15615–15628

    Article  Google Scholar 

  • Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R et al (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1874–1883

  • Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3147–3155

  • Timofte R, Agustsson E, Van Gool L, Yang M-H, Zhang L (2017) Ntire 2017 challenge on single image super-resolution: methods and results. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 114–125

  • Tong T, Li G, Liu X, & Gao Q. (2017). Image super-resolution using dense skip connections. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4799–4807.

  • Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, et al. (2017). Residual attention network for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3156–3164

  • Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803

  • Wang W, Zhu H, Dai J, Pang Y, Shen J, Shao L (2020) Hierarchical human parsing with typed part-relation reasoning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8929–8939

  • Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  • Wu Y, Zhang Y, Huang Y, Yang J (2019) Fast deconvolution method for angular super-resolution imaging based on sub-space embedding. J Eng 2019(19):6291–6294

    Article  Google Scholar 

  • Yan Y, Ren W, Hu X, Li K, Shen H, Cao X (2021) Srgat: single image super-resolution with graph attention network. IEEE Trans Image Process 30:4905–4918

    Article  Google Scholar 

  • Yang X, Li X, Li Z, Zhou D (2021) Image super-resolution based on deep neural network of multiple attention mechanism. J Vis Commun Image Represent 75:103019

    Article  Google Scholar 

  • Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: International conference on curves and surfaces, pp 711–730

  • Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018a) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV), pp 286–301

  • Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018b) Residual dense network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2472–2481

  • Zhang H, Goodfellow I, Metaxas D, Odena A (2019) Self-attention generative adversarial networks. In: International conference on machine learning, pp 7354–7363

  • Zhou F, Yang W, Liao Q (2012) Interpolation-based image super-resolution using multisurface fitting. IEEE Trans Image Process 21(7):3312–3318

    Article  MathSciNet  MATH  Google Scholar 

  • Zou WW, Yuen PC (2011) Very low resolution face recognition problem. IEEE Trans Image Process 21(1):327–340

    Article  MathSciNet  MATH  Google Scholar 

Download references

Funding

This work was supported by the Zhejiang Provincial Natural Science Foundation of China under Grant LY20E050011, and the National Natural Science Foundation of China under Grant 62071161.

Author information

Authors and Affiliations

  1. Institute of Intelligent Control and Robotics, School of Automation, Hangzhou Dianzi University, Hangzhou, 310018, China

    Farong Gao, Yong Wang, Zhangyi Yang, Yuliang Ma & Qizhong Zhang

Authors
  1. Farong Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhangyi Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuliang Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qizhong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All of the authors contribute to this manuscript, the detail contributions are as follows. FG and YW—conceptualization, methodology, writing, reviewing and editing draft; ZY and QZ—methodology, investigation, supervision; YW and YM—visualization, investigation, computational experiments, analyzing and interpretation of data.

Corresponding author

Correspondence to Qizhong Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

All authors have read this manuscript and are willing to process it for publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gao, F., Wang, Y., Yang, Z. et al. Single image super-resolution based on multi-scale dense attention network. Soft Comput 27, 2981–2992 (2023). https://doi.org/10.1007/s00500-022-07456-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-022-07456-3

Keywords

  • Super-resolution
  • Multi-scale convolution
  • Attention mechanism
  • Feature fusion