Skip to main content
SpringerLink
Log in

Attention hierarchical network for super-resolution

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Deep neural networks with attention mechanism for super-resolution (SR) have achieved good SR performance by focusing on the high-frequency components of images. However, during the SR process, it is difficult for these networks to obtain multi-level high-frequency features with different extraction difficulties from low-resolution images, resulting in the lack of textures and details in the reconstructed SR images. To solve this problem, we propose an attention hierarchical network (AHN) for SR. The proposed AHN separates and extracts high-frequency features with different extraction difficulties in a hierarchical way to obtain multi-level high-frequency features. In the process of separation and extraction, we separate high-frequency features into easy-to-extract features and difficult-to-extract features by attention block and extract the separated features by dense-residual module. Extensive experiments demonstrate that the proposed AHN is superior to the state-of-the-art SR methods and reconstructs better SR images that contain more textures and details.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Agustsson E, Ntire TR (2017) 2017 challenge on single image super-resolution: Dataset and study. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 126–135

  2. Anwar S, Khan S, Barnes N (2020) A deep journey into super-resolution: A survey. ACM Comput Surv 53(3):1–34. https://doi.org/10.1145/3390462

    Article  Google Scholar 

  3. Bevilacqua M, Roumy A, Guillemot C et al (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the 23rd British Machine Vision Conference pp 1–10

  4. Chen Y C, Li L, Yu L et al (2020) Uniter: Universal image-text representation learning. In: Proceedings of the European Conference on Computer Vision. Springer, pp 104–120

  5. Chen L, Zhang H, Xiao J, et al (2017) Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 5659–5667

  6. Choi J S, Kim M (2017) A deep convolutional neural network with selection units for super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 154–160

  7. Dong C, Loy CC, He K et al (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307. https://doi.org/10.1109/TPAMI.2015.2439281

    Article  Google Scholar 

  8. Dong C, Loy CC, Tang X et al (2016) Accelerating the Super-Resolution Convolutional Neural Network. In: Proceeding of the European Conference on Computer Vision. Springer, pp 391–407

  9. Garcia-Garcia JM, Lozano MD, Penichet VM et al (2022) Building a three-level multimodal emotion recognition framework. Multimedia Tools and Applications 1–31. https://doi.org/10.1007/s11042-022-13254-8

  10. Guo K, Guo H, Ren S et al (2020) Towards efficient motion-blurred public security video super-resolution based on back-projection networks. J Netw Comput Appl 166:102691. https://doi.org/10.1016/j.jnca.2020.102691

    Article  Google Scholar 

  11. Hu Y, Gao X, Li J et al (2021) Single image super-resolution with multi-scale information cross-fusion network. Signal Process 179:107831. https://doi.org/10.1016/j.sigpro.2020.107831

    Article  Google Scholar 

  12. Hu H, Gu J, Zhang Z et al (2018) Relation networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 3588–3597

  13. Hu Y, Li J, Huang Y et al (2019) Channel-wise and spatial feature modulation network for single image super-resolution. IEEE Trans Circuits Syst Video Technol 30(11):3911–3927. https://doi.org/10.1109/TCSVT.2019.2915238

    Article  Google Scholar 

  14. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 7132–7141

  15. Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 5197–5206

  16. Kim JH, Choi JH, Cheon M et al (2018) Ram: Residual attention module for single image super-resolution. arXiv:1811.12043

  17. Kim J, Lee JK, Lee KM et al (2016) Deeply-Recursive Convolutional Network for Image Super-Resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 1637–1645

  18. Kim J, Lee JK, Lee KM et al (2016) Accurate Image Super-Resolution Using Very Deep Convolutional Networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 1646–1654

  19. Kingma D P, Ba J (2014) Adam: A method for stochastic optimization. arXiv:14126980

  20. Lai WS, Huang JB, Ahuja N et al (2017) Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 5835–5843

  21. Ledig C, Theis L, Huszár F et al (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 4681–4690

  22. Lin Z, Feng M, Santos CND et al (2017) A structured self-attentive sentence embedding. arXiv:1703.03130

  23. Liu H, Gu Y (2022) Deep Joint Estimation Network for Satellite Video Super-Resolution With Multiple Degradations. IEEE Trans Geosci Remote Sens 60:1–15. https://doi.org/10.1109/TGRS.2022.3163790

    Article  Google Scholar 

  24. Martin D, Fowlkes C, Tal D et al (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 416–423

  25. Shi W, Caballero J, Huszar F et al (2016) Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 1874–1883

  26. Song Z, Zhao X, Hui Y et al (2021) Progressive back-projection network for COVID-CT super-resolution. Computer Methods and Programs in Biomedicine 106193. https://doi.org/10.1016/j.cmpb.2021.106193

  27. Song Z, Zhao X, Hui Y et al (2022) Fusing Attention Network based on Dilated Convolution for Super Resolution. IEEE Trans Cogn Dev Syst. https://doi.org/10.1109/TCDS.2022.3153090

    Article  Google Scholar 

  28. Song Z, Zhao X, Jiang H (2021) Gradual deep residual network for super-resolution. Multimed Tools Appl 80(7):9765–9778. https://doi.org/10.1007/s11042-020-10152-9

    Article  Google Scholar 

  29. Tai Y, Yang J, Liu X et al (2017) Memnet: A persistent memory network for image restoration. In: Proceedings of the IEEE International Conference on Computer Vision. IEEE, pp 4539–4547

  30. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 3147–3155

  31. Tong T, Li G, Liu X et al (2017) Image super-resolution using dense skip connections. In: Proceedings of the IEEE International Conference on Computer Vision. IEEE, pp 4799–4807

  32. Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. arXiv:1706.03762

  33. Wang Z, Bovik AC, Sheikh HR et al (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612. https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  34. Wang Z, Chen J, Hoi SCH (2020) Deep Learning for Image Super-resolution: A Survey. IEEE Trans Pattern Anal Mach Intell 43(10):3365–3387. https://doi.org/10.1109/TPAMI.2020.2982166

    Article  Google Scholar 

  35. Wang X, Yu K, Dong C et al (2018) Recovering realistic texture in image super-resolution by deep spatial feature transform. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 606–615

  36. Wang X, Yu K, Wu S et al (2018) Esrgan: Enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on Computer Vision. Springer, pp 1–10

  37. Wei D, Wang Z (2022) Channel rearrangement multi-branch network for image super-resolution. Digital Signal Process 120:103254. https://doi.org/10.1016/j.dsp.2021.103254

    Article  Google Scholar 

  38. Yang J, Wright J, Huang TS et al (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873. https://doi.org/10.1109/TIP.2010.2050625

    Article  MathSciNet  MATH  Google Scholar 

  39. Zeyde R, Elad M, Protter M et al (2010) On single image scale-up using sparse-representations. In: International Conference on Curves and Surfaces. Springer, pp 711–730

  40. Zhang H, Goodfellow I, Metaxas D et al (2019) Self-attention generative adversarial networks. In: International Conference on Machine Learning. IEEE, pp 7354–7363

  41. Zhang Y, Li K, Li K et al (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision. Springer, pp 286–301

  42. Zhang Y, Tian Y, Kong Y et al (2018) Residual dense network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 2472–2481

Download references

Acknowledgements

This work is supported by the National Key R&D Program (2020YFB1713600), the National Natural Science Foundation of China (61763029), the Science and Technology Program of Gansu Province (21YF5GA072, 21JR7RA206), the Education Industry Support Program of Gansu Provincial Department (2021CYZC-02), and the Natural Science Foundation of Gansu Province (21JR7RA206).

Author information

Authors and Affiliations

  1. College of Electrical Engineering and Information Engineering, Lanzhou University of Technology, Lanzhou, 730050, China

    Zhaoyang Song, Xiaoqiang Zhao, Yongyong Hui & Hongmei Jiang

  2. Key Laboratory of Gansu Advanced Control for Industrial Processes, Lanzhou, 730050, China

    Xiaoqiang Zhao, Yongyong Hui & Hongmei Jiang

  3. National Experimental Teaching Center of Electrical and Control Engineering, Lanzhou University of Technology, Lanzhou, 730050, China

    Xiaoqiang Zhao, Yongyong Hui & Hongmei Jiang

Authors
  1. Zhaoyang Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoqiang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yongyong Hui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongmei Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoqiang Zhao.

Ethics declarations

Conflicts of interests

The authors declare that they have no conflict of interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, Z., Zhao, X., Hui, Y. et al. Attention hierarchical network for super-resolution. Multimed Tools Appl 82, 46351–46369 (2023). https://doi.org/10.1007/s11042-023-15782-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15782-3

Keywords

Search

Navigation