Skip to main content
SpringerLink
Log in

GRAN: ghost residual attention network for single image super resolution

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recently, many works have designed wider and deeper networks to achieve higher image super-resolution performance. Despite their outstanding performance, they still suffer from high computational resources, preventing them from directly applying to embedded devices. To reduce the computation resources and maintain performance, we propose a novel Ghost Residual Attention Network (GRAN) for efficient super-resolution. This paper introduces Ghost Residual Attention Block (GRAB) groups to overcome the drawbacks of the standard convolutional operation, i.e., redundancy of the intermediate feature. GRAB consists of the Ghost Module and Channel and Spatial Attention Module (CSAM) to alleviate the generation of redundant features. Specifically, Ghost Module can reveal information underlying intrinsic features by employing linear operations to replace the standard convolutions. Reducing redundant features by the Ghost Module, our model decreases memory and computing resource requirements in the network. The CSAM pays more comprehensive attention to where and what the feature extraction is, which is critical to recovering the image details. Experiments conducted on the benchmark datasets demonstrate the superior performance of our method in both qualitative and quantitative. Compared to the baseline models, we achieve higher performance with lower computational resources, whose parameters and FLOPs have decreased by more than ten times.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Code Availability

After the paper is accepted, the code will be open-sourced.

References

  1. Ahn N, Kang B, Sohn KA (2018) Fast, accurate and lightweight super-resolution with cascading residual network

  2. Ai W, Tu X, Cheng S, Xie M (2020) Single image super-resolution via residual neuron attention networks. arXiv:2005.10455

  3. Barzegar S, Sharifi A, Manthouri M (2020) Super-resolution using lightweight detailnet network. Multimed Tools Appl

  4. Bevilacqua M, Roumy A, Guillemot C, Alberi-morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding

  5. Cai J, Gu S, Timofte R, Zhang L (2019) Ntire 2019 challenge on real image super-resolution: methods and results. In: CVPR

  6. Cai J, Zeng H, Yong H, Cao Z, Zhang L (2019) Toward real-world single image super-resolution: a new benchmark and a new model. In: ICCV

  7. Cao K, Liu Y, Duan L, Xie T (2020) Adaptive residual channel attention network for single image super-resolution. Sci Program

  8. Chen D, He Z, Cao Y, Yang J, Cao Y, Yang MY, Tang S, Zhuang Y (2019) Deep neural network for fast and accurate single image super-resolution via channel-attention-based fusion of orientation-aware features. arXiv:1912.04016

  9. Chen Y, Phonevilay V, Tao J, Chen X, Xia R, Zhang Q, Yang K, Xiong J, Xie J (2021) The face image super-resolution algorithm based on combined representation learning. Multimed Tools Appl

  10. Chen L, Zhang H, Xiao J, Nie L, Shao J, Liu W, Chua T-S (2017) Sca-cnn: spatial and channel-wise attention in convolutional networks for image captioning. In: CVPR

  11. Chu X, Zhang B, Xu R (2020) Multi-objective reinforced evolution in mobile neural architecture search. In: ECCV

  12. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell

  13. Fan Y, Yu J, Mei Y, Zhang Y, Fu Y, Liu D, Huang TS (2020) Neural sparse representation for image restoration. arXiv:2006.04357

  14. Fu J, Zheng H, Mei T (2017) Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition. In: CVPR

  15. Fukuda T, Suzuki M, Kurata G, Thomas S, Cui J, Ramabhadran B (2017) Efficient knowledge distillation from an ensemble of teachers. In: Interspeech

  16. Guo Y, Zhang C, Zhang C, Chen Y (2018) Sparse dnns with improved adversarial robustness neurIPS

  17. Han S, Mao H, Dally WJ (2015) Deep compression: compressing deep neural networks with pruning trained quantization and huffman coding

  18. Han K, Wang Y, Tian Q, Guo J, Xu C (2020) Ghostnet: more features from cheap operations. In: CVPR

  19. Haris M, Shakhnarovich G, Ukita N (2019) Deep back-projection networks for single image super-resolution. arXiv:1904.05677

  20. Heo B, Lee M, Yun S, Choi JY (2019) Knowledge distillation with adversarial samples supporting decision boundary. In: AAAI

  21. Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. arXiv:1503.02531

  22. Hu Y, Li J, Huang Y, Gao X (2019) Channel-wise and spatial feature modulation network for single image super-resolution. IEEE Trans Circuits Syst Vid Technol

  23. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: CVPR

  24. Hu S, Wang G, Wang Y, Chen C, Pan Z (2020) Accurate image super-resolution using dense connections and dimension reduction network. Multimed Tools Appl

  25. Huang J-B, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: CVPR

  26. Jaderberg M, Simonyan K, Zisserman A, Kavukcuoglu K (2015) Spatial transformer networks. arXiv:1506.02025

  27. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: CVPR

  28. Li X, Chen Z (2021) Single image super-resolution reconstruction based on fusion of internal and external features. Multimed Tools Appl

  29. Li J, Fang F, Mei K, Zhang G (2018) Multi-scale residual network for image super-resolution: 15th european conference, munich, germany, 8-14 Sept 2018, proceedings, part viii. In: ECCV

  30. Li R, He X, Zhu Y, Li X, Sun J, Zhang Y (2020) Enhancing self-supervised monocular depth estimation via incorporating robust constraints. In: Proceedings of the 28th ACM international conference on multimedia

  31. Li H, Kadav A, Durdanovic I, Samet H, Graf HP (2016) Pruning filters for efficient convnets. arXiv:1608.08710

  32. Li R, Sun J, Gong D, Zhu Y, Li H, Zhang Y (2019) Arsac: efficient model estimation via adaptively ranked sample consensus. Neurocomputing

  33. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution

  34. Liu M, Zhang Z, Hou L, Zuo W, Zhang L (2020) Deep adaptive inference networks for single image super-resolution. arXiv:2004.03915

  35. Lu X, Yuan Y, Yan P (2014) Alternatively constrained dictionary learning for image superresolution. IEEE Trans Cybern

  36. Lyn J (2020) Multi-level feature fusion mechanism for single image super-resolution. arXiv:2002.05962

  37. Ma H, Chu X, Zhang B, Wan S (2019) A matrix-in-matrix neural network for image super resolution. arXiv:1903.07949

  38. Mao Q, Li R, Zhu Y, Sun J, Zhang Y (2023) Enhancing 3d-2d representations for convolution occupancy networks. Pattern Recognit

  39. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV

  40. Matsui Y, Ito K, Aramaki Y, Fujimoto A, Ogawa T, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimed Tools Appl

  41. Muqeet A, Iqbal MTB, Bae S-H (2019) Hybrid residual attention network for single image super resolution. arXiv:1907.05514

  42. Niu A, Zhang K, Pham TX, Sun J, Zhu Y, Kweon IS, Zhang Y (2023) Cdpmsr: conditional diffusion probabilistic models for single image super-resolution. arXiv:2302.12831

  43. Niu A, Zhu Y, Zhang C, Sun J, Wang P, Kweon IS, Zhang Y (2022) Ms2net: multi-scale and multi-stage feature fusion for blurred image super-resolution. IEEE Transactions on Circuits and Systems for Video Technology

  44. Niu A, Zhang K, Pham TX, Sun J, Zhu Y, Kweon IS, Zhang Y (2023) Cdpmsr: conditional diffusion probabilistic models for single image super-resolution. arXiv:2202.05488

  45. Peng Y (2020) Super-resolution reconstruction using multiconnection deep residual network combined an improved loss function for single-frame image. Multimed Tools Appl

  46. Peng H, Wu J, Chen S, Huang J (2019) Collaborative channel pruning for deep networks. In: ICML

  47. Rajput SS, Arya K (2020) A robust face super-resolution algorithm and its application in low-resolution face recognition system. Multimed Tools Appl

  48. Salvador J, Perez-Pellitero E (2016) Naive bayes super-resolution forest. In: IEEE international conference on computer vision

  49. Schulter S, Leistner C, Bischof H (2015) Fast and accurate image upscaling with super-resolution forests. In: CVPR

  50. Shamsolmoali P, Zareapoor M, Jain DK, Jain VK, Yang J (2019) Deep convolution network for surveillance records super-resolution. Multimed Tool Appl

  51. Singh A, Singh J (2020) Survey on single image based super-resolution—implementation challenges and solutions. Multimed Tools Appl

  52. Song Z, Zhao X, Jiang H (2021) Gradual deep residual network for super-resolution. Multimed Tools Appl

  53. Tai YW, Liu S, Brown MS, Lin S (2010) Super resolution using edge prior and single image detail synthesis. In: CVPR

  54. Timofte R, Agustsson E, Van Gool L, Yang M-H, Zhang L (2017) Ntire 2017 challenge on single image super-resolution: methods and results. In: CVPR workshops

  55. Timofte R, De V, Gool LV (2014) Anchored neighborhood regression for fast example-based super-resolution. In: ICCV

  56. Wang Z, Chen J, Hoi SCH (2020) Deep learning for image super-resolution: a survey. IEEE Trans Pattern Anal Mach Intell

  57. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Loy CC, Qiao Y, Tang X (2018) Esrgan: enhanced super-resolution generative adversarial networks

  58. Woo S, Park J, Lee JY, Kweon IS (2018) Cbam: convolutional block attention module

  59. Woo S, Park J, Lee J-Y, So Kweon I (2018) Cbam: convolutional block attention module. In: ECCV

  60. Yang H, Tong J, Dou Q, Xiao L, Jeon G, Yang X (2021) Wide receptive field networks for single image super-resolution. Multimed Tools Appl

  61. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. TIP

  62. Yu J, Gao X, Tao D, Li X, Zhang K (2014) A unified learning framework for single image super-resolution. TNNLS

  63. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: International conference on curves and surfaces

  64. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: ECCV

  65. Zhao H, Jia J, Koltun V (2020) Exploring self-attention for image recognition. In: CVPR

  66. Zhu X, Cheng D, Zhang Z, Lin S, Dai J (2020) An empirical study of spatial attention mechanisms in deep networks. In: ICCV

Download references

Funding

This work was funded in part by the Project of the National Natural Science Foundation of China under Grant 61901384 and 61871328, the Natural Science Basic Research Program of Shaanxi under Grant 2021JCW-03, as well as the Joint Funds of the National Natural Science Foundation of China under Grant U19B2037.

Author information

Authors and Affiliations

  1. School of Computer Science, Northwestern Polytechnical University, Xi’an, 710129, State, China

    Axi Niu, Pei Wang, Yu Zhu, Qingsen Yan & Yanning Zhang

  2. School of Astronautics, Northwestern Polytechnical University, Xi’an, 710072, State, China

    Jinqiu Sun

Authors
  1. Axi Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinqiu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qingsen Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yanning Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, A.N, and Q.Y.; methodology, A.N.; software, A.N, P.W.; validation, A.N., P.W.; formal analysis, A.N.; investigation, P.W.; resources, Y.Z.; data curation, J.S.; writing—original draft preparation, A.N.; writing—review and editing, A.N., P.W., Y.Z, S.J.; visualization, P.W.; supervision, S.J.; project administration, A.N., Y.Z. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Jinqiu Sun.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Niu, A., Wang, P., Zhu, Y. et al. GRAN: ghost residual attention network for single image super resolution. Multimed Tools Appl 83, 28505–28522 (2024). https://doi.org/10.1007/s11042-023-15088-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15088-4

Keywords

Search

Navigation