Skip to main content
SpringerLink
Log in

CGA-Net: channel-wise gated attention network for improved super-resolution in remote sensing imagery

  • Research
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Super-resolution (SR) is a powerful technique for enhancing the quality of remote sensing imagery, which in turn can improve the accuracy of various computer vision tasks, such as object detection, classification, and segmentation. Deep convolutional neural networks (CNNs) have demonstrated significant progress in this field, and attention mechanisms are widely adopted in deep CNNs as they allow the models to assign weights to important areas within the feature map. In this paper, we propose the channel-wise gated attention (CGA) module, which integrates attention across the feature map channels and scales the resulting feature map through a gating parameter, leading to performance improvements. Furthermore, we present an SR framework that employs multiple attention blocks, with the CGA module serving as the core of each block, to enhance the spatial resolution of remote sensing imagery. Our proposed network, the channel-wise gated attention Network (CGA-Net), outperforms other attention-based deep SR models for 4\(\times \)- and 8\(\times \)-upsampling on two remote sensing datasets: Satellite Imagery Multi-Vehicles Dataset (SIMD), consisting of 5000 high-resolution remote sensing images, and DOTA, a large-scale satellite imagery dataset. We conduct several experiments to evaluate the effectiveness of our SR framework for object detection on the SIMD dataset. The code and trained weights for the proposed framework can be found at this link: https://github.com/Vision-At-SEECS/CGA-Net.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The code and trained weights for the proposed architecture can be found at this GitHub repository: https://github.com/Vision-At-SEECS/CGA-Net.

References

  1. Shermeyer, J., Van Etten, A.: The effects of super-resolution on object detection performance in satellite imagery. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0–0 (2019)

  2. Yang, D., Li, Z., Xia, Y., Chen, Z.: Remote sensing image super-resolution: challenges and approaches. In: 2015 IEEE International Conference on Digital Signal Processing (DSP), pp. 196–200, IEEE (2015)

  3. Anwar, S., Barnes, N.: Densely residual laplacian super-resolution. IEEE Trans. Patt. Anal. Mach. Intell. 44(3), 1192–1204 (2020)

    Article  Google Scholar 

  4. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Patt. Anal. Mach. Intell. 38(2), 295–307 (2016). https://doi.org/10.1109/TPAMI.2015.2439281

    Article  Google Scholar 

  5. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  6. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)

  7. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 286–301 (2018)

  8. Niu, B., Wen, W., Ren, W., Zhang, X., Yang, L., Wang, S., Zhang, K., Cao, X., Shen, H.: Single image super-resolution via a holistic attention network. In: European Conference on Computer Vision, pp. 191–207, Springer (2020)

  9. Xiao, Y., Yuan, Q., He, J., Zhang, Q., Sun, J., Su, X., Wu, J., Zhang, L.: Space-time super-resolution for satellite video: a joint framework based on multi-scale spatial-temporal transformer. Int. J. Appl. Earth Obs. Geoinf. 108, 102731 (2022). https://doi.org/10.1016/j.jag.2022.102731

    Article  Google Scholar 

  10. Jocher, G., Stoken, A., Borovec, J., NanoCode012, Chaurasia, A., TaoXie, Changyu, L., V, A., Laughing, tkianai, yxNONG, Hogan, A., lorenzomammana, AlexWang1900, Hajek, J., Diaconu, L., Marc, Kwon, Y., oleg, wanghaoyang0106, Defretin, Y., Lohia, A., ml5ah, Milanko, B., Fineran, B., Khromov, D., Yiwei, D., Doug, Durgesh, Ingham, F.: ultralytics/yolov5: V5.0 - YOLOv5-P6 1280 Models, AWS, Supervise.ly and YouTube Integrations. https://doi.org/10.5281/zenodo.4679653. https://doi.org/10.5281/zenodo.4679653

  11. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: European Conference on Computer Vision, pp. 391–407, Springer (2016)

  12. Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

  13. Huang, G., Liu, Z., Weinberger, K.Q.: Densely connected convolutional networks. CoRR abs/1608.06993 (2016) arXiv:1608.06993

  14. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3147–3155 (2017)

  15. Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: A persistent memory network for image restoration. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4539–4547 (2017)

  16. Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136–144 (2017)

  17. Ahn, N., Kang, B., Sohn, K.-A.: Image super-resolution via progressive cascading residual network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 791–799 (2018)

  18. Liang, J., Zhang, Y., Xue, J., Zhang, Y., Hu, Y.: Lightweight image super-resolution network using involution. Mach. Vis. Appl. 33(5), 68 (2022)

    Article  Google Scholar 

  19. Tong, T., Li, G., Liu, X., Gao, Q.: Image super-resolution using dense skip connections. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017)

  20. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

  21. Haris, M., Shakhnarovich, G., Ukita, N.: Deep back-projection networks for super-resolution. CoRR abs/1803.02735 (2018) arXiv:1803.02735

  22. Jiang, K., Wang, Z., Yi, P., Wang, G., Lu, T., Jiang, J.: Edge-enhanced GAN for remote sensing image superresolution. IEEE Trans. Geosci. Rem. Sens. 57(8), 5799–5812 (2019)

    Article  Google Scholar 

  23. Xiao, Y., Su, X., Yuan, Q., Liu, D., Shen, H., Zhang, L.: Satellite video super-resolution via multiscale deformable convolution alignment and temporal grouping projection. IEEE Trans. Geosci. Rem. Sens. 60, 1–19 (2022). https://doi.org/10.1109/TGRS.2021.3107352

    Article  Google Scholar 

  24. Xiao, Y., Yuan, Q., Jiang, K., He, J., Wang, Y., Zhang, L.: From degrade to upgrade: learning a self-supervised degradation guided adaptive network for blind remote sensing image super-resolution. Inf. Fus. 96, 297–311 (2023). https://doi.org/10.1016/j.inffus.2023.03.021

    Article  Google Scholar 

  25. Xiao, Y., Yuan, Q., Jiang, K., Jin, X., He, J., Zhang, L., Lin, C.-w.: Local-global temporal difference learning for satellite video super-resolution. arXiv preprint arXiv:2304.04421 (2023)

  26. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  27. Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  28. Kim, J.-H., Choi, J.-H., Cheon, M., Lee, J.-S.: Ram: Residual attention module for single image super-resolution. arXiv preprint arXiv:1811.12043 2, 1 (2018)

  29. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. CoRR abs/1609.05158 (2016) arXiv:1609.05158

  30. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process 13(4), 600–612 (2004)

    Article  Google Scholar 

  31. Haroon, M., Shahzad, M., Fraz, M.M.: Multisized object detection using spaceborne optical imagery. IEEE J. Sel. Top. Appl. Earth Obs. Rem. Sens. 13, 3032–3046 (2020)

    Article  Google Scholar 

  32. Ding, J., Xue, N., Xia, G.-S., Bai, X., Yang, W., Yang, M.Y., Belongie, S., Luo, J., Datcu, M., Pelillo, M., Zhang, L.: Object detection in aerial images: a large-scale benchmark and challenges. IEEE Trans. Patt. Anal. Mach. Intell. 44(11), 7778–7796 (2021)

    Article  Google Scholar 

  33. Sentinel 2. Earth Online. European Space Agency. https://sentinels.copernicus.eu/web/sentinel/missions/sentinel-2

  34. Dong, X., Sun, X., Jia, X., Xi, Z., Gao, L., Zhang, B.: Remote sensing image super-resolution using novel dense-sampling networks. IEEE Trans. Geosci. Rem. Sens. 59(2), 1618–1633 (2021). https://doi.org/10.1109/TGRS.2020.2994253

    Article  Google Scholar 

  35. Dong, X., Xi, Z., Sun, X., Yang, L.: Remote sensing image super-resolution via enhanced back-projection networks. In: IGARSS 2020–2020 IEEE International Geoscience and Remote Sensing Symposium, pp. 1480–1483 (2020). https://doi.org/10.1109/IGARSS39084.2020.9323316

  36. Lei, S., Shi, Z., Zou, Z.: Coupled adversarial training for remote sensing image super-resolution. IEEE Trans. Geosci. Rem. Sens. 58(5), 3633–3643 (2020). https://doi.org/10.1109/TGRS.2019.2959020

    Article  Google Scholar 

  37. Rabbi, J., Ray, N., Schubert, M., Chowdhury, S., Chao, D.: Small-object detection in remote sensing images with end-to-end edge-enhanced GAN and object detector network. Rem. Sens. 12(9), 1432 (2020)

    Article  Google Scholar 

Download references

Funding

No funding was received for conducting this study.

Author information

Authors and Affiliations

  1. National University of Sciences and Technology (NUST), Islamabad, Pakistan

    Bostan Khan, Zuhair Zafar & Muhammad Moazam Fraz

  2. Center of Excellence in Science and Applied Technologies (CESAT), Islamabad, Pakistan

    Adeel Mumtaz

  3. Staffordshire University, Stoke-on-Trent, ST4 2DF, UK

    Mohamed Sedkey & Elhadj Benkhelifa

Authors
  1. Bostan Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adeel Mumtaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zuhair Zafar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohamed Sedkey
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elhadj Benkhelifa
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Muhammad Moazam Fraz
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study’s conception and design. Experimentation and ablation studies were performed by BK, ZZ, and AM. Data analysis and review were conducted by MMF and MS. The first draft of the manuscript was written by BK, and all authors commented on previous versions of the manuscript. The project supervision is done by F. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Muhammad Moazam Fraz.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khan, B., Mumtaz, A., Zafar, Z. et al. CGA-Net: channel-wise gated attention network for improved super-resolution in remote sensing imagery. Machine Vision and Applications 34, 128 (2023). https://doi.org/10.1007/s00138-023-01477-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-023-01477-0

Keywords

Search

Navigation