Skip to main content
SpringerLink
Log in

Dual-stream GNN fusion network for hyperspectral classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Semi-supervised Graph Neural Networks (GNNs), as an effective data representation learning framework, have been applied to hyperspectral image (HSI) classification. Although GNNs can quickly capture the structural information of HSIs, traditional GNN-based methods use the entire graph as input, which consumes huge computational resources. Moreover, using a single form of GNN variant cannot effectively extract the correlation between two different dimensions, leading to insufficient improvement in the impact of noise and spectral differences in HSIs. To address these issues, a Dual-Stream GNN Fusion Network (named DGFNet) is proposed in this paper. This end-to-end network uses subcubes as input to reduce computational costs. The spatial branch uses Graph Attention Network (GAT) to capture the inherent relationships within the HSIs, and the risk of overfitting is reduced by combining a novel graph pooling and local guidance module to preserve important features in subcubes. The spectral branch weights the correlations among bands to obtain more reasonable spectral features. Finally, these spatial, spectral, and structural features are fused and classified using linear layers. Extensive comparative experiments on four benchmark datasets have demonstrated that our model achieves competitive performance, and its low computational cost advantage makes it more feasible and practical compared to most GNN-based methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Algorithm 2
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Data availability and access

The datasets generated and analysed in the current study comprise four publicly available datasets: Indian Pines, University of Pavia, Houston 2013, and Salinas datasets

References

  1. Ren J, Shi M, Chen J, Wang R, Wang X (2023) Hyperspectral image classification using multi-level features fusion capsule network with a dense structure. Appl Intell 53(11):14162–14181

    Article  Google Scholar 

  2. Wang M, Yan Z, Luo J, Ye Z, He P (2021) A band selection approach based on wavelet support vector machine ensemble model and membrane whale optimization algorithm for hyperspectral image. Appl Intell 1–15

  3. Tsai M-H, Chang Y-L, Shiau J-S, Wang S-M (2020) Exploring the effects of a serious game-based learning package for disaster prevention education: The case of battle of flooding protection. Int J Disaster Risk Reduction 43:101393

    Article  Google Scholar 

  4. Poland TM, Rassati D (2019) Improved biosecurity surveillance of non-native forest insects: a review of current methods. J Pest Sci 92(1):37–49

    Article  Google Scholar 

  5. Syed-Ab-Rahman SF, Hesamian MH, Prasad M (2022) Citrus disease detection and classification using end-to-end anchor-based deep learning model. Appl Intell 52(1):927–938

    Article  Google Scholar 

  6. Zhang J, Zou X, Kuang L-D, Wang J, Sherratt RS, Yu X (2022) Cctsdb 2021: a more comprehensive traffic sign detection benchmark. Hum-centric Comput Inf Sci 12

  7. Mounika K, Aravind K, Yamini M, Navyasri P, Dash S, Suryanarayana V (2021) Hyperspectral image classification using svm with pca. In: 2021 6th International Conference on Signal Processing, Computing and Control (ISPCC), pp. 470–475. IEEE

  8. Li J, Zhu Q (2020) A boosting self-training framework based on instance generation with natural neighbors for k nearest neighbor. Appl Intell 50(11):3535–3553

    Article  Google Scholar 

  9. Bukht TFN, Rahman H, Jalal A (2023) A novel framework for human action recognition based on features fusion and decision tree. In: 2023 4th International Conference on Advancements in Computational Sciences (ICACS), pp. 1–6. IEEE

  10. Ma KY, Chang C-I (2021) Kernel-based constrained energy minimization for hyperspectral mixed pixel classification. IEEE Trans Geosci Remote Sens 60:1–23

    Google Scholar 

  11. Tan X, Gao K, Liu B, Fu Y, Kang L (2021) Deep global-local transformer network combined with extended morphological profiles for hyperspectral image classification. Journal of Applied Remote Sensing 15(3):038509–038509

    Article  Google Scholar 

  12. Hong D, Wu X, Ghamisi P, Chanussot J, Yokoya N, Zhu XX (2020) Invariant attribute profiles: A spatial-frequency joint feature extractor for hyperspectral image classification. IEEE Trans Geosci Remote Sens 58(6):3791–3808

    Article  Google Scholar 

  13. Huang K-K, Ren C-X, Liu H, Lai Z-R, Yu Y-F, Dai D-Q (2021) Hyperspectral image classification via discriminant gabor ensemble filter. IEEE Trans Cybernet 52(8):8352–8365

    Article  Google Scholar 

  14. Vaddi R, Manoharan P (2020) Hyperspectral image classification using cnn with spectral and spatial features integration. Infrared Phys Technol 107:103296

    Article  Google Scholar 

  15. Huang L, Chen Y (2020) Dual-path siamese cnn for hyperspectral image classification with limited training samples. IEEE Geosci Remote Sens Lett 18(3):518–522

    Article  Google Scholar 

  16. Swain S, Banerjee A (2021) Evaluation of dimensionality reduction techniques on hybrid cnn-based hsi classification. Arabian J Geosci 14(24):2806

    Article  Google Scholar 

  17. Chen Y, Xia R, Zou K, Yang K (2023) Rnon: image inpainting via repair network and optimization network. Int J Mach Learn Cybernet 1–17

  18. Chen Y, Xia R, Zou K, Yang K (2023) Ffti: Image inpainting algorithm via features fusion and two-steps inpainting. J Visual Commun Image Representat 91:103776

    Article  Google Scholar 

  19. Zhou W, Kamata S-i, Wang H, Xue X (2023) Multiscanning-based rnn-transformer for hyperspectral image classification. IEEE Trans Geosci Remote Sens

  20. Zhao J, Hu L, Dong Y, Huang L, Weng S, Zhang D (2021) A combination method of stacked autoencoder and 3d deep residual network for hyperspectral image classification. Int J Appl Earth Observation Geoinf 102:102459

    Article  Google Scholar 

  21. Gao H, Yang Y, Lei S, Li C, Zhou H, Qu X (2019) Multi-branch fusion network for hyperspectral image classification. Knowl-Based Syst 167:11–25

    Article  Google Scholar 

  22. Zhang J, Huang H, Jin X, Kuang L-D, Zhang J (2023) Siamese visual tracking based on criss-cross attention and improved head network. Multimed Tools Appl 1–27

  23. Jia S, Lin Z, Xu M, Huang Q, Zhou J, Jia X, Li Q (2020) A lightweight convolutional neural network for hyperspectral image classification. IEEE Trans Geosci Remote Sens 59(5):4150–4163

    Article  Google Scholar 

  24. Han Z, Hong D, Gao L, Zhang B, Huang M, Chanussot J (2022) Autonas: Automatic neural architecture search for hyperspectral unmixing. IEEE Trans Geosci Remote Sens 60:1–14

    Google Scholar 

  25. Zhang J, Zheng Z, Xie X, Gui Y, Kim G-J (2022) Reyolo: A traffic sign detector based on network reparameterization and features adaptive weighting. J Ambient Intell Smart Environs (Preprint) 1–18

  26. Wang W, Dou S, Jiang Z, Sun L (2018) A fast dense spectral-spatial convolution network framework for hyperspectral images classification. Remote Sens 10(7):1068

    Article  Google Scholar 

  27. Wu H, Li D, Wang Y, Li X, Kong F, Wang Q (2021) Hyperspectral image classification based on two-branch spectral-spatial-feature attention network. Remote Sens 13(21):4262

    Article  Google Scholar 

  28. Chen Y, Xia R, Yang K, Zou K (2023) Mffn: Image super-resolution via multi-level features fusion network. Visual Computer 1–16

  29. Chen Y, Xia R, Yang K, Zou K (2023) Dargs: Image inpainting algorithm via deep attention residuals group and semantics. J King Saud Univ-Computer Inf Sci 35(6):101567

    Google Scholar 

  30. Zhang X, Chen S, Zhu P, Tang X, Feng J, Jiao L (2022) Spatial pooling graph convolutional network for hyperspectral image classification. IEEE Trans Geosci Remote Sens 60:1–15. https://doi.org/10.1109/TGRS.2022.3140353

    Article  Google Scholar 

  31. Bai J, Ding B, Xiao Z, Jiao L, Chen H, Regan AC (2021) Hyperspectral image classification based on deep attention graph convolutional network. IEEE Trans Geosci Remote Sens 60:1–16

    Google Scholar 

  32. Guo F, Li Z, Xin Z, Zhu X, Wang L (2021) Zhang J (2021) Dual graph u-nets for hyperspectral image classification. IEEE J Selected Topics Appl Earth Observations Remote Sens 14:8160–8170

    Article  Google Scholar 

  33. Zhang M, Luo H, Song W, Mei H, Su C (2021) Spectral-spatial offset graph convolutional networks for hyperspectral image classification. Remote Sens 13(21):4342

    Article  Google Scholar 

  34. Qin A, Shang Z, Tian J, Wang Y, Zhang T, Tang YY (2018) Spectral-spatial graph convolutional networks for semisupervised hyperspectral image classification. IEEE Geosci Remote Sens Lett 16(2):241–245

    Article  Google Scholar 

  35. Wan S, Gong C, Zhong P, Du B, Zhang L, Yang J (2019) Multiscale dynamic graph convolutional network for hyperspectral image classification. IEEE Trans Geosci Remote Sens 58(5):3162–3177

    Article  Google Scholar 

  36. Liu Q, Xiao L, Yang J, Wei Z (2020) Cnn-enhanced graph convolutional network with pixel-and superpixel-level feature fusion for hyperspectral image classification. IEEE Trans Geosci Remote Sens 59(10):8657–8671

    Article  Google Scholar 

  37. Sha A, Wang B, Wu X, Zhang L (2020) Semisupervised classification for hyperspectral images using graph attention networks. IEEE Geosci Remote Sens Lett 18(1):157–161

    Article  Google Scholar 

  38. Wang T, Wang G, Tan KE, Tan D (2020) Spectral pyramid graph attention network for hyperspectral image classification. arXiv:2001.07108

  39. Ding Y, Zhao X, Zhang Z, Cai W, Yang N (2021) Multiscale graph sample and aggregate network with context-aware learning for hyperspectral image classification. IEEE J Selected Topics Appl Earth Observations and Remote Sens 14:4561–4572

    Article  Google Scholar 

  40. Zhao Z, Wang H, Yu X (2022) Spectral-spatial graph attention network for semisupervised hyperspectral image classification. IEEE Geosci Remote Sens Lett 19

  41. Guo W, Xu G, Liu W, Liu B, Wang Y (2021) Cnn-combined graph residual network with multilevel feature fusion for hyperspectral image classification. IET Computer Vision 15(8):592–607

    Article  Google Scholar 

  42. Dong Y, Liu Q, Du B, Zhang L (2022) Weighted feature fusion of convolutional neural network and graph attention network for hyperspectral image classification. IEEE Trans Image Process 31:1559–1572

    Article  Google Scholar 

  43. Lu Z, Liang S, Yang Q, Du B (2022) Evolving block-based convolutional neural network for hyperspectral image classification. IEEE Trans Geosci Remote Sens 60:1–21

    Google Scholar 

  44. Qu L, Zhu X, Zheng J, Zou L (2021) Triple-attention-based parallel network for hyperspectral image classification. Remote Sens 13(2):324

    Article  Google Scholar 

  45. Maini DS, Aggarwal DAK (2018) Camera position estimation using 2d image dataset

  46. Bhardwaj VK, Maini S (2021) Improving snr of sm-ofi signal using rls adaptive filter algorithm. In: 2021 Sixth International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET), pp. 183–187. IEEE

  47. Ye Z, Li H, Song Y, Benediktsson JA, Tang YY (2016) Hyperspectral image classification using principal components-based smooth ordering and multiple 1-d interpolation. IEEE Trans Geosci Remote Sens 55(2):1199–1209

    Article  Google Scholar 

  48. Veličković, P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2017) Graph attention networks. arXiv:1710.10903

  49. Wang D, Du B, Zhang L (2021) Spectral-spatial graph reasoning network for hyperspectral image classification. arXiv:2106.13952

  50. Chauhan S, Singh M, Aggarwal AK (2023) Designing of optimal digital iir filter in the multi-objective framework using an evolutionary algorithm. Eng Appl Artif Intell 119:105803

    Article  Google Scholar 

  51. Chauhan S, Singh M, Aggarwal AK (2023) Investigative analysis of different mutation on diversity-driven multi-parent evolutionary algorithm and its application in area coverage optimization of wsn. Soft Comput 1–27

  52. Roy SK, Manna S, Song T, Bruzzone L (2021) Attention-based adaptive spec-tral-spatial kernel resnet for hyperspectral image classification. IEEE Trans Geosci Remote Sens 59(9):7831–7843. https://doi.org/10.1109/TGRS.2020.3043267

    Article  Google Scholar 

  53. Roy SK, Krishna G, Dubey SR, Chaudhuri BB (2019) Hybridsn: Exploring 3-d-2-d cnn feature hierarchy for hyperspectral image classification. IEEE Geosci Remote Sens Lett 17(2):277–281

    Article  Google Scholar 

  54. Sun L, Zhao G, Zheng Y, Wu Z (2022) Spectral-spatial feature tokenization transformer for hyperspectral image classification. IEEE Trans Geosci Remote Sens 60:1–14

    Article  Google Scholar 

  55. Ma W, Yang Q, Wu Y, Zhao W, Zhang X (2019) Double-branch multi-attention mechanism network for hyperspectral image classification. Remote Sens 11(11):1307

    Article  Google Scholar 

  56. Zhong Z, Li Y, Ma L, Li J, Zheng W-S (2021) Spectral-spatial transformer network for hyperspectral image classification: A factorized architecture search framework. IEEE Trans Geosci Remote Sens 60:1–15

    Article  Google Scholar 

  57. Li R, Zheng S, Duan C, Yang Y, Wang X (2020) Classification of hyperspectral image based on double-branch dual-attention mechanism network. Remote Sens 12(3):582

  58. Liu Q, Dong Y, Zhang Y, Luo H (2022) A fast dynamic graph convolutional network and cnn parallel network for hyperspectral image classification. IEEE Trans Geosci Remote Sens 60:1–15. https://doi.org/10.1109/TGRS.2022.3179419

    Article  Google Scholar 

  59. Xu Y, Li Z, Li W, Du Q, Liu C, Fang Z, Zhai L (2021) Dual-channel residual network for hyperspectral image classification with noisy labels. IEEE Trans Geosci Remote Sens 60:1–11

    Google Scholar 

  60. Zhang X, Shang S, Tang X, Feng J, Jiao L (2022) Spectral partitioning residual network with spatial attention mechanism for hyperspectral image classification. IEEE Trans Geosci Remote Sens 60:1–14. https://doi.org/10.1109/TGRS.2021.3074196

  61. Maaten L, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9(11)

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their detailed and constructive comments and suggestions, and thank Assoc. Prof. Cong’an Xu of NAU for their constructive suggestions. Thank Assoc. Prof. Shuaishuai FAN of STBU and Prof. Hongyang Bai of NJUST for the numerical calculations in this article

Funding

This research was supported by the National Natural Science Foundation of China (NSFC), grant number 62271499 and U2031138

Author information

Authors and Affiliations

  1. School of Information and Electronic Engineering, Shandong Technology and Business University (STBU), Yantai, 264000, China

    Weiming Li, Qikang Liu & Shuaishuai Fan

  2. Information Fusion Institute, Naval Aviation University (NAU), Yantai, 264000, China

    Cong’an Xu

  3. School of Energy and Power Engineering, Nanjing University of Science and Technology (NJUST), Nanjing, 210094, China

    Hongyang Bai

Authors
  1. Weiming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qikang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuaishuai Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cong’an Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hongyang Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Qikang Liu: Data curation, Methodology, Writing-Original draft preparation. Weiming Li: Conceptualization of this study, Software. Shuaishuai Fan: Data curation, Methodology. Cong’an Xu and Hongyang Bai: Writing-Original draft preparation

Corresponding author

Correspondence to Shuaishuai Fan.

Ethics declarations

Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper

Ethical and informed consent for data

All authors are informed and consent to publication

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Weiming Li, Qikang Liu, Shuaishuai Fan, Cong’an Xu, and Hongyang Bai these authors contributed equally to this work.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, W., Liu, Q., Fan, S. et al. Dual-stream GNN fusion network for hyperspectral classification. Appl Intell 53, 26542–26567 (2023). https://doi.org/10.1007/s10489-023-04960-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04960-3

Keywords

Search

Navigation