Skip to main content
SpringerLink
Log in

Multi-level relation learning for cross-domain few-shot hyperspectral image classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Cross-domain few-shot hyperspectral image classification focuses on learning prior knowledge from a large number of labeled samples from source domains and then transferring the knowledge to the tasks which contain few labeled samples in target domains. Following the metric-based manner, many current methods first extract the features of the query and support samples, and then directly predict the classes of query samples according to their distance to the support samples or prototypes. The relations between samples have not been fully explored and utilized. Different from current works, this paper proposes to learn sample relations on different levels and take them into the model learning process, to improve the cross-domain few-shot hyperspectral image classification. Building on current method of "Deep Cross-Domain Few-Shot Learning for Hyperspectral Image Classification" which adopts a domain discriminator to deal with domain-level distribution difference, the proposed method applies contrastive learning to learn the class-level sample relations to obtain more discriminable sample features. In addition, it adopts a transformer based cross-attention learning module to learn the set-level sample relations and acquire the attention from query samples to support samples. Our experimental results have demonstrated the contribution of the multi-level relation learning mechanism for few-shot hyperspectral image classification when compared with the state of the art methods. All the codes are available at github https://github.com/HENULWY/STBDIP.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Availability of data and materials

The datasets generated and analysed during the current study are available at https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes and http://naotoyokoya.com/Download.html.

Code availability

The codes are available at https://github.com/HENULWY/STBDIP.

References

  1. Li S, Song W, Fang L, Chen Y, Benediktsson JA (2019) Deep learning for hyperspectral image classification: An overview. IEEE Transactions on Geoscience and Remote Sensing PP (99) 1–20

  2. Jia S, Jiang S, Lin Z, Li N, Xu M, Yu S (2021) A survey: Deep learning for hyperspectral image classification with few labeled samples. Neurocomputing 448:179–204

    Article  Google Scholar 

  3. Melgani F, Bruzzone L (2004) Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing 42(8):1778–1790

    Article  Google Scholar 

  4. Li J, Bioucas-Dias JM, Plaza A (2012) Semisupervised hyperspectral image classification using soft sparse multinomial logistic regression. IEEE Geoscience and Remote Sensing Letters 10(2):318–322

    Google Scholar 

  5. Licciardi G, Marpu PR, Chanussot J, Benediktsson JA (2011) Linear versus nonlinear pca for the classification of hyperspectral data based on the extended morphological profiles. IEEE Geoscience and Remote Sensing Letters 9(3):447–451

    Article  Google Scholar 

  6. Zhang C, Zheng Y (2014) Hyperspectral remote sensing image classification based on combined svm and lda. Multispectral, Hyperspectral, and Ultraspectral Remote Sensing Technology, Techniques and Applications V 9263:462–468

    Google Scholar 

  7. Qian Y, Ye M, Zhou J (2012) Hyperspectral image classification based on structured sparse logistic regression and threedimensional wavelet texture features. IEEE Transactions on Geoscience and Remote Sensing 51(4):2276–2291

    Article  Google Scholar 

  8. Falco N, Benediktsson JA, Bruzzone L (2015) Spectral and spatial classification of hyperspectral images based on ica and reduced morphological attribute profiles. IEEE Transactions on Geoscience and Remote Sensing 53(11):6223–6240

    Article  Google Scholar 

  9. Yu S, Jia S, Xu C (2017) Convolutional neural networks for hyperspectral image classification. Neurocomputing 219:88–98

    Article  Google Scholar 

  10. Paoletti ME, Haut JM, Plaza J, Plaza A (2019) Deep learning classifiers for hyperspectral imaging: A review. ISPRS Journal of Photogrammetry and Remote Sensing 158(Dec.) 279–317

  11. Chen X, Li M, Yang X (2015) Stacked denoise autoencoder based feature extraction and classification for hyperspectral images. Journal of Sensors 2016(2015-11-30), 1–10

  12. Mei S, Ji J, Geng Y, Zhang Z, Li X, Du Q (2019) Unsupervised spatial-spectral feature learning by 3d convolutional autoencoder for hyperspectral classification. IEEE Transactions on Geoscience and Remote Sensing 57(9):6808–6820

    Article  Google Scholar 

  13. Lee H, Kwon H (2017) Going deeper with contextual cnn for hyperspectral image classification. IEEE Trans Image Process 26(10):4843–4855

    Article  MathSciNet  Google Scholar 

  14. Zhang H, Li Y, Zhang Y, Shen Q (2017) Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network. Remote Sensing Letters 8(4–6):438–447

    Article  Google Scholar 

  15. Zhong Z, Li J, Luo Z, Chapman M (2017) Spectral-spatial residual network for hyperspectral image classification: A 3-d deep learning framework. IEEE Transactions on Geoscience and Remote Sensing 56(2):847–858

    Article  Google Scholar 

  16. Liu B, Yu X, Zhang P, Tan X, Wang R, Zhi L (2018) Spectral-spatial classification of hyperspectral image using threedimensional convolution network. Journal of Applied Remote Sensing 12(1):016005–016005

    Google Scholar 

  17. Mou L, Ghamisi P, Zhu XX (2017) Deep recurrent neural networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 55(7):3639–3655

    Article  Google Scholar 

  18. Liu B, Yu X, Yu A, Zhang P, Wan G (2018) Spectral-spatial classification of hyperspectral imagery based on recurrent neural networks. Remote sensing letters 9(10–12):1118–1127

    Article  Google Scholar 

  19. Zhou F, Hang R, Liu Q, Yuan X (2019) Hyperspectral image classification using spectralspatial lstms - sciencedirect. Neurocomputing 328:39–47

    Article  Google Scholar 

  20. Qin A, Shang Z, Tian J, Wang Y, Zhang T, Tang YY (2018) Spectral-spatial graph convolutional networks for semisupervised hyperspectral image classification. IEEE Geoscience and Remote Sensing Letters 16(2):241–245

    Article  Google Scholar 

  21. Wan S, Gong C, Zhong P, Pan S, Li G, Yang J (2020) Hyperspectral image classification with context-aware dynamic graph convolutional network. IEEE Transactions on Geoscience and Remote Sensing 59(1):597–612

    Article  Google Scholar 

  22. Hong D, Gao L, Yao J, Zhang B, Plaza A, Chanussot J (2020) Graph convolutional networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 59(7):5966–5978

    Article  Google Scholar 

  23. Tang H, Li Y, Han X, Huang Q, Xie W (2019) A spatial-spectral prototypical network for hyperspectral remote sensing image. IEEE Geoscience and Remote Sensing Letters 17(1):167–171

    Article  Google Scholar 

  24. Ma X, Ji S, Wang J, Geng J, Wang H (2019) Hyperspectral image classification based on two-phase relation learning network. IEEE Transactions on Geoscience and Remote Sensing 57(12):10398–10409

    Article  Google Scholar 

  25. Li Z, Liu M, Chen Y, Xu Y, Du Q (2021) Deep cross-domain few-shot learning for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing PP(99), 1–18

  26. Bai J, Huang S, Xiao Z, Li X, Zhu Y, Regan AC, Jiao L (2022) Few-shot hyperspectral image classification based on adaptive subspaces and feature transformation. IEEE Transactions on Geoscience and Remote Sensing 60:1–17

    Google Scholar 

  27. Zhang Y, Li W, Zhang M, Wang S, Tao R, Du Q (2022) Graph information aggregation cross-domain few-shot learning for hyperspectral image classification. IEEE Transactions on Neural Networks and Learning Systems

  28. Khosla P, Teterwak P, Wang C, Sarna A, Tian Y, Isola P, Maschinot A, Liu C, Krishnan D (2020) Supervised contrastive learning. Advances in neural information processing systems 33:18661–18673

    Google Scholar 

  29. Koch G, Zemel R, Salakhutdinov R et al (2015) Siamese neural networks for one-shotimage recognition. ICML deep learning workshop 2(1)

  30. Vinyals O, Blundell C, Lillicrap T, Wierstra D et al (2016) Matching networks for one shot learning. Advances in neural information processing systems 29

  31. Snell J, Swersky K, Zemel R (2017) Prototypical networks for few-shot learning. Advances in neural information processing systems 30

  32. Sung F, Yang Y, Zhang L, Xiang T, Torr PH, Hospedales TM (2018) Learning to compare: Relation network for few-shot learning. Proceedings of the IEEE conference on computer vision and pattern recognition, 1199–1208

  33. Zhang B, Li X, Ye Y, Huang Z, Zhang L (2021) Prototype completion with primitive knowledge for few-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3754–3762

  34. Xue W, Wang W (2020) One-shot image classification by learning to restore prototypes. Proceedings of the AAAI Conference on Artificial Intelligence 34(04):6558–6565

    Article  MathSciNet  Google Scholar 

  35. Xu W, Xu Y, Wang H, Tu Z (2021) Attentional constellation nets for few-shot learning

  36. Finn C, Abbeel P, Levine S (2017) Modelagnostic meta-learning for fast adaptation of deep networks. International conference on machine learning, 1126–1135

  37. Jamal MA, Qi G-J (2019) Task agnostic metalearning for few-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 11719–11727

  38. Garcia V, Bruna J (2018) Few-shot learning with graph neural networks. 6th International Conference on Learning Representations, ICLR 2018

  39. Kim J, Kim T, Kim S, Yoo CD (2019) Edgelabeling graph neural network for few-shot learning. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11–20

  40. Chen C, Yang X, Xu C, Huang X, Ma Z (2021) Eckpn: Explicit class knowledge propagation network for transductive few-shot learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6596–6605

  41. Zhang C, Lyu X, Tang Z (2019) Tgg: Transferable graph generation for zero-shot and few-shot learning. Proceedings of the 27th ACM International Conference on Multimedia, 1641–1649

  42. Ma Y, Bai S, An S, Liu W, Liu A, Zhen X, Liu X (2020) Transductive relationpropagation network for few-shot learning. IJCAI 20:804–810

    Google Scholar 

  43. Wang W, Dou S, Jiang Z, Sun L (2018) A fast dense spectral-spatial convolution network framework for hyperspectral images classification. Remote sensing 10(7):1068

    Article  Google Scholar 

  44. Mou L, Lu X, Li X, Zhu XX (2020) Nonlocal graph convolutional networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 58(12):8246–8257

    Article  Google Scholar 

  45. Liu B, Yu X, Zhang P, Yu A, Fu Q, Wei X (2017) Supervised deep feature extraction for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 56(4):1909–1921

    Article  Google Scholar 

  46. Huang L, Chen Y (2020) Dual-path siamese cnn for hyperspectral image classification with limited training samples. IEEE Geoscience and Remote Sensing Letters 18(3):518–522

    Article  Google Scholar 

  47. Rao M, Tang P, Zhang Z (2020) A developed siamese cnn with 3d adaptive spatialspectral pyramid pooling for hyperspectral image classification. Remote Sensing 12(12):1964

  48. Sun J, Shen X, Sun Q (2022) Hyperspectral image few-shot classification network based on the earth mover’s distance. IEEE Transactions on Geoscience and Remote Sensing 60:1–14

    Article  Google Scholar 

  49. Gao K, Liu B, Yu X, Qin J, Zhang P, Tan X (2020) Deep relation network for hyperspectral image few-shot classification. Remote Sensing 12(6):923

    Article  Google Scholar 

  50. Rao M, Tang P, Zhang Z (2019) Spatial-spectral relation network for hyperspectral image classification with limited training samples. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12(12):5086–5100

    Article  Google Scholar 

  51. Zuo X, Yu X, Liu B, Zhang P, Tan X (2022) Fsl-egnn: Edge-labeling graph neural network for hyperspectral image few-shot classification. IEEE Transactions on Geoscience and Remote Sensing 60:1–18

    Article  Google Scholar 

  52. Xi B, Li J, Li Y, Song R, Hong D, Chanussot J (2022) Few-shot learning with classcovariance metric for hyperspectral image classification. IEEE Transactions on Image Processing 31:5079–5092

  53. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. Advances in neural information processing systems 30

  54. Long M, Cao Z, Wang J, Jordan MI (2018) Conditional adversarial domain adaptation. Advances in neural information processing systems 31

  55. Li Y, Zhang H, Shen Q (2017) Spectral-spatial classification of hyperspectral imagery with 3d convolutional neural network. Remote Sensing 9(1):67

    Article  Google Scholar 

  56. Yu C, Wang J, Chen Y, Huang M (2019) Transfer learning with dynamic adversarial adaptation network. IEEE Transactions on Geoscience and Remote Sensing pp. 778–786

  57. Zhu Y, Zhuang F, Wang J, Ke G, Chen J, Bian J, Xiong H, He Q (2020) Deep subdomain adaptation network for image classification. IEEE transactions on neural networks and learning systems 32(4):1713–1722

    Article  MathSciNet  Google Scholar 

  58. Liu B, Yu X, Yu A, Zhang P, Wan G, Wang R (2018) Deep few-shot learning for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 57(4):2290–2304

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank our anonymous reviewers for their valuable comments and suggestions. This work was supported by the Henan Province Science and Technology Research Project under Grant 232102110276, and National Natural Science Foundation of China under Grant 42371433.

Author information

Author notes

  1. Zheng Li, Wei Yang, Zhigang Han, Jianzhong Guo, and Junyong Yu These authors contributed equally to this work.

Authors and Affiliations

  1. School of Computer and Information Engineering, Henan University, Zhengzhou, 450046, Henan, China

    Chun Liu, Longwei Yang, Zheng Li & Wei Yang

  2. Henan Key Laboratory of Big Data Analysis and Processing, Henan University, Zhengzhou, 450046, Henan, China

    Chun Liu, Zheng Li & Wei Yang

  3. Henan Engineering Laboratory of Spatial Information Processing, Henan University, Zhengzhou, 450046, Henan, China

    Chun Liu, Zheng Li & Wei Yang

  4. Henan Industrial Technology Academy of Spatio-Temporal Big Data, Henan University, Zhengzhou, 450046, Henan, China

    Chun Liu, Zheng Li, Wei Yang, Zhigang Han & Jianzhong Guo

  5. College of Geography and Environmental Science, Henan University, Zhengzhou, 450046, Henan, China

    Zhigang Han & Jianzhong Guo

  6. Henan DeFan High-tech Software Co., Ltd, Zhengzhou, 450046, Henan, China

    Junyong Yu

Authors
  1. Chun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Longwei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhigang Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jianzhong Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Junyong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Chun Liu drafted the manuscript. Longwei Yang designed expriments. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Jianzhong Guo.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Ethics approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, C., Yang, L., Li, Z. et al. Multi-level relation learning for cross-domain few-shot hyperspectral image classification. Appl Intell 54, 4392–4410 (2024). https://doi.org/10.1007/s10489-024-05384-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05384-3

Keywords

Search

Navigation