Skip to main content
SpringerLink
Log in

Hierarchical capsule network for hyperspectral image classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Hyperspectral imaging is a highly advanced and sophisticated method for capturing images in hundreds of narrow, contiguous spectral bands. However, processing and analyzing such large amounts of data are challenging. Deep learning algorithms, especially those based on convolutional neural networks (CNNs), effectively extract rich feature representations from complex datasets, such as hyperspectral images (HSIs). These representations capture high-level patterns and characteristics that can facilitate accurate classification. The success of this approach has led to its widespread use in various remote sensing applications. However, the standard CNN inputs and outputs are scalars, ignoring the relative position relationships between features. In this paper, we propose a hierarchical capsule network for HSI classification. This network incorporates a multi-level convolutional structure for feature extraction and fusion. It utilizes convolutional feature maps of various depths to generate initial capsules, followed by vector computation using capsule neurons and a weight matrix to encode spatial location relationships among features. Furthermore, the shallow convolution of the hierarchical capsule network is pre-trained based on transfer learning to further improve the performance of HSI classification. According to experimental results, the proposed method for hyperspectral image classification has been found to outperform other state-of-the-art deep learning models on four benchmark datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Data availability

The datasets used during the work are available in the website “https://www.ehu.eus/ccwintco/index.php?title=Hyperspectral_Remote_Sensing_Scenes.”

References

  1. Ahmad M, Shabbir S, Roy SK, Hong D, Wu X, Yao J, Khan AM, Mazzara M, Distefano S, Chanussot J (2021) Hyperspectral image classification-traditional to deep models: a survey for future prospects. IEEE J Select Top Appl Earth Observ Remote Sens 15:968–999. https://doi.org/10.1109/igarss.2016.7730324

    Article  Google Scholar 

  2. Bhosle K, Musande V (2019) Evaluation of deep learning CNN model for land use land cover classification and crop identification using hyperspectral remote sensing images. J Indian Soc Remote Sens 47(11):1949–1958. https://doi.org/10.1007/s12524-019-01041-2

    Article  Google Scholar 

  3. Lu G, Fei B (2014) Medical hyperspectral imaging: a review. J Biomed Opt 19(1):010901. https://doi.org/10.1117/1.jbo.19.1.010901

    Article  Google Scholar 

  4. Tiwari K, Arora MK, Singh D (2011) An assessment of independent component analysis for detection of military targets from hyperspectral images. Int J Appl Earth Obs Geoinf 13(5):730–740. https://doi.org/10.1016/j.jag.2011.03.007

    Article  Google Scholar 

  5. Kuching S (2007) The performance of maximum likelihood, spectral angle mapper, neural network and decision tree classifiers in hyperspectral image analysis. J Comput Sci 3(6):419–423. https://doi.org/10.3844/jcssp.2007.419.423

    Article  Google Scholar 

  6. Gao L, Li J, Khodadadzadeh M, Plaza A, Zhang B, He Z, Yan H (2014) Subspace-based support vector machines for hyperspectral image classification. IEEE Geosci Remote Sens Lett 12(2):349–353. https://doi.org/10.1109/lgrs.2014.2341044

    Article  Google Scholar 

  7. Samaniego L, Bárdossy A, Schulz K (2008) Supervised classification of remotely sensed imagery using a modified \(k\)-nn technique. IEEE Trans Geosci Remote Sens 46(7):2112–2125. https://doi.org/10.1109/tgrs.2008.916629

    Article  Google Scholar 

  8. Gislason PO, Benediktsson JA, Sveinsson JR (2006) Random forests for land cover classification. Pattern Recogn Lett 27(4):294–300. https://doi.org/10.1016/j.patrec.2005.08.011

    Article  Google Scholar 

  9. Song W, Zhang L, Tian Y, Fong S, Liu J, Gozho A (2020) Cnn-based 3d object classification using hough space of lidar point clouds. HCIS 10:1–14. https://doi.org/10.1186/s13673-020-00228-8

    Article  Google Scholar 

  10. Chen J, Li K, Bilal K, Li K, Philip SY et al (2018) A bi-layered parallel training architecture for large-scale convolutional neural networks. IEEE Trans Parallel Distrib Syst 30(5):965–976. https://doi.org/10.1109/tpds.2018.2877359

    Article  Google Scholar 

  11. Cao D, Chen Z, Gao L (2020) An improved object detection algorithm based on multi-scaled and deformable convolutional neural networks. HCIS 10(1):1–22. https://doi.org/10.1186/s13673-020-00219-9

    Article  Google Scholar 

  12. Chen C, Li K, Teo SG, Zou X, Li K, Zeng Z (2020) Citywide traffic flow prediction based on multiple gated spatio-temporal convolutional neural networks. ACM Trans Knowl Discov Data (TKDD) 14(4):1–23. https://doi.org/10.1145/3385414

    Article  Google Scholar 

  13. Zhou S, Chen L, Sugumaran V (2020) Hidden two-stream collaborative learning network for action recognition. CMC-Comput Mater Contin 63(3):1545–1561. https://doi.org/10.32604/cmc.2020.09867

    Article  Google Scholar 

  14. Li S, Song W, Fang L, Chen Y, Ghamisi P, Benediktsson JA (2019) Deep learning for hyperspectral image classification: an overview. IEEE Trans Geosci Remote Sens 57(9):6690–6709. https://doi.org/10.1109/tgrs.2019.2907932

    Article  Google Scholar 

  15. Chen Y, Lin Z, Zhao X, Wang G, Gu Y (2014) Deep learning-based classification of hyperspectral data. IEEE J Select Top Appl Earth Observ Remote Sens 7(6):2094–2107. https://doi.org/10.1109/jstars.2014.2329330

    Article  Google Scholar 

  16. Chen Y, Zhao X, Jia X (2015) Spectral-spatial classification of hyperspectral data based on deep belief network. IEEE J Select Top Appl Earth Observ Remote Sens 8(6):2381–2392. https://doi.org/10.1109/jstars.2015.2388577

    Article  Google Scholar 

  17. Zhan Y, Tian H, Liu W, Yang Z, Wu K, Wang G, Chen P, Yu X (2017) A new hyperspectral band selection approach based on convolutional neural network. In: 2017 IEEE international geoscience and remote sensing symposium (IGARSS), pp 3660–3663. IEEE https://doi.org/10.1109/igarss.2017.8127792

  18. Paoletti ME, Haut JM, Plaza J, Plaza A (2020) Scalable recurrent neural network for hyperspectral image classification. J Supercomput 76(11):8866–8882. https://doi.org/10.1007/s11227-020-03187-0

    Article  Google Scholar 

  19. Du L, Hong L, Yang J, Xu T, Zhang J, Ai L, Zhou W (2021) Stellar spectral classification based on capsule network. Acta Astronom Sin 62(1):8. https://doi.org/10.1016/j.chinastron.2021.08.004

    Article  Google Scholar 

  20. Jia S, Jiang S, Lin Z, Li N, Xu M, Yu S (2021) A survey: deep learning for hyperspectral image classification with few labeled samples. Neurocomputing 448:179–204. https://doi.org/10.1016/j.neucom.2021.03.035

    Article  Google Scholar 

  21. Paoletti ME, Haut JM, Plaza J, Plaza A (2018) A new deep convolutional neural network for fast hyperspectral image classification. ISPRS J Photogramm Remote Sens 145:120–147. https://doi.org/10.1016/j.isprsjprs.2017.11.021

    Article  Google Scholar 

  22. Zhao W, Du S (2016) Learning multiscale and deep representations for classifying remotely sensed imagery. ISPRS J Photogramm Remote Sens 113:155–165. https://doi.org/10.1016/j.isprsjprs.2016.01.004

    Article  Google Scholar 

  23. Li Y, Zhang H, Shen Q (2017) Spectral-spatial classification of hyperspectral imagery with 3d convolutional neural network. Remote Sens 9(1):67. https://doi.org/10.3390/rs9010067

    Article  Google Scholar 

  24. Roy SK, Krishna G, Dubey SR, Chaudhuri BB (2020) Hybridsn: exploring 3-d-2-d cnn feature hierarchy for hyperspectral image classification. IEEE Geosci Remote Sens Lett. https://doi.org/10.1109/lgrs.2019.2918719

    Article  Google Scholar 

  25. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778. https://doi.org/10.1109/cvpr.2016.90

  26. Zhong Z, Li J, Luo Z, Chapman MA (2018) Spectral-spatial residual network for hyperspectral image classification: a 3-d deep learning framework. IEEE Trans Geosci Remote Sens. https://doi.org/10.1109/tgrs.2017.2755542

    Article  Google Scholar 

  27. Li H-C, Wang W-Y, Pan L, Li W, Du Q, Tao R (2020) Robust capsule network based on maximum correntropy criterion for hyperspectral image classification. IEEE J Select Top Appl Earth Observ Remote Sens 13:738–751. https://doi.org/10.1109/jstars.2020.2968930

    Article  Google Scholar 

  28. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. ArXiv arXiv:1710.09829

  29. Li J, Zhao Q, Li N, Ma L, Xia X, Zhang X, Ding N, Li N (2021) A survey on capsule networks: evolution, application, and future development. In: 2021 international conference on high performance big data and intelligent systems (HPBD &IS), pp 177–185. IEEE https://doi.org/10.1109/hpbdis53214.2021.9658349

  30. McIntosh B, Duarte K, Rawat YS, Shah M (2018) Multi-modal capsule routing for actor and action video segmentation conditioned on natural language queries. arXiv preprint arXiv:1812.00303

  31. Nguyen HH, Yamagishi J, Echizen I (2019) Capsule-forensics: using capsule networks to detect forged images and videos. In: ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2307–2311. IEEE https://doi.org/10.1109/icassp.2019.8682602

  32. Deng F, Pu S, Chen X, Shi Y, Yuan T, Pu S (2018) Hyperspectral image classification with capsule network using limited training samples. Sensors 18(9):3153. https://doi.org/10.3390/s18093153

    Article  Google Scholar 

  33. Zhang H, Meng L, Wei X, Tang X, Tang X, Wang X, Jin B, Yao W (2019) 1d-convolutional capsule network for hyperspectral image classification. arXiv preprint arXiv:1903.09834

  34. Paoletti ME, Haut JM, Fernandez-Beltran R, Plaza J, Plaza A, Li J, Pla F (2018) Capsule networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 57(4):2145–2160. https://doi.org/10.1109/tgrs.2018.2871782

    Article  Google Scholar 

  35. Yin J, Li S, Zhu H, Luo X (2019) Hyperspectral image classification using capsnet with well-initialized shallow layers. IEEE Geosci Remote Sens Lett 16(7):1095–1099. https://doi.org/10.1109/lgrs.2019.2891076

    Article  Google Scholar 

  36. Lei R, Zhang C, Du S, Wang C, Zhang X, Zheng H, Huang J, Yu M (2021) A non-local capsule neural network for hyperspectral remote sensing image classification. Remote Sens Lett 12(1):40–49. https://doi.org/10.1080/2150704x.2020.1864052

    Article  Google Scholar 

  37. Wang X, Tan K, Du Q, Chen Y, Du P (2019) Caps-triplegan: Gan-assisted capsnet for hyperspectral image classification. IEEE Trans Geosci Remote Sens 57(9):7232–7245. https://doi.org/10.1109/tgrs.2019.2912468

    Article  Google Scholar 

  38. Jiang X, Liu W, Zhang Y, Liu J, Li S, Lin J (2020) Spectral-spatial hyperspectral image classification using dual-channel capsule networks. IEEE Geosci Remote Sens Lett 18(6):1094–1098. https://doi.org/10.1109/lgrs.2020.2991405

    Article  Google Scholar 

  39. Wang J, Guo S, Huang R, Li L, Zhang X, Jiao L (2021) Dual-channel capsule generation adversarial network for hyperspectral image classification. IEEE Trans Geosci Remote Sens 60:1–16. https://doi.org/10.1186/s13673-020-00228-8

    Article  Google Scholar 

  40. Mei Z, Yin Z, Kong X, Wang L, Ren H (2022) Cascade residual capsule network for hyperspectral image classification. IEEE J Select Top Appl Earth Observ Remote Sens 15:3089–3106. https://doi.org/10.1109/jstars.2022.3166972

    Article  Google Scholar 

  41. Ryu J, Jang Y (2022) Hyperspectral image classification using 3d capsule-net based architecture. Appl Sci 12(21):11299. https://doi.org/10.3390/app122111299

    Article  Google Scholar 

  42. Liu J, Jiang X, Liu W (2022) Classifying hyperspectral images with capsule network and active learning. IGARSS 2022-2022 IEEE international geoscience and remote sensing symposium, pp 3624–3627. https://doi.org/10.1109/igarss46834.2022.9883239

  43. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. JMLR.org

  44. Xu Y, Zhang L, Du B, Zhang F (2018) Spectral-spatial unified networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens. https://doi.org/10.1109/tgrs.2018.2827407

    Article  Google Scholar 

  45. Yin J, Li S, Zhu H, Luo X (2019) Hyperspectral image classification using capsnet with well-initialized shallow layers. IEEE Geosci Remote Sens Lett 7:1–5. https://doi.org/10.1109/LGRS.2019.2891076

    Article  Google Scholar 

  46. Zhu K, Chen Y, Ghamisi P, Jia X, Benediktsson JA (2019) Deep convolutional capsule network for hyperspectral image spectral and spectral-spatial classification. Remote Sens. https://doi.org/10.3390/rs11030223

    Article  Google Scholar 

Download references

Funding

This paper was supported by the Open Fund of Hubei Key Laboratory of Intelligent Geo-Information Processing (Grant No. ZRIGIP-201801).

Author information

Authors and Affiliations

  1. School of Computer Science, China University of Geosciences, Wuhan, 430078, China

    Meilin Shi, Ruoxiang Wang & Jiansi Ren

  2. Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan, 430078, China

    Jiansi Ren

Authors
  1. Meilin Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruoxiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiansi Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiansi Ren.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, M., Wang, R. & Ren, J. Hierarchical capsule network for hyperspectral image classification. Neural Comput & Applic 35, 18417–18443 (2023). https://doi.org/10.1007/s00521-023-08664-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08664-0

Keywords

Search

Navigation