Skip to main content
SpringerLink
Log in

Hybrid 3D/2D Complete Inception Module and Convolutional Neural Network for Hyperspectral Remote Sensing Image Classification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Classification in hyperspectral remote sensing images (HRSIs) is a challenging process in image analysis and one of the most popular topics. In recent years, many methods have been proposed to solve the HRSIs classification problem. Compared to traditional machine learning methods, deep learning, especially convolutional neural networks (CNNs), is commonly used in the classification of HRSIs. Deep learning-based methods based on CNNs show remarkable performance in HRSIs classification and greatly support the development of classification technology. In this study, a method in which the Hybrid 3D/2D Complete Inception module and the Hybrid 3D/2D CNN method are used together has been proposed to solve the HRSIs classification problem. In the proposed method, multi-level feature extraction is performed by using multiple convolution layers with the Inception module. This improves the performance of the network. Conventional CNN-based methods use 2D CNN for feature extraction. However, only spatial features are extracted with 2D CNN. 3D CNN is used to extract spatial-spectral features. However, 3D CNN is computationally complex. Therefore, in the proposed method, a hybrid approach is used by first using 3D CNN and then 2D CNN. This reduces computational complexity and extracts more spatial features. In addition, PCA is used as a preprocessing step for optimum spectral band extraction in the proposed method. The proposed method has been tested using Indian pines, Salinas, University of Pavia, HyRANK-Loukia and Houston datasets, which are frequently used in studies for HRSIs classification. The overall accuracy of the proposed method in these five datasets are 99.83%, 100%, 100%, 90.47% and 98.93%, respectively. These results reveal that the proposed method provides higher classification performance compared to state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Chen C, Jiang F, Yang C et al (2018) Hyperspectral classification based on spectral–spatial convolutional neural networks. Eng Appl Artif Intell 68:165–171. https://doi.org/10.1016/j.engappai.2017.10.015

    Article  Google Scholar 

  2. Roy SK, Chatterjee S, Bhattacharyya S et al (2020) Lightweight spectral-spatial squeeze-and- excitation residual bag-of-features learning for hyperspectral classification. IEEE Trans Geosci Remote Sens 58:5277–5290. https://doi.org/10.1109/TGRS.2019.2961681

    Article  Google Scholar 

  3. Fırat H, Hanbay D (2021) 4CF-Net: Hiperspektral uzaktan algılama görüntülerinin spektral uzamsal sınıflandırılması için yeni 3B evrişimli sinir ağı. Gazi Üniversitesi Mühendislik-Mimarlık Fakültesi Derg 1:439–453. https://doi.org/10.17341/gazimmfd.901291

  4. Firat H, Hanbay D (2021) 3B ESA Tabanlı ResNet50 Kullanılarak Hiperspektral Görüntülerin Sınıflandırılması Classification of Hyperspectral Images Using 3D CNN Based ResNet50. In: 2021 29th Signal Process Commun Appl Conf 6–9. https://doi.org/10.1109/SIU53274.2021.9477899

  5. Ahmad M, Mazzara M, Distefano S (2021) Regularized cnn feature hierarchy for hyperspectral image classification. Remote Sens 13:1–11. https://doi.org/10.3390/rs13122275

    Article  Google Scholar 

  6. Firat H, Uçan M, Hanbay D (2021) Classification of hyperspectral remote sensing ımages using hybrid 3D-2D CNN architecture. J Comput Sci IDAP-2021:132–140

  7. Firat H, Uçan M, Hanbay D (2021) Hyperspectral ımage classification using MiniVGGNet. J Comput Sci IDAP-2021:295–303

  8. Ibarrola-Ulzurrun E, Drumetz L, Marcello J et al (2019) Hyperspectral classification through unmixing abundance maps addressing spectral variability. IEEE Trans Geosci Remote Sens 57:4775–4788. https://doi.org/10.1109/TGRS.2019.2892903

    Article  Google Scholar 

  9. Mohan A, Meenakshi Sundaram V (2020) V3O2: hybrid deep learning model for hyperspectral image classification using vanilla-3D and octave-2D convolution. J Real-Time Image Process. https://doi.org/10.1007/s11554-020-00966-z

    Article  Google Scholar 

  10. Gowtham B, Kumar IA, Reddy TS et al (2021) Hyperspectral image analysis using principal component analysis and siamese network. Turkish J Comput Math Educ 12:1191–1198

    Google Scholar 

  11. Ahmad M, Khan A, Khan AM et al (2019) Spatial prior fuzziness pool-based interactive classification of hyperspectral images. Remote Sens 11:1–19. https://doi.org/10.3390/rs11091136

    Article  Google Scholar 

  12. Li J, Bioucas-Dias JM, Plaza A (2010) Semisupervised hyperspectral image segmentation using multinomial logistic regression with active learning. IEEE Trans Geosci Remote Sens 48:4085–4098. https://doi.org/10.1109/TGRS.2010.2060550

    Article  Google Scholar 

  13. Wang Y, Yu W, Fang Z (2020) Multiple Kernel-based SVM classification of hyperspectral images by combining spectral, spatial, and semantic information. Remote Sens. https://doi.org/10.3390/RS12010120

    Article  Google Scholar 

  14. Ham JS, Chen Y, Crawford MM, Ghosh J (2005) Investigation of the random forest framework for classification of hyperspectral data. IEEE Trans Geosci Remote Sens 43:492–501. https://doi.org/10.1109/TGRS.2004.842481

    Article  Google Scholar 

  15. Li Y, Zhang H, Shen Q (2017) Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sens. https://doi.org/10.3390/rs9010067

    Article  Google Scholar 

  16. Fang L, Li S, Duan W et al (2015) Classification of hyperspectral images by exploiting spectral-spatial information of superpixel via multiple kernels. IEEE Trans Geosci Remote Sens 53:6663–6674. https://doi.org/10.1109/TGRS.2015.2445767

    Article  Google Scholar 

  17. Fang L, Li S, Kang X, Benediktsson JA (2014) Spectral-spatial hyperspectral image classification via multiscale adaptive sparse representation. IEEE Trans Geosci Remote Sens 52:7738–7749. https://doi.org/10.1109/TGRS.2014.2318058

    Article  Google Scholar 

  18. Camps-Valls G, Bruzzone L (2005) Kernel-based methods for hyperspectral image classification. IEEE Trans Geosci Remote Sens 43:1351–1362. https://doi.org/10.1109/TGRS.2005.846154

    Article  Google Scholar 

  19. Palmason JA, Benediktsson JA, Sveinsson JR (2005) Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Trans Geosci Remote Sens 43:480–491. https://doi.org/10.1109/TGRS.2004.842478

    Article  Google Scholar 

  20. Mohan A, Venkatesan M (2020) HybridCNN based hyperspectral image classification using multiscale spatiospectral features. Infrared Phys Technol. https://doi.org/10.1016/j.infrared.2020.103326

    Article  Google Scholar 

  21. Özdemir MF, Hanbay D (2021) Comparison of optimization algorithms for multi-object tracking FairMOT algorithm. J Comput Sci IDAP-2021:147–153

  22. Sel İ, Hanbay D (2021) Creating a parallel corpora for Turkish-English academic translations. J Comput Sci IDAP-2021:335–340

  23. Uzen H, Turkoglu M, Hanbay D (2021) Texture defect classification with multiple pooling and filter ensemble based on deep neural network. Expert Syst Appl 175:114838. https://doi.org/10.1016/j.eswa.2021.114838

    Article  Google Scholar 

  24. Üzen H, Fırat H, Karcİ A, Hanbay D (2019) Automatic thresholding method developed with entropy for fabric defect detection. In: 2019 ınternational artificial ıntelligence and data processing symposium (IDAP), pp 14–17

  25. Zhao C, Wan X, Zhao G et al (2017) Spectral-spatial classification of hyperspectral imagery based on stacked sparse autoencoder and random forest. Eur J Remote Sens 50:47–63. https://doi.org/10.1080/22797254.2017.1274566

    Article  Google Scholar 

  26. Data H, Chen Y, Lin Z et al (2014) Deep learning-based classification of hyperspectral data. IEEE J Sel Top Appl Earth Obs Remote Sens 7:2094–2107. https://doi.org/10.1109/JSTARS.2014.2329330

    Article  Google Scholar 

  27. Mughees A, Tao L (2017) Efficient deep auto-encoder learning for the classification of hyperspectral images. Proc - 2016 Int Conf Virtual Real Vis ICVRV 2016, pp 44–51. https://doi.org/10.1109/ICVRV.2016.16

  28. Zhong P, Gong Z, Li S, Schonlieb CB (2017) Learning to diversify deep belief networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 55:3516–3530. https://doi.org/10.1109/TGRS.2017.2675902

    Article  Google Scholar 

  29. Chen Y, Zhao X, Jia X (2015) Spectral-spatial classification of hyperspectral data based on deep belief network. IEEE J Sel Top Appl Earth Obs Remote Sens 8:2381–2392. https://doi.org/10.1109/JSTARS.2015.2388577

    Article  Google Scholar 

  30. Li J, Xi B, Li Y et al (2018) Hyperspectral classification based on texture feature enhancement and deep belief networks. Remote Sens. https://doi.org/10.3390/rs10030396

    Article  Google Scholar 

  31. Zhang C, Sargent I, Pan X et al (2019) Joint deep learning for land cover and land use classification. Remote Sens Environ 221:173–187. https://doi.org/10.1016/j.rse.2018.11.014

    Article  Google Scholar 

  32. Iyer P, A S, Lal S, (2021) Deep learning ensemble method for classification of satellite hyperspectral images. Remote Sens Appl Soc Environ 23:100580. https://doi.org/10.1016/j.rsase.2021.100580

    Article  Google Scholar 

  33. Roy SK, Krishna G, Dubey SR, Chaudhuri BB (2019) HybridSN: exploring 3D-2D CNN feature hierarchy for hyperspectral ımage classification. arXiv 17:277–281

  34. Ahmad M, Khan AM, Mazzara M et al (2020) A fast and compact 3-D CNN for hyperspectral image classification. IEEE Geosci Remote Sens Lett. https://doi.org/10.1109/LGRS.2020.3043710

    Article  Google Scholar 

  35. Ge Z, Cao G, Li X, Fu P (2020) Hyperspectral image classification method based on 2D–3D CNN and multibranch feature fusion. IEEE J Sel Top Appl Earth Obs Remote Sens 13:5776–5788. https://doi.org/10.1109/JSTARS.2020.3024841

    Article  Google Scholar 

  36. He M, Bo Li HC (2017) Multi-scale 3D deep convolutional neural network for hyperspectral image classification. IEEE Int Conf Image Process 2017:3904–3908

    Google Scholar 

  37. Venugopal N (2020) Automatic semantic segmentation with DeepLab dilated learning network for change detection in remote sensing images. Neural Process Lett 51:2355–2377. https://doi.org/10.1007/s11063-019-10174-x

    Article  Google Scholar 

  38. Mu C, Guo Z, Liu Y (2020) A multi-scale and multi-level spectral-spatial feature fusion network for hyperspectral image classification. Remote Sens. https://doi.org/10.3390/RS12010125

    Article  Google Scholar 

  39. Zhong Z, Li J, Luo Z, Chapman M (2018) Spectral-spatial residual network for hyperspectral image classification: a 3-D deep learning framework. IEEE Trans Geosci Remote Sens 56:847–858. https://doi.org/10.1109/TGRS.2017.2755542

    Article  Google Scholar 

  40. Roy SK, Dubey SR, Chatterjee S, Chaudhuri BB (2020) FuSENet: fused squeeze-and-excitation network for spectral-spatial hyperspectral image classification. IET Image Process 14:1653–1661. https://doi.org/10.1049/iet-ipr.2019.1462

    Article  Google Scholar 

  41. Roy SK, Manna S, Song T, Bruzzone L (2020) Attention-based adaptive spectral-spatial kernel resnet for hyperspectral image classification. IEEE Trans Geosci Remote Sens 59:7831–7843. https://doi.org/10.1109/TGRS.2020.3043267

    Article  Google Scholar 

  42. Meng Z, Li L, Tang X et al (2019) Multipath residual network for spectral-spatial hyperspectral image classification. Remote Sens 11:1–19. https://doi.org/10.3390/rs11161896

    Article  Google Scholar 

  43. Song W, Li S, Fang L (2018) Hyperspectral image classification with deep feature fusion network. IEEE Trans Geosci Remote Sens 99:3173–3184. https://doi.org/10.1109/IGARSS.2019.8898520

    Article  Google Scholar 

  44. Liu Y, Gao L, Xiao C et al (2020) Hyperspectral image classification based on a shuffled group convolutional neural network with transfer learning. Remote Sens 12:1–18. https://doi.org/10.3390/rs12111780

    Article  Google Scholar 

  45. Roy S, Mondal R, Paoletti ME et al (2021) Morphological convolutional neural networks for hyperspectral image classification. IEEE J Sel Top Appl Earth Obs Remote Sens 14:8689–8702. https://doi.org/10.1109/JSTARS.2021.3088228

    Article  Google Scholar 

  46. Ghosh P, Roy SK, Koirala B et al (2022) Deep hyperspectral unmixing using transformer network. https://doi.org/10.48550/arXiv.2203.17076

  47. Roy SK, Deria A, Hong D et al (2022) Multimodal fusion transformer for remote sensing ımage classification. https://doi.org/10.48550/arXiv.2203.16952

  48. Ghaderizadeh S, Abbasi-Moghadam D, Sharifi A et al (2021) Hyperspectral image classification using a hybrid 3D–2D convolutional neural networks. IEEE J Sel Top Appl Earth Obs Remote Sens 14:7570–7588. https://doi.org/10.1109/JSTARS.2021.3099118

    Article  Google Scholar 

  49. Gao H, Yang Y, Li C et al (2021) Multiscale residual network with mixed depthwise convolution for hyperspectral image classification. IEEE Trans Geosci Remote Sens 59:3396–3408. https://doi.org/10.1109/TGRS.2020.3008286

    Article  Google Scholar 

  50. Ahmad M, Shabbir S, Raza RA et al (2021) Hyperspectral ımage classification: artifacts of dimension reduction on hybrid CNN. Optik Int J Light Electron Opt 246:167757. https://doi.org/10.1016/j.ijleo.2021.167757

    Article  Google Scholar 

  51. Jiang L, Zhu B, Tao Y (2010) Hyperspectral image classification methods. Hyperspectral Imaging Food Qual Anal Control. https://doi.org/10.1016/B978-0-12-374753-2.10003-6

    Article  Google Scholar 

  52. Firat H, Asker ME, Hanbay D (2022) Classification of hyperspectral remote sensing images using different dimension reduction methods with 3D/2D CNN. Remote Sens Appl Soc Environ. https://doi.org/10.1016/j.rsase.2022.100694

    Article  Google Scholar 

  53. Patel M, Parmar M, Doshi S, et al (2019) Novel ınception-GAN for whispered-to-normal speech conversion. In: 10th ISCA Work Speech Synth (SSW 10), pp 87–92. https://doi.org/10.21437/ssw.2019-16

  54. Makantasis K, Karantzalos K, Doulamis A, Doulamis N (2015) Deep supervised learning for hyperspectral data classification through convolutional neural networks. In: Int Geosci Remote Sens Symp 2015-Novem:4959–4962. https://doi.org/10.1109/IGARSS.2015.7326945

  55. Ben Hamida A, Benoit A, Lambert P, Ben Amar C (2018) 3-D deep learning approach for remote sensing image classification. IEEE Trans Geosci Remote Sens 56:4420–4434. https://doi.org/10.1109/TGRS.2018.2818945

    Article  Google Scholar 

  56. 2013 ieee grss data fusion contest. http://www.grss-ieee.org/community/technical-committees/data-fusion/2013-ieee-grss-data-fusion-contest.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Technology, Vocational School of Technical Sciences, Dicle University, Diyarbakır, Turkey

    Hüseyin Fırat

  2. Department of Electricity and Energy, Vocational School of Technical Sciences, Dicle University, Diyarbakır, Turkey

    Mehmet Emin Asker

  3. Department of Electronics and Automation, Vocational School of Technical Sciences, Fırat University, Elazig, Turkey

    Mehmet İlyas Bayındır

  4. Department of Computer Engineering, Engineering Faculty, İnonu University, Malatya, Turkey

    Davut Hanbay

Authors
  1. Hüseyin Fırat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mehmet Emin Asker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mehmet İlyas Bayındır
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Davut Hanbay
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hüseyin Fırat.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fırat, H., Asker, M.E., Bayındır, M.İ. et al. Hybrid 3D/2D Complete Inception Module and Convolutional Neural Network for Hyperspectral Remote Sensing Image Classification. Neural Process Lett 55, 1087–1130 (2023). https://doi.org/10.1007/s11063-022-10929-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10929-z

Keywords

Search

Navigation