Skip to main content
SpringerLink
Log in

Hyperspectral remote sensing image classification based on dense residual three-dimensional convolutional neural network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Data-driven deep learning techniques set the current state of the art in image classification for hyperspectral remote sensing images. The lack of labeled training data and high dimensionality of hyperspectral images, results in these techniques being far from satisfactory with respect to accuracy and efficiency. To address the deficiencies of the existing approaches, we proposed a novel neural network technique, namely, dense residual three-dimensional convolutional neural network (DR-3D-CNN). Tailored for hyperspectral images, this network used 3D convolution instead of the conventional 2D convolution for more effective spectral feature extraction. It also employed dense residual connections to alleviate the problem of gradient dispersion. After the initial classification by the network, the proposed technique further refined the result using multi-label conditional random field optimization. Experimental results on various hyperspectral image datasets showed that the proposed model outperforms existing deep learning techniques with respect to accuracy by a large margin while requiring less training time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Bengio Y (2009) Learning deep architectures for AI foundations and trends[J]. Mach Learn 2(1):1–55

    MathSciNet  MATH  Google Scholar 

  2. Bengio Y, Ducharme R, Vincent P (2001) A neural probabilistic language model. Advances in Neural Information Processing Systems(NIPS), 933–938

  3. Boykov Y, Kolmogorov V (2004) An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision[J]. IEEE Trans Pattern Anal Mach Intell 26(9):1124–1137

    MATH  Google Scholar 

  4. Camps-Valls G, Bruzzone L (2005) Kernel-based methods for hyperspectral image classification[J]. IEEE Trans Geosci Remote Sens 43(6):1351–1362

    Google Scholar 

  5. Chang C-I (2007) Hyperspectral data exploitation: theory and applications. John Wiley & Sons, Hoboken

    Google Scholar 

  6. Chen L, Yang M (2017). Semi-supervised dictionary learning with label propagation for image classification[C]. CVM

  7. Chen Y, Nasrabadi NM, Tran TD (2013) Hyperspectral image classification via kernel sparse representation[J]. IEEE Trans Geosci Remote Sens 51(1):217–231

    Google Scholar 

  8. Chen Y, Lin Z, Zhao X, Wang G, Gu Y (2014) Deep learning-based classification of Hyperspectral data[J]. IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing 7:2094–2107

    Google Scholar 

  9. Chen Y, Jiang H, Li C, Jia X, Ghamisi P (2016) Deep feature extraction and classification of Hyperspectral images based on convolutional neural networks[J]. IEEE Trans Geosci Remote Sens 54:6232–6251

    Google Scholar 

  10. Cheng-Hsuan L, Bor-Chen K, Chin-Teng L, Chih-Sheng H (2012) A spatial-contextual support vector machine for remotely sensed image classification[J]. IEEE Trans Geosci Remote Sens 50(3):784–799

    Google Scholar 

  11. Dong Y, Zhang Z, Hong W-C (2018) A hybrid seasonal mechanism with a chaotic cuckoo search algorithm with a support vector regression model for electric load forecasting. Energies 11(4):1009

    Google Scholar 

  12. Fauvel M, Chanussot J, Benediktsson JA (2011) A spatial-spectral kernel based approach for the classification of remote sensing images[J]. Pattern Recogn 45(1):281–392

    Google Scholar 

  13. Fulkerson B, Vedaldi A, Soatto S (2009) Class segmentation and object localization with superpixel neighborhoods [J]. Proc IEEE 20(2):670–677

    Google Scholar 

  14. Geiss C, Thoma M, Taubenbock H (2018) Cost-sensitive multitask active learning for characterization of urban environments with remote sensing[J]. IEEE Geoscience & Remote Sensing Letters 99:1–5

    Google Scholar 

  15. Ghamisi P, Benediktsson JA, Sveinsson JR (2014) Automatic spectral-spatial classification framework based on attribute profiles and supervised feature extraction[J]. IEEE Trans Geosci Remote Sens 52(9):5771–5782

    Google Scholar 

  16. Ghamisi P, Benediktsson JA, Cavallaro G, Plaza A (2014) Automatic framework for spectral-spatial classification based on supervised feature extraction and morphological attribute profiles[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 7(6):2147–2160

    Google Scholar 

  17. Glorot X, Bengio Y (2010) Understanding the Difficulty of Training Deep Feedforward Neural Networks[J]. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, AISTATS 2010, Sardinia, Italy 5:249–256

    Google Scholar 

  18. Gong C, Tao D, Wei L et al. (2015) Saliency propagation from simple to difficult[C]. CVPR

  19. Gu Y, Wang C, You D, Zhang Y, Wang S, Zhang Y (2012) Representative multiple kernel learning for classification in hyper-spectral imagery[J]. IEEE Trans Geosci Remote Sens 50(7):2852–2865

    Google Scholar 

  20. He K, Zhang X, Ren S et al (2015) Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification[J]. In Proceedings of the 15th IEEE International Conference on Computer Vision, ICCV 2015 12:1026–1034

    Google Scholar 

  21. Heeker C et al (2008) Assessing the influence of reference spectral on synthetic SAM classification results[J]. IEEE Trans Geosci Remote Sens 46(12):4162–4172

    Google Scholar 

  22. Hinton G, Salakhutdinov R (2006) Reducing the dimensionality of data with neural networks[J]. Science 313(5786):504–507

    MathSciNet  MATH  Google Scholar 

  23. Hinton G, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets[J]. Neural Comput 18(7):1527–1554

    MathSciNet  MATH  Google Scholar 

  24. Hong WC, Li MW, Geng J, Zhang Y (2019) Novel chaotic bat algorithm for forecasting complex motion of floating platforms. Appl Math Model 72:425–443

    MathSciNet  MATH  Google Scholar 

  25. Hsieh PF (2009) Impact and realization of increased classes separability on the small sample size problem in the hyperspectral classification[J]. Can J Remote Sens 35(3):248–261

    Google Scholar 

  26. Huang G, Liu Z, Weinberger KQ (2017) Densely Connected Convolutional Networks[J]. In Proceedings of the 2017 IEEE conference on pattern recognition and computer vision(CVPR), College Park, MD, USA 6:1–9

    Google Scholar 

  27. Ioffe S, Szegedy C (2015) Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[J]. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015 7:1–9

    Google Scholar 

  28. Jia X, Richards JA (1994) Effieient maximum likelihood classification for imaging spectrometer datasets[J]. IEEE Trans Geosci Remote Sens 32(2):274–281

    Google Scholar 

  29. Keren F, Zhao Q, Gu IY-H, Yang J (2019) Deepside: A general deep framework for salient object detection[J]. Neurocomputing 356(13):69–82

    Google Scholar 

  30. Kingma D, Ba J (2014) Adam: a method for stochastic optimization[J]. Computer Science

  31. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet Classification with Deep Convolutional Neural Networks[J]. In Proceedings of the 26th Annual Conference on Neural Information Processing Systems 2012 12:1097–1105

    Google Scholar 

  32. Kundra H, Sadawarti H (2015) Hybrid algorithm of cuckoo search and particle swarm optimization for natural terrain feature extraction. Res J Inf Technol 7:58–69

    Google Scholar 

  33. Li J, Bioucas-Dias JM, Plaza A (2013) Spectral-spatial classification of hyperspectral data using loopy belief propagation and active learning[J]. IEEE Trans Geosci Remote Sens 51(2):844–856

    Google Scholar 

  34. Li J, Zhang H, Zhang L (2014) Supervised segmentation of very high resolution images by the use of extended morphological attribute profiles and a sparse transform[J]. IEEE Geosci Remote Sens Lett 11(8):1409–1413

    Google Scholar 

  35. Li Y, Zhang H, Shen Q (2016) Spectral-spatial classification of Hyperspectral imagery with 3D convolutional neural network[J]. Remote Sens 8:438–447

    Google Scholar 

  36. Li Y, Zhang H, Shen Q (2017) Spectral-spatial classification of Hyperspectral imagery with 3D convolutional neural network[J]. Remote Sens 9:67

    Google Scholar 

  37. Makantasis K, Karantzalos K, Doulamis A et al (2015) Deep Supervised Learning for Hyperspectral Data Classification through Convolutional Neural Networks[J]. In Proceedings of the 2015 IEEE International Geoscience and Remote Sensing Symposium 6:4959–4962

    Google Scholar 

  38. Meyer A, Paglieroni D, Astaneh C (2003) K-means reclustering: algorithmic options with quantifiable performance comparisons[J]. Proc SPIE 5001:84–92

    Google Scholar 

  39. Moser G, Serpico SB (2013) Combining support vector machines and Markov random fields in an integrated framework for contextual image classification[J]. IEEE Trans Geosci Remote Sens 51(5):2734–2752

    Google Scholar 

  40. Moser G, Serpico SB, Benediktsson JA (2013) Land-cover mapping by Markov modeling of spatial-contextual information in very-high-resolution remote sensing images[J]. Proc IEEE 101(3):631–651

    Google Scholar 

  41. Murray NJ, Keith DA, Simpson D et al (2018) Remap: an online remote sensing application for land cover classification and monitoring[J]. Methods Ecol Evol 9:1–9

    Google Scholar 

  42. Nie G-Y, Cheng M-M, Liu Y, Liang Z, Fan D-P, Liu Y, Wang Y (2019) Multi-Level Context Ultra-Aggregation for Stereo Matching, CVPR

  43. Pinto N, Doukhan D, Dicarlo J et al (2009) A high-throughout screening approach to discovering good forms of biologically inspired visual representation[J]. PLoS Comput Biol 5(11):e1000579

    Google Scholar 

  44. Song B, Li J, Dalla Mura M et al (2014) Remotely sensed image classification using sparse representations of morphological attribute profiles[J]. IEEE Transactions on Geoscience & Remote Sensing 52(8):5122–5136

    Google Scholar 

  45. Srivastava N, Hinton G, Krizhevsky A et al (2014) Dropout: a simple way to prevent neural networks from Overfitting[J]. J Mach Learn Res 15:1929–1958

    MathSciNet  MATH  Google Scholar 

  46. Willett RM, Duarte MF, Davenport MA, Baraniuk RG (2014) Sparsity and structure in hyperspectral imaging: sensing, reconstruction, and target detection[J]. IEEE Signal Process Mag 31(1):116–126

    Google Scholar 

  47. Zhang Z-C, Hong W-C (2019) Electric load forecasting by complete ensemble empirical model decomposition adaptive noise and support vector regression with quantum-based dragonfly algorithm. Nonlinear Dynamics 98:1107–1136

    Google Scholar 

  48. Zhang L, Tao D, Huang X (2012) On combining multiple features for hyperspectral remote sensing image classification[J]. IEEE Trans Geosci Remote Sens 50(3):879–893

    Google Scholar 

  49. Zhang L, Tao D, Huang X (2013) Tensor discriminative locality alignment for hyperspectral image spectral-spatial feature extraction[J]. IEEE Trans Geosci Remote Sens 51(1):242–256

    Google Scholar 

  50. Zhao W, Du S (2016) Spectral-spatial feature extraction for Hyperspectral image classification: a dimension reduction and deep learning approach[J]. IEEE Trans Geosci Remote Sens 54:4544–4554

    Google Scholar 

  51. Zhao WZ, Du SH (2016) Learning multiscale and deep representations for classifying remotely sensed imagery[J]. ISPPS Journal of Photogrammetry and Remote Sensing 113:155–165

    Google Scholar 

  52. Zhao JX, Liu J, Fan DP et al. (2019) EGNet:edge guidance network for salient object detection[J]

Download references

Acknowledgments

This work was supported by National Natural Science Foundation of China (61705019).

Author information

Authors and Affiliations

  1. Jiangsu Key Laboratory of Meteorological Observation and Information Processing, Nanjing University of Information Science & Technology, Nanjing, 210044, China

    Suting Chen & Meng Jin

  2. Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science & Technology, Nanjing, 210044, China

    Suting Chen & Jie Ding

Authors
  1. Suting Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meng Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Data curation, Suting Chen; Methodology, Meng Jin; Supervision, Suting Chen and Jie Ding; Writing – original draft, Meng Jin; Writing – review & editing, Suting Chen and Meng Jin.

Corresponding author

Correspondence to Suting Chen.

Ethics declarations

Conflict of interest

There is no conflict of interest.

The authors declared that they have no conflicts of interest to this work.

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, S., Jin, M. & Ding, J. Hyperspectral remote sensing image classification based on dense residual three-dimensional convolutional neural network. Multimed Tools Appl 80, 1859–1882 (2021). https://doi.org/10.1007/s11042-020-09480-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09480-7

Keywords

Search

Navigation