A Deep Network Based on Multiscale Spectral-Spatial Fusion for Hyperspectral Classification

  • Zhaokui LiEmail author
  • Lin Huang
  • Deyuan Zhang
  • Cuiwei LiuEmail author
  • Yan Wang
  • Xiangbin Shi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11062)


In this paper, we propose a deep network based on multiscale spectral-spatial fusion (MSS-Net) for Hyperspectral Image (HSI) classification. For the purpose of extracting better joint spectral-spatial features, the proposed network adopts multiscale spectral-spatial fusion method because different scale regions contain different spatial structure, texture features and more abundant neighborhood correlation which are helpful for classification. For every scale of input, we take the 3-D cubes from the raw data to the spatial and spectral learning module respectively. These two learning modules can extract the features with more abundant and original spectral-spatial correlation from the 3-D raw input data and then these features are combined as fusion spectral-spatial features. And we can get multiscale fusion spectral-spatial features which are fed to the two consequent residual learning block. Every residual block contains two 3-D convolutional layers and it can make full use of fusion features to learn more discriminative and high-level features. Furthermore, it also can help the network maintain higher accuracy when the network is deeper. After residual learning, multiscale fusion spectral-spatial features are concatenated and sent to fully convolutional layer for classification. The validation of our method is proved on three HSI data sets and the experimental results show that our method outperforms the other state-of-the-art methods.


Hyperspectral Image classification Multiscale Spectral-spatial fusion Deep network 



This work was supported in part by the Natural Science Foundation of China (NSFC) under Grant No. 61602320, and the Natural Science Foundation of Liaoning under Grant Nos. 201601180 and 201601172.


  1. 1.
    Jia, X., Kuo, B.-C., Crawford, M.M.: Feature mining for hyperspectral image classification. Proc. IEEE 101(3), 676–697 (2013)CrossRefGoogle Scholar
  2. 2.
    Fang, X., et al.: Locality and similarity preserving embedding for feature selection. Neurocomputing 128, 304–315 (2014)CrossRefGoogle Scholar
  3. 3.
    Li, W., Wu, G., Zhang, F., Du, Q.: Hyperspectral image classification using deep pixel-pair features. IEEE Trans. Geosci. Remote Sens. 55(2), 844–853 (2017)CrossRefGoogle Scholar
  4. 4.
    Chen, Y., Jiang, H., Li, C., Jia, X., Ghamisi, P.: Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans. Geosci. Remote Sens. 54(10), 6232–6251 (2016)CrossRefGoogle Scholar
  5. 5.
    Zhao, W., Du, S.: Spectral–spatial feature extraction for hyperspectral image classification: a dimension reduction and deep learning approach. IEEE Trans. Geosci. Remote Sens. 54(8), 4544–4554 (2016)CrossRefGoogle Scholar
  6. 6.
    Tao, C., Pan, H., Li, Y., Zou, Z.: Unsupervised spectral–spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification. IEEE Geosci. Remote Sens. Lett. 12(12), 2438–2442 (2015)CrossRefGoogle Scholar
  7. 7.
    Szegedy, C., et al.: Going deeper with convolutions. In: CVPR (2015)Google Scholar
  8. 8.
    Li, Y., Zhang, H., Shen, Q.: Spectral–spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sens. 9(1), 67 (2017)CrossRefGoogle Scholar
  9. 9.
    Yang, J., Zhao, Y.Q., Chan, C.W.: Learning and transferring deep joint spectral-spatial features for hyperspectral classification. IEEE Trans. Geosci. Remote Sens. 55(8), 1–14 (2017)CrossRefGoogle Scholar
  10. 10.
    Zhong, Z, Li, J., Luo, Z., et al.: Spectral-spatial residual network for hyperspectral image classification: a 3-d deep learning framework. IEEE Trans. Geosci. Remote Sens., 1–12 (2017)Google Scholar
  11. 11.
    He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition, pp. 770–778 (2015)Google Scholar
  12. 12.
    Waske, B., van der Linden, S., Benediktsson, J.A., Rabe, A., Hostert, P.: Sensitivity of support vector machines to random feature selection in classification of hyperspectral data. IEEE Trans. Geosci. Remote Sens. 48(7), 2880–2889 (2010)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer ScienceShenyang Aerospace UniversityShenyangPeople’s Republic of China

Personalised recommendations