Abstract
In recent years, hyperspectral image (HSI) classification methods based on generative adversarial networks (GANs) have been proposed and have made great progress, which can alleviate the dilemma of limited training samples. However, GAN-based HSI classification methods are heavily affected by the problem of imbalanced training data. The discriminator always tries to associate false labels with a few samples, which will reduce the classification accuracy. Another problem is the mode collapse based on the GAN network, which hinders the classification performance of HSI. A combined Transformer and GAN (TransGAN) model for HSI classification is proposed in this paper. First, in order to solve the problem of reduced classification accuracy caused by imbalanced training data, the discriminator is adjusted to a classifier with only one output. Second, the generator is constructed by using the Transformer, and the discriminator is added with a multi-scale pooling module (MSPM) to alleviate the problem of GAN model collapse. Experimental results on two HSI datasets show that the proposed TransGAN achieves better performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Audebert, N., Le Saux, B., Lefèvre, S.: Deep learning for classification of hyperspectral data: A comparative review. IEEE Geosci. Remote Sens. Mag. 7(2), 159–173 (2019)
Bera, S., Shrivastava, V.K.: Analysis of various optimizers on deep convolutional neural network model in the application of hyperspectral remote sensing image classification. Int. J. Remote Sens. 41(7), 2664–2683 (2020)
Chang, C.-I., Wang, S.: Constrained band selection for hyperspectral imagery. IEEE Trans. Geosci. Remote Sens. 44(6), 1575–1585 (2006)
Feng, J., Haipeng, Y., Wang, L., Cao, X., Zhang, X., Jiao, L.: Classification of hyperspectral images based on multiclass spatial-spectral generative adversarial networks. IEEE Trans. Geosci. Remote Sens. 57(8), 5329–5343 (2019)
Goodfellow, I.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, vol. 27 (2014)
Hamida, A.B., Benoit, A., Lambert, P., Amar, C.B.: 3-d deep learning approach for remote sensing image classification. IEEE Trans. Geosci. Remote Sens. 56(8), 4420–4434 (2018)
He, M., Li, B., Chen, H.: Multi-scale 3d deep convolutional neural network for hyperspectral image classification. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3904–3908. IEEE (2017)
Hong, D.: Spectralformer: Rethinking hyperspectral image classification with transformers. IEEE Trans. Geosci. Remote Sens. (2021)
Wei, H., Huang, Y., Wei, L., Zhang, F., Li, H.: Deep convolutional neural networks for hyperspectral image classification. J. Sens. 15(2), 1–12 (2015)
Jiang, J., Ma, J., Chen, C., Wang, Z., Cai, Z., Wang, L.: Superpca: A superpixelwise pca approach for unsupervised feature extraction of hyperspectral imagery. IEEE Trans. Geosci. Remote Sens. 56(8), 4581–4593 (2018)
Jiang, Y., Chang, S., Wang, Z.: Transgan: Two pure transformers can make one strong gan, and that can scale up. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P.S., Wortman Vaughan, J., (eds.) Advances in Neural Information Processing Systems, vol. 34, pp. 14745–14758. Curran Associates Inc. (2021)
Kumar, B., Dikshit, O., Gupta, A., Singh, M.K.: Feature extraction for hyperspectral image classification: A review. Int. J. Remote Sens. 41(16), 6248–6287 (2020)
Liu, B., Xuchu, Y., Zhang, P., Tan, X., Wang, R., Zhi, L.: Spectral-spatial classification of hyperspectral image using three-dimensional convolution network. J. Appl. Remote Sens. 12(1), 016005 (2018)
Odena, A., Olah, Shlens, J.: Conditional image synthesis with auxiliary classifier gans. In: International Conference on Machine Learning, pp. 2642–2651. PMLR (2017)
Parmar, N., et al.: Image transformer. In: International Conference on Machine Learning, pp. 4055–4064. PMLR (2018)
Plaza, A., Martínez, P., Plaza, J., Pérez, R.: Dimensionality reduction and classification of hyperspectral image data using sequences of extended morphological transformations. IEEE Trans. Geosci. Remote Sens. 43(3), 466–479 (2005)
Roy, S.K., Krishna, G., Dubey, S.R., Chaudhuri, B.B.: Hybridsn: Exploring 3-d-2-d cnn feature hierarchy for hyperspectral image classification. IEEE Geosci. Remote Sens. Lett. 17(2), 277–281 (2019)
Song, H., Kim, M., Park, D., Shin, Y., Lee, J.-G.: A survey. IEEE Trans. Neural Netw. Learn. Syst. Learn. Noisy Labels Deep Neural Netw. (2022)
Tun, N.L., et al.: Hyperspectral remote sensing images classification using fully convolutional neural network. In: 2021 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (ElConRus), pp. 2166–2170. IEEE (2021)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, J., Gao, F., Dong, J., Qian, D.: Adaptive dropblock-enhanced generative adversarial networks for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 59(6), 5040–5053 (2020)
Xia, J., Ghamisi, P., Yokoya, N., Iwasaki, A.: Random forest ensembles and extended multiextinction profiles for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 56(1), 202–216 (2017)
Zhan, Y., Dan, H., Wang, Y., Xianchuan, Y.: Semisupervised hyperspectral image classification based on generative adversarial networks. IEEE Geosci. Remote Sens. Lett. 15(2), 212–216 (2017)
Zhang, L., Zhang, L., Tao, D., Huang, X.: On combining multiple features for hyperspectral remote sensing image classification. IEEE Trans. Geosci. Remote Sens. 50(3), 879–893 (2011)
Zhong, Z., Li, J., Clausi, D.A., Wong, A.: Generative adversarial networks and conditional random fields for hyperspectral image classification. IEEE Trans. Cybern. 50(7), 3318–3329 (2019)
Zhu, L., Chen, Y., Ghamisi, P., Benediktsson, J.: Generative adversarial networks for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 56(9), 5046–5063 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Y., Shi, Z., Han, S., Wei, Z. (2022). Hyperspectral Image Classification Based on Transformer and Generative Adversarial Network. In: Khanna, S., Cao, J., Bai, Q., Xu, G. (eds) PRICAI 2022: Trends in Artificial Intelligence. PRICAI 2022. Lecture Notes in Computer Science, vol 13631. Springer, Cham. https://doi.org/10.1007/978-3-031-20868-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-20868-3_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20867-6
Online ISBN: 978-3-031-20868-3
eBook Packages: Computer ScienceComputer Science (R0)