Skip to main content
View expanded cover

Generative Adversarial Learning: Architectures and Applications pp 115–144Cite as

Generative Adversarial Networks for Data Augmentation in Hyperspectral Image Classification

Part of the Intelligent Systems Reference Library book series (ISRL,volume 217)

Abstract

Hyperspectral Imaging (HSI), or imaging spectroscopy, is an imaging method with numerous applications. Unlike conventional images, hyperspectral data contain information collected in narrow wavelength intervals at hundreds of bands across the electromagnetic spectrum. The resulting image has a high spatial and spectral resolution allowing for multiple features to be extracted from machine learning applications. However, HSI acquisition is expensive, resulting in low data availability. Especially in the field of remote sensing, hyperspectral datasets only have a limited number of labeled samples, and significant class imbalances are common. Therefore, synthetic samples are useful in complementing existing datasets used in classification tasks. To this end, a modification of a Gradient Penalty Wasserstein Generative Adversarial Network (WGAN-GP) is proposed for conditional generation of realistic hyperspectral data cubes that refrains from commonly used computationally intense model architectures. Dimensionality reduction is introduced as a preprocessing step that further reduces complexity while retaining only important information. The efficacy of the model is proven by verifying the similarity of the synthetic to the real samples, evaluated by comparing their spectral information and their performance on classification outcomes using spatio-spectral models.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-91390-8_6
  • Chapter length: 30 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   149.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-91390-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book
USD   199.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 6.1
Fig. 6.2
Fig. 6.3
Fig. 6.4
Fig. 6.5
Fig. 6.6
Fig. 6.7
Fig. 6.8
Fig. 6.9

Notes

  1. 1.

    http://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes#Indian_Pines.

References

  1. Antoniou, A., Storkey, A., Edwards, H.: Data augmentation generative adversarial networks (2017). arXiv:1711.04340

  2. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: International Conference on Machine Learning, pp. 214–223. PMLR (2017)

    Google Scholar 

  3. Audebert, N., Le Saux, B., Lefèvre, S.: Generative adversarial networks for realistic synthesis of hyperspectral samples. In: IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium, pp. 4359–4362. IEEE (2018)

    Google Scholar 

  4. Batista, G.E., Prati, R.C., Monard, M.C.: A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explor. Newsl. 6(1), 20–29 (2004)

    CrossRef  Google Scholar 

  5. Borengasser, M., Hungate, W.S., Watkins, R.: Hyperspectral Remote Sensing: Principles and Applications. CRC Press (2007)

    Google Scholar 

  6. Buda, M., Maki, A., Mazurowski, M.A.: A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw. 106, 249–259 (2018)

    CrossRef  Google Scholar 

  7. Calimeri, F., Marzullo, A., Stamile, C., Terracina, G.: Biomedical data augmentation using generative adversarial neural networks. In: International Conference on Artificial Neural Networks, pp. 626–634. Springer (2017)

    Google Scholar 

  8. Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)

    CrossRef  Google Scholar 

  9. Chen, Y., Jiang, H., Li, C., Jia, X., Ghamisi, P.: Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans. Geosci. Remote Sens. 54(10), 6232–6251 (2016)

    CrossRef  Google Scholar 

  10. Chen, Y., Lin, Z., Zhao, X., Wang, G., Gu, Y.: Deep learning-based classification of hyperspectral data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 7(6), 2094–2107 (2014)

    CrossRef  Google Scholar 

  11. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  12. Douzas, G., Bacao, F.: Effective data generation for imbalanced learning using conditional generative adversarial networks. Expert Syst. Appl. 91, 464–471 (2018)

    CrossRef  Google Scholar 

  13. Farajzadeh-Zanjani, M., Hallaji, E., Razavi-Far, R., Saif, M.: Generative adversarial dimensionality reduction for diagnosing faults and attacks in cyber-physical systems. Neurocomputing 440, 101–110 (2021)

    CrossRef  Google Scholar 

  14. Feng, W., Huang, W., Bao, W.: Imbalanced hyperspectral image classification with an adaptive ensemble method based on smote and rotation forest with differentiated sampling rates. IEEE Geosci. Remote Sens. Lett. 16(12), 1879–1883 (2019)

    CrossRef  Google Scholar 

  15. Frid-Adar, M., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: Synthetic data augmentation using gan for improved liver lesion classification. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 289–293. IEEE (2018)

    Google Scholar 

  16. Goetz, A.F., Vane, G., Solomon, J.E., Rock, B.N.: Imaging spectrometry for earth remote sensing. Science 228(4704), 1147–1153 (1985)

    Google Scholar 

  17. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks (2014). arXiv:1406.2661

  18. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.: Improved training of wasserstein gans (2017). arXiv:1704.00028

  19. Haut, J.M., Paoletti, M.E., Plaza, J., Plaza, A., Li, J.: Hyperspectral image classification using random occlusion data augmentation. IEEE Geosci. Remote Sens. Lett. 16(11), 1751–1755 (2019)

    CrossRef  Google Scholar 

  20. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)

    Google Scholar 

  21. Heinonen, J.: Lipschitz Functions, pp. 43–48. Springer New York, New York, NY (2001). https://doi.org/10.1007/978-1-4613-0131-8_6

  22. Huang, Y., Jin, Y., Li, Y., Lin, Z.: Towards imbalanced image classification: a generative adversarial network ensemble learning method. IEEE Access 8, 88399–88409 (2020)

    CrossRef  Google Scholar 

  23. Kantorovich, L., Rubinstein, G.S.: Vestnik leningrad (1958)

    Google Scholar 

  24. Kotsiantis, S., Kanellopoulos, D., Pintelas, P., et al.: Handling imbalanced datasets: a review. GESTS Int. Trans. Comput. Sci. Eng. 30(1), 25–36 (2006)

    Google Scholar 

  25. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)

    Google Scholar 

  26. Landgrebe, D.: Hyperspectral image data analysis. IEEE Signal Process. Mag. 19(1), 17–28 (2002)

    CrossRef  Google Scholar 

  27. Leevy, J.L., Khoshgoftaar, T.M., Bauder, R.A., Seliya, N.: A survey on addressing high-class imbalance in big data. J. Big Data 5(1), 1–30 (2018)

    CrossRef  Google Scholar 

  28. Li, J., Du, Q., Li, Y., Li, W.: Hyperspectral image classification with imbalanced data based on orthogonal complement subspace projection. IEEE Trans. Geosci. Remote Sens. 56(7), 3838–3851 (2018)

    CrossRef  Google Scholar 

  29. Li, W., Chen, C., Zhang, M., Li, H., Du, Q.: Data augmentation for hyperspectral image classification with deep CNN. IEEE Geosci. Remote Sens. Lett. 16(4), 593–597 (2018)

    CrossRef  Google Scholar 

  30. Lim, S.K., Loo, Y., Tran, N.T., Cheung, N.M., Roig, G., Elovici, Y.: Doping: Generative data augmentation for unsupervised anomaly detection with gan. In: 2018 IEEE International Conference on Data Mining (ICDM), pp. 1122–1127. IEEE (2018)

    Google Scholar 

  31. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization (2017). arXiv:1711.05101

  32. Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of ICML, vol. 30, p. 3. Citeseer (2013)

    Google Scholar 

  33. Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., Frey, B.: Adversarial autoencoders (2015). arXiv:1511.05644

  34. Palade, V.: Class imbalance learning methods for support vector machines. Imbalanced learning: foundations, algorithms, and applications, p. 83. Wiley, Hoboken, NJ, USA (2013)

    Google Scholar 

  35. Pearson, K.: Liii. on lines and planes of closest fit to systems of points in space. Lond., Edinb., Dublin Philos. Mag. J. Sci. 2(11), 559–572 (1901)

    Google Scholar 

  36. Plaza, A., Benediktsson, J.A., Boardman, J.W., Brazile, J., Bruzzone, L., Camps-Valls, G., Chanussot, J., Fauvel, M., Gamba, P., Gualtieri, A., et al.: Recent advances in techniques for hyperspectral image processing. Remote Sens. Environ. 113, S110–S122 (2009)

    CrossRef  Google Scholar 

  37. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks (2015). arXiv:1511.06434

  38. Rahman, M.M., Davis, D.N.: Addressing the class imbalance problem in medical datasets. Int. J. Mach. Learn. Comput. 3(2), 224 (2013)

    CrossRef  Google Scholar 

  39. Razavi-Far, R., Farajzadeh-Zanjani, M., Saif, M.: An integrated class-imbalanced learning scheme for diagnosing bearing defects in induction motors. IEEE Trans. Ind. Inform. 13(6), 2758–2769 (2017)

    CrossRef  Google Scholar 

  40. Rodarmel, C., Shan, J.: Principal component analysis for hyperspectral image classification. Surv. Land Inf. Sci. 62(2), 115–122 (2002)

    Google Scholar 

  41. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training gans (2016). arXiv:1606.03498

  42. Schmidhuber, J.: A possibility for implementing curiosity and boredom in model-building neural controllers. In: Proceedings of the International Conference on Simulation of Adaptive Behavior: From Animals to Animats, pp. 222–227 (1991)

    Google Scholar 

  43. Schmidhuber, J.: Generative adversarial networks are special cases of artificial curiosity (1990) and also closely related to predictability minimization (1991). Neural Netw. 127, 58–66 (2020)

    CrossRef  Google Scholar 

  44. Shaw, G., Manolakis, D.: Signal processing for hyperspectral image exploitation. IEEE Signal Process. Mag. 19(1), 12–16 (2002)

    CrossRef  Google Scholar 

  45. Shoemake, K.: Animating rotation with quaternion curves. In: Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques, pp. 245–254 (1985)

    Google Scholar 

  46. Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 1–48 (2019)

    CrossRef  Google Scholar 

  47. Sun, C., Shrivastava, A., Singh, S., Gupta, A.: Revisiting unreasonable effectiveness of data in deep learning era. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 843–852 (2017)

    Google Scholar 

  48. Yu, X., Wu, X., Luo, C., Ren, P.: Deep learning in remote sensing scene classification: a data augmentation enhanced convolutional neural network framework. GIScience Remote Sens. 54(5), 741–758 (2017)

    CrossRef  Google Scholar 

  49. Zhu, L., Chen, Y., Ghamisi, P., Benediktsson, J.A.: Generative adversarial networks for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 56(9), 5046–5063 (2018)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Centre for Scientific Research “Demokritos”, Institute of Informatics & Telecommunications, Athens, Greece

    Dimitra Koumoutsou & Eleni Charou

  2. Department of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece

    Georgios Siolas & Georgios Stamou

Authors
  1. Dimitra Koumoutsou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Georgios Siolas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eleni Charou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Georgios Stamou
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering and School of Computer Science, University of Windsor, Windsor, ON, Canada

    Assist. Prof. Roozbeh Razavi-Far

  2. SeeChange.ai, Manchester, UK

    Dr. Ariel Ruiz-Garcia

  3. Centre for Computational Science and Mathematical Modelling, Coventry University, Coventry, Switzerland

    Prof. Vasile Palade

  4. The Swiss AI Lab, IDSIA, University of Lugano, USI & SUPSI, Lugano, Switzerland

    Prof. Juergen Schmidhuber

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Koumoutsou, D., Siolas, G., Charou, E., Stamou, G. (2022). Generative Adversarial Networks for Data Augmentation in Hyperspectral Image Classification. In: Razavi-Far, R., Ruiz-Garcia, A., Palade, V., Schmidhuber, J. (eds) Generative Adversarial Learning: Architectures and Applications. Intelligent Systems Reference Library, vol 217. Springer, Cham. https://doi.org/10.1007/978-3-030-91390-8_6

Download citation