Skip to main content
SpringerLink
Log in

Semantic Segmentation of Hyperspectral Imaging Using Convolutional Neural Networks

  • Published:
Optical Memory and Neural Networks Aims and scope Submit manuscript

Abstract

using neural networks in hyperspectral imaging helps to get through the obstruction to solving data analysis, classification, and segmentation problems. There are problems, such as vegetations analysis in agriculture, which cannot be solved using classic RGB images due to lack of information. Applying neural networks to hyperspectral images is a sophisticated problem. The aim of this study is to examine concerns about using convolutional neural networks for the semantic segmentation of hyperspectral data. The following problems were considered: large spatial resolution, the influence of neural network’s input size on accuracy and performance; hyperspectral data preprocessing, the influence of dimensionality reduction and brightness equalization; neural network architecture influence on analyzing hyperspectral imaging. Also, the accuracy of neural networks was compared to classic approaches: multinominal logistic regression, random forest algorithm, discriminant analysis. As the result of the study the importance of choosing neural network’s architecture and hyperspectral data preprocessing methods are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.

Similar content being viewed by others

REFERENCES

  1. Asrar, G.Q., Fuchs, M., Kanemasu, Hatfield, En.J.L., Estimating absorbed photosynthetic radiation and leaf area index from spectral reflectance in wheat 1, Agron. J., 1984, vol. 76, no. 2, pp. 300–306,

    Article  Google Scholar 

  2. Yang, W., Yang, C., Hao, Z., Xie, C., Li, en M., Diagnosis of plant cold damage based on hyperspectral imaging and convolutional neural network, IEEE Access, 2019, vol. 7, pp. 118239–118248.

    Article  Google Scholar 

  3. Kulcke, A., Holmer, A., Wahl, P., Siemers, F., Wild, T., Daeschlein, en G., A compact hyperspectral camera for measurement of perfusion parameters in medicine, Biomed. Eng./Biomed. Technik, 2018, vol. 63, no. 5, pp. 519–527.

    Article  Google Scholar 

  4. Fabelo, H. et al., Spatio-spectral classification of hyperspectral images for brain cancer detection during surgical operations, PloS One, 2018, vol. 13, no. 3, p. e0193721.

    Article  Google Scholar 

  5. Halicek, M. et al., Deep convolutional neural networks for classifying head and neck cancer using hyperspectral imaging, J. Biomed. Opt., 2017, vol. 22, no. 6, p. 060503.

    Article  Google Scholar 

  6. Reis, M.M. et al., Chemometrics and hyperspectral imaging applied to assessment of chemical, textural and structural characteristics of meat, Meat Sci., 2018, vol. 144, pp. 100–109.

    Article  Google Scholar 

  7. Barton, I.F., Gabriel, M.J., Lyons-Baral, J., Barton, M.D., Duplessis, L., and Roberts, en C., Extending geometallurgy to the mine scale with hyperspectral imaging: A pilot study using drone-and ground-based scanning, Min., Metall. Explor., 2021, vol. 38, no. 2, pp. 799–818.

    Google Scholar 

  8. Li, J., Bioucas-Dias, J.M., Plaza, en A., Semisupervised hyperspectral image segmentation using multinomial logistic regression with active learning, IEEE Trans. Geosci. Remote Sens., 2010, vol. 48, no. 11, pp. 4085–4098.

    Google Scholar 

  9. Li, J., Bioucas-Dias, J.M., Plaza, and en A., Spectral–spatial hyperspectral image segmentation using subspace multinomial logistic regression and Markov random fields, IEEE Trans. Geosci. Remote Sens., 2011, vol. 50, no. 3, pp. 809–823.

    Article  Google Scholar 

  10. Wu, Z., Wang, Q., Plaza, A., Li, J., Sun, L., and Wei, en Z., Real-time implementation of the sparse multinomial logistic regression for hyperspectral image classification on GPUs, IEEE Geosci. Remote Sens. Lett., 2015, vol. 12, no. 7, pp. 1456–1460.

    Article  Google Scholar 

  11. Amini, S., Homayouni, S., Safari, A., and Darvishsefat, en A.A., Object-based classification of hyperspectral data using Random Forest algorithm, Geo-spatial Inf. Sci., 2018, vol. 21, no. 2, pp. 127–138.

    Google Scholar 

  12. Zimichev, E.A., Kazanskiy, N.L., and Serafimovich, P.G., Spectral-spatial classification with k-means++ particional clustering, Comput. Opt., 2014, vol. 38, no. 2, pp. 281–286.

    Article  Google Scholar 

  13. Myasnikov, E.V., Hyperspectral image segmentation using dimensionality reduction and classical segmentation approaches, Comput. Opt., 2017, vol. 41, no. 4, pp. 564–572.

    Article  Google Scholar 

  14. Villa, A., Benediktsson, J.A., Chanussot, J., and Jutten, en C., Hyperspectral image classification with independent component discriminant analysis, IEEE Trans. Geosci. Remote Sens., 2011, vol. 49, no. 12, pp. 4865–4876.

    Article  Google Scholar 

  15. Graña, M., Veganzons, M.A., and Ayerdi, B., Hyperspectral Remote Sensing Scenes. Accessed on March 1, 2022. [Online]. Available: http://www.ehu.eus/ccwintco/index.php?title=Hyperspectral_Remote_Sensing_Scenes.

  16. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, en L., Imagenet: A large-scale hierarchical image database”, in 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.

  17. Khodadadzadeh, M., Li, J., Plaza, A., and Bioucas-Dias, en J.M., A subspace-based multinomial logistic regression for hyperspectral image classification, IEEE Geosci. Remote Sens. Lett., 2014, vol. 11, no. 12, pp. 2105–2109.

    Article  Google Scholar 

  18. Xia, J., Ghamisi, P., Yokoya, N., and Iwasaki, en A., Random forest ensemp.es and extended multiextinction profiles for hyperspectral image classification, IEEE Trans. Geosci. Remote Sens., 2017, vol. 56, no. 1, pp. 202–216.

    Article  Google Scholar 

  19. Bandos, T.V., Bruzzone, L., and Camps-Valls, en G., Classification of hyperspectral images with regularized linear discriminant analysis, IEEE Trans. Geosci. Remote Sens., 2009, vol. 47, no. 3, pp. 862–873.

    Article  Google Scholar 

  20. Liao, W., Pizurica, A., Scheunders, P., Philips, W., and Pi, en Y., Semisupervised local discriminant analysis for feature extraction in hyperspectral images, IEEE Trans. Geosci. Remote Sens., 2012, vol. 51, no. 1, pp. 184–198.

    Article  Google Scholar 

  21. Paringer, R.A., Mukhin, A.V., and Kupriyanov, A.V., Formation of an informative index for recognizing specified objects in hyperspectral data, Comput. Opt., 2021, vol. 45, no. 6, pp. 873–878.

    Article  Google Scholar 

  22. Feng, L. et al., Detection of subtle bruises on winter jujube using hyperspectral imaging with pixel-wise deep learning method, IEEE Access, 2019, vol. 7, pp. 64494–64505.

    Article  Google Scholar 

  23. Wang, R. et al., Classification and segmentation of hyperspectral data of hepatocellular carcinoma samples using 1D convolutional Neural Network, Cytometry, Part A, 2020, vol. 97, no. 1, pp. 31–38.

    Article  Google Scholar 

  24. Sarker, Y., Fahim, S.R., Sarker, S.K., Badal, F.R., Das, S.K., and Mondal, en M.N.I., A multidimensional pixel-wise convolutional neural network for hyperspectral image classification, in 2019 IEEE International Conference on Robotics, Automation, Artificial-Intelligence and Internet-of-Things (RAAICON), 2019, pp. 104–107.

  25. Krizhevsky, A., Sutskever, I., and Hinton, en G.E., Imagenet classification with deep convolutional neural networks, Adv. Neural Inf. Process. Syst., 2012, vol. 25.

  26. Simonyan, K. and Zisserman, en A., Very deep convolutional networks for large-scale image recognition, 2014. arXiv preprint arXiv:1409. 15564.

  27. Szegedy, C. et al., Going deeper with convolutions, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1–9.

  28. Ronneberger, O., Fischer, P., and Brox, en T., U-net: Convolutional networks for biomedical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015, pp. 234–241.

  29. Liu, Z., Jiang, J., Qiao, X., Qi, X., Pan, Y., and Pan, en X., Using convolution neural network and hyperspectral image to identify moldy peanut kernels, LWT, 2020, vol. 132, p. 109815.

  30. Chen, S.-Y., Cheng, Y.-C., Yang, W.-L., and Wang, en M.-Y., Surface defect detection of Wet-P.ue leather using hyperspectral imaging, IEEE Access, 2021, vol. 9, pp. 127685–127702.

    Article  Google Scholar 

  31. Trajanovski, S., Shan, C., Weijtmans, P.J.C., de Koning, S.G.B., and Ruers, en T.J.M., Tongue tumor detection in hyperspectral images using deep learning semantic segmentation, IEEE Trans. Biomed. Eng., 2020, vol. 68, no. 4, pp. 1330–1340.

    Article  Google Scholar 

  32. HSI-Dataset-API. Accessed on March 1, 2022. [Online]. Available: https://pypi.org/project/HSI-Dataset-API.

  33. Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollár, en P., Focal loss for dense object detection, in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2980–2988.

  34. Loshchilov, I. and Hutter, en F., Sgdr: Stochastic gradient descent with warm restarts, 2016. arXiv preprint arXiv:1608. 03983.

  35. Paszke, A. et al., PyTorch: An Imperative Style, high-performance deep learning library, in Advances in Neural Information Processing Systems 32, Wallach, H., Larochelle, H., Beygelzimer, A., d\textquotesingle Alché-Buc, F., Fox, E., Garnett, en R., Ed., Reds Curran Associates, Inc., 2019, pp. 8024–8035.

    Google Scholar 

  36. Tian, Y., Fan, B., and Wu, en F., L2-net: Deep learning of discriminative patch descriptor in Euclidean space, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 661–669.

  37. Wold, S., Esbensen, K., and Geladi, en P., Principal component analysis, Chemom. Intell. Lab. Syst., 1987, vol. 2, no. 1–3, pp. 37–52.

    Article  Google Scholar 

Download references

Funding

The work was supported by the Ministry of Science and Higher Education of the Russian Federation, project no. FSSS-2021-0016 in the framework of the research performed by the laboratory “Photonics for a smart home and smart city” (state contract with the Samara University (theoretical research and software development) and as part of the “Priority 2030” federal strategic academic leadership program under “2021–2030 Samara University Development Program” by the Government of the Samara Region (experiments).

Author information

Authors and Affiliations

  1. Samara National Research University, 443086, Samara, Russia

    A. Mukhin, G. Danil & R. Paringer

  2. IPSI RAS—Branch of FSRC Crystallography and Photonics, Russian Academy of Sciences, 443001, Samara, Russia

    R. Paringer

Authors
  1. A. Mukhin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. Danil
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. R. Paringer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to A. Mukhin, G. Danil or R. Paringer.

Ethics declarations

The authors declare that they have no conflicts of interest.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mukhin, A., Danil, G. & Paringer, R. Semantic Segmentation of Hyperspectral Imaging Using Convolutional Neural Networks. Opt. Mem. Neural Networks 31 (Suppl 1), 38–47 (2022). https://doi.org/10.3103/S1060992X22050071

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S1060992X22050071

Keywords:

Search

Navigation