Skip to main content

Advertisement

SpringerLink
Log in

Classification of multi-spectral data with fine-tuning variants of representative models

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Due to rapid urbanization, agriculture drought, and environmental pollution, significant efforts have been focused on land use and land cover (LULC) multi-spectral scene classification. Identifying the changes in land use and land cover can facilitate updating the geographical maps. Besides, the technical challenges in multi-spectral images with implicit deep learning models due to the nature of multi-modal, it tackles real-life issues such as the collection of large-scale high-resolution data. The limited training samples are considered a crucial challenge in LULC deep learning classification as requiring a huge number of training samples to ensure the optimal learning procedure. The present work has focused on considering the fraction of multi-spectral data (EuroSAT data) and evaluated the exemplary CNN architectures such as shallow network (VGG16) and deep network (ResNet152V2) with different tuning variants along with the additional layers prior to classification layer to improve the optimal training of the networks to classify the multi-spectral data. The performance of the thirteen spectral bands of EuroSAT dataset that contain ten scene classes of land use and land cover were analyzed band-wise and combination of spectral bands. For the scene class ‘Sea & lake’ the best accuracy obtained was 96.17% with individual band B08A and 95.7% with Color Infra Red (CIR) band combination. The analysis provided in this work enables the remote sensing research community to boost performance even if the multi-spectral dataset size is small.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The EuroSAT dataset used in this work is collected from Sentinel-2 satellite and is openly available at https://www.kaggle.com/datasets/apollo2506/eurosat- dataset. The software with the given versions used in the current study is Python 3.6, PyTorch 0.4.1, NumPy 1.15, Pandas 0.23.4, Scikit-learn 0.19.2, and Pillow 5.2.0. In the current work, no custom algorithms were used. Demon strations were conducted on the existing representative CNN models with the additional layers (discussed in Section 3) to classify EuroSAT data.

Abbreviations

AdaM:

Adaptive Momentum

CNN:

Convolutional Neural Networks

GAN:

Generative Adversial Network

GIST:

Gradiant Information Scale Translation

HOG:

Histogram of Oriented Gradients

LULC:

Land use and Land Cover

PCA:

Principal Component Analysis

ReLU:

Rectified Linear Unit

ResNet:

Residual Network

RMSProp:

Root Mean Square Propagation

RNN:

Recurrent Neural Network

SIFT:

Scale Invariant Feature Transform

SGD:

Stochastic Gradient Descent

SURF:

Speeded-Up Robust Features

SWIR:

Short-Wave Infra Red

VGG:

Visual Geometry Group

References

  1. Alem A, Kumar S (2020) Deep learning methods for land cover and land use classification in remote sensing: A review, pp 903–908

  2. Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, Santamaría J, Fadhel MA, Al-Amidie M, Farhan L (2021) Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data 8 (1), art. no. 53. https://doi.org/10.1186/s40537-021-00444-8

  3. Bay H, Tuytelaars T, Van Gool L (2006) Surf: Speeded up robust features. In European conference on computer vision. Springer,  pp 404–417

  4. Cerreta M, Mele R, Poli G (2020) Urban ecosystem services (ues) assessment within a 3d virtual environment: A methodological approach for the larger urban zones (luz) of Naples, Italy. Appl Sci 10(18):6205

    Article  CAS  Google Scholar 

  5. Chaib S, Gu Y, Yao H (2015) An informative feature selection method based on sparse pca for vhr scene classification. IEEE Geosci Remote Sens Lett 13(2):147–151

    Article  ADS  Google Scholar 

  6. Chen Y-C, Lei T-C, Yao S, Wang H-P (2020) Pm2 5 prediction model based on combinational hammerstein recurrent neural networks. Mathematics 8(12):2178

    Article  CAS  Google Scholar 

  7. Chen C, Zhang B, Su H, Li W, Wang L (2016) Land-use scene classification using multi-scale completed local binary patterns. SIViP 10(4):745–752

    Article  Google Scholar 

  8. Chen G, Zhang X, Tan X, Cheng Y, Dai F, Zhu K, Gong Y, Wang Q (2018) Training small networks for scene classification of remote sensing images via knowledge distilla- tion. Remote Sensing 10(5):719

    Article  ADS  Google Scholar 

  9. Cheng G, Han J, Lu X (2017) Remote sensing image scene classification: Benchmark and state of the art. Proc IEEE 105(10):1865–1883

    Article  Google Scholar 

  10. Cheng D,Yuan Z, Wu Y (2018) Remote sensing image target recognition based on pruned deep neural network models. pp 1734–1738, cited By 1

  11. Cheng G, Zhou P, Yao X, Yao C, Zhang Y, Han J (2016) Object detection in vhropti- cal remote sensing images via learning rotation-invariant hog feature. In 2016 4th Inter- national Workshop on Earth Observation and Remote Sensing Applications (EORSA). IEEE, 2016, pp 433–436

  12. Cheriyadat AM (2013) Unsupervised feature learning for aerial scene classification. IEEE Trans Geosci Remote Sens 52(1):439–451

    Article  ADS  Google Scholar 

  13. Dai D, Yang W (2010) Satellite image classification via two-layer sparse coding with biased image representation. IEEE Geosci Remote Sens Lett 8(1):173–176

    Article  ADS  Google Scholar 

  14. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection.In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), 1, Ieee, pp 886–893

  15. Dong J, Yin R, Sun X, Li Q, Yang Y, Qin X (2018) Inpainting of remote sensing sst images with deep convolutional generative adversarial network. IEEE Geosci Remote Sens Lett 16(2):173–177

    Article  ADS  Google Scholar 

  16. dos Santos OABPJA, da Silva Torres R (2010) Evaluating the potential of texture and color descriptors for remote sensing image retrieval and classification. In Proceedings of VISAPP, pp 203–208

  17. Douze M, egou HJ, Sandhawalia H, Amsaleg L, Schmid C (2009) Evaluation of gist descriptors for web-scale image search. In Proceedings of the ACM International Con- ference on Image and Video Retrieval, pp 1–8

  18. Du B, Xiong W, Wu J, Zhang L, Zhang L, Tao D (2016) Stacked convolutional denoising auto-encoders for feature representation. IEEE Trans Cybernet 47(4):1017–1027

    Article  Google Scholar 

  19. Faqeerzada MA, Perez M, Lohumi S, Lee H, Kim G, Wakholi C, Joshi R, Cho B-K (2020) Online application of a hyperspectral imaging system for the sorting of adulterated almonds. Appl Sci 10(18):6569

    Article  CAS  Google Scholar 

  20. Ghiasi G, Lin T-Y, Le QV (2018) Dropblock: A regularization method for convolu- tional networks

  21. Gorban AN, Mirkes EM, Tyukin IY (2020) How deep should be the depth of convolutional neural networks: a backyard dog case study. Cogn Comput 12(2):388–397

    Article  Google Scholar 

  22. Haralick RM, Shanmugam K, Dinstein IH (1973) Textural features for image clas- sification. IEEE Trans Syst Man Cybern 6:610–621

    Article  Google Scholar 

  23. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778

  24. Helber P, Bischke B, Dengel A, Borth D (2018) Introducing eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. In IGARSS 2018–2018 IEEE International Geoscience and Remote Sensing Symposium. IEEE, 2018, pp. 204–207

  25. Helber P, Bischke B, Dengel A, Borth D (2019) Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE J Select Top Appl Earth Observ Remote Sens 12(7):2217–2226

    Article  ADS  Google Scholar 

  26. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  ADS  MathSciNet  CAS  PubMed  Google Scholar 

  27. Hu F, Xia G-S, Wang Z, Huang X, Zhang L, Sun H (2015) Unsupervised feature learning via spectral clustering of multidimensional patches for remotely sensed scene classification. IEEE J Sel Top Appl Earth Observ Remote Sensing 8(5):2030

    Google Scholar 

  28. Huang L, Chen C, Li W, Du Q (2016) Remote sensing image scene classification using multi-scale completed local binary patterns and fisher vectors. Remote Sensing 8(6):483

    Article  ADS  Google Scholar 

  29. Ince IF (2020) Performance boosting of scale and rotation invariant human activity recog- nition (har) with lstm networks using low dimensional 3d posture data in egocentric coordinates. Appl Sci 10(23):8474

    Article  CAS  Google Scholar 

  30. Kandel I, Castelli M, Popoviˇc A (2020) Comparative study of first order optimizers for image classification using convolutional neural networks on histopathology images. J Imaging 6(9):92

    Article  PubMed  PubMed Central  Google Scholar 

  31. Ke Y, Sukthankar R (2004) Pca-sift: A more distinctive representation for local image descriptors. In Proceedings of the 2004 IEEE Computer Society Conference on Com- puter Vision and Pattern Recognition, 2004. CVPR 2004, vol. 2. IEEE, 2004, pp II–II

  32. Kim M, Madden M, Warner TA (2009) Forest type mapping using object-specific texture measures from multispectral ikonos imagery. Photogramm Eng Remote Sens 75(7):819–829

    Article  Google Scholar 

  33. Lakshmi TRV (2018) Reduction of features to identify characters from degraded historical manuscripts. Alex Eng J 57(4):2393–2399

    Article  Google Scholar 

  34. Lakshmi TRV, Reddy CVK, Padmavathi K, Swaraja K, Meenakshi K (2022) Entropy based single image dehazing with refined transmission using holistic edges. Multimed Tools Appl 81(14):20229–20253

    Article  Google Scholar 

  35. Lakshmi TRV, Sastry PN, Rajinikanth T (2017) A novel 3d approach to recognize telugu palm leaf text. Eng Sci Technol Int J 20(1):143–150

    Google Scholar 

  36. Li Z, Itti L (2010) Saliency and gist features for target detection in satellite images. IEEE Trans Image Process 20(7):2017–2029

    ADS  MathSciNet  PubMed  Google Scholar 

  37. Li Z, Xu K, Xie J, Bi Q, Qin K (2020) Deep multiple instance convolutional neural networks for learning robust scene representations. IEEE Trans Geosci Remote Sensing 58(5):3685–3702

    Article  ADS  Google Scholar 

  38. Lakshmi TV, Krishna Reddy CV (2023) Classification of skin lesions by incorporating drop-block and batch normalization layers in representative CNN models. Arab J Sci Eng 1–14 (in press)

  39. Lu Z, Xu C, Du B, Ishida T, Zhang L, Sugiyama M (2021) Localdrop: A hybrid regularization for deep neural networks. IEEE Trans Patt Anal Mach Intell

  40. Narmatha P, Gupta S, Lakshmi TV, Manikavelan D (2023) Skin cancer detection from dermoscopic images using Deep Siamese domain adaptation convolutional Neural Network optimized with Honey Badger Algorithm. Biomed Signal Process Control 86:105264

    Article  Google Scholar 

  41. Mishra NB, Crews KA (2014) Mapping vegetation morphology types in a dry savanna ecosystem: Integrating hierarchical object-based image analysis with random forest. Int J Remote Sens 35(3):1175–1198

    Article  Google Scholar 

  42. Vijaya Lakshmi TR, Sastry PN, Rajinikanth TV (2017) Feature optimization to recognize Telugu handwritten characters by implementing DE and PSO techniques. In Proceedings of the 5th International Conference on Frontiers in Intelligent Computing: Theory and Applications: FICTA 2016, vol 2 . Springer, Singapore, pp 397–405

  43. Othman E, Bazi Y, Alajlan N, Alhichri H, Melgani F (2016) Using convolutional fea- tures and a sparse autoencoder for land-use scene classification. Int J Remote Sens 37(10):2149–2167

    Article  Google Scholar 

  44. Park S, Song A (2020) Discrepancy analysis for detecting candidate parcels requiring update of land category in cadastral map using hyperspectral uav images: A case study in jeonju, south korea. Remote Sensing 12(3):354

    Article  ADS  Google Scholar 

  45. Passah A, Amitab K, Kandar D (2021) Sar image despeckling using deep cnn. IET Image Processing, 15(6):1285–1297, cited By 0

  46. Penatti OA, Nogueira K, Dos Santos JA (2015) Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 44–51

  47. Qi K, Xiaochun Z, Baiyan W, Wu H (2016) Sparse coding-based correlaton model for land-use scene classification in high-resolution remote-sensing images. J Appl Remote Sens 10(4):042005

    Article  ADS  Google Scholar 

  48. Qian X, Li J, Cheng G, Yao X, Zhao S, Chen Y, Jiang L (2018) Evaluation of the effect of feature extraction strategy on the performance of high-resolution remote sensing image scene classification. Yaogan Xuebao/J Remote Sensing 22(5):758–776

    Google Scholar 

  49. Sheng G, Yang W, Xu T, Sun H (2012) High-resolution satellite scene classification using a sparse coding based multiple feature combination. Int J Remote Sens 33(8):2395–2412

    Article  Google Scholar 

  50. Shyu C-R, Klaric M, Scott GJ, Barb AS, Davis CH, Palaniappan K (2007) Geoiris: Geospatial information retrieval and indexing system—content mining, se- mantics modeling, and complex queries. IEEE Trans Geosci Remote Sens 45(4):839–852

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  51. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  52. Stumpf A, Kerle N (2011) Object-oriented mapping of landslides using random forests. Remote Sens Environ 115(10):2564–2577

    Article  ADS  Google Scholar 

  53. Swain MJ, Ballard DH (1991) Color indexing. Int J Comput Vision 7(1):11–32

    Article  Google Scholar 

  54. Thirumala VL, Karanam VS, Lankireddy PR, Kakumani AK, Yacharam RK (2021) Haze-level prior approach to enhance object visibility under atmospher- icdegradation. Turk J Electr Eng Comput Sci 29(2):994–1014

    Article  Google Scholar 

  55. Vijaya Lakshmi TR, Sastry PN, Rajinikanth T (2018) Feature selection to recognize text from palm leaf manuscripts. Signal Image Vid Process 12(2):223–229

    Article  Google Scholar 

  56. Vijaya Lakshmi TR, Venkata Krishna Reddy C (2017) Object classification using sift algorithm and transformation techniques. In Cognitive Informatics and Soft Comput- ing: Proceeding of CISC 2017. Springer, 2019, pp 403–408

  57. Walker JS, Briggs JM (2007) An object-oriented approach to urban forest mapping in phoenix. Photogramm Eng Remote Sens 73(5):577–583

    Article  Google Scholar 

  58. Xia G-S, Hu J, Hu F, Shi B, Bai X, Zhong Y, Zhang L, Lu X (2017) Aid: A benchmark data set for performance evaluation of aerial scene classification. IEEE Trans Geosci Remote Sens 55(7):3965–3981

    Article  ADS  Google Scholar 

  59. Yang Y, Newsam S (2010) Bag-of-visual-words and spatial extensions for land-use classi- fication. In Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems, pp 270–279

  60. Yuksel M, Basturk N, Badem H, Caliskan A, Basturk A (2018) Classification of high resolution hyperspectral remote sensing data using deep neural networks. J Intell Fuzzy Syst 34:2273–2285

  61. Zhang F, Du B, Zhang L (2014) Saliency-guided unsupervised feature learning for scene classification. IEEE Trans Geosci Remote Sens 53(4):2175–2184

    Article  ADS  Google Scholar 

  62. Zhang W, Sun X, Fu K, Wang C, Wang H (2013) Object detection in high-resolution remote sensing images using rotation invariant parts based model. IEEE Geosci Remote Sens Lett 11(1):74–78

    Article  ADS  Google Scholar 

  63. Zhao L, Tang P, Huo L (2016) Feature significance-based multibag-of-visual-words model for remote sensing image scene classification. J Appl Remote Sens 10(3):035004

    Article  ADS  Google Scholar 

  64. Zhao B, Zhong Y, Xia G-S, Zhang L (2015) Dirichlet-derived multiple topic scene clas- sification model for high spatial resolution remote sensing imagery. IEEE Trans Geosci Remote Sens 54(4):2108–2123

    Article  ADS  Google Scholar 

  65. Zheng X, Sun X, Fu K, Wang H (2012) Automatic annotation of satellite images via multifeature joint sparse coding with spatial relation constraint. IEEE Geosci Remote Sens Lett 10(4):652–656

    Article  ADS  Google Scholar 

  66. Zhou W, Newsam S, Li C, Shao Z (2018) Patternnet: A benchmark dataset for performance evaluation of remote sensing image retrieval. ISPRS J Photogramm Remote Sens 145:197–209

    Article  ADS  CAS  Google Scholar 

  67. Zou J, Li W, Chen C, Du Q (2016) Scene classification using local and global features with collaborative representation fusion. Inf Sci 348:209–226

    Article  ADS  MathSciNet  Google Scholar 

  68. Zou Q, Ni L, Zhang T, Wang Q (2015) Deep learning based feature selection for remote sensing scene classification. IEEE Geosci Remote Sens Lett 12(11):2321–2325

    Article  ADS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of ECE, Mahatma Gandhi Institute of Technology, Hyderabad, India

    T. R. Vijaya Lakshmi

  2. Department of EEE, Chaitanya Bharathi Institute of Technology, Hyderabad, India

    Ch. Venkata Krishna Reddy

  3. Department of ECE, GRIET, Hyderabad, India

    Padmavathi Kora, K. Swaraja, K. Meenakshi & Ch. Usha Kumari

  4. Department of ECE, JNTUH CESTH, Hyderabad, India

    L. Pratap Reddy

Authors
  1. T. R. Vijaya Lakshmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ch. Venkata Krishna Reddy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Padmavathi Kora
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. K. Swaraja
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. K. Meenakshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ch. Usha Kumari
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. L. Pratap Reddy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to T. R. Vijaya Lakshmi.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lakshmi, T.R.V., Reddy, C.V.K., Kora, P. et al. Classification of multi-spectral data with fine-tuning variants of representative models. Multimed Tools Appl 83, 23465–23487 (2024). https://doi.org/10.1007/s11042-023-16291-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16291-z

Keywords

Search

Navigation