Abstract
Due to rapid urbanization, agriculture drought, and environmental pollution, significant efforts have been focused on land use and land cover (LULC) multi-spectral scene classification. Identifying the changes in land use and land cover can facilitate updating the geographical maps. Besides, the technical challenges in multi-spectral images with implicit deep learning models due to the nature of multi-modal, it tackles real-life issues such as the collection of large-scale high-resolution data. The limited training samples are considered a crucial challenge in LULC deep learning classification as requiring a huge number of training samples to ensure the optimal learning procedure. The present work has focused on considering the fraction of multi-spectral data (EuroSAT data) and evaluated the exemplary CNN architectures such as shallow network (VGG16) and deep network (ResNet152V2) with different tuning variants along with the additional layers prior to classification layer to improve the optimal training of the networks to classify the multi-spectral data. The performance of the thirteen spectral bands of EuroSAT dataset that contain ten scene classes of land use and land cover were analyzed band-wise and combination of spectral bands. For the scene class ‘Sea & lake’ the best accuracy obtained was 96.17% with individual band B08A and 95.7% with Color Infra Red (CIR) band combination. The analysis provided in this work enables the remote sensing research community to boost performance even if the multi-spectral dataset size is small.
Similar content being viewed by others
Data availability
The EuroSAT dataset used in this work is collected from Sentinel-2 satellite and is openly available at https://www.kaggle.com/datasets/apollo2506/eurosat- dataset. The software with the given versions used in the current study is Python 3.6, PyTorch 0.4.1, NumPy 1.15, Pandas 0.23.4, Scikit-learn 0.19.2, and Pillow 5.2.0. In the current work, no custom algorithms were used. Demon strations were conducted on the existing representative CNN models with the additional layers (discussed in Section 3) to classify EuroSAT data.
Abbreviations
- AdaM:
-
Adaptive Momentum
- CNN:
-
Convolutional Neural Networks
- GAN:
-
Generative Adversial Network
- GIST:
-
Gradiant Information Scale Translation
- HOG:
-
Histogram of Oriented Gradients
- LULC:
-
Land use and Land Cover
- PCA:
-
Principal Component Analysis
- ReLU:
-
Rectified Linear Unit
- ResNet:
-
Residual Network
- RMSProp:
-
Root Mean Square Propagation
- RNN:
-
Recurrent Neural Network
- SIFT:
-
Scale Invariant Feature Transform
- SGD:
-
Stochastic Gradient Descent
- SURF:
-
Speeded-Up Robust Features
- SWIR:
-
Short-Wave Infra Red
- VGG:
-
Visual Geometry Group
References
Alem A, Kumar S (2020) Deep learning methods for land cover and land use classification in remote sensing: A review, pp 903–908
Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, Santamaría J, Fadhel MA, Al-Amidie M, Farhan L (2021) Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data 8 (1), art. no. 53. https://doi.org/10.1186/s40537-021-00444-8
Bay H, Tuytelaars T, Van Gool L (2006) Surf: Speeded up robust features. In European conference on computer vision. Springer, pp 404–417
Cerreta M, Mele R, Poli G (2020) Urban ecosystem services (ues) assessment within a 3d virtual environment: A methodological approach for the larger urban zones (luz) of Naples, Italy. Appl Sci 10(18):6205
Chaib S, Gu Y, Yao H (2015) An informative feature selection method based on sparse pca for vhr scene classification. IEEE Geosci Remote Sens Lett 13(2):147–151
Chen Y-C, Lei T-C, Yao S, Wang H-P (2020) Pm2 5 prediction model based on combinational hammerstein recurrent neural networks. Mathematics 8(12):2178
Chen C, Zhang B, Su H, Li W, Wang L (2016) Land-use scene classification using multi-scale completed local binary patterns. SIViP 10(4):745–752
Chen G, Zhang X, Tan X, Cheng Y, Dai F, Zhu K, Gong Y, Wang Q (2018) Training small networks for scene classification of remote sensing images via knowledge distilla- tion. Remote Sensing 10(5):719
Cheng G, Han J, Lu X (2017) Remote sensing image scene classification: Benchmark and state of the art. Proc IEEE 105(10):1865–1883
Cheng D,Yuan Z, Wu Y (2018) Remote sensing image target recognition based on pruned deep neural network models. pp 1734–1738, cited By 1
Cheng G, Zhou P, Yao X, Yao C, Zhang Y, Han J (2016) Object detection in vhropti- cal remote sensing images via learning rotation-invariant hog feature. In 2016 4th Inter- national Workshop on Earth Observation and Remote Sensing Applications (EORSA). IEEE, 2016, pp 433–436
Cheriyadat AM (2013) Unsupervised feature learning for aerial scene classification. IEEE Trans Geosci Remote Sens 52(1):439–451
Dai D, Yang W (2010) Satellite image classification via two-layer sparse coding with biased image representation. IEEE Geosci Remote Sens Lett 8(1):173–176
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection.In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), 1, Ieee, pp 886–893
Dong J, Yin R, Sun X, Li Q, Yang Y, Qin X (2018) Inpainting of remote sensing sst images with deep convolutional generative adversarial network. IEEE Geosci Remote Sens Lett 16(2):173–177
dos Santos OABPJA, da Silva Torres R (2010) Evaluating the potential of texture and color descriptors for remote sensing image retrieval and classification. In Proceedings of VISAPP, pp 203–208
Douze M, egou HJ, Sandhawalia H, Amsaleg L, Schmid C (2009) Evaluation of gist descriptors for web-scale image search. In Proceedings of the ACM International Con- ference on Image and Video Retrieval, pp 1–8
Du B, Xiong W, Wu J, Zhang L, Zhang L, Tao D (2016) Stacked convolutional denoising auto-encoders for feature representation. IEEE Trans Cybernet 47(4):1017–1027
Faqeerzada MA, Perez M, Lohumi S, Lee H, Kim G, Wakholi C, Joshi R, Cho B-K (2020) Online application of a hyperspectral imaging system for the sorting of adulterated almonds. Appl Sci 10(18):6569
Ghiasi G, Lin T-Y, Le QV (2018) Dropblock: A regularization method for convolu- tional networks
Gorban AN, Mirkes EM, Tyukin IY (2020) How deep should be the depth of convolutional neural networks: a backyard dog case study. Cogn Comput 12(2):388–397
Haralick RM, Shanmugam K, Dinstein IH (1973) Textural features for image clas- sification. IEEE Trans Syst Man Cybern 6:610–621
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778
Helber P, Bischke B, Dengel A, Borth D (2018) Introducing eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. In IGARSS 2018–2018 IEEE International Geoscience and Remote Sensing Symposium. IEEE, 2018, pp. 204–207
Helber P, Bischke B, Dengel A, Borth D (2019) Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE J Select Top Appl Earth Observ Remote Sens 12(7):2217–2226
Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507
Hu F, Xia G-S, Wang Z, Huang X, Zhang L, Sun H (2015) Unsupervised feature learning via spectral clustering of multidimensional patches for remotely sensed scene classification. IEEE J Sel Top Appl Earth Observ Remote Sensing 8(5):2030
Huang L, Chen C, Li W, Du Q (2016) Remote sensing image scene classification using multi-scale completed local binary patterns and fisher vectors. Remote Sensing 8(6):483
Ince IF (2020) Performance boosting of scale and rotation invariant human activity recog- nition (har) with lstm networks using low dimensional 3d posture data in egocentric coordinates. Appl Sci 10(23):8474
Kandel I, Castelli M, Popoviˇc A (2020) Comparative study of first order optimizers for image classification using convolutional neural networks on histopathology images. J Imaging 6(9):92
Ke Y, Sukthankar R (2004) Pca-sift: A more distinctive representation for local image descriptors. In Proceedings of the 2004 IEEE Computer Society Conference on Com- puter Vision and Pattern Recognition, 2004. CVPR 2004, vol. 2. IEEE, 2004, pp II–II
Kim M, Madden M, Warner TA (2009) Forest type mapping using object-specific texture measures from multispectral ikonos imagery. Photogramm Eng Remote Sens 75(7):819–829
Lakshmi TRV (2018) Reduction of features to identify characters from degraded historical manuscripts. Alex Eng J 57(4):2393–2399
Lakshmi TRV, Reddy CVK, Padmavathi K, Swaraja K, Meenakshi K (2022) Entropy based single image dehazing with refined transmission using holistic edges. Multimed Tools Appl 81(14):20229–20253
Lakshmi TRV, Sastry PN, Rajinikanth T (2017) A novel 3d approach to recognize telugu palm leaf text. Eng Sci Technol Int J 20(1):143–150
Li Z, Itti L (2010) Saliency and gist features for target detection in satellite images. IEEE Trans Image Process 20(7):2017–2029
Li Z, Xu K, Xie J, Bi Q, Qin K (2020) Deep multiple instance convolutional neural networks for learning robust scene representations. IEEE Trans Geosci Remote Sensing 58(5):3685–3702
Lakshmi TV, Krishna Reddy CV (2023) Classification of skin lesions by incorporating drop-block and batch normalization layers in representative CNN models. Arab J Sci Eng 1–14 (in press)
Lu Z, Xu C, Du B, Ishida T, Zhang L, Sugiyama M (2021) Localdrop: A hybrid regularization for deep neural networks. IEEE Trans Patt Anal Mach Intell
Narmatha P, Gupta S, Lakshmi TV, Manikavelan D (2023) Skin cancer detection from dermoscopic images using Deep Siamese domain adaptation convolutional Neural Network optimized with Honey Badger Algorithm. Biomed Signal Process Control 86:105264
Mishra NB, Crews KA (2014) Mapping vegetation morphology types in a dry savanna ecosystem: Integrating hierarchical object-based image analysis with random forest. Int J Remote Sens 35(3):1175–1198
Vijaya Lakshmi TR, Sastry PN, Rajinikanth TV (2017) Feature optimization to recognize Telugu handwritten characters by implementing DE and PSO techniques. In Proceedings of the 5th International Conference on Frontiers in Intelligent Computing: Theory and Applications: FICTA 2016, vol 2 . Springer, Singapore, pp 397–405
Othman E, Bazi Y, Alajlan N, Alhichri H, Melgani F (2016) Using convolutional fea- tures and a sparse autoencoder for land-use scene classification. Int J Remote Sens 37(10):2149–2167
Park S, Song A (2020) Discrepancy analysis for detecting candidate parcels requiring update of land category in cadastral map using hyperspectral uav images: A case study in jeonju, south korea. Remote Sensing 12(3):354
Passah A, Amitab K, Kandar D (2021) Sar image despeckling using deep cnn. IET Image Processing, 15(6):1285–1297, cited By 0
Penatti OA, Nogueira K, Dos Santos JA (2015) Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 44–51
Qi K, Xiaochun Z, Baiyan W, Wu H (2016) Sparse coding-based correlaton model for land-use scene classification in high-resolution remote-sensing images. J Appl Remote Sens 10(4):042005
Qian X, Li J, Cheng G, Yao X, Zhao S, Chen Y, Jiang L (2018) Evaluation of the effect of feature extraction strategy on the performance of high-resolution remote sensing image scene classification. Yaogan Xuebao/J Remote Sensing 22(5):758–776
Sheng G, Yang W, Xu T, Sun H (2012) High-resolution satellite scene classification using a sparse coding based multiple feature combination. Int J Remote Sens 33(8):2395–2412
Shyu C-R, Klaric M, Scott GJ, Barb AS, Davis CH, Palaniappan K (2007) Geoiris: Geospatial information retrieval and indexing system—content mining, se- mantics modeling, and complex queries. IEEE Trans Geosci Remote Sens 45(4):839–852
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Stumpf A, Kerle N (2011) Object-oriented mapping of landslides using random forests. Remote Sens Environ 115(10):2564–2577
Swain MJ, Ballard DH (1991) Color indexing. Int J Comput Vision 7(1):11–32
Thirumala VL, Karanam VS, Lankireddy PR, Kakumani AK, Yacharam RK (2021) Haze-level prior approach to enhance object visibility under atmospher- icdegradation. Turk J Electr Eng Comput Sci 29(2):994–1014
Vijaya Lakshmi TR, Sastry PN, Rajinikanth T (2018) Feature selection to recognize text from palm leaf manuscripts. Signal Image Vid Process 12(2):223–229
Vijaya Lakshmi TR, Venkata Krishna Reddy C (2017) Object classification using sift algorithm and transformation techniques. In Cognitive Informatics and Soft Comput- ing: Proceeding of CISC 2017. Springer, 2019, pp 403–408
Walker JS, Briggs JM (2007) An object-oriented approach to urban forest mapping in phoenix. Photogramm Eng Remote Sens 73(5):577–583
Xia G-S, Hu J, Hu F, Shi B, Bai X, Zhong Y, Zhang L, Lu X (2017) Aid: A benchmark data set for performance evaluation of aerial scene classification. IEEE Trans Geosci Remote Sens 55(7):3965–3981
Yang Y, Newsam S (2010) Bag-of-visual-words and spatial extensions for land-use classi- fication. In Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems, pp 270–279
Yuksel M, Basturk N, Badem H, Caliskan A, Basturk A (2018) Classification of high resolution hyperspectral remote sensing data using deep neural networks. J Intell Fuzzy Syst 34:2273–2285
Zhang F, Du B, Zhang L (2014) Saliency-guided unsupervised feature learning for scene classification. IEEE Trans Geosci Remote Sens 53(4):2175–2184
Zhang W, Sun X, Fu K, Wang C, Wang H (2013) Object detection in high-resolution remote sensing images using rotation invariant parts based model. IEEE Geosci Remote Sens Lett 11(1):74–78
Zhao L, Tang P, Huo L (2016) Feature significance-based multibag-of-visual-words model for remote sensing image scene classification. J Appl Remote Sens 10(3):035004
Zhao B, Zhong Y, Xia G-S, Zhang L (2015) Dirichlet-derived multiple topic scene clas- sification model for high spatial resolution remote sensing imagery. IEEE Trans Geosci Remote Sens 54(4):2108–2123
Zheng X, Sun X, Fu K, Wang H (2012) Automatic annotation of satellite images via multifeature joint sparse coding with spatial relation constraint. IEEE Geosci Remote Sens Lett 10(4):652–656
Zhou W, Newsam S, Li C, Shao Z (2018) Patternnet: A benchmark dataset for performance evaluation of remote sensing image retrieval. ISPRS J Photogramm Remote Sens 145:197–209
Zou J, Li W, Chen C, Du Q (2016) Scene classification using local and global features with collaborative representation fusion. Inf Sci 348:209–226
Zou Q, Ni L, Zhang T, Wang Q (2015) Deep learning based feature selection for remote sensing scene classification. IEEE Geosci Remote Sens Lett 12(11):2321–2325
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Lakshmi, T.R.V., Reddy, C.V.K., Kora, P. et al. Classification of multi-spectral data with fine-tuning variants of representative models. Multimed Tools Appl 83, 23465–23487 (2024). https://doi.org/10.1007/s11042-023-16291-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-16291-z