Abstract
Satellite image time series provide essential information to closely monitor vegetation dynamics, revealing growth pattern and phenological characteristics of crops. In this study, we explored a one-dimension Temporal Convolutional Neural Network (1D-TempCNN) model for land use and land cover classification from Sentinel-2 image time series. The main goal was to evaluate the generalization ability by comparing 1D-TempCNN with two classical methods, Support Vector Machine (SVM) and Random Forest (RF). The experiments were conducted in three different sites, each one with its own crop cycles and time series compositions, and we performed cross-site and cross-year tests. These cross-testing approaches allow examining the ability of the classification models to generalize temporal (and spectral) patterns. Two data augmentation techniques, sliding window and scaling, were used to increase the amount and diversity of data. Our results show that data augmentation technique was essential to handle the data variability, contributing to the generalization of models and overall performance, resulting in an increase of up to 19.72 percentage points in overall accuracy (OA). Additionally, results show that 1D-TempCNN achieved superior OA (94.34–98.67%) in cross-testing, outperforming SVM (88.32–97.45%) and RF (90.25–98.12%). This demonstrates that the proposed 1D-TempCNN model, combined with data augmentation techniques, is capable of higher generalization for land use and land cover classification.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdollahi A, Pradhan B (2021) Integrated technique of segmentation and classification methods with connected components analysis for road extraction from orthophoto images. Expert Syst Appl 176:1–10. https://doi.org/10.1016/j.eswa.2021.114908
Al-Najjar HAH, Kalantar B, Pradhan B et al (2019) Land cover classification from fused DSM and UAV images using convolutional neural networks. Remote Sens (Basel) 11(12):1–18. https://doi.org/10.3390/rs11121461
Alzubaidi L, Zhang J, Humaidi AJ et al (2021) Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data 8:1–74. https://doi.org/10.1186/s40537-021-00444-8
Arrechea-Castillo DA, Solano-Correa YT, Muñoz-Ordóñez JF et al (2023) Multiclass land use and land cover classification of Andean sub-basins in Colombia with Sentinel-2 and deep learning. Remote Sens (Basel) 15(10):1–20. https://doi.org/10.3390/rs15102521
Audebert N, Le Saux B, Lefèvre S (2017) Segment-before-detect: vehicle detection and classification through semantic segmentation of aerial images. Remote Sens (Basel) 9(4):1–18. https://doi.org/10.3390/rs9040368
Awokuse TO, Xie R (2015) Does agriculture really matter for economic growth in developing countries? Can J Agr Econ 63:77–99. https://doi.org/10.1111/cjag.12038
Baillarin SJ, Meygret A, Dechoz C et al (2012) Sentinel-2 level 1 products and image processing performances. Int Arch Photogramm XXXIX-B1:197–202. https://doi.org/10.5194/isprsarchives-XXXIX-B1-197-2012
Becker WR, Silva LCDA, Richetti J et al (2021) Harvest date forecast for soybeans from maximum vegetative development using satellite images. Int J Remote Sens 42(3):1121–1138. https://doi.org/10.1080/01431161.2020.1823042
Bin WuW, Yu QY, Peter VH et al (2014) How could agricultural land systems contribute to raise food production under global change? J Integr Agric 13:1432–1442. https://doi.org/10.1016/S2095-3119(14)60819-4
Breiman L (2001) Random forests. Mach Learn 45:5–32. https://doi.org/10.1023/A:1010933404324
Cai Y, Guan K, Peng J et al (2018) A high-performance and in-season classification system of field-level crop types using time-series Landsat data and a machine learning approach. Remote Sens Environ 210:35–47. https://doi.org/10.1016/j.rse.2018.02.045
Carranza-García M, García-Gutiérrez J, Riquelme JC (2019) A framework for evaluating land use and land cover classification using convolutional neural networks. Remote Sens (Basel) 11(3):1–23. https://doi.org/10.3390/rs11030274
Chen Y, Lin Z, Zhao X et al (2014) Deep learning-based classification of hyperspectral data. IEEE J Sel Top Appl Earth Obs 7(6):2094–2107. https://doi.org/10.1109/JSTARS.2014.2329330
Cima EG, Uribe-opazo MA, Johann JA et al (2018) Analysis of spatial autocorrelation of grain production and agricultural storage in Paraná. Eng Agric 4430:395–402. https://doi.org/10.1590/1809-4430-Eng.Agric.v38n3p395-402/2018
Colkesen I, Ozturk MY (2022) A comparative evaluation of state-of-the-art ensemble learning algorithms for land cover classification using WorldView-2, Sentinel-2 and ROSIS imagery. Arab J Geosci 15:1–21. https://doi.org/10.1007/s12517-022-10243-x
Companhia Nacional de Abastecimento - CONAB (2023) Acompanhamento da Safra Brasileira - Grãos - Safra 2022/23 - 11º-Levantamento. https://www.conab.gov.br/info-agro/safras/graos/boletim-da-safra-de-graos/item/download/48638_05c752c731ffa8d773cdfde231c4071e. Accessed 1 Aug 2023
Copernicus (2018) Open Access Hub. In: European Space Agency. https://scihub.copernicus.eu/. Accessed 1 Aug 2023
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297. https://doi.org/10.1023/A:1022627411411
de Aparecido LEO, de Rolim GS, Richetti J et al (2016) Köppen, Thornthwaite and Camargo climate classifications for climatic zoning in the State of Paraná, Brazil. Cienc e Agrotec 40:405–417. https://doi.org/10.1590/1413-70542016404003916
De Brébisson A, Vincent P (2016) An exploration of softmax alternatives belonging to the spherical loss family. ICLR 2016:1–9. https://doi.org/10.48550/arXiv.1511.05042
Debella-Gilo M, Gjertsen AK (2021) Mapping seasonal agricultural land use types using deep learning on sentinel-2 image time series. Remote Sens (basel) 13:1–17. https://doi.org/10.3390/rs13020289
Digra M, Dhir R, Sharma N (2022) Land use land cover classification of remote sensing images based on the deep learning approaches: A statistical analysis and review. Arab J Geosci 15:1–24. https://doi.org/10.1007/s12517-022-10246-8
Eberhardt IDR, Schultz B, Rizzi R et al (2016) Cloud cover assessment for operational crop monitoring systems in tropical areas. Remote Sens (basel) 8:1–14. https://doi.org/10.3390/rs8030219
ESA (2020) Sen2Cor | STEP. In: Step.esa.int. http://step.esa.int/main/snap-supported-plugins/sen2cor/. Accessed 1 Aug 2023
ESA (2022) User Guides - Sentinel-2 MSI - Sentinel Online. In: Sentinel-2 MSI. https://sentinel.esa.int/web/sentinel/user-guides/sentinel-2-msi. Accessed 1 Aug 2023
FAOSTAT (2023) Food and Agriculture Organization, Statistics Data. http://www.fao.org/faostat/en/#data/QL. Accessed 1 Aug 2023
Gómez C, White JC, Wulder MA (2016) Optical remotely sensed time series data for land cover classification: A review. ISPRS J Photogramm 116:55–72. https://doi.org/10.1016/j.isprsjprs.2016.03.008
Hao P, Zhan Y, Wang L et al (2015) Feature selection of time series MODIS data for early crop classification using random forest: a case study in Kansas, USA. Remote Sens (basel) 7:5347–5369. https://doi.org/10.3390/rs70505347
Hinton GE, Srivastava N, Krizhevsky A et al (2012) Improving neural networks by preventing co-adaptation of feature detectors. Preprint at arXiv https://doi.org/10.48550/arXiv.1207.0580
Hu F, Xia GS, Hu J, Zhang L (2015) Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery. Remote Sens (basel) 7:14680–14707. https://doi.org/10.3390/rs71114680
Hu WS, Li HC, Pan L et al (2020) Spatial-spectral feature extraction via deep ConvLSTM neural networks for hyperspectral image classification. IEEE T Geosci Remote 58(6):4237–4250. https://doi.org/10.1109/TGRS.2019.2961947
IBGE (2023) Instituto Brasileiro de Geografia e Estatística. In: Banco de dados agregados. http://www.sidra.ibge.gov.br. Accessed 1 Aug 2023
Inglada J, Arias M, Tardy B et al (2015) Assessment of an operational system for crop type map production using high temporal and spatial resolution satellite optical imagery. Remote Sens (basel) 7:12356–12379. https://doi.org/10.3390/rs70912356
Inglada J, Vincent A, Arias M et al (2017) Operational high resolution land cover map production at the country scale using satellite image time series. Remote Sens (basel) 9:95. https://doi.org/10.3390/rs9010095
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. ICML 1:448–456. https://doi.org/10.48550/arXiv.1502.03167
Ji S, Zhang C, Xu A et al (2018) 3D convolutional neural networks for crop classification with multi-temporal remote sensing images. Remote Sens (Basel) 10(1):1–17. https://doi.org/10.3390/rs10010075
Khaliq A, Peroni L, Chiaberge M (2018) Land cover and crop classification using multitemporal Sentinel-2 images based on crops phenological cycle. EESMS 2018:1–5. https://doi.org/10.1109/EESMS.2018.8405830
Khatami R, Mountrakis G, Stehman SV (2016) A meta-analysis of remote sensing research on supervised pixel-based land-cover image classification processes: general guidelines for practitioners and future research. Remote Sens Environ 177:89–100. https://doi.org/10.1016/j.rse.2016.02.028
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. 1-15. Preprint at arXiv https://doi.org/10.48550/arXiv.1412.6980
Kumar S, Meena RS, Sheoran S et al (2021) Remote sensing for agriculture and resource management. In: Natural Resources Conservation and Advances for Sustainability, pp 91–135. https://doi.org/10.1016/B978-0-12-822976-7.00012-0
Kussul N, Lavreniuk M, Skakun S, Shelestov A (2017) Deep learning classification of land cover and crop types using remote sensing data. IEEE Geosci Remote S 14:778–782. https://doi.org/10.1109/LGRS.2017.2681128
Laban N, Abdellatif B, Ebeid HM et al (2019) Seasonal multi-temporal pixel based crop types and land cover classification for satellite images using convolutional neural networks. ICCES 2018:21–26. https://doi.org/10.1109/ICCES.2018.8639232
Li W, Fu H, Yu L et al (2016) Stacked Autoencoder-based deep learning for remote-sensing image classification: a case study of African land-cover mapping. Int J Remote Sens 37(23):5632–5646. https://doi.org/10.1080/01431161.2016.1246775
Li Y, Zhang H, Shen Q (2017) Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sens (Basel) 9(1):1–21. https://doi.org/10.3390/rs9010067
Liang H, Li Q (2016) Hyperspectral imagery classification using sparse representations of convolutional neural network features. Remote Sens (Basel) 8(2):1–16. https://doi.org/10.3390/rs8020099
Liao C, Wang J, Xie Q et al (2020) Synergistic use of multi-temporal RADARSAT-2 and VENμS data for crop classification based on 1D convolutional neural network. Remote Sens (basel) 12:1–17. https://doi.org/10.3390/rs12050832
Löw F, Michel U, Dech S, Conrad C (2013) Impact of feature selection on the accuracy and spatial uncertainty of per-field crop classification using Support Vector Machines. ISPRS J Photogramm 85:102–119. https://doi.org/10.1016/j.isprsjprs.2013.08.007
Maggiori E, Tarabalka Y, Charpiat G, Alliez P (2017) Convolutional neural networks for large-scale remote-sensing image classification. IEEE T Geosci Remote 55(2):645–657. https://doi.org/10.1109/TGRS.2016.2612821
Mou L, Ghamisi P, Zhu XX (2018) Unsupervised spectral-spatial feature learning via deep residual conv-deconv network for hyperspectral image classification. IEEE T Geosci Remote 56(2):391–406. https://doi.org/10.1109/TGRS.2017.2748160
Orynbaikyzy A, Gessner U, Conrad C (2019) Crop type classification using a combination of optical and radar remote sensing data: a review. Int J Remote Sens 40(17):6553–6595. https://doi.org/10.1080/01431161.2019.1569791
Pedregosa F, Michel V, Grisel O et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830. https://doi.org/10.48550/arXiv.1201.0490
Pelletier C, Valero S, Inglada J et al (2016) Assessing the robustness of Random Forests to map land cover with high resolution satellite image time series over large areas. Remote Sens Environ 187:156–168. https://doi.org/10.1016/j.rse.2016.10.010
Pelletier C, Webb GI, Petitjean F (2019) Temporal convolutional neural network for the classification of satellite image time series. Remote Sens (basel) 11:1–25. https://doi.org/10.3390/rs11050523
Postadjian T, Le Bris A, Sahbi H, Mallet C (2017) Investigating the potential of deep neural networks for large-scale classification of very high resolution satellite images. SPRS Ann Photogramm Remote Sens 4:183–190. https://doi.org/10.5194/isprs-annals-IV-1-W1-183-2017
Pradhan B, Al-Najjar HAH, Sameen MI et al (2020) Unseen land cover classification fromhigh-resolution orthophotos using integration of zero-shot learning and convolutional neural networks. Remote Sens (Basel) 12(10):1–26. https://doi.org/10.3390/rs12101676
Ren Y, Yu Y, Guan H (2020) Da-capsunet: A dual-attention capsule u-net for road extraction from remote sensing imagery. Remote Sens (Basel) 12. https://doi.org/10.3390/rs12182866
Rußwurm M, Körner M (2020) Self-attention for raw optical Satellite Time Series Classification. ISPRS J Photogramm 169:421–435. https://doi.org/10.1016/j.isprsjprs.2020.06.006
Santos HG dos, Bhering SB, Bognola IA et al (2007) Mapa de Solos do Estado do Paraná. Embrapa Solos. https://www.embrapa.br/busca-de-publicacoes/-/publicacao/513293/mapa-de-solos-do-estado-do-parana-legenda-atualizada. Accessed 1 Aug 2023
Shorten C, Khoshgoftaar TM (2019) A survey on Image Data Augmentation for Deep Learning. J Big Data 6:1–48. https://doi.org/10.1186/s40537-019-0197-0
Simón Sánchez AM, González-Piqueras J, de la Ossa L, Calera A (2022) Convolutional neural networks for agricultural land use classification from Sentinel-2 image time series. Remote Sens (Basel) 14(21):1–23. https://doi.org/10.3390/rs14215373
Sonobe R, Tani H, Wang X et al (2014) Parameter tuning in the support vector machine and random forest and their performances in cross- and same-year crop classification using TerraSAR-X. Int J Remote Sens 35(23):7898–7909. https://doi.org/10.1080/01431161.2014.978038
Tatsumi K, Yamashiki Y, Canales Torres MA, Taipe CLR (2015) Crop classification of upland fields using Random Forest of time-series Landsat 7 ETM+ data. Comput Electron Agric 115:171–179. https://doi.org/10.1016/j.compag.2015.05.001
Tian S, Lu Q, Wei L (2022) Multiscale superpixel-based fine classification of crops in the UAV-manned hyperspectral imagery. Remote Sens (Basel) 14(14):1–21. https://doi.org/10.3390/rs14143292
Volpi M, Tuia D (2017) Dense semantic labeling of subdecimeter resolution images with convolutional neural networks. IEEE T Geosci Remote 55(2):881–893. https://doi.org/10.1109/TGRS.2016.2616585
Wang Z, Yan W, Oates T (2017) Time series classification from scratch with deep neural networks: A strong baseline. IJCNN 2017:1578–1585. https://doi.org/10.1109/IJCNN.2017.7966039
Wang S, Azzari G, Lobell DB (2019) Crop type mapping without field-level labels: random forest transfer and unsupervised clustering techniques. Remote Sens Environ 222:303–317. https://doi.org/10.1016/j.rse.2018.12.026
Wang S, Di Tommaso S, Deines JM, Lobell DB (2020) Mapping twenty years of corn and soybean across the US Midwest using the Landsat archive. Sci Data 7:1–14. https://doi.org/10.1038/s41597-020-00646-4
Whitcraft AK, Becker-Reshef I, Justice CO (2015) A framework for defining spatially explicit earth observation requirements for a global agricultural monitoring initiative (GEOGLAM). Remote Sens (basel) 7:1461–1481. https://doi.org/10.3390/rs70201461
Wu W Bin, Yu QY, Peter VH et al (2014) How could agricultural land systems contribute to raise food production under global change? J Integr Agric 13:1432–1442. https://doi.org/10.1016/S2095-3119(14)60819-4
Xiao J, Wu H, Wang C, Xia H (2018) Land cover classification using features generated from annual time-series Landsat data. IEEE Geosci Remote S 15(5):739–743. https://doi.org/10.1109/LGRS.2018.2805715
Xu J, Zhu Y, Zhong R et al (2020) DeepCropMapping: A multi-temporal deep learning approach with improved spatial generalizability for dynamic corn and soybean mapping. Remote Sens Environ 247:111946. https://doi.org/10.1016/j.rse.2020.111946
Xue H, Xu X, Zhu Q et al (2023) Object-oriented crop classification using time series Sentinel images from Google Earth Engine. Remote Sens (Basel) 15. https://doi.org/10.3390/rs15051353
Zhao H, Chen Z, Jiang H et al (2019) Evaluation of three deep learning models for early crop classification using Sentinel-1A imagery time series—a case study in Zhanjiang. China Remote Sens (basel) 11:2673. https://doi.org/10.3390/RS11222673
Zhong L, Gong P, Biging GS (2014) Efficient corn and soybean mapping with temporal extendability: a multi-year experiment using Landsat imagery. Remote Sens Environ 140:1–13. https://doi.org/10.1016/j.rse.2013.08.023
Zhong L, Hu L, Yu L et al (2016) Automated mapping of soybean and corn using phenology. ISPRS J Photogramm 119:151–164. https://doi.org/10.1016/j.isprsjprs.2016.05.014
Zhong L, Hu L, Zhou H (2019) Deep learning based multi-temporal crop classification. Remote Sens Environ 221:430–443. https://doi.org/10.1016/j.rse.2018.11.032
Acknowledgements
The authors thank the Paraná Federal Institute and Western Paraná State University and Federal University of Pelotas for the use of laboratories and equipment.
Author information
Authors and Affiliations
Contributions
Thiago Berticelli Ló: Conceptualization, Methodology, Software, Writing—Original Draft. Ulisses Brisolara Corrêa: Writing—Original Draft, Methodology, Software. Ricardo Matsumura Araujo: Writing—Original Draft, Methodology. Jerry Adriani Johann: Conceptualization, Supervision.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Responsible Editor: Biswajeet Pradhan
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ló, T.B., Corrêa, U.B., Araújo, R.M. et al. Temporal convolutional neural network for land use and land cover classification using satellite images time series. Arab J Geosci 16, 585 (2023). https://doi.org/10.1007/s12517-023-11688-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s12517-023-11688-4