Skip to main content
SpringerLink
Log in

Filling missing data and smoothing altered data in satellite imagery with a spatial functional procedure

  • Original Paper
  • Published:
Stochastic Environmental Research and Risk Assessment Aims and scope Submit manuscript

Abstract

Outliers and missing data are commonly found in satellite imagery. These are usually caused by atmospheric or electronic failures, hampering the correct monitoring of remote-sensing data. To avoid distorted data, we propose a procedure called “spatial functional prediction” (SFP). The SFP procedure consists of the following: (1) aggregating remote-sensing data for reducing the number of missing data and/or outliers; (2) additively decomposing the time series of images into a trend, a seasonal, and an error component; (3) defining the spatial functional data and predicting the trend component using an ordinary kriging; and (4) adding back the seasonal and error components to the predicted trend. The benefits of the SFP procedure are illustrated in the following scenarios: introducing random outliers, random missing data, mixtures of both, and artificial clouds in an extensive simulation study of composite images, and using daily images with real clouds. The following two derived variables are considered: land surface temperature (LST day) and normalized vegetation index (NDVI), which are obtained as remote-sensing data in a region in northern Spain during 2003–2016. The performance of SFP was checked using the root mean squared error (RMSE). A comparison with a procedure based on predicting with thin-plate splines (TpsP) is also made. We conclude that SFP is simpler and faster than TpsP, and provides smaller values of RMSE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Addink E (1999) A comparison of conventional and geostatistical methods to replace clouded pixels in NOAA-AVHRR images. Int J Remote Sens 20(5):961–977

    Google Scholar 

  • Aguilera-Morillo M, Durbán M, Aguilera A (2017) Prediction of functional data with spatial dependence: a penalized approach. Stoch Environ Res Risk Assess 31(1):7–22

    Google Scholar 

  • Benali A, Carvalho A, Nunes J, Carvalhais N, Santos A (2012) Estimating air surface temperature in Portugal using MODIS LST data. Remote Sens Environ 124:108–121

    Google Scholar 

  • Bernardi MS, Sangalli LM, Mazza G, Ramsay JO (2017) A penalized regression model for spatial functional data with application to the analysis of the production of waste in Venice province. Stoch Environ Res Risk Assess 31(1):23–38

    Google Scholar 

  • Bhavya V, Jaleel S, Anu Scree NC (2014) Could removal from multi-temporal satellite images using informatuion cloning and information reconstruction. Int J Emerg Trends Sci Technol 1(04):546–551

    Google Scholar 

  • Boer EP, de Beurs KM, Hartkamp AD (2001) Kriging and thin plate splines for mapping climate variables. Int J Appl Earth Observ Geoinf 3(2):146–154

    Google Scholar 

  • Brooks E, Wynne R, Thomas V (2018) Using window regression to gap-fill landsat etm+ post slc-off data. Remote Sens 10(10):1502

    Google Scholar 

  • Cai Z, Jönsson P, Jin H, Eklundh L (2017) Performance of smoothing methods for reconstructing ndvi time-series and estimating vegetation phenology from modis data. Remote Sens 9(12):1271

    Google Scholar 

  • Cleveland RB, Cleveland WS, McRae JE, Terpenning I (1990) STL: a seasonal-trend decomposition procedure based on loess. J Off Stat 6(1):3–73

    Google Scholar 

  • Cressie N, Wikle CK (2015) Statistics for spatio-temporal data. Wiley, Hoboken

    Google Scholar 

  • Delicado P, Giraldo R, Comas C, Mateu J (2010) Statistics for spatial functional data: some recent contributions. Environmetrics 21(3–4):224–239

    Google Scholar 

  • Durbin J, Koopman SJ (2002) A simple and efficient simulation smoother for state space time series analysis. Biometrika 89(3):603–616

    Google Scholar 

  • Eklundh L, Jönsson P (2012) TIMESAT 3.2 with parallel processing-Software Manual. Lund University

  • Gao Y, Xie H, Yao T, Xue C (2010) Integrated assessment on multi-temporal and multi-sensor combinations for reducing cloud obscuration of MODIS snow cover products of the Pacific Northwest USA. Remote Sens Environ 114(8):1662–1675

    Google Scholar 

  • Gerber F, de Jong R, Schaepman ME, Schaepman-Strub G, Reinhard F (2018) Predicting missing values in spatio-temporal remote sensing data. IEEE Trans Geosci Remote Sens 56(5):2841–2853

    Google Scholar 

  • Gerber F, Furrer R, Schaepman-Strub G, de Jong R, Schaepman ME (2016) Predicting missing values in spatio-temporal satellite data. ArXiv e-prints

  • Giraldo R, Delicado P, Mateu J (2010) Continuous time-varying kriging for spatial prediction of functional data: an environmental application. J Agric Biol Environ Stat 15(1):66–82

    Google Scholar 

  • Giraldo R, Delicado P, Mateu J (2011) Ordinary kriging for function-valued spatial data. Environ Ecol Stat 18(3):411–426

    Google Scholar 

  • Giraldo R, Mateu J, Delicado P (2012) geofd: an R package for function-valued geostatistical prediction. Revista Colombiana de Estadística 35(3):385–407

    Google Scholar 

  • Goitía A, Medina MR, Angulo J (2005) Joint estimation of spatial deformation and blurring in environmental data. Stoch Environ Res Risk Assess 19(1):1–7

    Google Scholar 

  • Goulard M, Voltz M (1993) Geostatistical interpolation of curves: a case study in soil science. In: Geostatistics Tróia’92. Springer, Berlin, pp 805–816

    Google Scholar 

  • Holben BN (1986) Characteristics of maximum-value composite images from temporal AVHRR data. Int J Remote Sens 7(11):1417–1434

    Google Scholar 

  • Hou J, Huang C, Zhang Y, Guo J, Gu J (2019) Gap-filling of modis fractional snow cover products via non-local spatio-temporal filtering based on machine learning techniques. Remote Sens 11(1):90

    Google Scholar 

  • Masek JG, Goward SN, Kennedy RE, Cohen WB, Moisen GG, Schleeweis K, Huang C (2013) United States forest disturbance trends observed using Landsat time series. Ecosystems 16(6):1087–1104

    Google Scholar 

  • Masek JG, Vermote EF, Saleous NE, Wolfe R, Hall FG, Huemmrich KF, Gao F, Kutler J, Lim T-K (2006) A Landsat surface reflectance dataset for North America, 1990–2000. IEEE Geosci Remote Sens Lett 3(1):68–72

    Google Scholar 

  • Matheron G (1981) Splines and kriging: their formal equivalence. Down-to-earth-statistics: solutions looking for geological problems, pp 77–95

  • Militino AF, Ugarte MD, Pérez-Goya U (2017) Stochastic spatio-temporal models for analysing NDVI distribution of GIMMS NDVI3g images. Remote Sens 9(1):76

    Google Scholar 

  • Militino AF, Ugarte MD, Pérez-Goya U (2018) Improving the quality of satellite imagery based on ground-truth data from rain gauge stations. Remote Sens 10(3):398

    Google Scholar 

  • MODIS (2017). https://modis.gsfc.nasa.gov/about/

  • Müller H, Rufin P, Griffiths P, Siqueira AJB, Hostert P (2015) Mining dense Landsat time series for separating cropland and pasture in a heterogeneous Brazilian savanna landscape. Remote Sens Environ 156:490–499

    Google Scholar 

  • Nychka D, Furrer R, Paige J, Sain S (2015) fields: tools for spatial data. R package version 9

  • Park S-Y, Sur C, Kim J-S, Lee J-H (2018) Evaluation of multi-sensor satellite data for monitoring different drought impacts. Stoch Environ Res Risk Assess 39:1–13

    Google Scholar 

  • Poggio L, Gimona A, Brown I (2012) Spatio-temporal MODIS EVI gap filling under cloud cover: an example in Scotland. ISPRS J Photogr Remote Sens 72:56–72

    Google Scholar 

  • Qiu B, Feng M, Tang Z (2016) A simple smoother based on continuous wavelet transform: comparative evaluation based on the fidelity, smoothness and efficiency in phenological estimation. Int J Appl Earth Observ Geoinf 47:91–101

    Google Scholar 

  • Ramsay J (2005) Functional data analysis. Encyclopedia of statistics in behavioral science. Wiley

  • Ramsay JO, Silverman BW (2007) Applied functional data analysis: methods and case studies. Springer, Berlin

    Google Scholar 

  • Richter R (1996) A spatially adaptive fast atmospheric correction algorithm. Int J Remote Sens 17(6):1201–1214

    Google Scholar 

  • Rossi RE, Dungan JL, Beck LR (1994) Kriging in the shadows: geostatistical interpolation for remote sensing. Remote Sens Environ 49(1):32–40

    Google Scholar 

  • Rouse J Jr, Haas R, Schell J, Deering D (1974) Monitoring vegetation systems in the Great Plains with ERTS. NASA Spec Publ 351:309

    Google Scholar 

  • Ruiz-Medina M, Espejo R (2012) Spatial autoregressive functional plug-in prediction of ocean surface temperature. Stoch Environ Res Risk Assess 26(3):335–344

    Google Scholar 

  • Schneider A (2012) Monitoring land cover change in urban and peri-urban areas using dense time stacks of Landsat satellite data and a data mining approach. Remote Sens Environ 124:689–704

    Google Scholar 

  • Shen H, Li H, Qian Y, Zhang L, Yuan Q (2014) An effective thin cloud removal procedure for visible remote sensing images. ISPRS J Photogram Remote Sens 96:224–235

    Google Scholar 

  • Slayback DA, Pinzon JE, Los SO, Tucker CJ (2003) Northern hemisphere photosynthetic trends 1982–99. Glob Change Biol 9(1):1–15

    Google Scholar 

  • Stein ML (1991) A kernel approximation to the kriging predictor of a spatial process. Ann Inst Stat Math 43(1):61–75

    Google Scholar 

  • Tseng D-C, Tseng H-T, Chien C-L (2008) Automatic cloud removal from multi-temporal SPOT images. Appl Math Comput 205(2):584–600

    Google Scholar 

  • Tucker CJ, Pinzon JE, Brown ME, Slayback DA, Pak EW, Mahoney R, Vermote EF, El Saleous N (2005) An extended AVHRR 8-km NDVI dataset compatible with MODIS and SPOT vegetation NDVI data. Int J Remote Sens 26(20):4485–4498

    Google Scholar 

  • Van de Kassteele J, Koelemeijer R, Dekkers A, Schaap M, Homan C, Stein A (2006) Statistical mapping of PM10 concentrations over Western Europe using secondary information from dispersion modeling and MODIS satellite observations. Stoch Environ Res Risk Assess 21(2):183–194

    Google Scholar 

  • van Wijk MT, Williams M (2005) Optical instruments for measuring leaf area index in low vegetation: application in arctic ecosystems. Ecol Appl 15(4):1462–1470

    Google Scholar 

  • Vera JF, Angulo JM, Roldán JA (2017) Stability analysis in nonstationary spatial covariance estimation. Stoch Environ Res Risk Assess 31(3):815–828

    Google Scholar 

  • Verhoef W, Menenti M, Azzali S (1996) Cover A colour composite of NOAA-AVHRR-NDVI based on time series analysis (1981–1992). Int J Remote Sens 17(2):231–235

    Google Scholar 

  • Wan Z, Dozier J (1996) A generalized split-window algorithm for retrieving land-surface temperature from space. IEEE Trans Geosci Remote Sens 34(4):892–905

    Google Scholar 

  • Wan Z, Zhang Y, Zhang Q, Li Z-L (2002) Validation of the land-surface temperature products retrieved from Terra Moderate Resolution Imaging Spectroradiometer data. Remote Sens Environ 83(1):163–180

    Google Scholar 

  • Wood SN (2003) Thin plate regression splines. J R Stat Soc: Ser B (Stat Methodol) 65(1):95–114

    Google Scholar 

  • Wood SN (2017) Generalized additive models: an introduction with R. Chapman and Hall/CRC, Boca Raton

    Google Scholar 

  • Xie H, Wang X, Liang T (2009) Development and assessment of combined Terra and Aqua snow cover products in Colorado Plateau, USA and Northern Xinjiang, China. J Appl Remote Sens 3(1):033559–033559

    Google Scholar 

  • Xu H, Xu C-Y, Sælthun NR, Zhou B, Xu Y (2015) Evaluation of reanalysis and satellite-based precipitation datasets in driving hydrological models in a humid region of Southern China. Stoch Environ Res Risk Assess 29(8):2003–2020

    Google Scholar 

  • Yang G, Shen H, Zhang L, He Z, Li X (2015) A moving weighted harmonic analysis method for reconstructing high-quality SPOT VEGETATION NDVI time-series data. IEEE Trans Geosci Remote Sens 53(11):6008–6021

    Google Scholar 

  • Yin G, Mariethoz G, McCabe MF (2016) Gap-filling of landsat 7 imagery using the direct sampling method. Remote Sens 9(1):12

    Google Scholar 

  • Zhang C, Li W, Civco D (2014) Application of geographically weighted regression to fill gaps in SLC-off Landsat ETM+ satellite imagery. Int J Remote Sens 35(22):7650–7672

    Google Scholar 

Download references

Acknowledgements

This research was supported by Project MTM2017-82553-R (AEI/FEDER, UE), and by “la Caixa” Foundation (ID 1000010434), Caja Navarra Foundation, and UNED Pamplona, under agreement LCF/PR/PR15/51100007.

Author information

Authors and Affiliations

  1. Department of Statistics, Computer Science and Mathematics, and InaMat, Public University of Navarre, 31006, Pamplona, Spain

    A. F. Militino, M. D. Ugarte & M. Montesino

Authors
  1. A. F. Militino
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. D. Ugarte
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Montesino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. D. Ugarte.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Militino, A.F., Ugarte, M.D. & Montesino, M. Filling missing data and smoothing altered data in satellite imagery with a spatial functional procedure. Stoch Environ Res Risk Assess 33, 1737–1750 (2019). https://doi.org/10.1007/s00477-019-01711-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00477-019-01711-0

Keywords

Search

Navigation