Spatio-temporal estimation of climatic variables for gap filling and record extension using Reanalysis data

Abstract

The availability of reliable meteorological records is crucial for the development of a number of environmental studies. Unfortunately, these records are not always complete, usually show errors and/or have an insufficient length. This paper presents a gap filling and data record extension methodology for minimum temperature, maximum temperature, and precipitation. It uses climatic information from the NCEP-NCAR Reanalysis project, identifying pixels (grid cells) within a Reanalysis domain that have the highest Pearson’s correlation coefficient with the variable of interest. Nine stations in the Maipo River basin (Santiago, Chile) were selected for a reconstruction experiment (from 1950 to 1970) and a subsequent gap filling experiment (from 1970 to 2012). A generalized linear mixed model with a bidirectional stepwise fit procedure was used to model temperature, whereas precipitation occurrence was represented using a generalized linear mixed model with binomial distribution, and precipitation amount used an exponential generalized linear model. The performance of the algorithm was compared with inverse distance weighting and spline interpolation methods and further evaluated using the Standardized Precipitation Evapotranspiration Index, contrasting real versus modeled data. Values of the coefficient of determination averaged 0.76 (0.74–0.84) minimum temperature, 0.73 (0.73–0.81) for maximum temperature, and 0.68 (0.51–0.78) for precipitation. Root-mean-squared error was around 1.5 °C and 5 mm for temperature and precipitation, respectively. The model explains local variation of climatic variables and indicators and can be replicated anywhere, as the Reanalysis data are easily accessible and have a worldwide coverage.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

References

  1. Akaike H (1974) A new look at the statistical model identification. IEEE Trans Autom Control 19:716–723

    Article  Google Scholar 

  2. Bao X, Zhang F (2012) Evaluation of NCEP–CFSR, NCEP–NCAR, ERA-Interim, and ERA-40 Reanalysis datasets against independent sounding observations over the Tibetan Plateau. J Clim 26:206–214

    Article  Google Scholar 

  3. Bastola S, Misra V (2014) Evaluation of dynamically downscaled Reanalysis precipitation data for hydrological application. Hydrol Process 28:1989–2002

    Article  Google Scholar 

  4. Bates D, Maechler M, Bolker B (2012) lme4: linear mixed-effects models using S4 classes. R package version 0.999999–0

  5. Begert M, Schlegel T, Kirchhofer W (2005) Homogeneous temperature and precipitation series of Switzerland from 1864 to 2000. Int J Climatol 25:65–80

    Article  Google Scholar 

  6. Beguería S, Vicente-Serrano SM (2013) SPEI: calculation of the Standardized Precipitation-Evapotranspiration Index. R package version 1.3

  7. Bengtsson L, Hagemann S, Hodges KI (2004) Can climate trends be calculated from Reanalysis data? J Geophys Res Atmos 109:1984–2012

    Article  Google Scholar 

  8. Beniston M, Stoffel M, Harding R, Kernan M, Ludwig R, Moors E, Samuels P, Tockner K (2012) Obstacles to data access for research related to climate and water: implications for science and EU policy-making. Environ Sci Policy 17:41–48

    Article  Google Scholar 

  9. Betts AK, Ball JH, Barr AG, Black TA, McCaughey JH, Viterbo P (2006) Assessing land-surface-atmosphere coupling in the ERA-40 Reanalysis with boreal forest data. Agric For Meteorol 140:365–382

    Article  Google Scholar 

  10. Blender R, Fraedrich K, Sienz F (2008) Extreme event return times in long-term memory processes near 1/f. Nonlinear Process Geophys 15(4):557–565

    Article  Google Scholar 

  11. Bojanowski JS, Vrieling A, Skidmore AK (2014) A comparison of data sources for creating a long-term time series of daily gridded solar radiation for Europe. Sol Energy 99:152–171

    Article  Google Scholar 

  12. Brands S, Gutiérrez JM, Herrera S, Cofiño AS (2012) On the use of Reanalysis data for downscaling. J Clim 25:2517–2526

    Article  Google Scholar 

  13. Bustos E, Meza FJ (2014) A method to estimate maximum and minimum air temperature using MODIS surface temperature and vegetation data: application to the Maipo basin, Chile. Theor Appl Climatol 120:211–226. https://doi.org/10.1007/s00704-014-1167-2

    Article  Google Scholar 

  14. Buyadi SN, Mohd WM, Misni A (2013) Impact of land use changes on the surface temperature distribution of area surrounding the National Botanic Garden, Shah Alam. Procedia Soc Behav Sci 101:516–525

    Article  Google Scholar 

  15. Casanueva A, Herrera S, Fernandez J, Frias MD, Gutierrez JM (2012) Comparison of statistical and dynamical downscaling methods in representing temperature extremes. 12th Annual Meeting of the European Meteorological Society (EMS) and the 9th European Conference on Applied Climatology (ECAC), Poland, 10-14 September 2012

  16. Castro LM, Miranda M, Fernández B (2013) Evaluation of TRMM multi-satellite precipitation analysis (TMPA) in a mountainous region of the Central Andes range with a Mediterranean climate. Hydrol Res 46:89–105. https://doi.org/10.2166/nh.2013.096

    Article  Google Scholar 

  17. Chen SM, Hwang JR (2000) Temperature prediction using fuzzy time series. IEEE Trans Syst Man Cybern B Cybern 30:263–275

    Article  Google Scholar 

  18. Colle BA (2004) Sensitivity of orographic precipitation to changing ambient conditions and terrain geometries: an idealized modeling perspective. J Atmos Sci 61(5):588–606

    Article  Google Scholar 

  19. Daly C (2006) Guidelines for assessing the suitability of spatial climate data sets. Int J Climatol 26:707–721

    Article  Google Scholar 

  20. Diez E, Primo C, Garcia-Moya JA, Gutiérrez JM, Orfila B (2005) Statistical and dynamical downscaling of precipitation over Spain from DEMETER seasonal forecasts. Tellus A 57:409–423

    Article  Google Scholar 

  21. Doggers P, Allen RG (2002) Estimating reference evapotranspiration under inaccurate data conditions. Irrig Drain Syst 16:33–45

    Article  Google Scholar 

  22. Drosdowsky W, Chambers LE (2001) Near-global sea surface temperature anomalies as predictors of Australian seasonal rainfall. J Clim 14(7):1677–1687

    Article  Google Scholar 

  23. Feng L, Nowak G, O'Neill TJ, Welch AH (2014) CUTOFF: a spatio-temporal imputation method. J Hydrol 519:3591–3605

    Article  Google Scholar 

  24. Flannigan MD, Wotton BM (2001) Climate, weather, and area burned. In: Johnson E, Miyanishi K (eds) Forest fires, behavior and ecological effects. EE.UU. Academic Press, New York, pp 351–373

    Google Scholar 

  25. Fowler HJ, Blenkinsop S, Tebaldi C (2007) Linking climate change modelling to impacts studies: recent advances in downscaling techniques for hydrological modelling. Int J Climatol 27(12):1547–1578

    Article  Google Scholar 

  26. Fuka DR, Walter MT, MacAlister C, DeGaetano AT, Steenhuis TS, Easton ZM (2013) Using the climate forecast system Reanalysis as weather input data for watershed models. Hydrol Process 28:5613–5623. https://doi.org/10.1002/hyp.10073

    Article  Google Scholar 

  27. Gershunov A, Cayan DR (2003) Heavy daily precipitation frequency over the contiguous United States: sources of climatic variability and seasonal predictability. J Clim 16:2752–2765

    Article  Google Scholar 

  28. Harnik N, Chang EK (2003) Storm track variations as seen in radiosonde observations and reanalysis data. J Clim 16(3):480–495

    Article  Google Scholar 

  29. Hong Y, Nix H, Hutchinson M, Booth T (2005) Spatial interpolation of monthly mean climate data for China. Int J Climatol 25:1369–1379

    Article  Google Scholar 

  30. Hwang S, Graham WD, Adams A, Geurink J (2013) Assessment of the utility of dynamically-downscaled regional Reanalysis data to predict streamflow in west central Florida using an integrated hydrologic model. Reg Environ Chang 13:69–80

    Article  Google Scholar 

  31. Jung-Woo K, Yakov A (2010) Reconstructing missing daily precipitation data using regression trees and artificial neural networks for SWAT streamflow simulation. J Hydrol 394:305–314

    Article  Google Scholar 

  32. Kalnay E, Kanamitsu M, Kistler R, Collins W, Deaven D, Gandin L, Iredell M, Saha S, White G, Woollen J, Zhu Y, Chelliah M, Ebisuzaki W, Higgins W, Janowiak J, Mo KC, Ropelewski C, Wang J, Leetmaa A, Reynolds R, Jenne R, Joseph D (1996) The NCEP/NCAR 40-year Reanalysis project. Bull Am Meteorol Soc 77:437–471

    Article  Google Scholar 

  33. Kemp MU, Kemp MMU (2012) Package ‘RNCEP’

  34. Kistler R, Collins W, Saha S, White G, Woollen J, Kalnay E, Chelliah M, Ebisuzaki W, Kanamitsu M, Kousky V, van den Dool H, Jenne R, Fiorino M (2001) The NCEP-NCAR 50-year reanalysis: monthly means CD-ROM and documentation. Bull Am Meteorol Soc 82(2):247–267

    Article  Google Scholar 

  35. Kottek M, Grieser J, Beck C, Rudolf B, Rubel F (2006) World map of the Köppen-Geiger climate classification updated. Meteorol Z 15:259–263

    Article  Google Scholar 

  36. Kubik M, Brayshaw D, Coker P (2012) Reanalysis: an improved data set for simulating wind generation?. In: WREF 2012. Denver, CO. http://tinyurl.com/c4ge72x

  37. Kuznetsova A, Brockhoff PB, Bojesen RH (2013) lmerTest: tests for random and fixed effects for linear mixed effect models (lmer objects of lme4 package). R package version 1.2–0

  38. Laurikkala J, Juhola M, Kentala E, Lavrac N, Miksch S, Kavsek B (2000) Informal identification of outliers in medical data. In Fifth international workshop on intelligent data analysis in medicine and pharmacology (pp. 20–24)

  39. Linares-Rodriguez A, Ruiz-Arias JA, Pozo-Vasquez D, Tovar-Pescador J (2011) Generation of synthetic daily global solar radiation data based on ERA-Interim Reanalysis and artificial neural networks. Energy 36:5356–5365

    Article  Google Scholar 

  40. Lookingbill TR, Urban DL (2003) Spatial estimation of air temperature differences for landscape-scale studies in montane environments. Agric For Meteorol 114:141–151

    Article  Google Scholar 

  41. Maidment RI, Grimes DI, Allan RP, Greatrex H, Rojas O, Leo O (2012) Evaluation of satellite-based and model re-analysis rainfall estimates for Uganda. Meteorol Appl 20:308–317

    Article  Google Scholar 

  42. McCune B (2007) Improved estimates of incident radiation and heat load using non-parametric regression against topographic variables. J Veg Sci 18:751–754

    Article  Google Scholar 

  43. Misra V, DiNapoli SM, Bastola S (2012) Dynamic downscaling of the twentieth-century Reanalysis over the southeastern United States. Reg Environ Chang 13:15–23

    Article  Google Scholar 

  44. Montecinos A, Aceituno P (2003) Seasonality of the ENSO-related rainfall variability in Central Chile and associated circulation anomalies. J Clim 16:281–296

    Article  Google Scholar 

  45. Nagata K (2011) Quantitative precipitation estimation and quantitative precipitation forecasting by the Japan Meteorological Agency. RSMC Tokyo –Typhoon Center Technical Review 13:37–50

    Google Scholar 

  46. Paulo AA, Rosa RD, Pereira LS (2012) Climate trends and behaviour of drought indices based on precipitation and evapotranspiration in Portugal. Nat Hazards Earth Syst Sci 12:1481–1491

    Article  Google Scholar 

  47. Peres-Neto PR, Jackson DA, Somers KM (2003) Giving meaningful interpretation to ordination axes: assessing loading significance in principal component analysis. Ecology 84:2347–2363

    Article  Google Scholar 

  48. Perry M, Hollis D (2005) The development of a new set of long-term climate averages for the UK. Int J Climatol 25:1023–1039

    Article  Google Scholar 

  49. Pierce D (2011) ncdf: Interface to Unidata netCDF data files. R package version 16.6

  50. Pinheiro J, Bates D, DebRoy S, Sarkar D, the R Development Core Team. (2012). Nlme: linear and nonlinear mixed effects models. R package version 3.1–104

  51. Pörtner H (2001) Climate change and temperature-dependent biogeography: oxygen limitation of thermal tolerance in animals. Naturwissenschaften 88:137–146

    Article  Google Scholar 

  52. Ramella L, Haimberger L (2014) A global radiosonde and tracked balloon archive on 16 pressure levels (GRASP) back to 1905–part 2: homogeneity adjustments for PILOT and radiosonde wind data. Earth Syst Sci Data Discuss 7:335–383

    Article  Google Scholar 

  53. Ramos-Calzado P, Gomez-Camacho J, Perez-Bernal F, Pita-Lopez MF (2008) A novel approach to precipitation series completion in climatological datasets: application to Andalusia. Int J Climatol 28(11):1525–1534

    Article  Google Scholar 

  54. Refslund J, Dellwik E, Hahmann AN, Barlage MJ, Boegh E (2014) Development of satellite green vegetation fraction time series for use in mesoscale modeling: application to the European heat wave 2006. Theor Appl Climatol 117:377–392

    Article  Google Scholar 

  55. Rienecker MM, Suarez MJ, Gelaro R, Todling R, Bacmeister J, Liu E, Bosilovich MG, Schubert SD, Takacs L, Kim GK, Bloom S, Chen J, Collins D, Conaty A, da Silva A, Gu W, Joiner J, Koster RD, Lucchesi R, Molod A, Owens T, Pawson S, Pegion P, Redder CR, Reichle R, Robertson FR, Ruddick AG, Sienkiewicz M, Woollen J (2011) MERRA: NASA’s modern-era retrospective analysis for research and applications. J Clim 24:3624–3648

    Article  Google Scholar 

  56. Rojas E, Arce B, Peña A, Boshell F, Ayarza M (2010) Quantization and interpolation of local trends in temperature and precipitation in the high Andean areas of Cundinamarca and Boyaca (Colombia). Corpoica 11:173–182

    Article  Google Scholar 

  57. Royer A, Poirier S (2010) Surface temperature spatial and temporal variations in North America from homogenized satellite SMMR-SSM/I microwave measurements and Reanalysis for 1979–2008. J Geophys Res Atmos (1984–2012) 115:1–16

    Article  Google Scholar 

  58. Ruiz-Arias JA, Tovar-Pescador J, Pozo-Vázquez D, Alsamamra H (2009) A comparative study of DEM-based models to estimate solar radiation on mountainous terrains. Int J Geogr Inf Sci 23(8):1049–1076

    Article  Google Scholar 

  59. Saha S, Moorthi S, Pan HL, Wu X, Wang J, Nadiga S, Tripp P, Kistler R, Woollen J, Behringer D, Liu H, Stokes D, Grumbine R, Gayno G, Wang J, Hou YT, Chuang HY, Juang HMH, Sela J, Iredell M, Treadon R, Kleist D, van Delst P, Keyser D, Derber J, Ek M, Meng J, Wei H, Yang R, Lord S, van den Dool H, Kumar A, Wang W, Long C, Chelliah M, Xue Y, Huang B, Schemm JK, Ebisuzaki W, Lin R, Xie P, Chen M, Zhou S, Higgins W, Zou CZ, Liu Q, Chen Y, Han Y, Cucurull L, Reynolds RW, Rutledge G, Goldberg M (2010) The NCEP climate forecast system Reanalysis. Bull Am Meteorol Soc 91:1015–1057

    Article  Google Scholar 

  60. Sandholt I, Rasmussen K, Andersen J (2002) A simple interpretation of the surface temperature/vegetation index space for assessment of surface moisture status. Remote Sens Environ 79:213–224

    Article  Google Scholar 

  61. Schmidli J, Frei C, Vidale PL (2006) Downscaling from GCM precipitation: a benchmark for dynamical and statistical downscaling methods. Int J Climatol 26:679–689

    Article  Google Scholar 

  62. Sherwood SC, Titchner HA, Thorneb PW, McCarthyb MP (2008) Short communication how do we tell which estimates of past climate change are correct? Int J Climatol 29:1520–1523. https://doi.org/10.1002/joc.1825

    Article  Google Scholar 

  63. Simmons A, Uppala S, Dee D, Kobayashi S (2007) ERA-Interim: new ECMWF Reanalysis products from 1989 onwards. ECMWF Newsl 110:25–35

    Google Scholar 

  64. Sobrino JA, Jiménez-Muñoz JC, Paolini L (2004) Land surface temperature retrieval from LANDSAT TM 5. Remote Sens Environ 90(4):434–440

    Article  Google Scholar 

  65. Sterl A (2001) On the impact of gap-filling algorithms on variability patterns of reconstructed oceanic surface fields. Geophys Res Lett 28:2473–2476

    Article  Google Scholar 

  66. Suga Y, Ogawa H, Ohno K, Yamada K (2003) Detection of surface temperature from LANDSAT-7/ETM+. Adv Space Res 32:2235–2240

    Article  Google Scholar 

  67. Tardivo G, Berti A (2012) A dynamic method for gap filling in daily temperature datasets. J Appl Meteorol Climatol 51:1079–1086

    Article  Google Scholar 

  68. Tardivo G, Berti A (2014) The selection of predictors in a regression-based method for gap filling in daily temperature datasets. Int J Climatol 34:1311–1317

    Article  Google Scholar 

  69. Vicente-Serrano SM, Beguería S, López-Moreno JI (2010) A multiscalar drought index sensitive to global warming: the standardized precipitation evapotranspiration index. J Clim 23:1696–1718

    Article  Google Scholar 

  70. Vrac M, Naveau P (2007) Stochastic downscaling of precipitation: from dry events to heavy rainfalls. Water Resour Res 43(7). https://doi.org/10.1029/2006WR005308

  71. Wan Z, Li ZL (1997) A physics-based algorithm for retrieving land-surface emissivity and temperature from EOS/MODIS data. IEEE Trans Geosci Remote Sens 35:980–996

    Article  Google Scholar 

  72. Wang D, Murphy M (2004) Estimating optimal transformations for multiple regression using the ACE algorithm. J Data Sci 2(4):329–346

    Google Scholar 

  73. Weng Q, Lu D, Schubring J (2004) Estimation of land surface temperature–vegetation abundance relationship for urban heat island studies. Remote Sens Environ 89(4):467–483

    Article  Google Scholar 

  74. Wickham H (2007) Reshaping data with the reshape package. J Stat Softw 21:1–20

    Article  Google Scholar 

  75. Wilks DS (2006) Statistical methods in the atmospheric sciences, 2nd edn. Academic Press/Elsevier, New York 627 pp

    Google Scholar 

  76. Wright CK, de Beurs KM, Akhmadieva ZK, Groisman PY, Henebry GM (2009) Reanalysis data underestimate significant changes in growing season weather in Kazakhstan [Internet]. Environ Res Lett 2009:045020 Available from: http://iopscience.iop.org/1748-9326/4/4/045020

    Article  Google Scholar 

  77. Yoshimura K, Kanamitsu M (2008) Dynamical global downscaling of global Reanalysis. Mon Weather Rev 136:2983–2998

    Article  Google Scholar 

  78. Zavala MA (2004) Estructura, dinámica y modelos de ensamblaje del bosque mediterráneo: entre la necesidad y la contingencia. Ecología del bosque mediterráneo en un mundo cambiante. Organismo Autónomo de Parques Nacionales. Ministerio de Medio Ambiente, Madrid, pp 249–280

    Google Scholar 

Download references

Acknowledgments

Special thanks to Alvaro Paredes for his help on the algorithm generation and Shaw Lacy for edits. The authors would like to thank two anonymous reviewers for their comments and suggestions on how to improve the robustness of the method and for pointing out its limitations.

Funding

The authors would like to acknowledge the support from FONDECYT Project Nos. 1120713 and 1170429. This work was partly carried out with the aid of a grant from the Inter-American Institute for Global Change Research (IAI, CRN3056) which is supported by the US National Science Foundation (Grant GEO-1128040).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Francisco J. Meza.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Morales-Moraga, D., Meza, F.J., Miranda, M. et al. Spatio-temporal estimation of climatic variables for gap filling and record extension using Reanalysis data. Theor Appl Climatol 137, 1089–1104 (2019). https://doi.org/10.1007/s00704-018-2653-8

Download citation

Keywords

  • Temperature
  • Precipitation
  • Record extension
  • Gap filling
  • Reanalysis data