Climate Dynamics

, Volume 53, Issue 3–4, pp 1287–1305 | Cite as

Bias adjustment and ensemble recalibration methods for seasonal forecasting: a comprehensive intercomparison using the C3S dataset

  • R. ManzanasEmail author
  • J. M. Gutiérrez
  • J. Bhend
  • S. Hemri
  • F. J. Doblas-Reyes
  • V. Torralba
  • E. Penabad
  • A. Brookshaw


This work presents a comprehensive intercomparison of different alternatives for the calibration of seasonal forecasts, ranging from simple bias adjustment (BA)—e.g. quantile mapping—to more sophisticated ensemble recalibration (RC) methods—e.g. non-homogeneous Gaussian regression, which build on the temporal correspondence between the climate model and the corresponding observations to generate reliable predictions. To be as critical as possible, we validate the raw model and the calibrated forecasts in terms of a number of metrics which take into account different aspects of forecast quality (association, accuracy, discrimination and reliability). We focus on one-month lead forecasts of precipitation and temperature from four state-of-the-art seasonal forecasting systems, three of them included in the Copernicus Climate Change Service dataset (ECMWF-SEAS5, UK Met Office-GloSea5 and Météo France-System5) for boreal winter and summer over two illustrative regions with different skill characteristics (Europe and Southeast Asia). Our results indicate that both BA and RC methods effectively correct the large raw model biases, which is of paramount importance for users, particularly when directly using the climate model outputs to run impact models, or when computing climate indices depending on absolute values/thresholds. However, except for particular regions and/or seasons (typically with high skill), there is only marginal added value—with respect to the raw model outputs—beyond this bias removal. For those cases, RC methods can outperform BA ones, mostly due to an improvement in reliability. Finally, we also show that whereas an increase in the number of members only modestly affects the results obtained from calibration, longer hindcast periods lead to improved forecast quality, particularly for RC methods.


Seasonal forecasting C3S Bias adjustment Ensemble recalibration Forecast quality Reliability Ensemble size Hindcast length 



This work has been funded by the C3S activity on Evaluation and Quality Control for seasonal forecasts. JMG was partially supported by the project MULTI-SDM (CGL2015-66583-R, MINECO/FEDER). FJDR was partially funded by the H2020 EUCP project (GA 776613).


  1. Barnston AG (1994) Linear statistical short-term climate predictive skill in the northern hemisphere. J Clim 7(10):1513–1564.<1513:LSSTCP>2.0.CO;2 CrossRefGoogle Scholar
  2. Bedia J, Golding N, Casanueva A, Iturbide M, Buontempo C, Gutiérrez JM (2018) Seasonal predictions of Fire Weather Index: paving the way for their operational applicability in Mediterranean Europe. Clim Serv 9:101–110. CrossRefGoogle Scholar
  3. Dee DP, Uppala SM, Simmons AJ, Berrisford P, Poli P, Kobayashi S, Andrae U, Balmaseda MA, Balsamo G, Bauer P, Bechtold P, Beljaars ACM, van de Berg L, Bidlot J, Bormann N, Delsol C, Dragani R, Fuentes M, Geer AJ, Haimberger L, Healy SB, Hersbach H, Holm EV, Isaksen L, Kallberg P, Koehler M, Matricardi M, McNally AP, Monge-Sanz BM, Morcrette JJ, Park BK, Peubey C, de Rosnay P, Tavolato C, Thepaut JN, Vitart F (2011) The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Q J R Meteorol Soc 137(656):553–597. CrossRefGoogle Scholar
  4. Déqué M (2007) Frequency of precipitation and temperature extremes over France in an anthropogenic scenario: Model results and statistical correction according to observed values. Glob Planet Change 57(1–2):16–26. CrossRefGoogle Scholar
  5. Doblas-Reyes FJ, Hagedorn R, Palmer TN (2005) The rationale behind the success of multi-model ensembles in seasonal forecasting II. Calibration and combination. Tellus A 57(3):234–252. Google Scholar
  6. Doblas-Reyes FJ, García-Serrano J, Lienert F, Biescas AP, Rodrigues LRL (2013) Seasonal climate predictability and forecasting: status and prospects. Wiley Interdiscip Rev Clim Change 4(4):245–268. CrossRefGoogle Scholar
  7. Eade R, Smith D, Scaife A, Wallace E, Dunstone N, Hermanson L, Robinson N (2014) Do seasonal-to-decadal climate predictions underestimate the predictability of the real world? Geophys Res Lett 41(15):5620–5628. CrossRefGoogle Scholar
  8. Epstein ES (1969) A scoring system for probability forecasts of ranked categories. J Appl Meteorol 8(6):985–987.<0985:ASSFPF>2.0.CO;2 CrossRefGoogle Scholar
  9. Feldmann K, Scheuerer M, Thorarinsdottir TL (2015) Spatial postprocessing of ensemble forecasts for temperature using non-homogeneous Gaussian regression. Mon Weather Rev 143(3):955–971. CrossRefGoogle Scholar
  10. Gneiting T, Raftery AE, Westveld AH, Goldman T (2005) Calibrated probabilistic forecasting using Ensemble Model Output Statistics and minimum CRPS estimation. Mon Weather Rev 133(5):1098–1118. CrossRefGoogle Scholar
  11. Gutiérrez JM, Cano R, Cofiño AS, Sordo C (2005) Analysis and downscaling multi-model seasonal forecasts in Peru using self-organizing maps. Tellus A 57(3):435–447. CrossRefGoogle Scholar
  12. Gutiérrez JM, Maraun D, Widmann M, Huth R, Hertig E, Benestad R, Roessler O, Wibig J, Wilcke R, Kotlarski S, San Martín D, Herrera S, Bedia J, Casanueva A, Manzanas R, Iturbide M, Vrac M, Dubrovsky M, Ribalaygua J, Pórtoles J, Räty O, Räisänen J, Hingray B, Raynaud D, Casado MJ, Ramos P, Zerenner T, Turco M, Bosshard T, S̆tĕpánek P, Bartholy J, Pongracz R, Keller DE, Fischer AM, Cardoso RM, Soares PMM, Czernecki B, Pagé C (2018) An intercomparison of a large ensemble of statistical downscaling methods over Europe: results from the VALUE perfect predictor crossvalidation experiment. Int J Climatol.
  13. Herrera S, Kotlarski S, Soares PMM, Cardoso RM, Jaczewski A, Gutiérrez JM, Maraun D (2018) Uncertainty in gridded precipitation products: influence of station density, interpolation method and grid resolution. Int J Climatol.
  14. Hersbach H (2000) Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather Forecast 15(5):559–570.<0559:DOTCRP>2.0.CO;2 CrossRefGoogle Scholar
  15. Iturbide M, Bedia J, Herrera S, Baño J, Fernández J, Frías MD, Manzanas R, San-Martín D, Cimadevilla E, Cofiño AS, Gutiérrez JM (2019) The R-based climate4R open framework for reproducible climate data access and post-processing. Environ Model Softw 111:42–54. CrossRefGoogle Scholar
  16. Kharin VV, Zwiers FW (2003) On the ROC score of probability forecasts. J Clim 16(24):4145–4150.<4145:OTRSOP>2.0.CO;2 CrossRefGoogle Scholar
  17. Kotlarski S, Szabó P, Herrera S, Räty O, Keuler K, Soares PMM, Cardoso RM, Bosshard T, Pagé C, Boberg F, Gutiérrez JM, Isotta FA, Jaczewski A, Kreienkamp F, Liniger MA, Lussana C, Pianko-Kluczyńska K (2017) Observational uncertainty and regional climate model evaluation: a pan-European perspective. Int J Climatol.
  18. Lachenbruch PA, Mickey MR (1968) Estimation of error rates in discriminant analysis. Technometrics 10(1):1–11. CrossRefGoogle Scholar
  19. Leung LR, Hamlet AF, Lettenmaier DP, Kumar A (1999) Simulations of the ENSO hydroclimate signals in the Pacific Northwest Columbia river basin. Bull Am Meteorol Soc 80(11):2313–2330.<2313:SOTEHS>2.0.CO;2 CrossRefGoogle Scholar
  20. Manzanas R, Gutiérrez JM (2018) Process-conditioned bias correction for seasonal forecasting: a case-study with ENSO in Peru. Clim Dyn.
  21. Manzanas R, Frías MD, Cofiño AS, Gutiérrez JM (2014) Validation of 40 year multimodel seasonal precipitation forecasts: The role of ENSO on the global skill. J Geophys Res Atmos 119(4):1708–1719. CrossRefGoogle Scholar
  22. Manzanas R, Gutiérrez JM, Fernández J, van Meijgaard E, Calmanti S, Magariño ME, Cofiño AS, Herrera S (2017) Dynamical and statistical downscaling of seasonal temperature forecasts in Europe: added value for user applications. Clim Serv.
  23. Manzanas R, Lucero A, Weisheimer A, Gutiérrez JM (2018) Can bias correction and statistical downscaling methods improve the skill of seasonal precipitation forecasts? Clim Dyn 50(3):1161–1176. CrossRefGoogle Scholar
  24. Maraun D, Wetterhall F, Ireson AM, Chandler RE, Kendon EJ, Widmann M, Brienen S, Rust HW, Sauter T, Themessl M, Venema VKC, Chun KP, Goodess CM, Jones RG, Onof C, Vrac M, Thiele-Eich I (2010) Precipitation downscaling under climate change: recent developments to bridge the gap between dynamical models and the end user. Rev Geophys 48:3. CrossRefGoogle Scholar
  25. Maraun D, Shepherd TG, Widmann M, Zappa G, Walton D, Mayr GJ, Hagemann S, Richter I, Soares PMM, Hall A, Mearns LO, (2017) Towards process-informed bias correction of climate change simulations. Nat Clim Change 7:764–773.
  26. Marcos R, Llasat MC, Quintana-Seguí P, Turco M (2018) Use of bias correction techniques to improve seasonal forecasts for reservoirs: a case-study in northwestern Mediterranean. Scie Total Environ 610–611:64–74. CrossRefGoogle Scholar
  27. Markus D, Mayr GJ, Achim Z (2017) Spatial ensemble postprocessing with standardized anomalies. Q J R Meteorol Soc 143(703):909–916. CrossRefGoogle Scholar
  28. Molteni F, Stockdale T, Balmaseda M, Balsamo G, Buizza R, Ferranti L, Magnusson L, Mogensen K, Palmer T, Vitart F (2011) The new ECMWF seasonal forecast system (System 4). European Centre for Medium-Range Weather Forecasts.
  29. Murphy AH (1973) A new vector partition of the probability score. J Appl Meteorol 12(4):595–600.<0595:ANVPOT>2.0.CO;2 CrossRefGoogle Scholar
  30. Nikulin G, Asharafb S, Magariño ME, Calmanti S, Cardoso RM, Bhend J, Fernández J, Frías MD, Fröhlichb K, Frühb B, Herrera S, Manzanas R, Gutiérrez JM, Hanssona U, Kolaxa M, Liniger M, Soares PMM, Spirig C, Tome R, Wysera K (2018) Dynamical and statistical downscaling of a global seasonal hindcast in eastern Africa. Clim Serv 9:72–85. CrossRefGoogle Scholar
  31. Pavan V, Marchesi S, Morgillo A, Cacciamani C, Doblas-Reyes FJ (2005) Downscaling of DEMETER winter seasonal hindcasts over Northern Italy. Tellus A 57(3):424–434. CrossRefGoogle Scholar
  32. Piani C, Haerter JO, Coppola E (2010) Statistical bias correction for daily precipitation in regional climate models over Europe. Theor Appl Climatol 99(1–2):187–192. CrossRefGoogle Scholar
  33. Sansom PG, Ferro CAT, Stephenson DB, Goddard L, Mason SJ (2016) Best practices for postprocessing ensemble climate forecasts. Part I: Selecting appropriate recalibration methods. J Clim 29(20):7247–7264. CrossRefGoogle Scholar
  34. Scheuerer M, Möller D (2015) Probabilistic wind speed forecasting on a grid based on ensemble model output statistics. Ann Appl Stat 9(3):1328–1349. CrossRefGoogle Scholar
  35. Sheau TN, Tangang F, Juneng L (2017) Bias correction of global and regional simulated daily precipitation and surface mean temperature over Southeast Asia using quantile mapping method. Glob Planet Change 149:79–90. CrossRefGoogle Scholar
  36. Smith DM, Eade R, Pohlmann H (2013) A comparison of full-field and anomaly initialization for seasonal to decadal climate prediction. Clim Dyn 41(11–12):3325–3338. CrossRefGoogle Scholar
  37. Thorarinsdottir TL, Johnson MS (2012) Probabilistic wind gust forecasting using non-homogeneous Gaussian regression. Mon Weather Rev 140(3):889–897. CrossRefGoogle Scholar
  38. Tippett MK, Barnston AG (2008) Skill of multimodel ENSO probability forecasts. Mon Weather Rev 136(10):3933–3946. CrossRefGoogle Scholar
  39. Torralba V, Doblas-Reyes FJ, MacLeod D, Christel I, Davis M (2017) Seasonal climate prediction: a new source of information for the management of wind energy resources. J Appl Meteorol Climatol 56(5):1231–1247. CrossRefGoogle Scholar
  40. Weigel AP, Liniger MA, Appenzeller C (2009) Seasonal ensemble forecasts: are recalibrated single models better than multimodels? Mon Weather Rev 137(4):1460–1479. CrossRefGoogle Scholar
  41. Weisheimer A, Palmer TN (2014) On the reliability of seasonal climate forecasts. J R Soc Interface 11:96. CrossRefGoogle Scholar
  42. Wilks DS, Hamill TM (2007) Comparison of ensemble-MOS methods using GFS reforecasts. Mon Weather Rev 135(6):2379–2390. CrossRefGoogle Scholar
  43. Zhao T, Bennett JC, Wang QJ, Schepen A, Wood AW, Robertson DE, Ramos MH (2017) How suitable is quantile mapping for postprocessing GCM precipitation forecasts? J Clim 30(9):3185–3196. CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • R. Manzanas
    • 1
    Email author
  • J. M. Gutiérrez
    • 1
  • J. Bhend
    • 2
  • S. Hemri
    • 2
  • F. J. Doblas-Reyes
    • 3
    • 4
  • V. Torralba
    • 3
  • E. Penabad
    • 5
  • A. Brookshaw
    • 5
  1. 1.Meteorology Group, Institute of Physics of Cantabria (IFCA)CSIC-University of CantabriaSantanderSpain
  2. 2.Federal Office of Meteorology and Climatology MeteoSwissZurichSwitzerland
  3. 3.Barcelona Supercomputing Center (BSC)BarcelonaSpain
  4. 4.ICREA, Pg. Lluís CompanysBarcelonaSpain
  5. 5.European Centre for Medium-Range Weather Forecasts (ECMWF)ReadingUK

Personalised recommendations