Skip to main content

Advertisement

Log in

Calibration and combination of dynamical seasonal forecasts to enhance the value of predicted probabilities for managing risk

  • Published:
Climate Dynamics Aims and scope Submit manuscript

Abstract

Seasonal probability forecasts produced with numerical dynamics on supercomputers offer great potential value in managing risk and opportunity created by seasonal variability. The skill and reliability of contemporary forecast systems can be increased by calibration methods that use the historical performance of the forecast system to improve the ongoing real-time forecasts. Two calibration methods are applied to seasonal surface temperature forecasts of the US National Weather Service, the European Centre for Medium Range Weather Forecasts, and to a World Climate Service multi-model ensemble created by combining those two forecasts with Bayesian methods. As expected, the multi-model is somewhat more skillful and more reliable than the original models taken alone. The potential value of the multimodel in decision making is illustrated with the profits achieved in simulated trading of a weather derivative. In addition to examining the seasonal models, the article demonstrates that calibrated probability forecasts of weekly average temperatures for leads of 2–4 weeks are also skillful and reliable. The conversion of ensemble forecasts into probability distributions of impact variables is illustrated with degree days derived from the temperature forecasts. Some issues related to loss of stationarity owing to long-term warming are considered. The main conclusion of the article is that properly calibrated probabilistic forecasts possess sufficient skill and reliability to contribute to effective decisions in government and business activities that are sensitive to intraseasonal and seasonal climate variability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. The World Climate Service is a commercial seasonal forecast service that provides a variety of climate information, analog construction tools, a monthly forecast and diagnostic document, and calibrated monthly forecasts of the NWS CFSv2, the ECMWF SFSv4, and the WCS multimodel ensemble at http://www.worldclimateservice.com. The WCS is a joint enterprise of Prescient Weather Ltd and MeteoGroup, an international independent weather and climate information provider in Europe, the US, and Asia.

  2. The CFSv2 became operational in March 2011. Historical forecasts for 2010 were not available when these computations were initiated.

  3. Our results may appear to differ from other studies of these two models (e.g., Kim et al. 2012), but it is essential to observe that measures of quality will depend on model calibration methods and the verification datasets used. Kim et al. focus on model bias; we focus on the probabilistic forecasts.

  4. Some of the empty bins actually contained a small number of correct high probability forecasts that were eliminated by the criterion that bins must have 20 total forecasts or more, suggesting that longer historical periods might be advantageous.

  5. Taylor and Buizza (2006) applied a similar approach, using ten-day ensemble forecasts to determine the pay-off probabilities of weather derivatives.

References

  • Climate Prediction Center (2011a) Toward a national multi-model ensemble (NMME) system for operational intra-seasonal to interannual (ISI) climate forecasts. http://www.cpc.ncep.noaa.gov/products/ctb/MMEWhitePaperCPO_revised.pdf

  • Climate Prediction Center (2011b) NCEP-DOE AMIP-II Reanalysis. http://www.cpc.ncep.noaa.gov/products/wesley/reanalysis2/index.html

  • Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc 39B:1–39

    Google Scholar 

  • Doblas-Reyes FJ, Hagedorn R, Palmer TN (2005) The rationale behind multi-model ensembles in seasonal forecasting—II. Calibration and combination. Tellus 57A:234–252

    Google Scholar 

  • ECMWF (2012) The EUROSIP seasonal forecasting system. http://www.ecmwf.int/products/forecasts/seasonal/documentation/eurosip/index.html

  • Johnson C, Bowler N (2009) On the reliability and calibration of ensemble forecasts. Mon Wea Rev 137:1717–1720

    Article  Google Scholar 

  • Kim HM, Webster PJ, Curry JA (2012) Seasonal prediction skill of ECMWF System 4 and NCEP CFSv2 retrospective forecast for the Northern Hemisphere Winter. Clim Dyn 39:2957–2973

    Article  Google Scholar 

  • Livezey RE, Timofeyeva MM (2008) The first decade of long-lead US seasonal forecasts: insights from a skill analysis. Bull Am Meteorol Soc 89:843–853

    Article  Google Scholar 

  • Molteni F et al (2011) The new ECMWF seasonal forecast system (System 4), ECMWF Technical Memorandum 656. http://www.ecmwf.int/publications/library/do/references/list/14

  • Palmer TN (2004) Development of a European multi-model ensemble system for seasonal to inter-seasonal annual prediction (DEMETER). Bull Am Meteorol Soc 85:853–872

    Article  Google Scholar 

  • Raftery AE, Gneiting T, Balabdaoui F, Polakowski M (2005) Using Bayesian model averaging to calibrate forecast ensembles. Mon Weather Rev 133:1155–1174

    Article  Google Scholar 

  • Saha S et al (2010) The NCEP climate forecast system reanalysis. Bull Am Meteorol Soc 91:1015–1057

    Article  Google Scholar 

  • Saha S et al (2013) The NCEP climate forecast system version 2, submitted to J Clim manuscript available at http://cfs.ncep.noaa.gov/cfsv2.info/CFSv2_paper.pdf

  • Taylor JW, Buizza R (2006) Density forecasting for weather derivative pricing. Int J Forecast 22:29–42

    Article  Google Scholar 

  • Van den Dool H (2007) Empirical methods in short-term climate prediction. Oxford University Press, Oxford

    Google Scholar 

  • Wilks DS (2006) Statistical methods in the atmospheric sciences, 2nd edn. Academic Press, Amsterdam

    Google Scholar 

  • Xue Y, M Chen, A Kumar, Z Hu, W. Wang (2013) Prediction skill and bias of tropical pacific sea surface temperatures in the NCEP climate forecast system version 2. J Clim (accepted)

  • Zhang L, Kumar A, Wang W (2012) Influence of changes in observations on precipitation: a case study for the climate forecast system reanalysis (CFSR). J Geophys Res 117:D08105. doi:10.1029/2011JD017347

    Article  Google Scholar 

Download references

Acknowledgments

The research reported here was supported in part by Small Business Innovation Research (SBIR) Phase I and Phase II Grants from the National Oceanic and Atmospheric Administration (NOAA). Mark S. Roulston, now with Winton Capital in Oxford, introduced us to the Gaussian comb and Bayesian calibration method when we started calibrating seasonal forecasts some years ago. Comments by Roberto Buizza of the European Centre for Medium-Range Weather Forecasts (ECMWF) and an anonymous reviewer led to improvements in the article.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to John A. Dutton.

Additional information

This paper is a contribution to the Topical Collection on Climate Forecast System Version 2 (CFSv2), which is a coupled global climate model and was implemented by National Centers for Environmental Prediction (NCEP) in seasonal forecasting operations in March 2011. This Topical Collection is coordinated by Jin Huang, Arun Kumar, Jim Kinter and Annarita Mariotti.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dutton, J.A., James, R.P. & Ross, J.D. Calibration and combination of dynamical seasonal forecasts to enhance the value of predicted probabilities for managing risk. Clim Dyn 40, 3089–3105 (2013). https://doi.org/10.1007/s00382-013-1764-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00382-013-1764-2

Keywords

Navigation