Abstract
Seasonal probability forecasts produced with numerical dynamics on supercomputers offer great potential value in managing risk and opportunity created by seasonal variability. The skill and reliability of contemporary forecast systems can be increased by calibration methods that use the historical performance of the forecast system to improve the ongoing real-time forecasts. Two calibration methods are applied to seasonal surface temperature forecasts of the US National Weather Service, the European Centre for Medium Range Weather Forecasts, and to a World Climate Service multi-model ensemble created by combining those two forecasts with Bayesian methods. As expected, the multi-model is somewhat more skillful and more reliable than the original models taken alone. The potential value of the multimodel in decision making is illustrated with the profits achieved in simulated trading of a weather derivative. In addition to examining the seasonal models, the article demonstrates that calibrated probability forecasts of weekly average temperatures for leads of 2–4 weeks are also skillful and reliable. The conversion of ensemble forecasts into probability distributions of impact variables is illustrated with degree days derived from the temperature forecasts. Some issues related to loss of stationarity owing to long-term warming are considered. The main conclusion of the article is that properly calibrated probabilistic forecasts possess sufficient skill and reliability to contribute to effective decisions in government and business activities that are sensitive to intraseasonal and seasonal climate variability.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The World Climate Service is a commercial seasonal forecast service that provides a variety of climate information, analog construction tools, a monthly forecast and diagnostic document, and calibrated monthly forecasts of the NWS CFSv2, the ECMWF SFSv4, and the WCS multimodel ensemble at http://www.worldclimateservice.com. The WCS is a joint enterprise of Prescient Weather Ltd and MeteoGroup, an international independent weather and climate information provider in Europe, the US, and Asia.
The CFSv2 became operational in March 2011. Historical forecasts for 2010 were not available when these computations were initiated.
Our results may appear to differ from other studies of these two models (e.g., Kim et al. 2012), but it is essential to observe that measures of quality will depend on model calibration methods and the verification datasets used. Kim et al. focus on model bias; we focus on the probabilistic forecasts.
Some of the empty bins actually contained a small number of correct high probability forecasts that were eliminated by the criterion that bins must have 20 total forecasts or more, suggesting that longer historical periods might be advantageous.
Taylor and Buizza (2006) applied a similar approach, using ten-day ensemble forecasts to determine the pay-off probabilities of weather derivatives.
References
Climate Prediction Center (2011a) Toward a national multi-model ensemble (NMME) system for operational intra-seasonal to interannual (ISI) climate forecasts. http://www.cpc.ncep.noaa.gov/products/ctb/MMEWhitePaperCPO_revised.pdf
Climate Prediction Center (2011b) NCEP-DOE AMIP-II Reanalysis. http://www.cpc.ncep.noaa.gov/products/wesley/reanalysis2/index.html
Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc 39B:1–39
Doblas-Reyes FJ, Hagedorn R, Palmer TN (2005) The rationale behind multi-model ensembles in seasonal forecasting—II. Calibration and combination. Tellus 57A:234–252
ECMWF (2012) The EUROSIP seasonal forecasting system. http://www.ecmwf.int/products/forecasts/seasonal/documentation/eurosip/index.html
Johnson C, Bowler N (2009) On the reliability and calibration of ensemble forecasts. Mon Wea Rev 137:1717–1720
Kim HM, Webster PJ, Curry JA (2012) Seasonal prediction skill of ECMWF System 4 and NCEP CFSv2 retrospective forecast for the Northern Hemisphere Winter. Clim Dyn 39:2957–2973
Livezey RE, Timofeyeva MM (2008) The first decade of long-lead US seasonal forecasts: insights from a skill analysis. Bull Am Meteorol Soc 89:843–853
Molteni F et al (2011) The new ECMWF seasonal forecast system (System 4), ECMWF Technical Memorandum 656. http://www.ecmwf.int/publications/library/do/references/list/14
Palmer TN (2004) Development of a European multi-model ensemble system for seasonal to inter-seasonal annual prediction (DEMETER). Bull Am Meteorol Soc 85:853–872
Raftery AE, Gneiting T, Balabdaoui F, Polakowski M (2005) Using Bayesian model averaging to calibrate forecast ensembles. Mon Weather Rev 133:1155–1174
Saha S et al (2010) The NCEP climate forecast system reanalysis. Bull Am Meteorol Soc 91:1015–1057
Saha S et al (2013) The NCEP climate forecast system version 2, submitted to J Clim manuscript available at http://cfs.ncep.noaa.gov/cfsv2.info/CFSv2_paper.pdf
Taylor JW, Buizza R (2006) Density forecasting for weather derivative pricing. Int J Forecast 22:29–42
Van den Dool H (2007) Empirical methods in short-term climate prediction. Oxford University Press, Oxford
Wilks DS (2006) Statistical methods in the atmospheric sciences, 2nd edn. Academic Press, Amsterdam
Xue Y, M Chen, A Kumar, Z Hu, W. Wang (2013) Prediction skill and bias of tropical pacific sea surface temperatures in the NCEP climate forecast system version 2. J Clim (accepted)
Zhang L, Kumar A, Wang W (2012) Influence of changes in observations on precipitation: a case study for the climate forecast system reanalysis (CFSR). J Geophys Res 117:D08105. doi:10.1029/2011JD017347
Acknowledgments
The research reported here was supported in part by Small Business Innovation Research (SBIR) Phase I and Phase II Grants from the National Oceanic and Atmospheric Administration (NOAA). Mark S. Roulston, now with Winton Capital in Oxford, introduced us to the Gaussian comb and Bayesian calibration method when we started calibrating seasonal forecasts some years ago. Comments by Roberto Buizza of the European Centre for Medium-Range Weather Forecasts (ECMWF) and an anonymous reviewer led to improvements in the article.
Author information
Authors and Affiliations
Corresponding author
Additional information
This paper is a contribution to the Topical Collection on Climate Forecast System Version 2 (CFSv2), which is a coupled global climate model and was implemented by National Centers for Environmental Prediction (NCEP) in seasonal forecasting operations in March 2011. This Topical Collection is coordinated by Jin Huang, Arun Kumar, Jim Kinter and Annarita Mariotti.
Rights and permissions
About this article
Cite this article
Dutton, J.A., James, R.P. & Ross, J.D. Calibration and combination of dynamical seasonal forecasts to enhance the value of predicted probabilities for managing risk. Clim Dyn 40, 3089–3105 (2013). https://doi.org/10.1007/s00382-013-1764-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00382-013-1764-2