Abstract
A sequence of ensemble forecasts permits the estimation of the “predictable” and “unpredictable” components of the forecasts and hence the predictability of the system. Predictability is a feature of a physical and/or mathematical system which characterizes its “ability to be predicted” as measured, for instance, by the rate at which initially close states separate. Modern climate models may be used to predict climate evolution on seasonal to decadal timescales and they may also be used to estimate predictability. The predictability in this case measures the model’s ability to predict its own evolution and, for a well behaved climate model, this may be a useful measure of the predictability of the climate system itself. Forecast skill, where forecasts are compared to observations, indicates a forecast system’s “ability to predict” the evolution of the climate system. Common measures of skill include correlation, mean square error and mean square skill score. When the model is used to predict its own evolution, rather than the evolution of the actual climate system, the same measures of skill may be used but are then predictability measures and termed measures of “potential skill”. The expectation is that the model’s skill in predicting its own evolution will be greater than its ability to predict the actual evolution of the system in the face of observational uncertainty and model deficiencies. The further expectation is that the differences between potential and actual skill can give an indication of where and to what extent there is the potential for improvement in forecast skill. Predictability studies may be undertaken with available models but it may not be immediately obvious which statistical conditions are necessary if potential skill is to be a reasonable estimate of attainable skill. The purpose of this paper is to investigate and illustrate the spatial and temporal behaviour of the predictable and unpredictable components of surface air temperature in a decadal prediction experiment and to discuss some of the implications for predictability studies and for analyzing model behaviour. Formal relationships between potential, actual, and attainable skill are developed which, however, provide only modest constraints on model results if they are to be used to infer predictability. The results of various scalings of the forecasts on potential and actual skill and their relationship are also considered. Differences in the actual and potential forecast skill between models and/or model versions can be partitioned into components associated with known statistics as a guide to forecast improvement.
Similar content being viewed by others
References
Boer GJ, Kharin VV, Merryfield WJ (2018) Differences in potential and actual skill in a decadal prediction experiment. Clim Dyn (accepted for publication)
Boer GJ, Lambert SJ (2008) Multi-model decadal potential predictability of precipitation and temperature. Geophys Res Lett. https://doi.org/10.1029/2008GL033234
Boer GJ (2004) Long time-scale potential predictability in an ensemble of coupled climate models. Clim Dyn 23:29–44
Boer GJ (2009) Climate trends in a seasonal forecasting system. Atmos Ocean 47:123–138. https://doi.org/10.3137/AO1002.2009
Boer GJ et al (2016) The decadal climate prediction project (DCPP) contribution to CMIP6. Geosci Model Dev 9:3751–3777. https://doi.org/10.5194/gmd-9-3751-2016
Boer GJ, Kharin VV, Merryfield WJ (2013) Decadal predictability and forecast skill. Clim Dyn 41:1817–1833. https://doi.org/10.1007/s00382-013-1705-0
Deque M (2003) Continuous variables. In: Jolliffe I, Stephenson D (eds) Forecast verification: a practitioner’s guide in atmospheric science. Wiley, Chichester, pp 97–119
Doblas-Reyes FJ, Hagedorn R, Palmer TN, Morcrette JJ (2005) Impact of increasing greenhouse gas concentrations in seasonal ensemble forecasts. Geophys Res Lett 33:L07708. https://doi.org/10.1029/2005GL025061
Eade R, Smith D, Scaife A, Wallace E, Dunstone N, Hermanson L, Robinson N (2014) Do seasonal-to- decadal climate predictions under- estimate the predictability of the real world? Geophys Res Lett. https://doi.org/10.1002/2014GL061146
Eyring V, Bony S, Meehl G, Senior C, Stevens B, Stouffer R, Taylor K (2016) Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci Model Dev 9:1937–1958. https://doi.org/10.5194/gmd-9-1937-2016
Feng X, DelSole T, Houser P (2013) Comparison of statistical estimates of potential seasonal predictability. J Geophys Res Atmos. 118:6002–6016. https://doi.org/10.1002/jgrd.50498
Flato G et al (2013) Evaluation of climate models. In: Stocker TF, Qin D, Plattner G-K, Tignor M, Allen SK, Boschung J, Nauels A, Xia Y, Bex V, Midgley PM (eds) Climate change 2013: the physical science basis. Contribution of working group I to the fifth assessment report of the intergovernmental panel on climatechange. Cambridge University Press, Cambridge, New York
Hartmann DL et al (2013) Observations: Atmosphere and Surface. In: Stocker TF, Qin D, Plattner G-K, Tignor M, Allen SK, Boschung J, Nauels A, Xia Y, Bex V, Midgley PM (eds) Climate change 2013: the physical science basis. Contribution of Working Group I to the fifth assessment report of the intergovernmental panel on climate change. Cambridge University Press, Cambridge, New York
Jolliffe I, Stephenson D (eds) (2003) Forecast verification: a practitioner’s guide in atmospheric science. Wiley, Chichester
Kharin VV, Teng Q, Zwiers FW, Boer GJ, Derome J, Fontecilla JF (2009) Skill assessment of seasonal hindcasts from the Canadian historical forecast project. Atmos Ocean 47:204–223. https://doi.org/10.3137/AO1101.2009
Kharin VV, Merryfield WJ, Boer GJ, Lee WS (2017) A post-processing method for seasonal forecasts using temporally and spatially smoothed statistics. Mon Weather Rev 145:3545–3561. https://doi.org/10.1175/MWR-D-16-0337.1
Kumar A, Peng P, Chen M (2014) Is there a relationship between potential and actual skill? Mon Weather Rev 142:2220–2227. https://doi.org/10.1175/MWR-D-13-00287.1
Mailier PJ, Jolliffe IT, Stephenson DB (2006) Quality of weather forecasts: review and recommendations. Royal Meteorological Society Project Report, p 89
Merryfield WJ et al (2013) The Canadian seasonal to interannual prediction system. Part I: models and initialization. Mon Weather Rev 141:2910–2945. https://doi.org/10.1175/MWR-D-12-00216.1
National Research Council (U.S.) (2010) Assessment of intraseasonal to interannual climate prediction and predictability. National Academies Press, Washington, D.C
Palmer TN, Buizza R, Hagedorn R, Lawrence A, Leutbecher M, Smith L (2006) Ensemble prediction: a pedagogical perspective. ECMWF Newslett 106:10–17
Pohlmann H, Botzet M, Latif M, Rioesch A, Wild M, Schuck P (2004) Estimating the decadal predictability of a coupled AOGCM. J Clim 17:4463–4472. https://doi.org/10.1175/3209.1
Scaife AA et al (2014) Skillful long-range prediction of European and North American winters. Geophys Res Lett 41:2514–2519. https://doi.org/10.1002/2014GL059637
Stockdale TN, Molteni F, Ferranti L (2015) Atmospheric initial conditions and the predictability of the Arctic Oscillation. Geophys Res Lett 42:1173–1179. https://doi.org/10.1002/2014GL062681
Storch vH, Zwiers FW (1999) Statistical analysis in climate research. Cambridge University Press, Cambridge
Toth Z, Talagrand O, Candille G, Zhu Y, (2003) Chapter 7, Probability and ensemble forecasts. In: Forecast verification: a practitioner’s guide in atmospheric science. Wiley, Chichester, p 240
Weisheimer A, Doblas-Reyes FJ, Palmer TN, Alessandri A, Arribas A, Deque M, Keenlyside N, MacVean M, Navarra A, Rogel P (2009) ENSEMBLES—a new multi-model ensemble for seasonal-to-annual predictions: skill and progress beyond DEMETER in forecasting tropical Pacific SST. Geophys Res Lett 36:L21711. https://doi.org/10.1029/2009GL040896
Acknowledgements
We are pleased to acknowledge the important contributions of many members of the CCCma team in the development of the model and the forecasting system that has led to this investigation and to Woo-Sung Lee for her contribution in producing the forecasts.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Boer, G.J., Merryfield, W.J. & Kharin, V.V. Relationships between potential, attainable, and actual skill in a decadal prediction experiment. Clim Dyn 52, 4813–4831 (2019). https://doi.org/10.1007/s00382-018-4417-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00382-018-4417-7