Climate Dynamics

, Volume 41, Issue 3–4, pp 885–900 | Cite as

Climate model dependence and the replicate Earth paradigm

  • Craig H. Bishop
  • Gab AbramowitzEmail author


Multi-model ensembles are commonly used in climate prediction to create a set of independent estimates, and so better gauge the likelihood of particular outcomes and better quantify prediction uncertainty. Yet researchers share literature, datasets and model code—to what extent do different simulations constitute independent estimates? What is the relationship between model performance and independence? We show that error correlation provides a natural empirical basis for defining model dependence and derive a weighting strategy that accounts for dependence in experiments where the multi-model mean would otherwise be used. We introduce the “replicate Earth” ensemble interpretation framework, based on theoretically derived statistical relationships between ensembles of perfect models (replicate Earths) and observations. We transform an ensemble of (imperfect) climate projections into an ensemble whose mean and variance have the same statistical relationship to observations as an ensemble of replicate Earths. The approach can be used with multi-model ensembles that have varying numbers of simulations from different models, accounting for model dependence. We use HadCRUT3 data and the CMIP3 models to show that in out of sample tests, the transformed ensemble has an ensemble mean with significantly lower error and much flatter rank frequency histograms than the original ensemble.


Climate model ensembles Model independence Climate uncertainty quantification Climate model bias correction 



The CMIP3 modelling groups, PCMDI and the WCRP’s Working Group on Coupled Modelling (WGCM) for making the WCRP CMIP3 multi-model dataset available—support is provided by the Office of Science, U.S. Department of Energy. CHB was supported by the U.S. Office of Naval Research Grant# 4304-D-0-5. We also thank an anonymous reviewer for providing extensive and constructive feedback.

Supplementary material

382_2012_1610_MOESM1_ESM.docx (2.2 mb)
Supplementary material 1 (DOCX 2.18 MB)


  1. Annan JD, Hargreaves JC (2010) Reliability of the CMIP3 ensemble. Geophys Res Lett 37. doi: 10.1029/2009gl041994
  2. Annan JD, Hargreaves JC (2011) Understanding the CMIP3 ensemble. J Clim 24:4529–4538. doi: 10.1175/2011JCLI3873.1 CrossRefGoogle Scholar
  3. Brohan P, Kennedy JJ, Harris I, Tett SFB, Jones PD (2006) Uncertainty estimates in regional and global observed temperature changes: a new dataset from 1850. J Geophys Res 111:D12106CrossRefGoogle Scholar
  4. Collins M, Booth BB, Bhaskaran B, Harris GR, Murphy JM, Sexton DMH, Webb MJ (2010) Climate model errors, feedbacks and forcings: a comparison of perturbed physics and multi-model ensembles. Clim Dyn 36:1737–1766. doi: 10.1007/s00382-010-0808-0 CrossRefGoogle Scholar
  5. DelSole T (2007) A Bayesian framework for multimodel regression. J Clim 20:2810–2826CrossRefGoogle Scholar
  6. Doblas-Reyes FJ, Hagedorn R, Palmer TN (2005) The rationale behind the success of multi-model ensembles in seasonal forecasting. Part II: calibration and combination. Tellus 57A:234–252Google Scholar
  7. Furrer R, Knutti R, Sain SR, Nychka DW, Meehl GA (2007) Spatial patterns of probabilistic temperature change projections from a multi- variate Bayesian analysis. Geophys Res Lett 34:L06711. doi: 10.1029/2006GL027754 CrossRefGoogle Scholar
  8. Giorgi F, Mearns L (2002) Calculation of average, uncertainty range and reliability of regional climate changes from AOGCM simulations via the ‘reliability ensemble averaging’ (REA) method. J Clim 15:1141–1158CrossRefGoogle Scholar
  9. Glahn HR, Lowry DA (1972) The use of model output statistics (MOS) in objective weather forecasting. J Appl Meteor 11:1203–1211CrossRefGoogle Scholar
  10. Gleckler PJ, Taylor KE, Doutriaux C (2008) Performance metrics for climate models. J Geophys Res 113:D06104CrossRefGoogle Scholar
  11. Gneiting T, Raftery AE (2005) Weather forecasting with ensemble methods. Science 310:248–249CrossRefGoogle Scholar
  12. Greene AM, Goddard L, Lall U (2006) Probabilistic multimodel regional temperature change projections. J Clim 19:4326–4346CrossRefGoogle Scholar
  13. Jewson S, Hawkins E (2009) CMIP3 ensemble spread, model similarity, and climate prediction uncertainty.
  14. Jun M, Knutti R, Nychka D (2008) Spatial analysis to quantify numerical model bias and dependence: how many climate models are there? J Am Stat Assoc 103:934–947CrossRefGoogle Scholar
  15. Knutti R, Abramowitz G, Collins M, Eyring V, Gleckler PJ, Hewitson B, Mearns L (2010) Good practice guidance paper on assessing and combining multi model climate projections. In: Stocker TF, Qin D, Plattner G-K, Tignor M, Midgley PM (eds) Meeting report of the intergovernmental panel on climate change expert meeting on assessing and combining multi model climate projections. IPCC Working Group I Technical Support Unit, University of Bern, Bern, SwitzerlandGoogle Scholar
  16. Knutti R, Furrer R, Tebaldi C, Cermak J, Meehl GA (2010b) Challenges in combining projections from multiple models. J Clim 23:2739–2758CrossRefGoogle Scholar
  17. Krishnamurti TN, Kishtawal CM, Zhang Z, Larow T, Bachiochi D, Williford E, Gadgil S, Surendran S (2000) Multimodel ensemble forecasts for weather and seasonal climate. J Clim 13:4196–4216CrossRefGoogle Scholar
  18. Lambert SJ, Boer GJ (2001) CMIP1 evaluation and intercomparison of coupled climate models. Clim Dyn 17:83–106CrossRefGoogle Scholar
  19. Leutbecher M, Palmer TN (2008) Ensemble forecasting. J Comput Phys 227:3515–3539CrossRefGoogle Scholar
  20. Masson D, Knutti R (2011) Climate model genealogy. Geophys Res Lett 38:L08703. doi: 10.1029/2011GL046864 CrossRefGoogle Scholar
  21. Meehl GA et al (2007) Climate change 2007: the physical science basis. In: Solomon S et al (ed) Contribution of working group I to the fourth assessment report of the intergovernmental panel on climate change. Cambridge University Press, CambridgeGoogle Scholar
  22. Meehl GA, Covey C, Delworth T, Latif M, McAvaney B, Mitchell JFB, Stouffer RJ, Taylor KE (2007) The WCRP CMIP3 multi-model dataset: a new era in climate change research. Bull Am Meteorol Soc 88:1383–1394CrossRefGoogle Scholar
  23. Palmer TN, Doblas-Reyes FJ, Weisheimer A, Rodwell MJ (2008) Toward seamless prediction: calibration of climate change projections using seasonal forecasts. Bull Am Meteor Soc 89:459–470CrossRefGoogle Scholar
  24. Pennell C, Reichler T (2011) On the effective number of climate models. J Clim 24:2358–2367. doi: 10.1175/2010JCLI3814.1 CrossRefGoogle Scholar
  25. Perkins SE, Pitman AJ, Holbrook NJ, McAneney J (2007) Evaluation of the AR4 climate models’ simulated daily maximum temperature, minimum temperature, and precipitation over Australia using probability density functions. J Clim 20:4356–4376CrossRefGoogle Scholar
  26. Reifen C, Toumi R (2009) Climate projections: past performance no guarantee of future skill? Geophys Res Lett 36:L13704CrossRefGoogle Scholar
  27. Schär C, Vidale PL, Lüthi D, Frei C, Häberli C, Liniger MA, Appenzeller C (2004) The role of increasing temperature variability in European summer heatwaves. Nature 427:332–336CrossRefGoogle Scholar
  28. Smith RL, Tebaldi C, Nychka DW, Mearns LO (2009) Bayesian modeling of uncertainty in ensembles of climate models. J Am Stat Assoc 104:97–116CrossRefGoogle Scholar
  29. Snyder C, Bengtsson T, Bickel P, Anderson JL (2008) Obstacles to high-dimensional particle filtering. Mon Weather Rev 136:4629–4640CrossRefGoogle Scholar
  30. Tebaldi C, Knutti R (2007) The use of the multimodel ensemble in probabilistic climate projections. Philos Transact R Soc A 365:2053–2075CrossRefGoogle Scholar
  31. Tebaldi C, Smith RW, Nychka D, Mearns LO (2005) Quantifying uncertainty in projections of regional climate change: a Bayesian approach to the analysis of multi-model ensembles. J Clim 18:1524–1540CrossRefGoogle Scholar
  32. van Leeuwen PJ (2009) Particle filtering in geophysical systems. Mon Weather Rev 137:4089–4114CrossRefGoogle Scholar
  33. Weigel AP, Knutti R, Liniger MA, Appenzeller C (2010) Risks of model weighting in multimodel climate projections. J Clim 23:4175–4191CrossRefGoogle Scholar
  34. Wilson LJ, Vallée M (2002) The Canadian updateable model output statistics (UMOS) system: design and development tests. Weather Forecast 17:206–222CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Naval Research Laboratory, Marine Meteorology DivisionMontereyUSA
  2. 2.Climate Change Research Centre, ARC Centre of Excellence for Climate System ScienceUniversity of New South WalesSydneyAustralia

Personalised recommendations