Climate Dynamics

, Volume 43, Issue 7–8, pp 2297–2308 | Cite as

On the generation of climate model ensembles

  • Ned Haughton
  • Gab Abramowitz
  • Andy Pitman
  • Steven J. Phipps


Climate model ensembles are used to estimate uncertainty in future projections, typically by interpreting the ensemble distribution for a particular variable probabilistically. There are, however, different ways to produce climate model ensembles that yield different results, and therefore different probabilities for a future change in a variable. Perhaps equally importantly, there are different approaches to interpreting the ensemble distribution that lead to different conclusions. Here we use a reduced-resolution climate system model to compare three common ways to generate ensembles: initial conditions perturbation, physical parameter perturbation, and structural changes. Despite these three approaches conceptually representing very different categories of uncertainty within a modelling system, when comparing simulations to observations of surface air temperature they can be very difficult to separate. Using the twentieth century CMIP5 ensemble for comparison, we show that initial conditions ensembles, in theory representing internal variability, significantly underestimate observed variance. Structural ensembles, perhaps less surprisingly, exhibit over-dispersion in simulated variance. We argue that future climate model ensembles may need to include parameter or structural perturbation members in addition to perturbed initial conditions members to ensure that they sample uncertainty due to internal variability more completely. We note that where ensembles are over- or under-dispersive, such as for the CMIP5 ensemble, estimates of uncertainty need to be treated with care.


Climate model ensembles Ensemble generation Ensemble uncertainty 


  1. Annan JD, Hargreaves JC (2010) Reliability of the CMIP3 ensemble. Geophys Res Lett 37:5. doi: 10.1029/2009GL041994.
  2. Bishop CH, Abramowitz G (2013) Climate model dependence and the replicate earth paradigm. Clim Dyn 41(3-4):885–900. doi: 10.1007/s00382-012-1610-y CrossRefGoogle Scholar
  3. Brohan P, Kennedy JJ, Harris I, Tett SFB, Jones PD (2006) Uncertainty estimates in regional and global observed temperature changes: a new data set from 1850. J Geophys Res 111(D12). doi: 10.1029/2005JD006548.
  4. Ducharne A, Laval K (2000) Influence of the realistic description of soil water-holding capacity on the global water cycle in a GCM. J Clim 13(24):4393–4413. doi: 10.1175/1520-0442(2000)013<4393:IOTRDO>2.0.CO;2.<4393%3AIOTRDO>2.0.CO%3B2 Google Scholar
  5. Fernández-Donado L, González-Rouco JF, Raible CC, Ammann CM, Barriopedro D, García-Bustamante E, Jungclaus JH, Lorenz SJ, Luterbacher J, Phipps SJ, Servonnat J, Swingedouw D, Tett SFB, Wagner S, Yiou P, Zorita E (2013) Large-scale temperature response to external forcing in simulations and reconstructions of the last millennium. Clim Past 9(1):393–421. doi: 10.5194/cp-9-393-2013.
  6. Fischer EM, Lawrence DM, Sanderson BM (2010) Quantifying uncertainties in projections of extremesa perturbed land surface parameter experiment. Clim Dyn 37(7-8):1381–1398. doi: 10.1007/s00382-010-0915-y,CrossRefGoogle Scholar
  7. Hamill TM (2001) Interpretation of rank histograms for verifying ensemble forecasts. Mon Weather Rev 129(3):550–560. doi: 10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2.<0550:IORHFV>2.0.CO%3B2
  8. Knutti R, Abramowitz G, Collins M, Eyering V, Gleckler PJ, Hewitson B, Mearns LO (2010) Good practice guidance paper on assessing and combining multi model climate projections. In: Stocker TF, Qin D, Plattner GK, Tignor M, Midgley GF (eds) Meeting report of the intergovernmental panel on climate change expert meeting on assessing and combining multi model climate projections. IPCC Working Group I Technical Support Unit. University of Bern, Bern, SwitzerlandGoogle Scholar
  9. Macadam I, Pitman AJ, Whetton PH, Abramowitz G (2010) Ranking climate models by performance using actual values and anomalies: implications for climate change impact assessments. Geophys Res Lett 37(16). doi: 10.1029/2010GL043877.
  10. Meehl GA, Stocker TF, Collins WD, Friedlingstein P, Gaye AT, Gregory JM, Kitoh A, Knutti R, Murphy JM, Noda A, Raper SC, Watterson IG, Weaver AJ, Zhao ZC (2007) Global climate projections. In: Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt KB, Tignor M, Miller HL (eds) Climate change 2007: the physical science basis. Contribution of working group I to the fourth assessment report of the intergovernmental panel on climate change. Cambridge University Press, Cambridge, pp 747–845Google Scholar
  11. Murphy J, Sexton D, Barnett D, Jones G, Webb M, Collins M, Stainforth D (2004) Quantification of modelling uncertainties in a large ensemble of climate change simulations. Nature 430(7001):768–772. Google Scholar
  12. Oldenborgh GJv, Reyes FJD, Drijfhout SS, Hawkins E (2013) Reliability of regional climate model trends. Environ Res Lett 8:014-055. doi: 10.1088/1748-9326/8/1/014055.
  13. Perkins SE, Pitman AJ, Holbrook NJ, McAneney J (2007) Evaluation of the AR4 climate models simulated daily maximum temperature, minimum temperature, and precipitation over australia using probability density functions. J Clim 20(17):4356–4376. doi: 10.1175/JCLI4253.1 CrossRefGoogle Scholar
  14. Phipps SJ (2010) The CSIRO Mk3L climate system model v1.2, Technical report no. 4. The Antarctic Climate & Ecosystems CRC, Hobart, Tasmania, AustraliaGoogle Scholar
  15. Phipps SJ, Rotstayn LD, Gordon HB, Roberts JL, Hirst AC, Budd WF (2011) The CSIRO Mk3L climate system model version 1.0—part 1: description and evaluation. Geosci Model Dev 4(2):483–509. doi: 10.5194/gmd-4-483-2011.
  16. Phipps SJ, Rotstayn LD, Gordon HB, Roberts JL, Hirst AC, Budd WF (2012) The CSIRO Mk3L climate system model version 1.0—part 2: response to external forcings. Geosci Model Dev 5(3):649–682. doi: 10.5194/gmd-5-649-2012.
  17. Phipps SJ, McGregor HV, Gergis J, Gallant AJE, Neukom R, Stevenson S, Ackerley D, Brown JR, Fischer MJ, van Ommen TD (2013) Paleoclimate data-model comparison and the role of climate forcings over the past 1500 years. J Clim doi: 10.1175/JCLI-D-12-00108.1
  18. Pirani A (ed) (2008) WCRP Coupled Model Intercomparison Project—Phase 5, CLIVAR echanges, vol 16. National Oceanography Centre, Southampton, UK.
  19. Randall D, Wood R, Bony S, Colman R, Fichefet T, Fyfe J, Kattsov V, Pitman A, Shukla J, Srinivasan J, Stouffer R, Sumi A, Taylor K (2007) Climate models and their evaluation. In: Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt K, Tignor M, Miller H (eds) Climate change 2007: the physical science basis. Contribution of working group I to the fourth assessment report of the Intergovernmental Panel on Climate Change. Cambridge University Press, CambridgeGoogle Scholar
  20. Reichert P, Schervish M, Small MJ (2002) An efficient sampling technique for bayesian inference with computationally demanding models. Technometrics 44(4):318–327. doi: 10.1198/004017002188618518 CrossRefGoogle Scholar
  21. Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt K, Tignor M (eds) (2007) Contribution of working group I to the fourth assessment report of the Intergovernmental Panel on Climate Change. IPCC Fourth Assessment Report: climate change 2007. Cambridge University Press, Cambridge.
  22. Stainforth D, Aina T, Christensen C, Collins M, Faull N, Frame D, Kettleborough J, Knight S, Martin A, Murphy J, et al (2005) Uncertainty in predictions of the climate response to rising levels of greenhouse gases. Nature 433(7024):403–406. doi: 10.1038/nature03301. Google Scholar
  23. Tebaldi C, Knutti R (2007) The use of the multi-model ensemble in probabilistic climate projections. Philos T Roy Soc A 365(1857):2053–2075CrossRefGoogle Scholar
  24. Washington WM, Meehl GA (1989) Climate sensitivity due to increased CO2 experiments with a coupled atmosphere and ocean general circulation model. Clim Dyn 4(1):1–38. doi: 10.1007/BF00207397.
  25. WCRP (2013) CMIP5: overview.
  26. Yokohata T, Annan JD, Collins M, Jackson CS, Tobis M, Webb MJ, Hargreaves JC (2012) Reliability of multi-model and structurally different single-model ensembles. Clim Dyn pp 1–18. doi: 10.1007/s00382-011-1203-1.

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Ned Haughton
    • 1
  • Gab Abramowitz
    • 1
  • Andy Pitman
    • 1
  • Steven J. Phipps
    • 1
  1. 1.Climate Change Research Centre Level 4, Mathews BuildingUniversity of New South WalesSydneyAustralia

Personalised recommendations