Climate Dynamics

, Volume 36, Issue 9–10, pp 1737–1766 | Cite as

Climate model errors, feedbacks and forcings: a comparison of perturbed physics and multi-model ensembles

  • Matthew CollinsEmail author
  • Ben B. B. Booth
  • B. Bhaskaran
  • Glen R. Harris
  • James M. Murphy
  • David M. H. Sexton
  • Mark J. Webb


Ensembles of climate model simulations are required for input into probabilistic assessments of the risk of future climate change in which uncertainties are quantified. Here we document and compare aspects of climate model ensembles from the multi-model archive and from perturbed physics ensembles generated using the third version of the Hadley Centre climate model (HadCM3). Model-error characteristics derived from time-averaged two-dimensional fields of observed climate variables indicate that the perturbed physics approach is capable of sampling a relatively wide range of different mean climate states, consistent with simple estimates of observational uncertainty and comparable to the range of mean states sampled by the multi-model ensemble. The perturbed physics approach is also capable of sampling a relatively wide range of climate forcings and climate feedbacks under enhanced levels of greenhouse gases, again comparable with the multi-model ensemble. By examining correlations between global time-averaged measures of model error and global measures of climate change feedback strengths, we conclude that there are no simple emergent relationships between climate model errors and the magnitude of future global temperature change. Algorithms for quantifying uncertainty require the use of complex multivariate metrics for constraining projections.


Ensembles Uncertainty Model errors Climate feedbacks Observational constraints 



This work was supported by the Joint DECC and Defra Integrated Climate Programme—DECC/Defra (GA01101) and by the European Community ENSEMBLES (GOCE-CT-2003-505539). Hugo Lambert made useful comments on an earlier version of the manuscript and we thank three anonymous reviewers for their comments.


  1. Ackerley D, Highwood EJ, Frame D, Booth BBB (2009) Changes in the global sulfate burden due to perturbations in global CO2 concentrations. J Clim 20:5421–5432CrossRefGoogle Scholar
  2. Adler RF et al (2003) The Version 2 global precipitation climatology project (GPCP) Monthly precipitation analysis (1979-Present). J Hydrometeorol 4:1147–1167CrossRefGoogle Scholar
  3. Allan RJ, Ansell TJ (2006) A new globally complete monthly historical mean sea level pressure data set (HadSLP2): 1850–2004. J Clim 19:5816–5842CrossRefGoogle Scholar
  4. Allen MR, Kettleborough J, Stainforth DA (2002) Model error in weather and climate forecasting. In: Proceedings of the ECMWF seminar series.
  5. Annan JD, Hargreaves JC (2010) Reliability of the CMIP3 ensemble. Geophys Res Lett 37:L02703. doi: 10.1029/2009GL041994 CrossRefGoogle Scholar
  6. Annan JD, Hargreaves JC, Ohgaito R, Abe-Ouchi A, Emori S (2005) Efficiently constraining climate sensitivity with ensembles of Paleoclimate simulations. Sci On-line Lett Atmos 1:181–184Google Scholar
  7. Aumann HH et al (2003) AIRS/AMSU/HSB on the Aqua mission: design, science objectives, data products, and processing systems. IEEE Trans Geosci Remote Sens 41:253–264CrossRefGoogle Scholar
  8. Barnett DN, Brown SJ, Murphy JM, Sexton DMH, Webb MJ (2006) Quantifying uncertainty in changes in extreme event frequency in response to doubled CO2 using a large ensemble of GCM simulations. Clim Dyn 26:489–511CrossRefGoogle Scholar
  9. Boer G, Yu B (2003) Climate sensitivity and response. Clim Dyn 20:415–429Google Scholar
  10. Brierley CM, Thorpe AJ, Collins M (2009) An example of the dependence of the transient climate response on the temperature of the modelled climate state. Atmos Sci Lett 10:23–28CrossRefGoogle Scholar
  11. Brierley CM, Collins M, Thorpe AJ (2010) The impact of perturbations to ocean-model parameters on climate and climate change in a coupled model. Clim Dyn 34:325–343CrossRefGoogle Scholar
  12. Brohan P, Kennedy JJ, Harris I, Tett SFB, Jones PD (2006) Uncertainty estimates in regional and global observed temperature changes: a new dataset from 1850. J Geophys Res 111:D12106. doi: 10.1029/2005JD006548 CrossRefGoogle Scholar
  13. Cess RD et al (1990) Intercomparison and interpretation of climate feedback processes in 19 atmospheric general circulation models. J Geophys Res 95:16601–16615CrossRefGoogle Scholar
  14. Collins WV (2006) Radiative forcing by well-mixed greenhouse gases: Estimates from climate models in the IPCC AR4. J Geophys Res 111:D14317. doi: 10.1029/2005JD006713 CrossRefGoogle Scholar
  15. Collins M (2007) Ensembles and probabilities: a new era in the prediction of climate change. Philos Trans R Soc Lond A 365:1957–1970CrossRefGoogle Scholar
  16. Collins M, Booth BBB, Harris GR, Murphy JM, Sexton DMH, Webb MJ (2006) Towards quantifying uncertainty in transient climate change. Clim Dyn 27:127–147CrossRefGoogle Scholar
  17. Collins M, Brierley CM, MacVean M, Booth BBB, Harris GR (2007) The sensitivity of the rate of transient climate change to ocean physics perturbations. J Clim 20:2315–2320CrossRefGoogle Scholar
  18. Colman RA (2003) A comparison of climate feedbacks in general circulation models. Clim Dyn 20:865–873Google Scholar
  19. Da Silva A, Young C, Levitus S (1994) Atlas of surface marine data 1994, volume 1: algorithms and procedures. NOAA Atlas NESDIS 6. US Department of Commerce, WashingtonGoogle Scholar
  20. Dijkstra HA, Neelin JD (1999) Imperfections of the thermohaline circulation: multiple equilibria and flux correction. J Clim 12:1382–1392CrossRefGoogle Scholar
  21. Forest CE, Stone PH, Sokolov AP (2006) Estimated PDFs of climate system properties including natural and anthropogenic forcings. Geophys Res Lett 33:L01705CrossRefGoogle Scholar
  22. Forster PMdeF, Taylor KE (2006) Climate forcings and climate sensitivities diagnosed from coupled climate model integrations. J Clim 19:6181–6194CrossRefGoogle Scholar
  23. Forster PMdeF et al (2007) Changes in atmospheric constituents and in radiative forcing. In: Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt KB, Tignor M, Miller HL (eds) Climate change 2007: the physical science basis. Contribution of working Group I to the fourth assessment report of the intergovernmental panel on climate change. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USAGoogle Scholar
  24. Frame DJ et al (2009) The BBC climate change experiment part 1: design of the coupled model ensemble. Philos Trans R Soc Lond A 367:855–870CrossRefGoogle Scholar
  25. Gleckler PJ, Taylor KE, Doutriaux C (2008) Performance metrics for climate models. J Geophys Res 113:D06104. doi: 10.1029/2007JD008972 CrossRefGoogle Scholar
  26. Gordon CC et al (2000) The simulation of SST, sea ice extents and ocean heat transport in a version of the Hadley Centre coupled model without flux adjustments. Clim Dyn 16:147–168CrossRefGoogle Scholar
  27. Gregory JM, Webb MJ (2008) Tropospheric adjustment induces a cloud component in CO2 forcing. J Clim 21:58–71CrossRefGoogle Scholar
  28. Gregory JM et al (2004) A new method for diagnosing radiative forcing and climate sensitivity. Geophys Res Lett 31:L03205CrossRefGoogle Scholar
  29. Grist JP, Josey SA (2003) Inverse analysis adjustment of the SOC air–sea flux climatology using ocean heat transport constraints. J Clim 20:3274–3295CrossRefGoogle Scholar
  30. Hagedorn R, Doblas-Reyes FJ, Palmer TN (2005) The rationale behind the success of multimodel ensembles in seasonal forecasting. Part I. Basic concept. Tellus 57:219–233CrossRefGoogle Scholar
  31. Hansen J, Ruedy R, Sato M, Reynolds R (1996) Global surface air temperature in 1995: return to pre-Pinatubo level. Geophys Res Lett 23:1665–1668CrossRefGoogle Scholar
  32. Harris GR, Sexton DMH, Booth BBB, Collins M, Murphy JM, Webb MJ (2006) Frequency distributions of transient regional climate change from perturbed physics ensembles of general circulation model simulations. Clim Dyn 27:357–375CrossRefGoogle Scholar
  33. Harrison EF, Minnis P, Barkstrom BR, Ramanathan V, Cess R, Gibson CG (1990) Seasonal variation of cloud radiative forcing derived from the Earth Radiation Budget Experiment. J Geophys Res 95:687–703Google Scholar
  34. Held IM, Soden BJ (2006) Robust responses of the hydrological cycle to global warming. J Clim 19:5686–5699CrossRefGoogle Scholar
  35. Hibbard KA, Meehl GA, Cox PM, Friedlingsten P (2007) A strategy for climate change stabilization experiments. EOS 88:20. doi: 10.1029/2007EO200002 CrossRefGoogle Scholar
  36. Huntingford C, Cox PM (2000) An analogue model to derive additional climate change scenarios from existing GCM simulations. Clim Dyn 16:575–586CrossRefGoogle Scholar
  37. Jackson CS, Sen MK, Huerta G, Deng Y, Bowman KP (2008) Error reduction and convergence in climate prediction. J Clim 21:6698–6709CrossRefGoogle Scholar
  38. Jones A, Roberts DL, Woodage MJ, Johnson CE (2001) Indirect sulphate aerosol forcing in a climate model with an interactive sulphur cycle. J Geophys Res 106:20293–20310CrossRefGoogle Scholar
  39. Joshi MM, Gregory JM, Webb MJ, Sexton DMH, Johns TC (2008) Mechanisms for the land/sea warming exhibited by simulations of climate change. Clim Dyn 30:455–465CrossRefGoogle Scholar
  40. Jun M, Knutti R, Nychka DW (2008) Spatial analysis to quantify numerical model bias and dependence: how many climate models are there? J Am Stat Assoc Appl Case Stud 103:934–947CrossRefGoogle Scholar
  41. Knutti R, Meehl GA, Allen MR, Stainforth DA (2006) Constraining climate sensitivity from the seasonal cycle in surface temperature. J Clim 19:4224–4233CrossRefGoogle Scholar
  42. Knutti R, Furrer R, Tebaldi C, Cernak J, Meehl GA (2010) Challenges in combining projections from multiple climate models. J Clim (in press)Google Scholar
  43. Lambert SJ, Boer HJ (2001) CMIP1 evaluation and intercomparison of coupled climate models. Clim Dyn 17:83–106CrossRefGoogle Scholar
  44. Lambert FH, Chiang JCH (2007) Control of land–ocean temperature contrast by ocean heat uptake. Geophys Res Lett 34:L13704CrossRefGoogle Scholar
  45. Legates DR, Willmott CJ (1990) Mean seasonal and spatial variability in global surface air temperature. Theor Appl Climatol 41:11–21Google Scholar
  46. McKay MD, Conover WJ, Beckman RJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21:239–245CrossRefGoogle Scholar
  47. Meehl GA, Stocker T et al (2007a) Global climate projections. I. Climate Change 2007: the physical science basis. In: Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt KB, Tignor M, Miller HL (eds) Contribution of working Group I to the fourth assessment report of the intergovernmental panel on climate change. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USAGoogle Scholar
  48. Meehl GA et al (2007b) The WCRP CMIP3 multimodel dataset: a new era in climate change research. Bull Am Meteorol Soc 88:1383–1394CrossRefGoogle Scholar
  49. Min SK, Simonis D, Hense A (2007) Probabilistic climate change predictions applying Bayesian model averaging. Philos Trans R Soc Lond A 365:2103–2116CrossRefGoogle Scholar
  50. Molteni F, Buizza R, Palmer TN, Petroliagis T (2006) The ECMWF ensemble prediction system: methodology and validation. Quart J Roy Meteorol Soc 122:73–119CrossRefGoogle Scholar
  51. Moore B, Gates WL, Mata LJ, Underdal A (2001) Advancing our understanding. In: Houghton JT, Ding Y, Griggs DJ, Noguer M, van der Linden PJ, Dai X, Maskell K, Johnson CA (eds) Climate change 2001: the scientific basis. Contribution of working Group I to the third assessment report of the intergovernmental panel on climate change, Cambridge University PressGoogle Scholar
  52. Murphy JM (1995) Transient response of the Hadley Centre coupled ocean–atmosphere model to increasing carbon dioxide. Part III. Analysis of global mean response using simple models. J Clim 8:496–514CrossRefGoogle Scholar
  53. Murphy JM, Sexton DMH, Barnett DN, Jones GS, Webb MJ, Collins M, Stainforth DA (2004) Quantification of modelling uncertainties in a large ensemble of climate change simulations. Nature 430:768–772CrossRefGoogle Scholar
  54. Murphy JM, Booth BBB, Collins M, Harris GR, Sexton D, Webb MJ (2007) A methodology for probabilistic predictions of regional climate change from perturbed physics ensembles. Philos Trans R Soc Lond A 365:1993–2028CrossRefGoogle Scholar
  55. Murphy JM, Sexton DMH, Jenkins G, Boorman P, Booth BBB, Brown K, Clark R, Collins M, Harris GR, Kendon E (2009) Climate change projections. ISBN 978-1-906360-02-3Google Scholar
  56. Myhre G, Highwood EJ, Shine KP, Stordal F (1998) New estimates of radiative forcing due to well mixed greenhouse gases. Geophys Res Lett 25(14):2715–2718. doi: 10.1029/98GL01908 Google Scholar
  57. Niehörster F, Spangehl T, Fast I, Cubasch U (2006) Quantification of model uncertainties: parameter sensitivities of the coupled model ECHO-G with middle atmosphere. Geophys Res Abs 8, EGU06-A-08526Google Scholar
  58. Piani C, Frame DJ, Stainforth DA, Allen MR (2005) Constraints on climate change from a multi-thousand member ensemble of simulations. Geophys Res Lett 32:L23825. doi: 10.1029/2005GL024452 CrossRefGoogle Scholar
  59. Pope VD, Gallani ML, Rowntree PR, Stratton RA (2000) The impact pf new physical parametrizations in the Hadley Centre climate model-HadAM3. Clim Dyn 16:123–146CrossRefGoogle Scholar
  60. Raper SCB, Gregory JM, Stouffer RJ (2002) The role of climate sensitivity and ocean heat uptake on AOGCM transient temperature response. J Clim 15:124–130CrossRefGoogle Scholar
  61. Rayner NA et al (2003) Global analyses of sea surface temperature, sea ice, and night marine air temperature since the late nineteenth century. J Geophys Res 108, D14, 4407. doi: 10.1029/2002JD002670
  62. Reichler T, Kim J (2008) How well do climate models simulate today’s climate? Bull Am Meteorol Soc 89:303–311CrossRefGoogle Scholar
  63. Rossow WB, Walker AW, Beuschel DE, Roiter MD (1996) International Satellite Cloud Climatology Project (ISCCP) documentation of new cloud datasets World Meteorological Organisation WMO/TD 737, pp 115Google Scholar
  64. Rougier JC (2007) Probabilistic inference for future climate using an ensemble of climate model evaluations. Clim Change 81:247–264CrossRefGoogle Scholar
  65. Rougier JC, Sexton DMH, Murphy JM, Stainforth DA (2009) Analysing the climate sensitivity of the HadSM3 climate model using ensembles from different but related experiments. J Clim 22:3540–3557CrossRefGoogle Scholar
  66. Sanderson BM, Piani C (2007) Towards constraining climate sensitivity by linear analysis of feedback patterns in thousands of perturbed-physics GCM simulations. Clim Dyn 30:175–190CrossRefGoogle Scholar
  67. Sanderson BM et al (2008) Constraints on model response to greenhouse gas forcing and the role of subgrid-scale processes. J Clim 21:2384–2400CrossRefGoogle Scholar
  68. Sato M, Hansen JE, McCormick MP, Pollack JB (1993) Stratospheric aerosol optical depths (1850–1990). J Geophys Res 98:22987–22994CrossRefGoogle Scholar
  69. Schneider von Deimling T, Held H, Ganopolski A, Rahmstorf S (2006) Climate sensitivity estimated from ensemble simulations of glacial climates. Clim Dyn 27:149–163CrossRefGoogle Scholar
  70. Senior CA, Mitchell JFB (2000) The time dependence of climate sensitivity. Geophys Res Lett 27:2685–2688CrossRefGoogle Scholar
  71. Smith TM, Reynolds RW (2004) Improved extended reconstruction of SST (1854–1997). J Clim 17:2466–2477CrossRefGoogle Scholar
  72. Soden BJ, Held IM (2006) An assessment of climate feedbacks in coupled ocean–atmosphere models. J Clim 19:3354–3360CrossRefGoogle Scholar
  73. Soden BJ, Broccoli AJ, Hemler RS (2004) On the use of cloud forcing to estimate cloud feedback. J Clim 17(19):3661–3665CrossRefGoogle Scholar
  74. Sokolov AP et al (2009) Probabilistic forecast for 21st century climate based on uncertainties in emissions (without policy) and climate parameters. J Clim 22:5175–5204CrossRefGoogle Scholar
  75. Stainforth DA et al (2005) Uncertainty in predictions of the climate response to rising levels of greenhouse gases. Nature 433:403–406CrossRefGoogle Scholar
  76. Stocker TF (2004) Climate change: models change their tune. Nature 430:737–738Google Scholar
  77. Stott PA, Forest CE (2007) Ensemble climate predictions using climate models and observational constraints. Philos Trans R Soc Lond A 365:2029–2052CrossRefGoogle Scholar
  78. Sutton RT, Dong B-W, Gregory JM (2007) Land/sea warming ratio in response to climate change: IPCC AR4 model results and comparison with observations. Geophys Res Lett 34:L02701CrossRefGoogle Scholar
  79. Taylor KE (2001) Summarizing multiple aspects of model performance in a single diagram. J Geophys Res 106:7183–7192CrossRefGoogle Scholar
  80. Taylor KE, Crucifix M, Doutriaux C, Broccoli AJ, Mitchell JFB, Webb MJ (2007) Estimating shortwave radiative forcing and response in climate models. J Clim 20:2530–2543CrossRefGoogle Scholar
  81. Tziperman E, Toggweiler JR, Feliks Y, Bryan K (1994) Instability of the thermohaline circulation with respect to mixed boundary conditions: is it really a problem for realistic models? J Phys Oceanogr 24:217–232CrossRefGoogle Scholar
  82. Uppala SM et al (2005) The ERA-40 re-analysis. Quart J Roy Meteorol Soc 131:2961–3012CrossRefGoogle Scholar
  83. Webb MJ et al (2006) On the contribution of local feedback mechanisms to the range of climate sensitivity in two GCM ensembles. Clim Dyn 27:17–38CrossRefGoogle Scholar
  84. Webster MD et al (2002) Uncertainty in emissions projections for climate models. Atmos Environ 36:3659–3670CrossRefGoogle Scholar
  85. Wielicki BA, Barkstrom BR, Harrison EF, Lee RB III, Louis Smith G, Cooper JE (1996) Clouds and the Earth’s Radiant Energy System (CERES): an earth observing system experiment. Bull Am Meteorol Soc 77:853–868CrossRefGoogle Scholar
  86. Wylie DP, Menzel WP, Woolf HM, Strabala KI (1994) Four years of global cirrus cloud statistics using HIRS. J Clim 7:1972–1986CrossRefGoogle Scholar
  87. Xie P, Arkin PA (1997) Global precipitation: a 17-year monthly analysis based on gauge observations, satellite estimates, and numerical model outputs. Bull Am Meteorol Soc 78:2539–2558CrossRefGoogle Scholar
  88. Yokohata T et al (2008) Comparison of equilibrium and transient responses to CO2 increase in eight state-of-the-art climate models. Tellus 60:946–961CrossRefGoogle Scholar
  89. Yokohata T, Webb MJ, Collins M, Williams KD, Yoshimori M, Hargreaves JC, Annan JD (2010) Structural similarities and differences in climate responses to CO2 increase between two perturbed physics ensembles. J Clim 23(6):1392–1410Google Scholar
  90. Zhang MH, Cess RD, Hack JJ, Kiehl JT (1994) Diagnostic study of climate feedback processes in atmospheric GCMs. J Geophys Res 99:5525–5537CrossRefGoogle Scholar

Copyright information

© Crown Copyright 2010

Authors and Affiliations

  • Matthew Collins
    • 1
    Email author
  • Ben B. B. Booth
    • 1
  • B. Bhaskaran
    • 1
  • Glen R. Harris
    • 1
  • James M. Murphy
    • 1
  • David M. H. Sexton
    • 1
  • Mark J. Webb
    • 1
  1. 1.Met Office Hadley CentreExeterUK

Personalised recommendations