Climate Dynamics

, Volume 41, Issue 7–8, pp 1703–1729 | Cite as

History matching for exploring and reducing climate model parameter space using observations and a large perturbed physics ensemble

  • Daniel Williamson
  • Michael Goldstein
  • Lesley Allison
  • Adam Blaker
  • Peter Challenor
  • Laura Jackson
  • Kuniko Yamazaki


We apply an established statistical methodology called history matching to constrain the parameter space of a coupled non-flux-adjusted climate model (the third Hadley Centre Climate Model; HadCM3) by using a 10,000-member perturbed physics ensemble and observational metrics. History matching uses emulators (fast statistical representations of climate models that include a measure of uncertainty in the prediction of climate model output) to rule out regions of the parameter space of the climate model that are inconsistent with physical observations given the relevant uncertainties. Our methods rule out about half of the parameter space of the climate model even though we only use a small number of historical observations. We explore 2 dimensional projections of the remaining space and observe a region whose shape mainly depends on parameters controlling cloud processes and one ocean mixing parameter. We find that global mean surface air temperature (SAT) is the dominant constraint of those used, and that the others provide little further constraint after matching to SAT. The Atlantic meridional overturning circulation (AMOC) has a non linear relationship with SAT and is not a good proxy for the meridional heat transport in the unconstrained parameter space, but these relationships are linear in our reduced space. We find that the transient response of the AMOC to idealised CO2 forcing at 1 and 2 % per year shows a greater average reduction in strength in the constrained parameter space than in the unconstrained space. We test extended ranges of a number of parameters of HadCM3 and discover that no part of the extended ranges can by ruled out using any of our constraints. Constraining parameter space using easy to emulate observational metrics prior to analysis of more complex processes is an important and powerful tool. It can remove complex and irrelevant behaviour in unrealistic parts of parameter space, allowing the processes in question to be more easily studied or emulated, perhaps as a precursor to the application of further relevant constraints.


Bayesian uncertainty quantification History matching Implausibility Observations NROY space 



This research was funded by the NERC RAPID-RAPIT project (NE/G015368/1), and was supported by the Joint DECC/Defra Met Office Hadley Centre Climate Programme (GA01101). We would like to thank the CPDN team, in particular Andy Bowery, for their work in submitting our ensemble to the CPDN users. We’d also like to thank Richard Allan for helpful discussions regarding precipitation estimates, and all of the CPDN users around the world who contributed their spare computing resource as part of the generation of our ensemble. Finally, we’d like to thank both referees for their thoughtful and detailed comments.


  1. Acreman DM, Jeffery CD (2007) The use of Argo for validation and tuning of mixed layer models. Ocean Model 19:53–69CrossRefGoogle Scholar
  2. Berliner LM, Kim Y (2008) Bayesian design and analysis for superensemble-based climate forecasting. J Clim 21(9):1981–1910CrossRefGoogle Scholar
  3. Brierley CM, Collins M, Thorpe AJ (2010) The impact of perturbations to ocean-model parameters on climate and climate change in a coupled model. Clim Dyn 34:325–343CrossRefGoogle Scholar
  4. Broecker WS (1987) The biggest chill. Nat Hist Mag 97:74–82Google Scholar
  5. Brohan P, Kennedy JJ, Harris I, Tett SFB, Jones PD (2006) Uncertainty estimates in regional and global observed temperature changes: a new dataset from 1850. J Geophys Res 111:D12106CrossRefGoogle Scholar
  6. Challenor P, McNeall D, Gattiker J (2009) Assessing the probability of rare climate events. In: O’Hagan A, West M (eds) The handbook of applied Bayesian analysis, chap. 10. Oxford University Press, OxfordGoogle Scholar
  7. Collins M, Brierley CM, MacVean M, Booth BBB, Harris GR (2007) The sensitivity of the rate of transient climate change to ocean physics perturbations. J Clim 20:23315–2320CrossRefGoogle Scholar
  8. Collins M, Booth BBB, Bhaskaran B, Harris GR, Murphy JM, Sexton DMH, Webb MJ (2011) Climate model errors, feedbacks and forcings: a comparison of perturbed physics and multi-model experiments. Clim Dyn 36:1737–1766CrossRefGoogle Scholar
  9. Craig PS, Goldstein M, Seheult AH, Smith JA (1996) Bayes linear strategies for matching hydrocarbon reservoir history. In: Bernado JM, Berger JO, Dawid AP, Smith AFM (eds) Bayesian statistics 5. Oxford University Press, Oxford, pp 69–95Google Scholar
  10. Craig PS, Goldstein M, Seheult AH, Smith JA (1997) Pressure matching for hydrocarbon reservoirs: a case study in the use of Bayes linear strategies for large computer experiments. In: Gatsonis C, Hodges JS, Kass RE, McCulloch R, Rossi P, Singpurwalla ND (eds) Case studies in Bayesian statistics vol III. Springer, New York, pp 36–93Google Scholar
  11. Craig PS, Goldstein M, Rougier JC, Seheult AH (2001) Bayesian forecasting for complex systems using computer simulators. J Am Stat Assoc 96:717–729CrossRefGoogle Scholar
  12. Cumming JA, Goldstein M (2010) Bayes linear uncertainty analysis for oil reservoirs based on multiscale computer experiments. In: O’Hagan A, West M (eds) The Oxford handbook of applied Bayesian analysis. Oxford University Press, Oxford, pp 241–270Google Scholar
  13. de Finetti B (1974) Theory of probability, volume 1. Wiley, New YorkGoogle Scholar
  14. de Finetti B (1975) Theory of probability, volume 2. Wiley, New YorkGoogle Scholar
  15. Dickson RR, Brown J (1994) The production of North Atlantic deep water: sources, rates, and pathways. J Geophys Res Oceans 99:12,319–12,341CrossRefGoogle Scholar
  16. Draper NR, Smith H (1998) Applied regression analysis, 3rd edn. Wiley, New YorkGoogle Scholar
  17. Edwards NR, Cameron D, Rougier JC (2011) Precalibrating an intermediate complexity climate model. Clim Dyn 37:1469–1482CrossRefGoogle Scholar
  18. Frieler K, Meinshausen M, Schneider von Deimling T, Andrews T, Forster P (2011) Changes in global-mean precipitation in response to warming, greenhouse gas forcing and black carbon. Geophys Res Lett 38:L04702. doi: 10.1029/2010GL045953 CrossRefGoogle Scholar
  19. Furrer R, Sain SR, Nychka D, Meehl GA (2007) Multivariate Bayesian analysis of atmosphere–ocean general circulation models. Environ Ecol Stat 14:249–266CrossRefGoogle Scholar
  20. Goldstein M (1986) Exchangeable belief structures. J Am Stat Assoc 81:971–976CrossRefGoogle Scholar
  21. Goldstein M (1986) Prevision. In: Kotz S, Johnson NL (eds) Encyclopaedia of statistical sciences, vol 7. pp 175–176Google Scholar
  22. Goldstein M, Rougier JC (2004) Probabilistic formulations for transferring inferences from mathematical models to physical systems. SIAM J Sci Comput 26(2):467–487CrossRefGoogle Scholar
  23. Goldstein M, Rougier JC (2009) Reified Bayesian modelling and inference for physical systems. J Stat Plan Inference 139:1221–1239CrossRefGoogle Scholar
  24. Goldstein M, Wooff D (2007) Bayes linear statistics theory and methods. Wiley, New YorkCrossRefGoogle Scholar
  25. Gordon C, Cooper C, Senior CA, Banks H, Gregory JM, Johns TC, Mitchell JFB, Wood RA (2000) The simulation of SST, sea ice extents and ocean heat transports in a version of the Hadley Centre coupled model without flux adjustments. Clim Dyn 16:147–168CrossRefGoogle Scholar
  26. Gregory JM, Dixon KW, Stouffer RJ, Weaver AJ, Driesschaert E, Eby M, Fichefet T, Hasumi H, Hu A, Jungclaus JH,Kamenkovich IV, Levermann A, Montoya M, Murakami S, Nawrath S, Oka A, Sokolov AP, Thorpe, RB (2005) Subannual, seasonal and interannual variability of the North Atlantic meridional overturning circulation. Geophys Res Lett 32. doi: 10.1029/2005GL023209
  27. Huffman GJ, Adler RF, Bolvin DT, Gu G (2009) Improving the global precipitation record: GPCP Version 2.1. Geophys Res Lett 36:L17808. doi: 10.1029/2009GL040000 CrossRefGoogle Scholar
  28. Jackson L, Vellinga M, Harris G (2012) The sensitivity of the meridional overturning circulation to modelling uncertainty in a perturbed physics ensemble without flux adjustment. Clim Dyn. doi: 10.1007/s00382-011-1110-5
  29. Johns WE, Baringer MO, Beal LM, Cunningham SA, Kanzow T, Bryden HL, Hirschi JJM, Marotzke J, Meinen C, Shaw B, Curry R (2011) Continuous, array-based estimates of Atlantic Ocean heat transport at 26.5N. J Clim 24:2429–2449CrossRefGoogle Scholar
  30. Jones PD, New M, Parker DE, Martin S, Rigor IG (1999) Surface air temperature and its changes over the past 150 years. Rev Geophys 37(2):173–199CrossRefGoogle Scholar
  31. Joshi MM, Webb MJ, Maycock AC, Collins M (2010) Stratospheric water vapour and high climate sensitivity in a version of the HadSM3 climate model. Atmos Chem Phys 10:7161–7167CrossRefGoogle Scholar
  32. Kraus EB, Turner J (1967) A one dimensional model of the seasonal thermocline II. The general theory and its consequences. Tellus 19:98–106CrossRefGoogle Scholar
  33. Legates DR, Willmott CJ (1990) Mean seasonal and spatial variability in global surface air temperature. Theor Appl Climatol 41:11–21CrossRefGoogle Scholar
  34. Kennedy MC, O’Hagan A (2001) Bayesian calibration of computer models. J R Stat Soc Ser B 63:425–464CrossRefGoogle Scholar
  35. Kuhlbrodt T, Griesel A, Montoya M, Levermann A, Hofmann M, Rahmstorf S (2007) On the driving processes of the Atlantic meridional overturning circulation. Rev Geophys 45. doi: 10.1029/2004RG000166
  36. Liu C, Allan RP (2012) Multisatellite observed responses of precipitation and its extremes to interannual climate variability. J Geophys Res 117:D03101. doi: 10.1029/2011JD016568 CrossRefGoogle Scholar
  37. Manabe S, Stouffer RJ (1980) Sensitivity of a global climate model to an increase of CO2 concentration in the atmosphere. J Geophys Res 85:5529–5554CrossRefGoogle Scholar
  38. McManus JF, Francois R, Gherardi JM, Keigwin LD, Brown-Leger S (2004) Collapse and rapid resumption of Atlantic meridional circulation linked to deglacial climate changes. Nature 428:834–837. doi: 10.1038/nature02494 CrossRefGoogle Scholar
  39. Meehl GA, Covey C, Delworth T, Latif M, McAvaney B, Mitchell JFB, Stouffer RJ, Taylor KE (2007) The WCRP CMIP3 multi-model dataset: a new era in climate change research. Bull Am Meteorol Soc 88:1383–1394CrossRefGoogle Scholar
  40. Morris MD, Mitchell TJ (1995) Exploratory designs for computational experiments. J Stat Plan Inference 43:381–402CrossRefGoogle Scholar
  41. Murphy JM, Sexton DMH, Barnett DN, Jones GS, Webb MJ, Collins M, Stainforth DA (2004) Quantification of modelling uncertainties in a large ensemble of climate change simulations. Nature 430:768–772CrossRefGoogle Scholar
  42. Murphy JM, Sexton DMH, Jenkins GJ, Booth BBB, Brown CC, Clark RT, Collins M, Harris GR, Kendon EJ, Betts RA, Brown SJ, Humphrey KA, McCarthy MP, McDonald RE, Stephens A, Wallace C, Warren R, Wilby R, Wood R (2009) UK Climate Projections Science Report: Climate change projections. Met Office Hadley Centre, Exeter, UK.
  43. Pope VD, Gallani ML, Rowntree PR, Stratton RA (2000) The impact of new physical parameterizations in the Hadley Centre climate model: HadAM3. Clim Dyn 16:123–146CrossRefGoogle Scholar
  44. Pukelsheim F (1994) The three sigma rule. Am Stat 48:88–91Google Scholar
  45. Rhines PB, Häkkinen S (2003) Is the oceanic heat transport in the North Atlantic irrelevant to the climate in Europe? ASOF Newsl 13–17Google Scholar
  46. Rice JA (1995) Mathematical statistics and data analysis, 2nd edn. Duxbury Press, Wadsworth Publishing Company, Belmont, CaliforniaGoogle Scholar
  47. Rougier JC (2007) Probabilistic inference for future climate using an ensemble of climate model evaluations. Clim Change 81:247–264CrossRefGoogle Scholar
  48. Rougier JC, Sexton DMH, Murphy JM, Stainforth D (2009) Emulating the sensitivity of the HadSM3 climate model using ensembles from different but related experiments. J Clim 22:3540–3557CrossRefGoogle Scholar
  49. Rougier JC, Goldstein M, House L (2012) Second-order exchangeability analysis for multi-model ensembles. J Am Stat Assoc (to appear)Google Scholar
  50. Rowlands DJ, Frame DJ, Ackerley D, Aina T, Booth BBB, Christensen C, Collins M, Faull N, Forest CE, Grandey BS, Gryspeerdt E, Highwood EJ, Ingram WJ, Knight S, Lopez A, Massey N, McNamara F, Meinshausen N, Piani C, Rosier SM, Sanderson BJ, Smith LA, Stone DA, Thurston M, Yamazaki K, Yamazaki YH, Allen MR (2012) Broad range of 2050 warming from an observationally constrained large climate model ensemble. Nat Geosci, published online. doi: 10.1038/NGEO1430
  51. Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4:409–435CrossRefGoogle Scholar
  52. Sanderson BM, Shell KM, Ingram W (2010) Climate feedbacks determined using radiative kernels in a multi-thousand member ensemble of AOGCMs. Clim Dyn 35:1219–1236CrossRefGoogle Scholar
  53. Santner TJ, Williams BJ, Notz WI (2003) The design and analysis of computer experiments. Springer, New YorkCrossRefGoogle Scholar
  54. Sexton DMH, Murphy JM, Collins M (2011) Multivariate probabilistic projections using imperfect climate models part 1: outline of methodology. Clim Dyn. doi: 10.1007/s00382-011-1208-9
  55. Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt KB, Tignor M, Miller HL (eds) (2007) Contribution of working group I to the fourth assessment report of the intergovernmental panel on climate change, 2007. Cambridge University Press, CambridgeGoogle Scholar
  56. Stephens GL, Wild M, Stackhouse Jr PW, L’Ecuyer T, Kato S, Henderson DS (2012) The global character of the flux of downward longwave radiation. J Clim 25:2329–2340CrossRefGoogle Scholar
  57. Trenberth KE, Fasullo JT, Kiehl J (2009) Earth’s global energy budget. Bull Am Meteorol Soc 90:311–323. doi: 10.1175/2008BAMS2634.1 CrossRefGoogle Scholar
  58. Vernon I, Goldstein M, Bower RG (2010) Galaxy formation: a Bayesian uncertainty analysis. Bayesian Anal 5(4):619–846, with DiscussionGoogle Scholar
  59. Whittle P (1992) Probability via expectation, 3rd edn. Springer texts in statistics. Springer, New YorkGoogle Scholar
  60. Williamson D, Goldstein M, Blaker A (2012) Fast linked analyses for scenario based hierarchies. J R Stat Soc Ser C 61(5):665–692CrossRefGoogle Scholar
  61. Williamson D, Blaker AT (2013) Evolving Bayesian emulators for structured chaotic time series, with application to large climate models. SIAM J Uncertain Quantification (resubmitted)Google Scholar
  62. Zaglauer S (2012) The evolutionary algorithm SAMOA with use of design of experiments. In: Proceeding GECCO companion ’12. ACM, New York, pp 637–638Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Daniel Williamson
    • 1
  • Michael Goldstein
    • 1
  • Lesley Allison
    • 2
  • Adam Blaker
    • 3
  • Peter Challenor
    • 3
  • Laura Jackson
    • 4
  • Kuniko Yamazaki
    • 5
  1. 1.Department of Mathematical SciencesDurham UniversityDurhamUK
  2. 2.NCAS-Climate, Department of MeteorologyUniversity of ReadingReadingUK
  3. 3.National Oceanography CentreSouthamptonUK
  4. 4.Met Office Hadley CentreExeterUK
  5. 5.Atmosphere and Ocean Research InstituteThe University of TokyoKashiwaJapan

Personalised recommendations