Advertisement

Finding plausible and diverse variants of a climate model. Part II: development and validation of methodology

  • Ambarish V. KarmalkarEmail author
  • David M. H. Sexton
  • James M. Murphy
  • Ben B. B. Booth
  • John W. Rostron
  • Doug J. McNeall
Article

Abstract

The usefulness of a set of climate change projections largely depends on how well it spans a range of outcomes consistent with known uncertainties. Here, we present exploratory work towards developing a strategy to select variants of a state-of-the-art but expensive climate model suitable for climate projection studies. The strategy combines information from a set of relatively cheap, idealized perturbed parameter ensemble (PPE) and CMIP5 multi-model ensemble (MME) experiments, and uses two criteria as the basis to select model variants for a PPE suitable for future projections: (a) acceptable model performance at two different timescales, and (b) maintaining diversity in model response to climate change. This second part of a pair of papers builds upon Part I in which we established a strong relationship between model errors at weather and climate timescales across a PPE for a variety of key variables. This relationship is used to filter out parts of parameter space that do not give credible simulations of present day climate, while minimizing the impact on ranges in forcings and feedbacks that drive model responses to climate change. We use statistical emulation to explore the parameter space thoroughly, and demonstrate that about 90% can be filtered out without affecting diversity in global-scale climate change responses. This leads to the identification of plausible parts of parameter space from which model variants can be selected for projection studies. We selected and ran 50 variants from the plausible parameter combinations and validated the emulator predictions. Comparisons with the CMIP5 MME demonstrate that our approach can produce a set of plausible model variants that span a relatively wide range in model response to climate change. We also highlight how the prior expert-specified ranges for uncertain model parameters are constrained as a result of our methodology, and discuss recommendations for future work.

Keywords

Uncertainty Perturbed parameter ensemble Seamless assessment Statistical emulation Filtering parameter space Plausible model variants 

Notes

Acknowledgements

This research was supported by the Joint UK BEIS/Defra Met Office Hadley Centre Climate Programme (GA01101). John Rostron was supported by the UK-China Research & Innovation Partnership Fund through the Met Office Climate Science for Service Partnership (CSSP) China as part of the Newton Fund. We would like to thank Mark Webb, Mark Ringer, and Alejandro Bodas-Salcedo for their help with designing and setting up the experiments.

References

  1. Adler RF, Huffman GJ, Chang A, Ferraro R, Xie PP, Janowiak J, Rudolf B, Schneider U, Curtis S, Bolvin D (2003) The version-2 global precipitation climatology project (GPCP) monthly precipitation analysis (1979-present). J Hydrometeorol 4(6):1147–1167CrossRefGoogle Scholar
  2. Allan R, Ansell T (2006) A new globally complete monthly historical gridded mean sea level pressure dataset (HadSLP2): 1850–2004. J Clim 19(22):5816–5842CrossRefGoogle Scholar
  3. Andrews T, Gregory JM, Webb MJ, Taylor KE (2012) Forcing, feedbacks and climate sensitivity in CMIP5 coupled atmosphere-ocean climate models. Geophys Res Lett 39(9):L10702.  https://doi.org/10.1029/2012GL051942 CrossRefGoogle Scholar
  4. Bellouin N, Collins W, Culverwell I, Halloran P, Hardiman S, Hinton T, Jones C, McDonald R, McLaren A, O’Connor F (2011) The HadGEM2 family of Met Office Unified Model climate configurations. Geosci Model Dev 4(3):723–757CrossRefGoogle Scholar
  5. Bony S, Colman R, Kattsov VM, Allan RP, Bretherton CS, Dufresne JL, Hall A, Hallegatte S, Holland MM, Ingram W (2006) How well do we understand and evaluate climate change feedback processes? J Clim 19(15):3445–3482CrossRefGoogle Scholar
  6. Bony S, Webb M, Stevens B, Bretherton C, Klein S, Tselioudis G (2009) The cloud feedback model intercomparison project: summary of activities and recommendations for advancing assessments of cloud-climate feedbacks. CFMIP DocGoogle Scholar
  7. Booth BB, Jones CD, Collins M, Totterdell IJ, Cox PM, Sitch S, Huntingford C, Betts RA, Harris GR, Lloyd J (2012) High sensitivity of future global warming to land carbon cycle processes. Environ Res Lett 7(2):024,002CrossRefGoogle Scholar
  8. Brown A, Milton S, Cullen M, Golding B, Mitchell J, Shelly A (2012) Unified modeling and prediction of weather and climate: a 25-year journey. Bull Am Meteorol Soc 93(12):1865–1877CrossRefGoogle Scholar
  9. Carslaw K, Lee L, Reddington C, Pringle K, Rap A, Forster P, Mann G, Spracklen D, Woodhouse M, Regayre L et al (2013) Large contribution of natural aerosols to uncertainty in indirect forcing. Nature 503(7474):67–71CrossRefGoogle Scholar
  10. Cess RD, Potter G, Blanchet J, Boer G, Del Genio A, Deque M, Dymnikov V, Galin V, Gates W, Ghan S (1990) Intercomparison and interpretation of climate feedback processes in 19 atmospheric general circulation models. J Geophys Res Atmos 95(D10):16,601–16,615CrossRefGoogle Scholar
  11. Collins M (2007) Ensembles and probabilities: a new era in the prediction of climate change. Phil Trans R Soc A 365:1957–1970.  https://doi.org/10.1098/rsta.2007.2068 CrossRefGoogle Scholar
  12. Collins M, Booth BB, Harris GR, Murphy JM, Sexton DM, Webb MJ (2006) Towards quantifying uncertainty in transient climate change. Clim Dyn 27(2–3):127–147CrossRefGoogle Scholar
  13. Collins M, Brierley C, MacVean M, Booth B, Harris G (2007) The sensitivity of the rate of transient climate change to ocean physics perturbations. J Clim 20(10):2315–2320CrossRefGoogle Scholar
  14. Collins M, Booth BB, Bhaskaran B, Harris GR, Murphy JM, Sexton DM, Webb MJ (2011) Climate model errors, feedbacks and forcings: a comparison of perturbed physics and multi-model ensembles. Clim Dyn 36(9–10):1737–1766CrossRefGoogle Scholar
  15. Compo GP, Whitaker JS, Sardeshmukh PD, Matsui N, Allan RJ, Yin X, Gleason BE, Vose RS, Rutledge G, Bessemoulin P (2011) The twentieth century reanalysis project. Q J R Meteorol Soc 137(654):1–28CrossRefGoogle Scholar
  16. Covey C, Lucas DD, Tannahill J, Garaizar X, Klein R (2013) Efficient screening of climate model sensitivity to a large number of perturbed input parameters. J Adv Model Earth Syst 5(3):598–610CrossRefGoogle Scholar
  17. Dee DP, Uppala S, Simmons A, Berrisford P, Poli P, Kobayashi S, Andrae U, Balmaseda M, Balsamo G, Bauer P (2011) The ERA-Interim reanalysis: Configuration and performance of the data assimilation system. Q J R Meteorol Soc 137(656):553–597CrossRefGoogle Scholar
  18. Edwards NR, Cameron D, Rougier J (2011) Precalibrating an intermediate complexity climate model. Clim Dyn 37(7–8):1469–1482CrossRefGoogle Scholar
  19. Flato G, Marotzke J, Abiodun B, Braconnot P, Chou SC, Collins WJ, Cox P, Driouech F, Emori S, Eyring V, et al (2013) Evaluation of climate models. In: Climate change 2013: the physical science basis. contribution of Working Group I to the fifth assessment report of the intergovernmental panel on climate change. Climate Change 2013 5:741–866Google Scholar
  20. Gates WL, Boyle JS, Covey C, Dease CG, Doutriaux CM, Drach RS, Fiorino M, Gleckler PJ, Hnilo JJ, Marlais SM (1999) An overview of the results of the atmospheric model intercomparison project (AMIP I). Bull Am Meteorol Soc 80(1):29–55CrossRefGoogle Scholar
  21. Gleckler PJ, Taylor KE, Doutriaux C (2008) Performance metrics for climate models. J Geophys Res Atmos 113(D6):D06104.  https://doi.org/10.1029/2007JD008972 CrossRefGoogle Scholar
  22. Golaz JC, Larson VE, Hansen JA, Schanen DP, Griffin BM (2007) Elucidating model inadequacies in a cloud parameterization by use of an ensemble-based calibration framework. Mon Weather Rev 135(12):4077–4096CrossRefGoogle Scholar
  23. Gong W, Duan Q, Li J, Wang C, Di Z, Dai Y, Ye A, Miao C (2015) Multi-objective parameter optimization of common land model using adaptive surrogate modeling. Hydrol Earth Syst Sci 19(5):2409–2425CrossRefGoogle Scholar
  24. Gregoire LJ, Valdes PJ, Payne AJ, Kahana R (2011) Optimal tuning of a GCM using modern and glacial constraints. Clim Dyn 37(3–4):705–719CrossRefGoogle Scholar
  25. Gregory J, Webb M (2008) Tropospheric adjustment induces a cloud component in CO2 forcing. J Clim 21(1):58–71CrossRefGoogle Scholar
  26. Harris G, Collins M, Sexton D, Murphy J, Booth B (2010) Probabilistic projections for 21st century european climate. Nat Hazards Earth Syst Sci 10(9):2009–2020CrossRefGoogle Scholar
  27. Harris GR, Sexton DM, Booth BB, Collins M, Murphy JM (2013) Probabilistic projections of transient climate change. Clim Dyn 40(11–12):2937–2972CrossRefGoogle Scholar
  28. Hazeleger W, Van den Hurk B, Min E, Van Oldenborgh G, Petersen A, Stainforth D, Vasileiadou E, Smith L (2015) Tales of future weather. Nat Clim Change 5(2):107–113CrossRefGoogle Scholar
  29. Hewitt H, Copsey D, Culverwell I, Harris C, Hill R, Keen A, McLaren A, Hunke E (2011) Design and implementation of the infrastructure of HadGEM3: The next-generation Met Office climate modelling system. Geosci Model Dev 4(2):223–253CrossRefGoogle Scholar
  30. Hourdin F, Mauritsen T, Gettelman A, Golaz JC, Balaji V, Duan Q, Folini D, Ji D, Klocke D, Qian Y (2017) The art and science of climate model tuning. Bull Am Meteorol Soc 98(3):589–602CrossRefGoogle Scholar
  31. Huffman GJ, Bolvin DT, Nelkin EJ, Wolff DB, Adler RF, Gu G, Hong Y, Bowman KP, Stocker EF (2007) The TRMM multisatellite precipitation analysis (TMPA): Quasi-global, multiyear, combined-sensor precipitation estimates at fine scales. J Hydrometeorol 8(1):38–55CrossRefGoogle Scholar
  32. Jackson CS, Sen MK, Huerta G, Deng Y, Bowman KP (2008) Error reduction and convergence in climate prediction. J Clim 21(24):6698–6709CrossRefGoogle Scholar
  33. Jackson LC, Vellinga M, Harris GR (2012) The sensitivity of the meridional overturning circulation to modelling uncertainty in a perturbed physics ensemble without flux adjustment. Clim Dyn 39(1–2):277–285CrossRefGoogle Scholar
  34. Klocke D, Rodwell M (2014) A comparison of two numerical weather prediction methods for diagnosing fast-physics errors in climate models. Q J R Meteorol Soc 140(679):517–524CrossRefGoogle Scholar
  35. Klocke D, Pincus R, Quaas J (2011) On constraining estimates of climate sensitivity with present-day observations through model weighting. J Clim 24(23):6092–6099CrossRefGoogle Scholar
  36. Knutti R, Furrer R, Tebaldi C, Cermak J, Meehl GA (2010) Challenges in combining projections from multiple climate models. J Clim 23(10):2739–2758CrossRefGoogle Scholar
  37. Lambert FH, Harris GR, Collins M, Murphy JM, Sexton DM, Booth BB (2013) Interactions between perturbations to different earth system components simulated by a fully-coupled climate model. Clim Dyn 41(11–12):3055–3072CrossRefGoogle Scholar
  38. Lee L, Carslaw K, Pringle K, Mann G, Spracklen D (2011) Emulation of a complex global aerosol model to quantify sensitivity to uncertain parameters. Atmos Chem Phys 11(23):12,253–12,273CrossRefGoogle Scholar
  39. Linz M, Tziperman E, MacMartin DG (2014) Process-based analysis of climate model ENSO simulations: intermodel consistency and compensating errors. J Geophys Res Atmos 119(12):7396–7409CrossRefGoogle Scholar
  40. Loeb NG, Wielicki BA, Doelling DR, Smith GL, Keyes DF, Kato S, Manalo-Smith N, Wong T (2009) Toward optimal closure of the Earth’s top-of-atmosphere radiation budget. J Clim 22(3):748–766CrossRefGoogle Scholar
  41. Loeppky JL, Sacks J, Welch WJ (2009) Choosing the sample size of a computer experiment: a practical guide. Technometrics 51(4):366–376CrossRefGoogle Scholar
  42. Lucas D, Klein R, Tannahill J, Ivanova D, Brandon S, Domyancic D, Zhang Y (2013) Failure analysis of parameter-induced simulation crashes in climate models. Geosci Model Dev 6(4):1157–1171CrossRefGoogle Scholar
  43. Ma HY, Xie S, Klein S, Williams K, Boyle J, Bony S, Douville H, Fermepin S, Medeiros B, Tyteca S (2014) On the correspondence between mean forecast errors and climate errors in CMIP5 models. J Clim 27(4):1781–1798CrossRefGoogle Scholar
  44. Martin G, Milton S, Senior C, Brooks M, Ineson S, Reichler T, Kim J (2010) Analysis and reduction of systematic errors through a seamless approach to modeling weather and climate. J Clim 23(22):5933–5957CrossRefGoogle Scholar
  45. Mauritsen T, Stevens B, Roeckner E, Crueger T, Esch M, Giorgetta M, Haak H, Jungclaus J, Klocke D, Matei D et al (2012) Tuning the climate of a global model. J Adv Model Earth Syst 4:M00A01.  https://doi.org/10.1029/2012MS000154 CrossRefGoogle Scholar
  46. McKay MD, Beckman RJ, Conover WJ (1979) Comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245Google Scholar
  47. McNeall D, Williams J, Booth B, Betts R, Challenor P, Wiltshire A, Sexton D (2016) The impact of structural error on parameter constraint in a climate model. Earth Syst Dyn 7:917–935.  https://doi.org/10.5194/esd-7-917-2016 CrossRefGoogle Scholar
  48. Meehl GA, Covey C, Taylor KE, Delworth T, Stouffer RJ, Latif M, McAvaney B, Mitchell JF (2007) The WCRP CMIP3 multimodel dataset: a new era in climate change research. Bull Am Meteorol Soc 88(9):1383–1394CrossRefGoogle Scholar
  49. Min SK, Simonis D, Hense A (2007) Probabilistic climate change predictions applying bayesian model averaging. Philos Trans R Soc Lond A 365(1857):2103–2116CrossRefGoogle Scholar
  50. Murphy JM, Sexton DM, Barnett DN, Jones GS, Webb MJ, Collins M, Stainforth DA (2004) Quantification of modelling uncertainties in a large ensemble of climate change simulations. Nature 430(7001):768–772CrossRefGoogle Scholar
  51. Murphy JM, Booth BB, Collins M, Harris GR, Sexton DM, Webb MJ (2007) A methodology for probabilistic predictions of regional climate change from perturbed physics ensembles. Philos Trans R Soc Lond A 365(1857):1993–2028CrossRefGoogle Scholar
  52. Murphy J, Sexton D, Jenkins G, Boorman P, Booth B, Brown C, Clark R, Collins M, Harris G, Kendon E (2009) UK climate projections science report: UKCP09. Met Office Hadley Centre, ExeterGoogle Scholar
  53. Murphy JM, Booth BB, Boulton CA, Clark RT, Harris GR, Lowe JA, Sexton DM (2014) Transient climate changes in a perturbed parameter ensemble of emissions-driven earth system model simulations. Clim Dyn 43(9–10):2855–2885CrossRefGoogle Scholar
  54. Neelin JD, Bracco A, Luo H, McWilliams JC, Meyerson JE (2010) Considerations for parameter optimization and sensitivity in climate models. Proc Natl Acad Sci 107(50):21,349–21,354CrossRefGoogle Scholar
  55. Olson R, Sriver R, Chang W, Haran M, Urban N, Keller K (2013) What is the effect of unresolved internal climate variability on climate sensitivity estimates? J Geophys Res Atmos 118(10):4348–4358CrossRefGoogle Scholar
  56. Phillips TJ, Potter GL, Williamson DL, Cederwall RT, Boyle JS, Fiorino M, Hnilo JJ, Olson JG, Xie S, Yio JJ (2004) Evaluating parameterizations in general circulation models: climate simulation meets weather prediction. Bull Am Meteorol Soc 85(12):1903–1915CrossRefGoogle Scholar
  57. Piani C, Frame D, Stainforth D, Allen M (2005) Constraints on climate change from a multi-thousand member ensemble of simulations. Geophys Res Lett 32(23):L23825.  https://doi.org/10.1029/2005GL024452 CrossRefGoogle Scholar
  58. Räisänen J, Palmer T (2001) A probability and decision-model analysis of a multimodel ensemble of climate change simulations. J Clim 14(15):3212–3226CrossRefGoogle Scholar
  59. Reichler T, Kim J (2008) How well do coupled models simulate today’s climate? Bull Am Meteorol Soc 89(3):303–311CrossRefGoogle Scholar
  60. Rienecker MM, Suarez MJ, Gelaro R, Todling R, Bacmeister J, Liu E, Bosilovich MG, Schubert SD, Takacs L, Kim GK (2011) MERRA: NASA’s modern-era retrospective analysis for research and applications. J Clim 24(14):3624–3648CrossRefGoogle Scholar
  61. Ringer MA, Andrews T, Webb MJ (2014) Global-mean radiative feedbacks and forcing in atmosphere-only and coupled atmosphere-ocean climate change experiments. Geophys Res Lett 41(11):4035–4042CrossRefGoogle Scholar
  62. Rodwell M, Palmer T (2007) Using numerical weather prediction to assess climate models. Q J R Meteorol Soc 133(622):129–146CrossRefGoogle Scholar
  63. Ross SM (2003) Peirce’s criterion for the elimination of suspect experimental data. J Eng Technol 20(2):38–41Google Scholar
  64. Rossow WB, Schiffer RA (1999) Advances in understanding clouds from ISCCP. Bull Am Meteorol Soc 80(11):2261–2287CrossRefGoogle Scholar
  65. Rougier J (2007) Probabilistic inference for future climate using an ensemble of climate model evaluations. Clim Change 81(3):247–264CrossRefGoogle Scholar
  66. Rougier JC, Sexton DMH (2007) Inference in ensemble experiments. Phil Trans R Soc A 365:2133–2143.  https://doi.org/10.1098/rsta.2007.2071 CrossRefGoogle Scholar
  67. Rougier J, Sexton DM, Murphy JM, Stainforth D (2009) Analyzing the climate sensitivity of the hadsm3 climate model using ensembles from different but related experiments. J Clim 22(13):3540–3557CrossRefGoogle Scholar
  68. Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65CrossRefGoogle Scholar
  69. Roustant O, Ginsbourger D, Deville Y (2012) Dicekriging, Diceoptim: Two R packages for the analysis of computer experiments by kriging-based metamodelling and optimization. J Stat Softw 51(1):54pCrossRefGoogle Scholar
  70. Rowell DP (1998) Assessing potential seasonal predictability with an ensemble of multidecadal GCM simulations. J Clim 11(2):109–120CrossRefGoogle Scholar
  71. Rowlands DJ, Frame DJ, Ackerley D, Aina T, Booth BB, Christensen C, Collins M, Faull N, Forest CE, Grandey BS (2012) Broad range of 2050 warming from an observationally constrained large climate model ensemble. Nat Geosci 5(4):256–260CrossRefGoogle Scholar
  72. Saltelli A, Chan K, Scott EM et al (2000) Sensitivity analysis, vol 1. Wiley, New YorkGoogle Scholar
  73. Scaife AA, Copsey D, Gordon C, Harris C, Hinton T, Keeley S, O’Neill A, Roberts M, Williams K (2011) Improved atlantic winter blocking in a climate model. Geophys Res Lett 38(23):L23703.  https://doi.org/10.1029/2011GL049573 CrossRefGoogle Scholar
  74. Schmidt GA, Bader D, Donner LJ, Elsaesser GS, Golaz JC, Hannay C, Molod A, Neale RB, Saha S (2017) Practice and philosophy of climate model tuning across six US modeling centers. Geosci Model Dev 10(9):3207CrossRefGoogle Scholar
  75. Sexton DM, Murphy JM (2012) Multivariate probabilistic projections using imperfect climate models. Part ii: robustness of methodological choices and consequences for climate sensitivity. Clim Dyn 38(11–12):2543–2558CrossRefGoogle Scholar
  76. Sexton DM, Murphy JM, Collins M, Webb MJ (2012) Multivariate probabilistic projections using imperfect climate models part i: outline of methodology. Clim Dyn 38(11–12):2513–2542CrossRefGoogle Scholar
  77. Sexton DMH, Karmalkar AV, Murphy JM, Williams KD, Boutle IA, Morcrette CJ, Stirling AJ, Vosper SB (2019) Finding plausible and diverse variants of a climate model. Part I: establishing the relationship between errors at weather and climate time scales. Clim Dyn.  https://doi.org/10.1007/s00382-019-04625-3 Google Scholar
  78. Shindell DT, Lamarque JF, Schulz M, Flanner M, Jiao C, Chin M, Young P, Lee YH, Rotstayn L, Mahowald N (2013) Radiative forcing in the accmip historical and future climate simulations. Atmos Chem Phys 13(6):2939–2974CrossRefGoogle Scholar
  79. Shiogama H, Watanabe M, Yoshimori M, Yokohata T, Ogura T, Annan JD, Hargreaves JC, Abe M, Kamae Y, Oishi R (2012) Perturbed physics ensemble using the miroc5 coupled atmosphere-ocean gcm without flux corrections: experimental design and results. Clim Dyn 39(12):3041–3056CrossRefGoogle Scholar
  80. Stainforth DA, Aina T, Christensen C, Collins M, Faull N, Frame DJ, Kettleborough JA, Knight S, Martin A, Murphy J et al (2005) Uncertainty in predictions of the climate response to rising levels of greenhouse gases. Nature 433(7024):403–406CrossRefGoogle Scholar
  81. Suzuki K, Golaz JC, Stephens GL (2013) Evaluating cloud tuning in a climate model with satellite observations. Geophys Res Lett 40(16):4464–4468CrossRefGoogle Scholar
  82. Taylor KE, Stouffer RJ, Meehl GA (2012) An overview of CMIP5 and the experiment design. Bull Am Meteorol Soc 93(4):485–498CrossRefGoogle Scholar
  83. Tebaldi C, Knutti R (2007) The use of the multi-model ensemble in probabilistic climate projections. Philos Trans R Soc Lond A 365(1857):2053–2075CrossRefGoogle Scholar
  84. Tett SF, Mitchell JF, Parker DE, Allen MR (1996) Human influence on the atmospheric vertical temperature structure: Detection and observations. Science 274(5290):1170–1173CrossRefGoogle Scholar
  85. Tokmakian R, Challenor P (2014) Uncertainty in modeled upper ocean heat content change. Clim Dyn 42(3–4):823–842CrossRefGoogle Scholar
  86. Walters D, Williams K, Boutle I, Bushell A, Edwards J, Field P, Lock A, Morcrette C, Stratton R, Wilkinson J et al (2014) The Met Office Unified Model global atmosphere 4.0 and JULES global land 4.0 configurations. Geosci Model Dev 7(1):361–386CrossRefGoogle Scholar
  87. Webb MJ, Lock AP (2013) Coupling between subtropical cloud feedback and the local hydrological cycle in a climate model. Clim Dyn 41(7–8):1923–1939CrossRefGoogle Scholar
  88. Webb MJ, Senior C, Sexton D, Ingram W, Williams K, Ringer M, McAvaney B, Colman R, Soden B, Gudgel R (2006) On the contribution of local feedback mechanisms to the range of climate sensitivity in two GCM ensembles. Clim Dyn 27(1):17–38CrossRefGoogle Scholar
  89. Webb MJ, Lambert FH, Gregory JM (2013) Origins of differences in climate sensitivity, forcing and feedback in climate models. Clim Dyn 40(3–4):677–707CrossRefGoogle Scholar
  90. Williams K, Bodas-Salcedo A, Déqué M, Fermepin S, Medeiros B, Watanabe M, Jakob C, Klein S, Senior C, Williamson D (2013) The Transpose-AMIP II experiment and its application to the understanding of southern ocean cloud biases in climate models. J Clim 26(10):3258–3274CrossRefGoogle Scholar
  91. Williamson D, Goldstein M, Allison L, Blaker A, Challenor P, Jackson L, Yamazaki K (2013) History matching for exploring and reducing climate model parameter space using observations and a large perturbed physics ensemble. Clim Dyn 41(7–8):1703–1729CrossRefGoogle Scholar
  92. Williamson D, Blaker AT, Hampton C, Salter J (2015) Identifying and removing structural biases in climate models with history matching. Clim Dyn 45(5–6):1299–1324CrossRefGoogle Scholar
  93. Williamson DB, Blaker AT, Sinha B (2017) Tuning without over-tuning: parametric uncertainty quantification for the NEMO ocean model. Geosci Model Dev 10(4):1789CrossRefGoogle Scholar
  94. Willmott CJ, Matsuura K (2005) Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim Res 30(1):79–82CrossRefGoogle Scholar
  95. Xie P, Arkin PA (1996) Analyses of global monthly precipitation using gauge observations, satellite estimates, and numerical model predictions. J Clim 9(4):840–858CrossRefGoogle Scholar
  96. Xie S, Ma HY, Boyle JS, Klein SA, Zhang Y (2012) On the correspondence between short-and long-time-scale systematic errors in CAM4/CAM5 for the year of tropical convection. J Clim 25(22):7937–7955CrossRefGoogle Scholar
  97. Yamazaki K, Rowlands DJ, Aina T, Blaker AT, Bowery A, Massey N, Miller J, Rye C, Tett SF, Williamson D (2013) Obtaining diverse behaviors in a climate model without the use of flux adjustments. J Geophys Res Atmos 118(7):2781–2793CrossRefGoogle Scholar
  98. Yokohata T, Webb MJ, Collins M, Williams KD, Yoshimori M, Hargreaves JC, Annan JD (2010) Structural similarities and differences in climate responses to CO2 increase between two perturbed physics ensembles. J Clim 23(6):1392–1410CrossRefGoogle Scholar
  99. Zelinka MD, Andrews T, Forster PM, Taylor KE (2014) Quantifying components of aerosol-cloud-radiation interactions in climate models. J Geophys Res Atmos 119(12):7599–7615CrossRefGoogle Scholar
  100. Zhang X, Zhang S, Liu Z, Wu X, Han G (2015) Parameter optimization in an intermediate coupled climate model with biased physics. J Clim 28(3):1227–1247CrossRefGoogle Scholar

Copyright information

© Crown 2019

Authors and Affiliations

  1. 1.Met OfficeExeterUK
  2. 2.Northeast Climate Adaptation Science Center and Dept of GeosciencesUniversity of Massachusetts AmherstAmherstUSA

Personalised recommendations