Finding plausible and diverse variants of a climate model. Part II: development and validation of methodology
The usefulness of a set of climate change projections largely depends on how well it spans a range of outcomes consistent with known uncertainties. Here, we present exploratory work towards developing a strategy to select variants of a state-of-the-art but expensive climate model suitable for climate projection studies. The strategy combines information from a set of relatively cheap, idealized perturbed parameter ensemble (PPE) and CMIP5 multi-model ensemble (MME) experiments, and uses two criteria as the basis to select model variants for a PPE suitable for future projections: (a) acceptable model performance at two different timescales, and (b) maintaining diversity in model response to climate change. This second part of a pair of papers builds upon Part I in which we established a strong relationship between model errors at weather and climate timescales across a PPE for a variety of key variables. This relationship is used to filter out parts of parameter space that do not give credible simulations of present day climate, while minimizing the impact on ranges in forcings and feedbacks that drive model responses to climate change. We use statistical emulation to explore the parameter space thoroughly, and demonstrate that about 90% can be filtered out without affecting diversity in global-scale climate change responses. This leads to the identification of plausible parts of parameter space from which model variants can be selected for projection studies. We selected and ran 50 variants from the plausible parameter combinations and validated the emulator predictions. Comparisons with the CMIP5 MME demonstrate that our approach can produce a set of plausible model variants that span a relatively wide range in model response to climate change. We also highlight how the prior expert-specified ranges for uncertain model parameters are constrained as a result of our methodology, and discuss recommendations for future work.
KeywordsUncertainty Perturbed parameter ensemble Seamless assessment Statistical emulation Filtering parameter space Plausible model variants
This research was supported by the Joint UK BEIS/Defra Met Office Hadley Centre Climate Programme (GA01101). John Rostron was supported by the UK-China Research & Innovation Partnership Fund through the Met Office Climate Science for Service Partnership (CSSP) China as part of the Newton Fund. We would like to thank Mark Webb, Mark Ringer, and Alejandro Bodas-Salcedo for their help with designing and setting up the experiments.
- Bony S, Webb M, Stevens B, Bretherton C, Klein S, Tselioudis G (2009) The cloud feedback model intercomparison project: summary of activities and recommendations for advancing assessments of cloud-climate feedbacks. CFMIP DocGoogle Scholar
- Flato G, Marotzke J, Abiodun B, Braconnot P, Chou SC, Collins WJ, Cox P, Driouech F, Emori S, Eyring V, et al (2013) Evaluation of climate models. In: Climate change 2013: the physical science basis. contribution of Working Group I to the fifth assessment report of the intergovernmental panel on climate change. Climate Change 2013 5:741–866Google Scholar
- McKay MD, Beckman RJ, Conover WJ (1979) Comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245Google Scholar
- Murphy J, Sexton D, Jenkins G, Boorman P, Booth B, Brown C, Clark R, Collins M, Harris G, Kendon E (2009) UK climate projections science report: UKCP09. Met Office Hadley Centre, ExeterGoogle Scholar
- Ross SM (2003) Peirce’s criterion for the elimination of suspect experimental data. J Eng Technol 20(2):38–41Google Scholar
- Saltelli A, Chan K, Scott EM et al (2000) Sensitivity analysis, vol 1. Wiley, New YorkGoogle Scholar
- Sexton DMH, Karmalkar AV, Murphy JM, Williams KD, Boutle IA, Morcrette CJ, Stirling AJ, Vosper SB (2019) Finding plausible and diverse variants of a climate model. Part I: establishing the relationship between errors at weather and climate time scales. Clim Dyn. https://doi.org/10.1007/s00382-019-04625-3 Google Scholar