Uncertainty Quantification Using Multiple Models—Prospects and Challenges
Model evaluation for long-term climate predictions must be done on quantities other than the actual prediction, and a comprehensive uncertainty quantification is impossible. An ad hoc alternative is provided by coordinated model intercomparisons which typically use a “one model one vote” approach. The problem with such an approach is that it treats all models as independent and equally plausible. Reweighting all models of the ensemble for performance and dependence seems like an obvious way to improve on model democracy, yet there are open questions on what constitutes a “good” model, how to define dependency, how to interpret robustness, and how to incorporate background knowledge. Understanding those issues have the potential to increase confidence in model predictions in modeling efforts outside of climate science where similar challenges exist.
KeywordsEnsemble modeling Idealization Model independence Robustness Structural model uncertainty Uncertainty quantification
We thank Claus Beisbart, Keith Beven, Benedikt Knüsel, Wendy Parker, and Marius Wälchli for discussions and comments which have improved the quality of this manuscript. This project was supported by the European Union’s Horizon 2020 research and innovation program under grant agreement No 641816 (CRESCENDO). We acknowledge the World Climate Research Programme’s Working Group on Coupled Modelling, which is responsible for CMIP, and we thank the climate modeling groups for producing and making available their model output. For CMIP, the U.S. Department of Energy’s Program for Climate Model Diagnosis and Intercomparison provides coordinating support and led the development of software infrastructure in partnership with the Global Organization for Earth System Science Portals.
- Annan, J., & Hargreaves, J. (2016). On the meaning of independence in climate science. Earth System Dynamics Discussions, 1–17. https://doi.org/10.5194/esd-2016-34.
- Bellouin, N., et al. (2011). The HadGEM2 family of Met Office Unified Model climate configurations. Geoscientific Model Development, 4(3), 723–757. https://doi.org/10.5194/gmd-4-723-2011.
- Knutti, R., Abramowitz, G., et al. (2010b). Good practice guidance paper on assessing and combining multi model climate projections, meeting report of the intergovernmental panel on climate change expert meeting on assessing and combining multi model climate projections. In T. F. Stocker, et al. (Eds.), IPCC working group I technical support unit. Switzerland: University of Bern, Bern.Google Scholar
- Lenhard, J., & Winsberg, E. (2010). Holism, entrenchment, and the future of climate model pluralism. Studies in History and Philosophy of Science Part B: Studies in History and Philosophy of Modern Physics, 41(3), 253–262. (Elsevier). https://doi.org/10.1016/j.shpsb.2010.07.001.CrossRefGoogle Scholar
- Levins, R. (1966). The strategy of model building in population biology. American Naturalist, 421–431. https://doi.org/10.2307/27836590.
- Oreskes, N., Shrader-Frechette, K., & Belitz, K. (1994). Verification, validation, and confirmation of numerical models in the earth sciences. Science, 263(5147), 641. AAAS. Retrieved June 4, 2014, from http://www.sciencemag.org/cgi/content/abstract/sci;263/5147/641.
- Parker, W. S. (2006). Understanding pluralism in climate modeling. Foundations of Science, 11(4), 349–368. (Springer). http://www.springerlink.com/index/138424X1082M7277.pdf.
- Schupbach, J. N. (2016). Robustness analysis as explanatory reasoning. The British Journal for the Philosophy of Science, 69(February), axw008. https://doi.org/10.1093/bjps/axw008.
- Stegenga, J., & Menon, T. (2017). Robustness and independent evidence. 84(July), 414–435. http://www.journals.uchicago.edu/doi/10.1086/692141.
- Wimsatt, W. C. (2012). Robustness, reliability, and overdetermination (1981). In L. Soler et al. (Eds.), Characterizing the robustness of science: After the practice turn in philosophy of science (pp. 61–87). Dordrecht: Springer Netherlands. https://doi.org/10.1007/978-94-007-2759-5_2.CrossRefGoogle Scholar