Abstract
This chapter identifies conditions under which robust predictive modeling results have special epistemic significance—related to truth, confidence, and security—and considers whether those conditions are met in the context of climate modeling today. The findings are disappointing. When today’s climate models agree that an interesting hypothesis about future climate change is true, it cannot be inferred, via the arguments considered here anyway, that the hypothesis is likely to be true, nor that confidence in the hypothesis should be significantly increased, nor that a claim to have evidence for the hypothesis is now more secure. In some other modeling contexts, the prospects for such arguments are brighter.
This is a preview of subscription content, log in via an institution to check access.
Notes
- 1.
By an interesting predictive hypothesis , I mean a hypothesis about the future that scientists (i) do not already consider very likely to be true or very likely to be false and (ii) consider a priority for further investigation. In climate science today, these are typically, but not always, quantitative hypotheses about changes in global or regional climate on the timescale of several decades to centuries.
- 2.
When does an ensemble agree that a hypothesis is true? Assume that the values of model variables can be translated into statements regarding target system properties. Then a simulation indicates the truth (falsity) of some hypothesis H about a target system if its statements about the target system entail that H is true (false). For example, if H says that temperature will increase by between 1 °C and 1.5 °C, and each of the simulations in an ensemble indicates an increase between 1.2 °C and 1.4 °C, then each of those simulations indicates the truth of H and the ensemble is in agreement that H is true.
- 3.
I take agreement among modeling results to be synonymous with robustness, as is common in the climate modeling literature. Some authors define robustness differently (see e.g. Pirtle et al. 2010).
- 4.
Projections are predictions of what would happen under specified scenarios in which greenhouse gases and other climate forcing factors evolve in particular ways.
- 5.
- 6.
More precisely, average results for individual models were in agreement regarding the hypothesis; some models were run more than once with different initial conditions, and only average results for each model were shown in the main body of the report.
- 7.
Woodward (2006) notes the limited applicability of a related analysis.
- 8.
A simulation indicates correctly regarding a hypothesis H if it indicates the correct truth value for H.
- 9.
So, while we might not know which member(s) of the ensemble will indicate correctly regarding a given H, we have evidence that there is usually at least one such member in the ensemble.
- 10.
The “capturing truth” terminology is taken from Judd et al. (2007), which includes a related technical definition of the “bounding box” of an ensemble.
- 11.
Tuning a climate model involves making ad hoc changes to its parameter values or to the form of its equations in order to improve the fit between the model’s output and observational data.
- 12.
- 13.
It is important to recall the definition of “interesting hypotheses” given in Fn. 1. The conclusion here is fully compatible with there being some hypotheses about future climate that scientists can, with justification, consider likely to be true. The expectation that global climate will continue to warm, for instance, is grounded not just in agreement among predictions from complex climate models, but also in basic understanding of physical processes, theoretical analysis, observational data, and results from simpler models.
- 14.
Here I assume that p(H) takes a value between zero and one, i.e. it is not known to be certainly true or certainly false. From the updating rule, we see that p(H|e) > p(H) iff p(e|H)/p(e) > 1. But p(e|H)/p(e) > 1 iff p(e) < p(e|H). When is p(e) < p(e|H)? By the law of total probability, p(e) = p(e|H)×p(H) + p(e|~H)×p(~H). Since p(H) + p(~H) = 1, p(e) is in effect a weighted average of p(e|H) and p(e|~H); it takes a value between p(e|H) and p(e|~H). So p(e) will be smaller than p(e|H) iff p(e|H) > p(e|~H). So p(H|e) > p(H) iff p(e|H) > p(e|~H).
- 15.
The reasons given here are also discussed by Tebaldi and Knutti (2007).
- 16.
- 17.
For the sake of simplicity, the argument given here targets increased confidence, rather than significantly increased confidence. It is relatively easy to imagine how an analogous argument for significantly increased confidence might be given, once what counts as “significant” is defined in the case of interest.
- 18.
Note that from this it follows that p(e|H) > p(e|~H), so a Bayesian argument from robustness to increased confidence (similar to that of Sect. 9.4.1) could also be made.
- 19.
Odenbaugh (2012) considers how a relaxed version of the Condorcet Jury Theorem might be used to analyze the significance of scientific consensus (among experts, rather than models) regarding the existence and causes of global climate change.
- 20.
See Footnote 17.
- 21.
This is assuming that f can be defined for M B ; this issue is not addressed here. If f cannot be defined, then (5a) is also problematic.
- 22.
The present analysis expands upon the insightful but brief discussion given by Staley (2004, 474–475).
- 23.
Important questions about how the strength of evidence is defined and determined remain to be addressed; for the sake of discussion, it is assumed here that some reasonable and coherent analysis can be given.
- 24.
In fact, scientists may only believe that these assumptions are close enough to being true. For the sake of simplicity, this is ignored in the discussion above; including it would complicate but not undermine the argument.
- 25.
The mathematical logician typically uses a somewhat different notion of logical independence.
- 26.
Security can be enhanced more or less. Ceteris paribus, the closer the sets of auxiliary assumptions come to being fully logically independent of one another, the more security is enhanced. A set of assumptions A’ is fully logically independent of another set A if every assumption in A is such that, if that assumption is false, all of the assumptions in A’ could still be true. Security is also enhanced more, ceteris paribus, to the extent that it is not only possible that all of the assumptions in A’ could be true even while some assumption in A is false, but likely that all of the assumptions in A’ will be true if some assumption in A is false. For simplicity, the discussion above does not consider this quantitative aspect of enhanced security.
- 27.
Note that even if results from each climate model in an ensemble do have positive evidential relevance, this is not necessarily enough for the argument of Sect. 4.1 to work. That argument also depends upon the correlations among erroneous indications from the models, and even models that individually are more reliable than chance may nevertheless be more likely to agree in indicating that H is true when in fact it is false than when in fact it is true. Thanks to Dan Steel for reminding me to attend to connections between the discussion here and in Sect. 4.1.
- 28.
The claim here is not that individual modeling results have negative evidential relevance, but that their evidential status (with regard to interesting hypotheses about long-term climate change) is largely unknown.
- 29.
Of course, it does not follow that climate policy decisions should be put on hold. Expectations of a warmer world are well founded; the challenge is rather to make sensible decisions despite remaining uncertainties about the details of future climate change.
References
Abramowitz, Gabriel. 2010. Model Independence in Multi-Model Ensemble Prediction. Australian Meteorological and Oceanographic Journal 59: 3–6.
British Broadcasting Corporation (BBC). 2010. Climate Change Experiment Results. http://www.bbc.co.uk/sn/climateexperiment
Cartwright, Nancy, et al. 1991. Replicability, Reproducibility, and Robustness: Comments on Harry Collins. History of Political Economy 23 (1): 143–155.
Christensen, Jens Hesselbjerg, Bruce Hewitson, Aristita Busuioc, Anthony Chen, Xuejie Gao, R. Held, Richard Jones, et al. 2007. Chapter 11: Regional Climate Projections. In Climate Change, 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, 847–940. Cambridge: University Press.
Collins, M., R. Knutti, J. Arblaster, J.L. Dufresne, T. Fichefet, P. Friedlingstein, X. Gao, W.J. Gutowski, T. Johns, G. Krinner, and M. Shongwe. 2013. Long-Term Climate Change: Projections, Commitments and Irreversibility. In Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, 1029–1136. Cambridge: Cambridge University Press.
Edwards, Paul N. 1999. Global Climate Science, Uncertainty and Politics: Data-Laden Models, Model-Filtered Data. Science as Culture 8 (4): 437–472.
———. 2010. A Vast Machine: Computer Models, Climate Data, and the Politics of Global Warming. Cambridge, MA: MIT Press.
Gleckler, P.J., K.E. Taylor, and C. Doutriaux. 2008. Performance Metrics for Climate Models. Journal of Geophysical Research: Atmospheres 113 (D6): D06104. https://doi.org/10.1029/2007JD008972.
IPCC. 2013. In Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, ed. Thomas F. Stocker, Dahe Qin, Gian-Kasper Plattner, Melinda M.B. Tignor, Simon K. Allen, Judith Boschung, Alexander Nauels, Yu Xia, Vincent Bex, and Pauline M. Midgley. Cambridge: Cambridge University Press.
Judd, Kevin, Leonard A. Smith, and Antje Weisheimer. 2007. How Good Is an Ensemble at Capturing Truth? Using Bounding Boxes for Forecast Evaluation. Quarterly Journal of the Royal Meteorological Society 133 (626): 1309–1325.
Knutti, Reto, Myles R. Allen, Pierre Friedlingstein, Jonathan M. Gregory, Gabriele C. Hegerl, Gerald A. Meehl, Malte Meinshausen, et al. 2008. A Review of Uncertainties in Global Temperature Projections over the Twenty-First Century. Journal of Climate 21 (11): 2651–2663.
Knutti, Reto, et al. 2010. Challenges in Combining Projections from Multiple Climate Models. Journal of Climate 23: 2739–2758.
Ladha, Krishna K. 1992. The Condorcet Jury Theorem, Free Speech, and Correlated Votes. American Journal of Political Science 36 (3): 617–634.
———. 1995. Information Pooling through Majority-Rule Voting: Condorcet’s Jury Theorem with Correlated Votes. Journal of Economic Behavior & Organization 26 (3): 353–372.
Levins, Richard. 1966. The Strategy of Model Building in Population Biology. American Scientist 54 (4): 421–431.
———. 1993. A Response to Orzack and Sober: Formal Analysis and the Fluidity of Science. The Quarterly Review of Biology 68 (4): 547–555.
Lloyd, Elisabeth A. 2010. Confirmation and Robustness of Climate Models. Philosophy of Science. 77 (5): 971–984.
———. 2015. Model Robustness as a Confirmatory Virtue: The Case of Climate Science. Studies in History and Philosophy of Science Part A 49: 58–68.
Meehl, Gerard A., Thomas F. Stocker, William D. Collins, A.T. Friedlingstein, Amadou T. Gaye, Jonathan M. Gregory, Akio Kitoh, et al. 2007. Global Climate Projections. In Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, 747–845. New York: Cambridge University Press
Murphy, James M., Ben B.B. Booth, Mat Collins, Glen R. Harris, David M.H. Sexton, and Mark J. Webb. 2007. A Methodology for Probabilistic Predictions of Regional Climate Change from Perturbed Physics Ensembles. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 365 (1857): 1993–2028.
Odenbaugh, Jay. 2012. Climate, Consensus, and Contrarians. The Environment: Philosophy, Science, and Ethics, 137–150. Cambridge: MIT Press.
Orzack, Steven Hecht, and Elliott Sober. 1993. A Critical Assessment of Levins’s the Strategy of Model Building in Population Biology (1966). The Quarterly Review of Biology 68 (4): 533–546.
Owen, Guillermo, Bernard Grofman, and Scott L. Feld. 1989. Proving a Distribution-Free Generalization of the Condorcet Jury Theorem. Mathematical Social Sciences 17 (1): 1–16.
Parker, Wendy S. 2006. Understanding Pluralism in Climate Modeling. Foundations of Science 11 (4): 349–368.
———. 2009. II—Wendy S. Parker: Confirmation and Adequacy-for-Purpose in Climate Modelling. Aristotelian Society Supplementary Volume. 83: 233–249.
———. 2010. Whose Probabilities? Predicting Climate Change with Ensembles of Models. Philosophy of Science 77 (5): 985–997.
———. 2016. Reanalyses and Observations: What’s the Difference? Bulletin of the American Meteorological Society 97 (9): 1565–1572.
Pennell, Christopher, and Thomas Reichler. 2011. On the Effective Number of Climate Models. Journal of Climate 24 (9): 2358–2367.
Pirtle, Zachary, Ryan Meyer, and Andrew Hamilton. 2010. What Does It Mean When Climate Models Agree? A Case for Assessing Independence Among General Circulation Models. Environmental Science & Policy 13 (5): 351–361.
Randall, David A., Richard A. Wood, Sandrine Bony, Robert Colman, Thierry Fichefet, John Fyfe, Vladimir Kattsov, et al. 2007. Climate Models and Their Evaluation. In Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the IPCC, 589–662. Cambridge: Cambridge University Press.
Solomon, Susan, et al. 2007. Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. New York: Cambridge University Press.
Stainforth, David A., Tolu Aina, Carl Christensen, Mat Collins, Nick Faull, Dave J. Frame, Jamie A. Kettleborough, et al. 2005. Uncertainty in Predictions of the Climate Response to Rising Levels of Greenhouse Gases. Nature 433 (7024): 403–406.
Staley, Kent W. 2004. Robust Evidence and Secure Evidence Claims. Philosophy of Science 71 (4): 467–488.
Tebaldi, Claudia, and Reto Knutti. 2007. The Use of the Multi-Model Ensemble in Probabilistic Climate Projections. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 365 (1857): 2053–2075.
Tebaldi, Claudia, Richard L. Smith, Doug Nychka, and Linda O. Mearns. 2005. Quantifying Uncertainty in Projections of Regional Climate Change: A Bayesian Approach to the Analysis of Multimodel Ensembles. Journal of Climate 18 (10): 1524–1540.
Weisberg, Michael. 2006. Robustness Analysis. Philosophy of Science 73 (5): 730–742.
Wimsatt, William C. 2007. Robustness, Reliability, and Overdetermination. In Re-engineering Philosophy for Limited Beings: Piecewise Approximations to Reality. Cambridge: Harvard University Press.
Woodward, Jim. 2006. Some Varieties of Robustness. Journal of Economic Methodology 13 (2): 219–240.
Acknowledgments
This is a revised version of Parker, W.S. 2011. “When climate models agree: The significance of robust model predictions,” Philosophy of Science 78(4): 579–600. Thanks to University of Chicago Press for permission to republish substantial portions of that paper. I have benefitted from the suggestions and criticisms of Dan Steel, Reto Knutti, Kent Staley, Phil Ehrlich, Leonard Smith, Joel Katzav, Charlotte Werndl, and two anonymous referees for Philosophy of Science.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 The Author(s)
About this chapter
Cite this chapter
Parker, W.S. (2018). The Significance of Robust Climate Projections. In: A. Lloyd, E., Winsberg, E. (eds) Climate Modelling. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-319-65058-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-65058-6_9
Published:
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-319-65057-9
Online ISBN: 978-3-319-65058-6
eBook Packages: Earth and Environmental ScienceEarth and Environmental Science (R0)