Skip to main content

The Significance of Robust Climate Projections


This chapter identifies conditions under which robust predictive modeling results have special epistemic significance—related to truth, confidence, and security—and considers whether those conditions are met in the context of climate modeling today. The findings are disappointing. When today’s climate models agree that an interesting hypothesis about future climate change is true, it cannot be inferred, via the arguments considered here anyway, that the hypothesis is likely to be true, nor that confidence in the hypothesis should be significantly increased, nor that a claim to have evidence for the hypothesis is now more secure. In some other modeling contexts, the prospects for such arguments are brighter.

This is a preview of subscription content, access via your institution.


  1. 1.

    By an interesting predictive hypothesis , I mean a hypothesis about the future that scientists (i) do not already consider very likely to be true or very likely to be false and (ii) consider a priority for further investigation. In climate science today, these are typically, but not always, quantitative hypotheses about changes in global or regional climate on the timescale of several decades to centuries.

  2. 2.

    When does an ensemble agree that a hypothesis is true? Assume that the values of model variables can be translated into statements regarding target system properties. Then a simulation indicates the truth (falsity) of some hypothesis H about a target system if its statements about the target system entail that H is true (false). For example, if H says that temperature will increase by between 1 °C and 1.5 °C, and each of the simulations in an ensemble indicates an increase between 1.2 °C and 1.4 °C, then each of those simulations indicates the truth of H and the ensemble is in agreement that H is true.

  3. 3.

    I take agreement among modeling results to be synonymous with robustness, as is common in the climate modeling literature. Some authors define robustness differently (see e.g. Pirtle et al. 2010).

  4. 4.

    Projections are predictions of what would happen under specified scenarios in which greenhouse gases and other climate forcing factors evolve in particular ways.

  5. 5.

    In part, this is because it is difficult to determine what a model’s performance in simulating past and present climate indicates about its accuracy in predicting various quantities of interest (see Randall et al. 2007; Gleckler et al. 2008; Parker 2009).

  6. 6.

    More precisely, average results for individual models were in agreement regarding the hypothesis; some models were run more than once with different initial conditions, and only average results for each model were shown in the main body of the report.

  7. 7.

    Woodward (2006) notes the limited applicability of a related analysis.

  8. 8.

    A simulation indicates correctly regarding a hypothesis H if it indicates the correct truth value for H.

  9. 9.

    So, while we might not know which member(s) of the ensemble will indicate correctly regarding a given H, we have evidence that there is usually at least one such member in the ensemble.

  10. 10.

    The “capturing truth” terminology is taken from Judd et al. (2007), which includes a related technical definition of the “bounding box” of an ensemble.

  11. 11.

    Tuning a climate model involves making ad hoc changes to its parameter values or to the form of its equations in order to improve the fit between the model’s output and observational data.

  12. 12.

    See Edwards (1999, 2010) and Parker (2016), for non-technical discussions of reanalysis datasets.

  13. 13.

    It is important to recall the definition of “interesting hypotheses” given in Fn. 1. The conclusion here is fully compatible with there being some hypotheses about future climate that scientists can, with justification, consider likely to be true. The expectation that global climate will continue to warm, for instance, is grounded not just in agreement among predictions from complex climate models, but also in basic understanding of physical processes, theoretical analysis, observational data, and results from simpler models.

  14. 14.

    Here I assume that p(H) takes a value between zero and one, i.e. it is not known to be certainly true or certainly false. From the updating rule, we see that p(H|e) > p(H) iff p(e|H)/p(e) > 1. But p(e|H)/p(e) > 1 iff p(e) < p(e|H). When is p(e) < p(e|H)? By the law of total probability, p(e) = p(e|H)×p(H) + p(e|~H)×p(~H). Since p(H) + p(~H) = 1, p(e) is in effect a weighted average of p(e|H) and p(e|~H); it takes a value between p(e|H) and p(e|~H). So p(e) will be smaller than p(e|H) iff p(e|H) > p(e|~H). So p(H|e) > p(H) iff p(e|H) > p(e|~H).

  15. 15.

    The reasons given here are also discussed by Tebaldi and Knutti (2007).

  16. 16.

    Wimsatt (2007) discusses a case in biology in which an apparently robust modeling result turned out to be grounded in erroneous assumptions shared by the models. See also Orzack and Sober (1993, 539).

  17. 17.

    For the sake of simplicity, the argument given here targets increased confidence, rather than significantly increased confidence. It is relatively easy to imagine how an analogous argument for significantly increased confidence might be given, once what counts as “significant” is defined in the case of interest.

  18. 18.

    Note that from this it follows that p(e|H) > p(e|~H), so a Bayesian argument from robustness to increased confidence (similar to that of Sect. 9.4.1) could also be made.

  19. 19.

    Odenbaugh (2012) considers how a relaxed version of the Condorcet Jury Theorem might be used to analyze the significance of scientific consensus (among experts, rather than models) regarding the existence and causes of global climate change.

  20. 20.

    See Footnote 17.

  21. 21.

    This is assuming that f can be defined for M B ; this issue is not addressed here. If f cannot be defined, then (5a) is also problematic.

  22. 22.

    The present analysis expands upon the insightful but brief discussion given by Staley (2004, 474–475).

  23. 23.

    Important questions about how the strength of evidence is defined and determined remain to be addressed; for the sake of discussion, it is assumed here that some reasonable and coherent analysis can be given.

  24. 24.

    In fact, scientists may only believe that these assumptions are close enough to being true. For the sake of simplicity, this is ignored in the discussion above; including it would complicate but not undermine the argument.

  25. 25.

    The mathematical logician typically uses a somewhat different notion of logical independence.

  26. 26.

    Security can be enhanced more or less. Ceteris paribus, the closer the sets of auxiliary assumptions come to being fully logically independent of one another, the more security is enhanced. A set of assumptions A’ is fully logically independent of another set A if every assumption in A is such that, if that assumption is false, all of the assumptions in A’ could still be true. Security is also enhanced more, ceteris paribus, to the extent that it is not only possible that all of the assumptions in A’ could be true even while some assumption in A is false, but likely that all of the assumptions in A’ will be true if some assumption in A is false. For simplicity, the discussion above does not consider this quantitative aspect of enhanced security.

  27. 27.

    Note that even if results from each climate model in an ensemble do have positive evidential relevance, this is not necessarily enough for the argument of Sect. 4.1 to work. That argument also depends upon the correlations among erroneous indications from the models, and even models that individually are more reliable than chance may nevertheless be more likely to agree in indicating that H is true when in fact it is false than when in fact it is true. Thanks to Dan Steel for reminding me to attend to connections between the discussion here and in Sect. 4.1.

  28. 28.

    The claim here is not that individual modeling results have negative evidential relevance, but that their evidential status (with regard to interesting hypotheses about long-term climate change) is largely unknown.

  29. 29.

    Of course, it does not follow that climate policy decisions should be put on hold. Expectations of a warmer world are well founded; the challenge is rather to make sensible decisions despite remaining uncertainties about the details of future climate change.


  • Abramowitz, Gabriel. 2010. Model Independence in Multi-Model Ensemble Prediction. Australian Meteorological and Oceanographic Journal 59: 3–6.

    CrossRef  Google Scholar 

  • British Broadcasting Corporation (BBC). 2010. Climate Change Experiment Results.

  • Cartwright, Nancy, et al. 1991. Replicability, Reproducibility, and Robustness: Comments on Harry Collins. History of Political Economy 23 (1): 143–155.

    CrossRef  Google Scholar 

  • Christensen, Jens Hesselbjerg, Bruce Hewitson, Aristita Busuioc, Anthony Chen, Xuejie Gao, R. Held, Richard Jones, et al. 2007. Chapter 11: Regional Climate Projections. In Climate Change, 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, 847–940. Cambridge: University Press.

    Google Scholar 

  • Collins, M., R. Knutti, J. Arblaster, J.L. Dufresne, T. Fichefet, P. Friedlingstein, X. Gao, W.J. Gutowski, T. Johns, G. Krinner, and M. Shongwe. 2013. Long-Term Climate Change: Projections, Commitments and Irreversibility. In Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, 1029–1136. Cambridge: Cambridge University Press.

    Google Scholar 

  • Edwards, Paul N. 1999. Global Climate Science, Uncertainty and Politics: Data-Laden Models, Model-Filtered Data. Science as Culture 8 (4): 437–472.

    CrossRef  Google Scholar 

  • ———. 2010. A Vast Machine: Computer Models, Climate Data, and the Politics of Global Warming. Cambridge, MA: MIT Press.

    Google Scholar 

  • Gleckler, P.J., K.E. Taylor, and C. Doutriaux. 2008. Performance Metrics for Climate Models. Journal of Geophysical Research: Atmospheres 113 (D6): D06104.

    CrossRef  Google Scholar 

  • IPCC. 2013. In Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, ed. Thomas F. Stocker, Dahe Qin, Gian-Kasper Plattner, Melinda M.B. Tignor, Simon K. Allen, Judith Boschung, Alexander Nauels, Yu Xia, Vincent Bex, and Pauline M. Midgley. Cambridge: Cambridge University Press.

    Google Scholar 

  • Judd, Kevin, Leonard A. Smith, and Antje Weisheimer. 2007. How Good Is an Ensemble at Capturing Truth? Using Bounding Boxes for Forecast Evaluation. Quarterly Journal of the Royal Meteorological Society 133 (626): 1309–1325.

    CrossRef  Google Scholar 

  • Knutti, Reto, Myles R. Allen, Pierre Friedlingstein, Jonathan M. Gregory, Gabriele C. Hegerl, Gerald A. Meehl, Malte Meinshausen, et al. 2008. A Review of Uncertainties in Global Temperature Projections over the Twenty-First Century. Journal of Climate 21 (11): 2651–2663.

    CrossRef  Google Scholar 

  • Knutti, Reto, et al. 2010. Challenges in Combining Projections from Multiple Climate Models. Journal of Climate 23: 2739–2758.

    CrossRef  Google Scholar 

  • Ladha, Krishna K. 1992. The Condorcet Jury Theorem, Free Speech, and Correlated Votes. American Journal of Political Science 36 (3): 617–634.

    Google Scholar 

  • ———. 1995. Information Pooling through Majority-Rule Voting: Condorcet’s Jury Theorem with Correlated Votes. Journal of Economic Behavior & Organization 26 (3): 353–372.

    CrossRef  Google Scholar 

  • Levins, Richard. 1966. The Strategy of Model Building in Population Biology. American Scientist 54 (4): 421–431.

    Google Scholar 

  • ———. 1993. A Response to Orzack and Sober: Formal Analysis and the Fluidity of Science. The Quarterly Review of Biology 68 (4): 547–555.

    CrossRef  Google Scholar 

  • Lloyd, Elisabeth A. 2010. Confirmation and Robustness of Climate Models. Philosophy of Science. 77 (5): 971–984.

    Google Scholar 

  • ———. 2015. Model Robustness as a Confirmatory Virtue: The Case of Climate Science. Studies in History and Philosophy of Science Part A 49: 58–68.

    CrossRef  Google Scholar 

  • Meehl, Gerard A., Thomas F. Stocker, William D. Collins, A.T. Friedlingstein, Amadou T. Gaye, Jonathan M. Gregory, Akio Kitoh, et al. 2007. Global Climate Projections. In Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, 747–845. New York: Cambridge University Press

    Google Scholar 

  • Murphy, James M., Ben B.B. Booth, Mat Collins, Glen R. Harris, David M.H. Sexton, and Mark J. Webb. 2007. A Methodology for Probabilistic Predictions of Regional Climate Change from Perturbed Physics Ensembles. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 365 (1857): 1993–2028.

    CAS  CrossRef  Google Scholar 

  • Odenbaugh, Jay. 2012. Climate, Consensus, and Contrarians. The Environment: Philosophy, Science, and Ethics, 137–150. Cambridge: MIT Press.

    Google Scholar 

  • Orzack, Steven Hecht, and Elliott Sober. 1993. A Critical Assessment of Levins’s the Strategy of Model Building in Population Biology (1966). The Quarterly Review of Biology 68 (4): 533–546.

    CrossRef  Google Scholar 

  • Owen, Guillermo, Bernard Grofman, and Scott L. Feld. 1989. Proving a Distribution-Free Generalization of the Condorcet Jury Theorem. Mathematical Social Sciences 17 (1): 1–16.

    CrossRef  Google Scholar 

  • Parker, Wendy S. 2006. Understanding Pluralism in Climate Modeling. Foundations of Science 11 (4): 349–368.

    CrossRef  Google Scholar 

  • ———. 2009. II—Wendy S. Parker: Confirmation and Adequacy-for-Purpose in Climate Modelling. Aristotelian Society Supplementary Volume. 83: 233–249.

    Google Scholar 

  • ———. 2010. Whose Probabilities? Predicting Climate Change with Ensembles of Models. Philosophy of Science 77 (5): 985–997.

    Google Scholar 

  • ———. 2016. Reanalyses and Observations: What’s the Difference? Bulletin of the American Meteorological Society 97 (9): 1565–1572.

    CrossRef  Google Scholar 

  • Pennell, Christopher, and Thomas Reichler. 2011. On the Effective Number of Climate Models. Journal of Climate 24 (9): 2358–2367.

    CrossRef  Google Scholar 

  • Pirtle, Zachary, Ryan Meyer, and Andrew Hamilton. 2010. What Does It Mean When Climate Models Agree? A Case for Assessing Independence Among General Circulation Models. Environmental Science & Policy 13 (5): 351–361.

    CrossRef  Google Scholar 

  • Randall, David A., Richard A. Wood, Sandrine Bony, Robert Colman, Thierry Fichefet, John Fyfe, Vladimir Kattsov, et al. 2007. Climate Models and Their Evaluation. In Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the IPCC, 589–662. Cambridge: Cambridge University Press.

    Google Scholar 

  • Solomon, Susan, et al. 2007. Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. New York: Cambridge University Press.

    Google Scholar 

  • Stainforth, David A., Tolu Aina, Carl Christensen, Mat Collins, Nick Faull, Dave J. Frame, Jamie A. Kettleborough, et al. 2005. Uncertainty in Predictions of the Climate Response to Rising Levels of Greenhouse Gases. Nature 433 (7024): 403–406.

    CAS  CrossRef  Google Scholar 

  • Staley, Kent W. 2004. Robust Evidence and Secure Evidence Claims. Philosophy of Science 71 (4): 467–488.

    CrossRef  Google Scholar 

  • Tebaldi, Claudia, and Reto Knutti. 2007. The Use of the Multi-Model Ensemble in Probabilistic Climate Projections. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 365 (1857): 2053–2075.

    CrossRef  Google Scholar 

  • Tebaldi, Claudia, Richard L. Smith, Doug Nychka, and Linda O. Mearns. 2005. Quantifying Uncertainty in Projections of Regional Climate Change: A Bayesian Approach to the Analysis of Multimodel Ensembles. Journal of Climate 18 (10): 1524–1540.

    CrossRef  Google Scholar 

  • Weisberg, Michael. 2006. Robustness Analysis. Philosophy of Science 73 (5): 730–742.

    CrossRef  Google Scholar 

  • Wimsatt, William C. 2007. Robustness, Reliability, and Overdetermination. In Re-engineering Philosophy for Limited Beings: Piecewise Approximations to Reality. Cambridge: Harvard University Press.

    Google Scholar 

  • Woodward, Jim. 2006. Some Varieties of Robustness. Journal of Economic Methodology 13 (2): 219–240.

    CrossRef  Google Scholar 

Download references


This is a revised version of Parker, W.S. 2011. “When climate models agree: The significance of robust model predictions,” Philosophy of Science 78(4): 579–600. Thanks to University of Chicago Press for permission to republish substantial portions of that paper. I have benefitted from the suggestions and criticisms of Dan Steel, Reto Knutti, Kent Staley, Phil Ehrlich, Leonard Smith, Joel Katzav, Charlotte Werndl, and two anonymous referees for Philosophy of Science.

Author information

Authors and Affiliations


Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 The Author(s)

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Parker, W.S. (2018). The Significance of Robust Climate Projections. In: A. Lloyd, E., Winsberg, E. (eds) Climate Modelling. Palgrave Macmillan, Cham.

Download citation