Climatic Change

, Volume 116, Issue 2, pp 427–436 | Cite as

Do probabilistic expert elicitations capture scientists’ uncertainty about climate change?

  • Antony Millner
  • Raphael Calel
  • David A. Stainforth
  • George MacKerron


Expert elicitation studies have become important barometers of scientific knowledge about future climate change (Morgan and Keith, Environ Sci Technol 29(10), 1995; Reilly et al., Science 293(5529):430–433, 2001; Morgan et al., Climate Change 75(1–2):195–214, 2006; Zickfeld et al., Climatic Change 82(3–4):235–265, 2007, Proc Natl Acad Sci 2010; Kriegler et al., Proc Natl Acad Sci 106(13):5041–5046, 2009). Elicitations incorporate experts’ understanding of known flaws in climate models, thus potentially providing a more comprehensive picture of uncertainty than model-driven methods. The goal of standard elicitation procedures is to determine experts’ subjective probabilities for the values of key climate variables. These methods assume that experts’ knowledge can be captured by subjective probabilities—however, foundational work in decision theory has demonstrated this need not be the case when their information is ambiguous (Ellsberg, Q J Econ 75(4):643–669, 1961). We show that existing elicitation studies may qualitatively understate the extent of experts’ uncertainty about climate change. We designed a choice experiment that allows us to empirically determine whether experts’ knowledge about climate sensitivity (the equilibrium surface warming that results from a doubling of atmospheric CO2 concentration) can be captured by subjective probabilities. Our results show that, even for this much studied and well understood quantity, a non-negligible proportion of climate scientists violate the choice axioms that must be satisfied for subjective probabilities to adequately describe their beliefs. Moreover, the cause of their violation of the axioms is the ambiguity in their knowledge. We expect these results to hold to a greater extent for less understood climate variables, calling into question the veracity of previous elicitations for these quantities. Our experimental design provides an instrument for detecting ambiguity, a valuable new source of information when linking climate science and climate policy which can help policy makers select decision tools appropriate to our true state of knowledge.

Supplementary material

10584_2012_620_MOESM1_ESM.pdf (156 kb)
(PDF 155 KB)


  1. Allen M et al (2006) Observational constraints on climate sensitivity. In: Schellnhuber H, Cramer W, Nakicenovic N, Wigley T, Yohe G (eds) Avoiding dangerous climate change. Cambridge University Press, Cambridge, UK, p 406Google Scholar
  2. Barnard GA (1945) A new test for 2 × 2 tables. Nature 156(3954):177–177CrossRefGoogle Scholar
  3. Binmore K (2009). Rational decisions. Princeton University PressGoogle Scholar
  4. Camerer C, Weber M (1992) Recent developments in modeling preferences: uncertainty and ambiguity. J Risk Uncertain 5(4):325–370CrossRefGoogle Scholar
  5. De Finetti B (1937) La prévision : ses lois logiques, ses sources subjectives. Annales de l’Institut Henri Poincaré. Institut Henri Poincaré, ParisGoogle Scholar
  6. Ellsberg D (1961) Risk, ambiguity, and the Savage axioms. Q J Econ 75(4):643–669CrossRefGoogle Scholar
  7. Ellsberg D (2001) Risk, ambiguity and decision. RoutledgeGoogle Scholar
  8. Gilboa I (2009) Theory of decision under uncertainty, 1 edn. Cambridge University PressGoogle Scholar
  9. Gilboa I et al (2009) Is it always rational to satisfy Savage’s axioms? Econ Philos 25(3):285–296CrossRefGoogle Scholar
  10. Heath C, Tversky A (1991) Preference and belief: ambiguity and competence in choice under uncertainty. J Risk Uncertain 4(1):5–28CrossRefGoogle Scholar
  11. Kahneman D et al (1982) Judgment under uncertainty: heuristics and biases. Cambridge University PressGoogle Scholar
  12. Knutti R (2008) Should we believe model predictions of future climate change? Philos Trans R Soc A: Math Phys Sci 366(1885):4647–4664CrossRefGoogle Scholar
  13. Kriegler E et al (2009) Imprecise probability assessment of tipping points in the climate system. Proc Natl Acad Sci 106(13):5041–5046CrossRefGoogle Scholar
  14. Lemoine D, Traeger CP (2012) Tipping points and ambiguity in the economics of climate change. NBER working paper no 18230Google Scholar
  15. Machina MJ, Schmeidler D (1992) A more robust definition of subjective probability. Econometrica 60(4):745–780CrossRefGoogle Scholar
  16. Meinshausen M et al (2009) Greenhouse-gas emission targets for limiting global warming to 2C. Nature 458(7242):1158–1162CrossRefGoogle Scholar
  17. Millner A et al (2012) Scientific ambiguity and climate policy. Environ Resour Econ (forthcoming). doi:10.1007/s10640-012-9612-0 Google Scholar
  18. Morgan MG et al (2006) Elicitation of expert judgments of aerosol forcing. Climate Change 75(1–2):195–214CrossRefGoogle Scholar
  19. Morgan MG, Henrion M (1992) Uncertainty: a guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge University PressGoogle Scholar
  20. Morgan MG, Keith DW (1995) Subjective judgements by climate experts. Environ Sci Technol 29(10):468A–476AGoogle Scholar
  21. Nordhaus WD (2008) A question of balance. Yale University PressGoogle Scholar
  22. Ramsey F (1931) Truth and probability. In: Braithwaite R (ed) The foundations of mathematics and other logical essays. Kegan, Paul, Trench, Trubner & Co., London, Harcout, Brace and Company, New YorkGoogle Scholar
  23. Reilly J et al (2001) Uncertainty and climate change assessments. Science 293(5529):430–433CrossRefGoogle Scholar
  24. Resnik MD (1987) Choices: an introduction to decision theory. University of Minnesota PressGoogle Scholar
  25. Roe GH, Baker MB (2007) Why is climate sensitivity so unpredictable? Science 318(5850):629–632CrossRefGoogle Scholar
  26. Savage LJ (1954) The foundations of statistics. WileyGoogle Scholar
  27. Slovic P, Tversky A (1974) Who accepts Savage’s axiom? Behav Sci 19(6):368–373CrossRefGoogle Scholar
  28. Smith LA (2002) What might we learn from climate forecasts? Proc Natl Acad Sci USA 99(Suppl 1):2487–2492CrossRefGoogle Scholar
  29. Smith LA (2007) Chaos: a very short introduction, vol 159. Oxford University Press, OxfordGoogle Scholar
  30. Stainforth D et al (2007) Confidence, uncertainty and decision-support relevance in climate predictions. Philos Trans R Soc A: Math Phys Sci 365(1857):2145–2161CrossRefGoogle Scholar
  31. Stern NH (2007) The economics of climate change: the Stern review. Cambridge University Press, CambridgeGoogle Scholar
  32. Walley P (1990) Statistical reasoning with imprecise probabilities. Monographs on statistics and applied probability, vol 42. Chapman & HallGoogle Scholar
  33. Woodward RT, Bishop RC (1997) How to decide when experts disagree: uncertainty-based choice rules in environmental policy. Land Econ 73(4):492–507CrossRefGoogle Scholar
  34. Zickfeld K et al (2007) Expert judgements on the response of the Atlantic meridional overturning circulation to climate change. Climatic Change 82(3–4):235–265CrossRefGoogle Scholar
  35. Zickfeld K et al (2010) Expert judgments about transient climate response to alternative future trajectories of radiative forcing. Proc Natl Acad Sci 107(28):12451–12456CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2012

Authors and Affiliations

  • Antony Millner
    • 1
    • 2
  • Raphael Calel
    • 2
  • David A. Stainforth
    • 2
  • George MacKerron
    • 2
  1. 1.Department of Agricultural and Resource EconomicsUniversity of CaliforniaBerkeleyUSA
  2. 2.Grantham Research Institute on Climate Change and the EnvironmentLondon School of Economics and Political ScienceLondonUK

Personalised recommendations