Skip to main content

Wisdom of the experts: Using survey responses to address positive and normative uncertainties in climate-economic models


The social cost of carbon (SCC) and the climate-economic models underlying this prominent US climate policy instrument are heavily affected by modeler opinion and therefore may not reflect the views of most climate economists. To test whether differences exist, we recalibrate key uncertain model parameters using formal expert elicitation: a multi-question online survey of individuals who have published scholarship on the economics of climate change, with 165 to 216 respondents, depending on the question. Survey questions on the magnitude of climate impacts and appropriate discount rates revealed that prevailing views differ from prominent IAMs, including DICE. We calibrate the DICE damage functions and discount rates to reflect the mean and median survey responses, respectively, recognizing these two parameters’ differing sources of uncertainty (positive versus normative). We find a 16-fold higher SCC than the base DICE-2013R assumptions, with a range of 11- to 24-fold under alternative modeling assumptions (using the DICE-2016R2 model version and calibrating damages to median rather than mean responses). Our findings support a 7- to 13-fold SCC increase for different respondent subgroups even when we exclude the potential for catastrophic climate impact shocks. Our results reveal a significant disparity between IAMs and the broader community of scholars publishing in this field.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2


  1. 1.

    As an alternative, Pindyck (2019) develops a “simplified” model whose calibration requires economists to provide information outside their area of expertise, specifically the CO2 emission reductions necessary to avoid a 20% loss of GDP; this is problematic as expert judgment is “not appropriate when a field does not have relevant scientific expertise and related measurements” (Colson and Cooke 2018). In comparison, maintaining IAM structure allows our survey to avoid overly complex questions for which respondents do not have a prior benchmark.

  2. 2.

    The social choice–based approach starts with a group of individuals with heterogeneous preferences who must dynamically manage a consumption asset. To bridge the divide between the competing policy positions, a “representative” SDR is derived to ensure a Pareto-efficient consumption allocation over time. Differing setups and initial assumptions result in differing aggregation rules (Gollier and Zeckhauser 2005; Heal and Millner 2014).

  3. 3.

    In the setup of Millner and Heal (2018), a series of committees must each come to an agreement over the pure rate of time preference over which they disagree. As all committee members share the same elasticity of marginality utility of consumption by assumption, this is equivalent to selecting the social discount rate (by the simple Ramsey rule). According to the Millner-Heal voting procedure, the committee selects the median pure rate of time preference of its committee members.

  4. 4.

    Therefore, we calculate the more policy-relevant marginal SCC instead of an average SCC.

  5. 5.

    The DICE-2013R’s warming scenario is a 3 °C increase by 2080, regardless of the preference and damage function assumptions that we apply. Therefore, our damage function (and corresponding SCC estimates) should be considered a lower bound, as society will have 10 years less to adapt to a 3 °C change than our survey implies.

  6. 6.

    Placing open-ended questions on the second page of the survey likely leads to a reduced response rate for these questions, as some respondents did not complete the entire survey.

  7. 7.

    While DICE-2013R and DICE-2016R2 employ quadratic damage functions, earlier DICE models employed an inverse quadratic function that ensured damages were less than output. Nordhaus calibrated these earlier models using an enumerative method, which included a measurement of certainty-equivalent damages of catastrophic events. For DICE-2013R and DICE-2016R2, Nordhaus moved to a meta-analysis strategy that essentially omits catastrophic impacts (Howard 2014). As we did not ask respondents about the appropriate functional form or about climate damages for a higher temperature scenario (enabling the calibration of the DICE damage function’s exponent), we believe that it is prudent to maintain Nordhaus’ current assumptions.

  8. 8.

    Applying the arithmetic mean is the standard approach for aggregating survey responses and distributions (Lorenz et al. 2011) including for climate damage surveys (Nordhaus 1994b; Roughgarden and Schneider 1999; Gerst et al. 2010; Pindyck 2019). The popularity of the mean is based on the approach’s ability to minimize forecast error: (1) the sample mean collapses around the true value as sample size increases, according to the central limit theorem (Freeman and Groom 2015), and (2) recent work demonstrates that aggregating expert assessments using the mean outperforms the median in terms of accuracy of prediction (Colson and Cooke 2018).

  9. 9.

    This assumption is in line with 97% of responses to Question 5, but not strictly with the 3% of responses predicting that climate change will most likely have a 0% impact on GDP this century (according to their responses to Questions 5 and 13).

  10. 10.

    A potential interpretation of our calibrations of δ and η, which internalize these alternative discount rules, is as rough approximations of the representative rule (see Online Resource 4).

  11. 11.

    Additionally, Millner and Heal (2018) have a simple and transparent decision rule, and its setup of committees deciding on a “representative” SDR matches our structure of a survey of experts.

  12. 12.

    In Drupp et al. (2018), the coefficient of variation is 1.3 for δ and 0.6 for η. Additionally, in DICE, growth rates decline over time to steady state making uncertainty over η less relevant over time to the uncertainty of the social discount rate. Given the 300-year time horizon of DICE, the uncertainty over δ should be expected to dominate.

  13. 13.

    For example, our survey scenario focuses on a 3 °C increase by 2090 following Nordhaus (1994a); this 3 °C comes earlier in DICE-2016R2 than in DICE-2013R, i.e., 2075 instead of 2080, moving it further from our survey scenario.

  14. 14.

    Running DICE-2016R2 on its optimal setting, we find that a maximum temperature increase of 2 °C is impossible for any level of climate damages and preference parameter values. Exploring the damage and preference parameters, we find a minimum of 2.32° for the maximum temperature increase. This infeasibility limit is the result of Nordhaus (2018) recalibrating DICE-2016R2 to replicate long-run climate dynamics instead of short-run climate dynamics. As the infeasibility of a 2 °C maximum temperature breaks from mainstream global policy and diverges drastically from DICE-2013R (published only 3 years earlier), we are wary of this recalibration. As recalibrating the climate model is beyond the scope of this paper, we believe providing an SCC range with modeling from both DICE-2013R and DICE-2016R2 is more transparent for policymakers.

  15. 15.

    While Crost and Traeger (2013) find that running Monte Carlo simulations over uncertain parameters in deterministic IAMs can lead to incorrect optimal taxes, their results do not apply to the BAU emissions path upon which the SCC is calculated.

  16. 16.

    We find that our results are robust to recalibrating the initial level of capital; see Online Resource 7.

  17. 17.

    When we run DICE with a normative discount rate, the model shifts slightly off its traditional BAU scenario. As we cannot impose a descriptive rate, an alternative is to maintain DICE’s socioeconomic scenario (capital, output, population, and emissions) by constraining the savings rate to its base value. Essentially, this allows for a difference between private and social discount rates. We find our SCC estimates are robust (see Online Resource 7).

  18. 18.

    The Drupp et al. (2018) SDR survey had 181 respondents, while 157 economists (of 543 total respondents) responded to the Pindyck (2019) survey on climate economics.

  19. 19.

    There was almost universal agreement that there will be a negative effect by the end of the century (97%).

  20. 20.

    The mean responses for different subgroups of respondents (as defined by the number, type, and topic of their publications) ranged from − 6.3 to − 10.6%.

  21. 21.

    The mean responses for different subgroups of respondents ranged from 8.6 to 16%.

  22. 22.

    Assuming a normal distribution, we estimate this probability using the mean and standard deviation provided in the Supplementary Material of Nordhaus (2018). As Nordhaus never conducted a similar uncertainty analysis for DICE-2013R, a comparison with DICE-2013R is not possible.

  23. 23.

    Except for Question 15 on the probability of a catastrophic climate outcome, the above damage results are relatively unaffected by trimming inconsistent responses. For Question 15, trimming decreases the untrimmed mean of 22% by one-third. This highlights the need for careful attention in construction of climate surveys related to catastrophic impacts.

  24. 24.

    Given the equivalence of our results and Drupp et al. (2018) with respect to the SDR, it is safe to assume that the mean and median found in Drupp et al. (2018) with respect to the pure rate of time preference (δ) also apply to our sample. This supports our earlier assumption that the median δ equals the median value in Drupp et al. (2018).

  25. 25.

    For economist who responded, Pindyck (2019) finds mean and median discount rates of 2.7% and 2.0%, respectively.

  26. 26.

    The median value is unaffected by alternative trimming assumptions. However, the mean value is sensitive to trimming, with a value of 3.1% with no trimming.

  27. 27.

    This should be less of a concern for the SDR given our results match SDR experts from Drupp et al. (2018).

  28. 28.

    The ideal parameter is “measurable in theory but not in practice” with “related measurements” available (Colson and Cooke 2018).


  1. Ackerman F, Stanton EA, Bueno R (2010) Fat tails, exponents, extreme uncertainty: simulating catastrophe in DICE. Ecol Econ 69:1657–1665.

    Article  Google Scholar 

  2. Anthoff D, Tol RS (2014) The income elasticity of the impact of climate change. In: Tiezzi S, Martini C (eds) Is the environment a luxury? An inquiry into the relationship between environment and income. Routledge, New York, pp 34–47

    Google Scholar 

  3. Colson AR, Cooke RM (2018) Expert elicitation: using the classical model to validate experts’ judgments. Rev Environ Econ Policy 12:113–132.

    Article  Google Scholar 

  4. Cooke RM (2013) Uncertainty analysis comes to integrated assessment models for climate change and conversely. Clim Chang 117:467–479.

    Article  Google Scholar 

  5. Cooke RM, Goossens LL (2008) TU Delft expert judgment data base. Reliab Eng Syst Saf 93:657–674.

    Article  Google Scholar 

  6. Council of Economic Advisers (2017) Discounting for public policy: theory and recent evidence on the merits of updating the discount rate. Council of Economic Advisers. Accessed 26 June 2019

  7. Cropper ML, Freeman MC, Groom B, Pizer WA (2014) Declining discount rates. Am Econ Rev 104(5):538–543.

    Article  Google Scholar 

  8. Crost B, Traeger CP (2013) Optimal climate policy: uncertainty versus Monte Carlo. Econ Lett 120:552–558.

    Article  Google Scholar 

  9. Dasgupta P (2008) Discounting climate change. J Risk Uncertain 37:141–169.

    Article  Google Scholar 

  10. Drupp MA, Freeman MC, Groom B, Nesje F (2018) Discounting disentangled. Am Econ J Econ Pol 10:109–134.

    Article  Google Scholar 

  11. Fan W, Yan Z (2010) Factors affecting response rates of the web survey: a systematic review. Comput Hum Behav 26:132–139.

    Article  Google Scholar 

  12. Freeman MC, Groom B (2015) Positively gamma discounting: combining the opinions of experts on the social discount rate. Econ J 125:1015–1024.

    Article  Google Scholar 

  13. Gerst MD, Howarth RB, Borsuk ME (2010) Accounting for the risk of extreme outcomes in an integrated assessment of climate change. Energy Policy 38:4540–4548.

    Article  Google Scholar 

  14. Gollier C, Hammitt JK (2014) The long-run discount rate controversy. Ann Rev Resour Econ 6:273–295.

    Article  Google Scholar 

  15. Gollier C, Zeckhauser R (2005) Aggregation of heterogeneous time preferences. J Polit Econ 113:878–896.

    Article  Google Scholar 

  16. Heal G (2017) The economics of the climate. J Econ Lit 55:1046–1063.

    Article  Google Scholar 

  17. Heal G, Millner A (2013) Discounting under disagreement. National Bureau of Economic Research Working Paper 18999.

  18. Heal G, Millner A (2014) Agreeing to disagree on climate policy. Proc Natl Acad Sci U S A 111:3695–3698.

    Article  Google Scholar 

  19. Hope C (2013) Critical issues for the calculation of the social cost of CO 2: why the estimates from PAGE09 are higher than those from PAGE2002. Clim Chang 117:531–543.

    Article  Google Scholar 

  20. Howard P (2014) Omitted damages: what’s missing from the social cost of carbon. Institute for Policy Integrity, New York University School of Law. Accessed 13 March 2014

  21. Howard P, Sterner T (2017) Few and not so far between: a meta-analysis of climate damage estimates. Environ Resour Econ 68:197–225.

    Article  Google Scholar 

  22. Howard P, Sylvan D (2015) Expert consensus on the economics of climate change. Institute for Policy Integrity, New York University School of Law. Accessed December 2015

  23. Interagency Working Group on Social Cost of Carbon (2010) Social cost of carbon for regulatory impact analysis under executive order 12866. US Environmental Protection Agency. Accessed 26 June 2019

  24. Jones MS, House LA, Gao Z (2015) Respondent screening and revealed preference axioms: testing quarantining methods for enhanced data quality in web panel surveys. Public Opin Q 79:687–709.

    Article  Google Scholar 

  25. Kalaitzidakis P, Mamuneas TP, Stengos T (2003) Rankings of academic journals and institutions in economics. J Eur Econ Assoc 1:1346–1366.

    Article  Google Scholar 

  26. Kalaitzidakis P, Mamuneas TP, Stengos T (2011) An updated ranking of academic journals in economics. Can J Econ 44:1525–1538.

    Article  Google Scholar 

  27. Lorenz J, Rauhut H, Schweitzer F, Helbing D (2011) How social influence can undermine the wisdom of crowd effect. Proc Natl Acad Sci U S A 108:9020–9025

    Article  Google Scholar 

  28. Marten AL, Kopits EA, Griffiths CW, Newbold SC, Wolverton A (2015) Incremental CH4 and N2O mitigation benefits consistent with the US Government’s SC-CO2 estimates. Clim Pol 15:272–298.

    Article  Google Scholar 

  29. Miller J (2006) Research reveals alarming incidence of ‘undesirable’ online panelists. Research Conference Report, RFL Communications, Inc. Accessed 4 August 2016

  30. Millner A, Heal G (2018) Discounting by committee. J Public Econ 167:91–104

    Article  Google Scholar 

  31. Nordhaus WD (1994a) Expert opinion on climatic change. Am Sci 82:45–51

    Google Scholar 

  32. Nordhaus WD (1994b) Managing the global commons: the economics of climate change, vol 31. MIT press, Cambridge

    Google Scholar 

  33. Nordhaus WD (2014) Estimates of the social cost of carbon: concepts and results from the DICE-2013R model and alternative approaches. J Assoc Environ Resour Econ 1:273–312.

    Article  Google Scholar 

  34. Nordhaus WD (2017) Revisiting the social cost of carbon. Proc Natl Acad Sci U S A 144:1518–1523.

    Article  Google Scholar 

  35. Nordhaus W (2018) Evolution of modeling of the economics of global warming: changes in the DICE model, 1992–2017. Clim Chang 148:623–640

    Article  Google Scholar 

  36. Nordhaus W (2019) Climate change: the ultimate challenge for economics. Am Econ Rev 109(6):1991–2014

    Article  Google Scholar 

  37. Oppenheimer M, Little CM, Cooke RM (2016) Expert judgement and uncertainty quantification for climate change. Nat Clim Chang 6:445–451.

    Article  Google Scholar 

  38. Pindyck RS (2017) The use and misuse of models for climate policy. Rev Environ Econ Policy 11:100–114.

    Article  Google Scholar 

  39. Pindyck RS (2019) The social cost of carbon revisited. J Environ Econ Manag 94:140–160.

    Article  Google Scholar 

  40. Revesz RL, Howard PH, Arrow K, Goulder LH, Kopp RE, Livermore MA, Sterner T (2014) Global warming: improve economic models of climate change. Nature 508:173–175.

    Article  Google Scholar 

  41. Roughgarden T, Schneider SH (1999) Climate change policy: quantifying uncertainties for damages and optimal carbon taxes. Energy Policy 27:415–429.

    Article  Google Scholar 

  42. Rousseau S (2008) Journal evaluation by environmental and resource economists: a survey. Scientometrics 77:223–233.

    Article  Google Scholar 

  43. Rousseau S, Verbeke T, Rousseau R (2009) Evaluating environmental and resource economics journals: a TOP-curve approach. Rev Environ Econ Policy 3:270–287.

    Article  Google Scholar 

  44. Schauer MJ (1995) Estimation of the greenhouse gas externality with uncertainty. Environ Resour Econ 5:71–82.

    Article  Google Scholar 

  45. Sheehan KB (2001) E-mail survey response rates: a review. J Comp-Mediat Commun 6.

  46. Tol RSJ (2013) The economic impact of climate change in the 20th and 21st centuries. Clim Chang 117:795–808.

    Article  Google Scholar 

  47. Weitzman ML (2001) Gamma discounting. Am Econ Rev 91:260–271.

    Article  Google Scholar 

  48. Weitzman ML (2007) A review of the stern review of the economics of climate change. J Econ Lit 45:703–724.

    Article  Google Scholar 

  49. Weitzman ML (2012) GHG targets as insurance against catastrophic climate damages. J Public Econ Theory 14(2):221–244.

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Peter Harrison Howard.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material


(DO 17 kb)


(DO 16 kb)


(XLSX 209688 kb)


(XLSX 36 kb)


(XLSX 734 kb)


(XLSX 24 kb)


(XLSX 3669 kb)


(XLSX 277 kb)


(XLSX 157 kb)

ESM 10

(PDF 1004 kb)

ESM 11

(DO 8 kb)

ESM 12

(XLSX 19 kb)

ESM 13

(DO 3 kb)

ESM 14

(XLSX 14 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Howard, P.H., Sylvan, D. Wisdom of the experts: Using survey responses to address positive and normative uncertainties in climate-economic models. Climatic Change 162, 213–232 (2020).

Download citation


  • Social cost of carbon (SCC)
  • Climate-economic models
  • IAMs
  • Expert elicitation