Abstract
Given the vast uncertainty surrounding climate impacts, metaanalyses of global climate damage estimates are a key tool for determining the relationship between temperature and climate damages. Due to limited data availability, previous metaanalyses of global climate damages potentially suffered from multiple sources of coefficient and standard error bias: duplicate estimates, omitted variables, measurement error, overreliance on published estimates, dependent errors, and heteroskedasticity. To address and test for these biases, we expand on previous datasets to obtain sufficient degrees of freedom to make the necessary model adjustments, including dropping duplicate estimates and including methodological variables. Estimating the relationship between temperature and climate damages using weighted least squares with clusterrobust standard errors, we find strong evidence that duplicate and omitted variable biases flatten the relationship. However, the magnitude of the bias greatly depends on the treatment of speculative hightemperature (>4 \(^{\circ }\)C) damage estimates. Replacing the DICE2013R damage function with our preferred estimate of the temperature–damage relationship, we find a three to fourfold increase in the 2015 SCC relative to DICE, depending on the treatment of productivity. When catastrophic impacts are also factored in, the SCC increases by four to fivefold.
1 Introduction
Climate change is one of the preeminent policy issues of our day, and the Paris agreement signals some international agreement that action is urgently needed. The social cost of carbon (SCC) is one of the primary tools for calibrating the socially optimal policy response, particularly in the U.S. under the Obama Administration. The SCC is estimated using integrated assessment models (IAMs), which capture the various steps in the climate and economic processes that translate a marginal unit of \(\hbox {CO}_{2}\) emissions into a measurement of economic damage. While accuracy within each of these steps is necessary to precisely estimate the SCC, accurately calibrating the climate damage function—which we define as translating a temperature change into a percentage change in GDP—is critical.^{Footnote 1} Given the considerable uncertainty in climate impacts, metaanalyses of climate damage estimates are a key tool for depicting the relationship between temperature and climate damages, so as to communicate the current state of knowledge to model developers (Van den Bergh and Button 1999). By clarifying this relationship and the uncertainty underlying it, metaanalyses also explain to policymakers arguably the best assessment of the risks that climate change poses to the global economic system and to human wellbeing.^{Footnote 2}
Due to a dearth of global damage estimates, past metaanalyses (Newbold and Marten 2014; Nordhaus 2013; Tol 2009, 2014) conducted relatively limited analyses of the temperature–damage relationship. It is fair to say that the first damage studies made the very reasonable choice of using simple “back of the envelope” estimates for damages. Gradually, the number of studies increased as methodologies improved, and authors were able to gather small datasets varying from 13 to 43 damage estimates (acquired from 13 to 17 early studies) as a basis for assigning damage values. However, we believe that these earlier analyses are quite limited and potentially biased because these datasets were characterized by nonindependent observations (Aldy et al. 2010; Van den Bergh and Botzen 2015). Many observations were estimated using similar methods and models, and very often undertaken by the same authors and thus had some element of duplication.
The idea of using metaanalyses to summarize the state of knowledge is well established. However, it becomes problematic to work with averages of earlier values if those estimates come from only a very limited number of pioneering authors who have published several estimates each. In fact, 5 of 13 studies used in the regression by Nordhaus (2013) were Nordhaus’ own earlier estimates, and an additional four were from the other two main modelers in the IAM literature. Given the small sample sizes available in the literature, authors of these prior metaanalyses simply regressed global climate damages (measured as a percentage of global GDP) on global temperature change (assumed to be relative to the preindustrial period) and calculated basic ordinary least squares (OLS) standard errors. In doing so, they did not account for duplication or dependency of damage estimates. The consequence is an exaggerated sense of knowledge about the temperature–damage relationship and a greater weighting of estimates from established authors in the field (who tend toward the enumerative strategy) and away from new authors such as Burke et al. (2015) (who tend toward statisticalbased strategies); see also Sterner (2015).
These earlier metaanalyses find that global climate damages increase at an increasing rate, though they differ in the exact rate of change; see Fig. 1. Specifically, the models predict a wide range of potential damage estimates from 1.9 to 17.3% of GDP for a 3 \(^{\circ }\hbox {C}\) increase in global average surface temperature. While these earlier estimates represent strong first steps in understanding the temperature–damage relationship, they are potentially biased due to these data limitations and the limiting effect of the estimation techniques employed. This is particularly true when they are used to predict the damages from higher temperatures, since there is a large sensitivity to the choice of functional form.
There are several sources of bias in previous empirical estimates of the temperature–damage relationship. First, because the data are dependent on one another due to similar underlying estimation methods, authors, and models (Tol 2009; Aldy et al. 2010; Dell et al. 2014; Van den Bergh and Botzen 2015; National Academies of Sciences, Engineering, and Medicine 2017), OLS is inefficient and the resulting standard errors are biased. More importantly, as indicated above, due to the common practice of citing earlier estimates and updating previous estimates in the literature, the resulting estimates suffer from multiple (duplicate) publication bias. Ours is, to the best of our knowledge, the first paper to apply this rigorous metaanalysis concept fully in climate economics.^{Footnote 3} Given that older (i.e., less uptodate) damage estimates tend to be cited multiple times (Van den Bergh and Botzen 2015), we expect this to bias downward the temperature–damage relationship. Second, earlier estimates also suffer from omitted variable bias due to a failure to account for whether the underlying damage estimates captured nonmarket and catastrophic climate impacts in addition to market impacts (Van den Bergh and Botzen 2015; JEP 2015) and to control for whether market impact estimates included potential impacts on economic productivity. Third, earlier studies potentially suffer from publication bias by relying exclusively on published studies (Ackerman and Stanton 2010; Havranek et al. 2015). Fourth, there are several citation errors with respect to damage estimates (JEP 2015),^{Footnote 4} in particular with respect to the measurement of temperature (Gelman 2014). Last, hightemperature damage estimates (corresponding to temperature changes above 4 \(^{\circ }\)C) have an undue impact on the estimated temperature–damage relationship over the critical time period of the twentyfirst century, despite these estimates’ empirical shakiness and overreliance on extrapolation (Van den Bergh and Botzen 2015); for example, Tol (2014)’s results are extremely sensitive to the inclusion of its sole hightemperature damage estimate (see later discussion). By dealing systematically with these biases, this paper identifies a significant downward bias in the literature of the effect of temperature change on global climate damages.
Our aim is thus to improve understanding of the temperature–damage relationship by advancing the metaanalysis techniques pioneered by Tol and Nordhaus. A metaanalysis is not necessarily synonymous with “truth,” but we presumably all agree that if we are going to do a metaanalysis, every effort should be spent on trying to do it is as well as possible. By conducting an exhaustive search of the published and grey literature to assemble a larger dataset, we are able to conduct a metaanalysis using more dataintensive techniques that can more accurately capture this relationship and the uncertainty underlying it. Specifically, using weighted least squares and calculating clusterrobust standard errors at the model level, we regress climate damages on an adjusted measurement of temperature change squared and its interaction with indicator variables for whether the model captures nonmarket, productivity, and catastrophic climate impacts. By dropping duplicate damage estimates and hightemperature damage estimates, we explore the impact of duplicate publication and other biases on the temperature–damage relationship. In doing so, we are able to better characterize the current state of knowledge with respect to the temperature–damage relationship. To understand the potential effect of the previous bias on the SCC, we recalibrate the Dynamic Integrated ClimateEconomy (DICE) model, which is generally seen as the first and most respected IAM, by replacing its damage function with the preferred specification of the temperature–damage relationship and reestimating the SCC.
Our paper is structured as follows. First, we discuss the history of global climate damage estimates. Second, we discuss the relevant metaanalysis literature in the context of global climate damages. Third, we describe the creation of our datasets. Fourth, we present the econometric model used for our metaanalysis of willingness to pay. Fifth, we present our regression results with respect to the temperature–damage relationship. Sixth, we provide a short sensitivity analysis. Seventh, we discuss the interpretation of our results as a damage function. Finally, we conclude.
2 History of Global Climate Damage Estimates
Since 1994, when the first serious attempts at empirically estimating global damages from climate change began (Tol 2009), analysts have developed multiple methodologies to identify the monetary value of climate impacts. These methodologies—each with relative strengths and weaknesses—were introduced in essentially four waves. Each methodology continues to be employed, often by the same authors(s).
The first wave of serious empirical estimates consists of applications of the enumerative strategy and expert elicitation. Fankhauser (1995) is credited with the first extensive application of the enumerative strategy (i.e., the bottomup approach) to estimate global climate damages (Tol 2009). This quickly became the goto methodology for IAM modelers to calibrate their climate damage function(s), most notably DICE (Nordhaus 1994a), the Framework for Uncertainty, Negotiation and Distribution (FUND) (Tol 1995), and Policy Analysis of the Greenhouse Effect (PAGE) (Plambeck and Hope 1996). Relying heavily on author discretion (Pindyck 2013), IAM modelers assemble and construct regionalsector damage estimates (for example, estimates of damage to U.S. agriculture), citing economic impact studies, valuing physical impact estimates from the sciences, or applying benefittransfer (Tol 2009; Van den Bergh and Botzen 2015).^{Footnote 5} While the enumerative strategy can capture market, nonmarket, and catastrophic damages from climate change, enumerative studies often omit significant impacts ignored by the modelers (Revesz et al. 2014; Howard 2014; Interagency Working Group on Social Cost of Carbon 2016) and general equilibrium impacts (Hoel and Sterner 2007). The range of enumerative damage estimates is 2.3 to −11.5% of GDP for temperature increases of 1–3 \(^{\circ }\)C.
Also early on in the development of IAMs, model developers (Nordhaus 1994b; Schauer 1995) conducted several smallsample expert elicitation studies on global damages with the aim of calibrating their model’s climate damage function(s). Recent work (Howard 2015) extends this methodology to a large sample of objectively selected experts. Given this method’s reliance on subjective opinion, the failure of these studies to assign weights based on accuracy of predictions (using seed questions) is an ongoing shortcoming (Cooke 2013; Oppenheimer et al. 2016). Expert elicitation captures a range of market and nonmarket impacts with an estimate range of 0 to −10.2% of GDP for temperature increases of 1–6 \(^{\circ }\)C.
Starting in the early 2000s, authors introduced crosssectional regression to identify the impact of the current climate on national GDP and wellbeing, and then projected future impacts by inputting future climate scenarios. Authors used several regression techniques, including Ricardian analysis (Mendelsohn et al. 2000b),^{Footnote 6} regressing climate variables directly on GDP (e.g., Choiniere and Horowitz 2006; Nordhaus 2006), regressing happiness on climate variables and income (Rehdanz and Maddison 2005; Maddison and Rehdanz 2011), and the household production function approach (Maddison 2003). While the statistical approach is advantageous in that it is datadriven and captures adaptation using current technologies (Mendelsohn et al. 2000b; Burke et al. 2015), omitted variable bias is potentially a serious problem; all but one study (Choiniere and Horowitz 2006) attempt to address this issue using unique identification strategies. Crosssectional analyses, however, capture only a narrow set of climate impacts. The first two regression techniques omit nonmarket impacts, while the latter two omit market impacts. Additionally, these studies omit impacts that vary only a little spatially (e.g., \(\hbox {CO}_{2}\) fertilization and sea level rise), have not yet occurred (e.g., tipping points), or are not captured by variation in average temperature (e.g., extreme weather) (Tol 2009; Van den Bergh and Botzen 2015). The range of crosssectional climate damage estimates is 0.1 to −16.3% of GDP for temperature increases of 0.7–3.2 \(^{\circ }\)C.
Starting around 2010, climate economists employed computable general equilibrium analysis (CGE) for the first time to estimate global damage estimates from climate change. As in enumerative studies, modelers employed regionsector impact estimates from the economics literature to calibrate their climateeconomic models; the difference from enumerative studies is that they used estimated impacts to the supply of production inputs and demand for outputs in their calibration, instead of estimated impacts to output produced. Examples of CGE models include the Intertemporal Computable Equilibrium System (ICES) (Bosello et al. 2012) and Environmental Impact and Sustainability Applied General Equilibrium Model (ENVISAGE) (Roson and Van der Mensbrugghe 2012). Unlike traditional IAMs, CGE models capture the impacts of climate change on economic growth through changes in the stock of inputs (capital and land), factor productivity (capital, labor, land, and/or total), and demand for services (energy, health, and/or tourism) (Eboli et al. 2010; Roson and Van der Mensbrugghe 2012). While the CGE model framework allows climate impacts to interact, it comes with the shortcomings of the enumerative approach. The methodology omits nonmarket^{Footnote 7} and catastrophic impacts, in addition to the transitional costs of climate change.^{Footnote 8} The range of CGE climate damage estimates is −0.2 to −4.6% of GDP for temperature increases of 1.5–4.8 \(^{\circ }\)C.
More recently, climate economists introduced panel regression and scientific estimates. To avoid omitted variable bias, Dell et al. (2012) and Burke et al. (2015) exploited annual variation in weather data to develop panel data of GDP and weather over time. By including country and time fixed effects to control for time and spatial varying unobservables, these studies analyzed the impact of changes in weather on changes in GDP assuming that impacts of weather and climate are identical (Dell et al. 2014; Burke et al. 2015). While there is an ongoing debate over the underlying identification strategy, we avoid the role of arbiter to prevent the introduction of selection bias (instead we test the sensitivity of our results to the exclusion of this and other estimation strategies).^{Footnote 9} The mechanisms for climate’s impact on economic growth are unclear, though likely pathways include declines in input (capital, labor, and land) and total factor productivity (Dell et al. 2014; Howard 2014; Burke et al. 2015). Panel estimates omit nonmarket, tipping point, and general equilibrium impacts (Dell et al. 2014; Burke et al. 2015), and have a range of −0.3 to −23% of GDP for temperature increases of 3.4–4.3 \(^{\circ }\)C.
Following the lead of Nordhaus (1994b), who included scientists in his landmark survey on climate change to capture a wider range of future possibilities, we also include sciencebased damage estimates. Scientific estimates use physical thresholds—limits of human physical adaptability to heat stress (Weitzman 2012) and the 2 \(^{\circ }\)C limit on global warming recommended by a large body of scientists and international organizations (Nordhaus 2014)—to derive global damage estimates from climate change.^{Footnote 10} The methodology potentially captures market, nonmarket, and tipping point impacts (Nordhaus 2015). Its range of damage estimates is −4.9 to −99% of GDP for temperature increases of 3–12 \(^{\circ }\)C.
With each wave of estimates, IAM and CGE modelers released updated damage estimates based on their own or others’ previous work. This has led to many duplicate and interdependent estimates in the literature. Given that enumerative and CGE studies together represent the majority of studies in previous metaanalyses (e.g., Tol 2014), these interrelationships are serious problems; see Fig. 2. A further consequence is that the data underlying even the most uptodate enumerative damage estimates date back to the 1990s (Revesz et al. 2014).
While predicting the magnitude of climate damages is fraught with uncertainty, climate damage estimates become even more speculative above 3–4 \(^{\circ }\)C. If we look at a scatter plot of the estimates (see Figure A1 in Appendix A), a 3 \(^{\circ }\)C increase—the most likely value of the climate sensitivity parameter (IPCC 2007)—is a natural break in the data; only eight estimates correspond to temperature changes exceeding 3 \(^{\circ }\)C. Similarly, 4 \(^{\circ }\)C is another natural cutoff, as it represents the approximate increase in temperature by the end of the century under Business As Usual [see Scenario A2 in IPCC (2007) and RCP 8.5 in IPCC 2013] and most climate research focuses on impacts occurring during this century. While uncertainty in climate damage estimates increases with temperature regardless of the estimation strategy,^{Footnote 11} estimates become highly uncertain and contentious above this temperature threshold of 3–4 \(^{\circ }\)C.^{Footnote 12} In fact, our limited knowledge of impacts may still be overstated, with three of the five estimates above 4 \(^{\circ }\)C being potential extrapolations of other damage estimates.
3 Conducting Metaanalyses of Global Climate Damage Estimates
Metaanalysis (and specifically metaregression) has become a common tool in environmental economics. Nelson and Kennedy (2009) provide a thorough review of this literature up until 2009, and find a wide range of quality in the literature. Metaregression is commonly used to study climate damages to a particular economic sector (e.g., Challinor et al. (2014) analyze global agricultural production) and/or region (e.g., Houser et al. 2014 analyze U.S. climate damages). Some of these metaanalyses reveal complex relationships between climate change and society, including Hsiang et al. (2013), who find that climate change will lead to more human conflicts. Only recently has metaregression been employed at the macroscale (Tol 2009, 2014; Nordhaus 2013; Newbold and Marten 2014), though none have met the standards laid out in Nelson and Kennedy (2009).
In this study, we are interested in the factual and methodological determinants of global willingness to pay to avoid total impacts of climate change as measured as a percentage of global GDP (D). As in previous metaanalyses, increase in global average surface temperature (T) is included as the sole factual cause of observed heterogeneity. The idea behind this decision is not that temperature is the singular climaterelated driver of impacts, but that many of the other drivers, such as precipitation change and increased storm intensity, are strongly correlated with temperature change. Unlike previous metaanalyses of global climate damage estimates, we also control for methodological causes of the observed heterogeneity in the damage impact estimates, in order to avoid omitted variable bias. We divide these methodological variables into those that directly (R) and indirectly (W) impact the relationship between temperature and damages.
Generally, the metaanalysis regression is
where \(\varepsilon \) is the error term. This error term can be subdivided into unobservables (\(\mu \)) and measurement errors (e) from studies where \(\mu \sim \,N\left( {0,\sigma _\mu ^2 } \right) \) and \(e\sim N\left( {0,\sigma _e^2 } \right) \).^{Footnote 13} While \(\sigma _e^2 \) is often reported in statistical studies (Rhodes 2012), this is unobserved for nonstatistical damage estimates. Following the common assumption of a linear function, we assume that the temperature–damage relationship becomes
where Z and X are the unobserved factual and methodological causes of the unobserved heterogeneity of damages.
Heteroskedasticity and dependent errors are common issues in metaanalyses (Nelson and Kennedy 2009; Stanley et al. 2013; U.S. Environmental Protection Agency (EPA) 2006). Unaddressed, these estimation issues result in inconsistent and inefficient coefficient estimates—which are particularly important given the small sample sizes of most metaanalyses—and biased estimates of standard errors. Heteroskedasticity arises because of significant unobserved heterogeneity in the damage estimates. Although weighted least squares (WLS) is the preferred estimator for small sample sizes, WLS as traditionally specified (i.e., using \(\frac{1}{\sigma _e^2 }\) as weights) cannot be used,^{Footnote 14} because the standard errors (i.e., \(\sigma _e^2 )\) are unobserved for nonstatistical global climate damage estimates. Instead, Davidson and MacKinnon (1993) or other heteroskedasticity robust standard errors are recommended (U.S. Environmental Protection Agency (EPA) 2006; Nelson and Kennedy 2009; Rhodes 2012). Dependence of the error terms arises because of correlations in the underlying damage estimate—including similar authorship, estimation method, or model—which result in clustering of error terms. Due to small sample size, the use of the preferred estimators to address dependence—generalized least squares (GLS) and panel estimators—is limited (Cameron and Trivedi 2010, pp. 274–275; Nelson and Kennedy 2009; Van den Bergh and Button 1999), and a combination of clusterrobust standard errors, carefully chosen control variables, and the use of only one estimate per study are recommended (Nelson and Kennedy 2009; Cameron and Trivedi 2010).
Outlier estimates constitute another key economic issue in metaanalysis due to their potential overly influential impact on study findings (Nelson and Kennedy 2009). The metaanalysis literature recommends analyzing sensitivity to outliers by dropping potential outliers and using alternative estimators that are robust to outliers (e.g., Huber weights) (U.S. Environmental Protection Agency (EPA) 2006; Nelson and Kennedy 2009).
Duplicate publication is an identification issue, raised in the medical literature (Tramèr et al. 1997; Gøtzsche 1989), that is currently unaddressed in the climate economics literature. Unlike the medical literature that discusses the potential bias from overt and covert republication of scientific results (e.g., with and without crossreferences to original reports or data), we are concerned with potential bias from the common practices of updating climatedamage estimates over time and calibrating climatemodel damage functions based on previous estimates in the climate damage literature. Tramèr et al. (1997) addresses duplication by excluding duplicate estimates.
4 Data
For our study’s dataset, we combine the studies included in the most recent analyses by Tol (2014, 2015b)^{Footnote 15} and Newbold and Marten (2014). We doublecheck estimates—including reaggregating damage estimates using GDP weights (see Appendix A1)—and correct errors when present (see Appendix A2). We also searched for new damage estimates by: (1) including updates of previous damage estimates; (2) reviewing publications of alreadyincluded authors; (3) reviewing the damage estimates underlying the studies cited in the most uptodate metaanalysis of the SCC (Havranek et al. 2015); and (4) searching Google Scholar and Econlit.^{Footnote 16} We include an estimate if it meets the following conditions: global, unique (i.e., not a rerunning of a previously cited IAM or an IAM copying a previously cited IAM), and dating after 1993 [the start of modern climate damage estimates (Tol 2009)] and before 2016. Following the U.S. Environmental Protection Agency’s (EPA) (2006) recommendation, we include only one estimate per study,^{Footnote 17} unless multiple estimates were deemed potentially independent.^{Footnote 18} We assembled 49 damage estimates from 41 studies, using six estimation strategies; 24 primary authors (including ten estimates from Nordhaus); and 22–27 models (depending on the definition). Of these estimates, 11 are drawn from the gray literature as a means of addressing publication bias.^{Footnote 19}
After assembling the data, we drop estimates that do not meet our a priori selection criteria. We are interested solely in global willingness to pay to avoid climate change, and drop all studies that measure impacts in terms of compensating surplus (CS): Maddison (2003), Rehdanz and Maddison (2005), and Maddison and Rehdanz (2011); we conduct sensitivity analysis to this assumption in Appendix C in the Supplementary Material.^{Footnote 20} In the case of Meyer and Cooper (1995), who apply the value of statistical life from OECD nations and use parity unit damage valuation to derive an estimate of the amount society is willing to accept for climate damages, we undo these adjustments to derive a corresponding willingness to pay estimate; see Appendix A3. Additionally, we exclude all estimates that arbitrarily cap damages based on author discretion: Nordhaus (1994a) and Nordhaus and Yang (1996).^{Footnote 21} Finally, we drop crosssectional studies at the national level that do not address omitted variable bias: Choiniere and Horowitz (2006). This leaves 43 data points from 35 studies.
Given that many of the estimates are updates or citations of previous estimates—giving a false sense of precision—we further limit our observations to prevent duplication bias (i.e., double counting) (Gøtzsche 1989; Tramèr et al. 1997; Nelson and Kennedy 2009). We define a study as a duplicate if it is not the most uptodate estimate by an author utilizing a particular method or if the estimate cites alreadyincluded estimates.^{Footnote 22} After dropping estimates that correspond to this definition, 26 nonduplicate estimates from 20 studies remain.
We code multiple damage, temperature, and methodological variables. D_new is the damage estimate cited in the reviewed paper. T_new is global average mean surface temperature increase in degrees Celsius relative to the preindustrial or current time. For studies that estimate the impact of an increase in global land temperature, we multiply their temperature change by the ratio between global land and surface temperatures to convert to global surface temperature; see Appendix A2. We take care to distinguish studies that include nonmarket damages or catastrophic damages or allow for an effect through changes in productivity.^{Footnote 23} Similarly, we distinguish between crosssectional and panel data (i.e., the inclusion of country fixed effects) and we account for the type and date of each study. Finally, we classify studies by author, estimation method, and model type. See Table 1 for a summary of the data before and after we remove duplicate estimates.
The mean damage estimate increases from 6.3% of GDP to 9.5% when we remove duplicate estimates (a similar increase occurs for the median and temperatureweighted mean damage estimates). Looking at Figure A1 in Appendix A, we can also see that the bulk of estimates for both datasets are below 4 \(^{\circ }\)C even after eliminating duplicate estimates, highlighting the potential for hightemperature estimates to have a disproportionate effect on metaregression results. In particular, local polynomial smoothing of damages on temperature change (see Figure A2 in Appendix A) shows that the impact of duplicate estimates is potentially dependent on the treatment of hightemperature damage estimates.
Damage estimates also differ by author, estimation method, and model. In particular, damage estimates derived using sciencebased and survey methodologies tend to be higher, though there are a handful of high enumerative and statistical damage estimates. Furthermore, damage estimates derived by Tol using the enumerative strategy appear to be lower than other enumerative estimates. Given that the estimates from the early waves tend to be lower than estimates from later waves, it is unsurprising that—contrary to the findings of Tol (2009)—damage estimates appear to be increasing over time, though the results are generally insignificant.^{Footnote 24}
5 Econometric Specification
Following historical precedent (of climate damages increasing at an increasing rate in temperature), and to preserve degrees of freedom, we assume a quadratic damage function with no initial benefits from climate change, such that \(g\left( T \right) =T^{2}\) in Eq. (2). Unlike previous papers, we control for whether temperature is measured relative to preindustrial or current temperatures by including an adjustment term \(\theta _j \)—as measured by the difference in temperature in the base year (or period) of the study relative to the preindustrial period^{Footnote 25}—such that Eq. (2) simplifies to
where we assume that there are no climate damages for a 0 \(^{\circ }\)C increase [a standard assumption implying \(\gamma =0\) in Eq. (2)] and t2 is our new temperaturesquared variable that accounts for its corresponding estimate’s reference period. If we relax the implicit assumption in Eq. (3) that there are no methodological variables, and if we account for clustering (i.e., dependence), we can rewrite this expression as
where k is the cluster level (author, estimation method and model) and \(X_{j,k,l} \) is an indicator variable for estimate jand category \(l \epsilon \left\{ { mkt,cat,prod,cross } \right\} \), which is equal to one if estimate j captures only market impacts (i.e., omits nonmarket impacts), includes catastrophic impacts, accounts for productivity, and corresponds to a crosssectional estimate without country fixed effects, respectively.^{Footnote 26} This expression simplifies to
where \(\varepsilon _{j,k} \sim N\left( {0,\sigma ^{2}} \right) \).
Given our study’s small sample size, we estimate Eq. (4) using WLS with clustered standard errors assuming that we reject the null hypotheses of homoskedasticity and independent observations. In terms of addressing heteroskedasticity, we do not observe measurement error or the number of observations for many global climate damage estimates. Since the literature finds that damage estimates are more uncertain for higher temperature changes (Tol 2009, 2014; Burke et al. 2015), we instead weight estimates by the inverse of temperature change to place lower weight on the more uncertain damage estimates (Day 1999). In terms of addressing dependence of estimates, there are multiple scales of potential dependence: author, estimation method, and model. Therefore, we cluster at the scale for which there is the strongest empirical evidence of dependence, in addition to carefully choosing control variables and selecting only one estimate per study (except when potential independence cannot be rejected).
To analyze the impacts of duplication and hightemperature damage estimates, we estimate expression (4) including and excluding duplicate and hightemperature damage estimates. Given our hypothesis of duplication bias, regressions that include duplicate data are used for purposes of producing a counterfactual; it should be assumed that we are referring to estimates that exclude duplicate damage estimates unless explicitly stated. Given the evidence that hightemperature damage estimates are speculative and potentially extrapolations of already included estimates, we explore the impact of a 4 \(^{\circ }\)C increase as a cutoff for lower temperature damage estimates. For analyzing the potential impact of climate change in this century, the preferred dataset excludes duplicate and hightemperature damage estimates.
Finally, given concerns about overfitting our model due to its small sample size—particularly in the preferred dataset consisting of 21 observations (excluding duplicate and hightemperature damage estimates)—we conduct our analysis using two sets of exogenous variables: a smaller set that includes only temperature squared and its interaction with the market and catastrophic indicator variables and an extended set that also includes the indicator variable for crosssectional studies without country fixed effects and the interaction of temperature squared with the productivity indicator variable. The latter two variables are chosen for exclusion because both variables have lower correlation with damages across all datasets—particularly the preferred dataset—and productivity is strongly correlated with the market indicator variable.^{Footnote 27}
6 Results
We test for heteroskedasticity and dependent error terms using Cameron and Trivedi’s decomposition of IM test (see Table B1 in Appendix B) and the Breusch–Pagan test of independence (see Table B2), respectively. We consistently fail to reject the null hypothesis of homoskedasticity when we exclude hightemperature estimates and consistently reject the null hypothesis when we include them. The Cameron and Trivedi’s decomposition of IM test also tests for nonnormal kurtosis and skewness, which we consistently reject (the only exception is skewness when we include duplicate estimates and drop hightemperature damage estimates).^{Footnote 28} For purposes of simplicity, we present results using WLS regardless of whether we exclude hightemperature estimates; these results only slightly differ from the OLS results. We also consistently reject the null hypothesis of independent damage estimates, including at the model scale, for which we reject at the 1% significance level.^{Footnote 29} For all datasets, we cluster error terms at the model scale.
When we regress climate damages on temperature squared and its interaction with indicator variables, the results indicate a strong relationship between temperature and climate damages; see Table 2. In particular, the regressions are all jointly significant at the 1% significance level, and the coefficient corresponding to temperature squared (i.e., noncatastrophic climate damages) is positive and significant at the 5% significance level. Additionally, catastrophic impacts are positive, but only significant at the 1% level when damage estimates for hightemperature increases (i.e., above 4 \(^{\circ }\)C) are included. As expected, we generally find that the exclusion of nonmarket impacts has a negative and sometimes significant effect on damages. In the extended specifications, we find that productivity has an insignificant, positive effect on climate damages, as well as strong evidence of crosssectional bias.
Our results appear split with respect to our other two main statistical concerns: overspecification and hightemperature estimates. On the one hand, overspecification appears not to be of particular concern, given the small change in coefficients corresponding to temperaturesquared and catastrophic impacts that occurs with the inclusion of the productivity and crosssectional controls. While there are some significant changes in the coefficient corresponding to the market indicator variable, this is due to the strong correlation between the market and productivity indicator variables that results from the exclusion of nonmarket impacts and productivity modeling in all CGE and panel models. On the other hand, hightemperature damage estimates appear to be a critical issue, given the large change in coefficient magnitudes between Specifications (4) and (8), though only the coefficients corresponding to temperaturesquared and its interaction with the market indicator variable are statistically different; this highlights a common theme in this paper that total damages are robust.^{Footnote 30} The signs of coefficients are almost identical when we include and exclude hightemperature estimates; again, the only difference is the coefficient corresponding to the market indicator variable when productivity is excluded due to the high level of correlation between the market variable and productivity. As a consequence of this correlation between the market and productivity impacts of climate change in the underlying damage estimates and an increase in adjustedR squared with the inclusion of productivity and crosssectional controls, we choose the extended specification as the preferred specification [i.e., Specifications (4) and (8) in Table 2].
Our damage estimates are statistically higher than those of Nordhaus (2013). We reject the null hypothesis that noncatastrophic damages (the coefficient corresponding to t2) and total damages (the sum of the coefficients corresponding to temperaturesquared and its interaction with the catastrophic dummy) are equivalent to the Nordhaus (2013) damage estimate of 0.2136; the only exception is for noncatastrophic damages when we include hightemperature estimates. If we also consider climate change’s impact on productivity jointly with noncatastrophic and total impacts (i.e., add the coefficient corresponding to the interaction of temperaturesquared and productivity), this difference remains statistically significant.^{Footnote 31}
In support of the main hypothesis of our paper, we find strong evidence of duplication bias when we exclude damage estimates above 4 \(^{\circ }\)C. Compared to the counterfactual dataset of lowtemperature damage estimates that includes duplicate estimates, the damage coefficient corresponding to temperaturesquared increases and the coefficient corresponding to catastrophic impacts declines when we drop duplicate estimates. As expected, the standard errors corresponding to these variables also increase—resulting in catastrophic impacts becoming statistically insignificant—emphasizing the false precision from duplication. Though less substantial, we find similar changes for the larger dataset that includes hightemperature estimates. Using seemingly unrelated regression (SUR) to test for duplicate publication bias (a test of whether the exclusion restriction for duplicate estimates significantly impacts coefficients), we reject the null of no duplication bias at the 1% significance level for the preferred dataset that considers only damage estimates corresponding to lowtemperature increase (i.e., 4 \(^{\circ }\)C or less) and fail to reject the null for the dataset containing hightemperature damage estimates.^{Footnote 32} Interestingly, while we find strong evidence of duplication bias, we find no evidence of publication bias.^{Footnote 33}
We also find evidence of omitted variable bias. In particular, we consistently find the previously omitted variables (mkt_t2, cat_t2, prod_t2, and cross, as defined above in Eq. (4)) jointly significant. The adjusted Rsquared statistic also increases when we include these previously omitted variables. Furthermore, we reject the null of no omitted variable bias using the Ramsey regression specificationerror test for omitted variables when we include only temperaturesquared and the shorter set of explanatory variables (t2, mkt_t2, and cat_t2) at the 5 and 10% significance levels. We fail to reject the null hypothesis when we use the extended set of variables (including prod_t2 and cross) and when we exclude hightemperature estimates, though not when we include hightemperature estimates. Finally, documentation on the construction of estimates also indicates that these indicator variables should be included based on theory (see Sect. 2).
For the preferred regression of impacts this century (i.e., Regression (4) in Table 2, corresponding to the regression of damages on the extended set of variables excluding hightemperature estimates), we have shown that the coefficient corresponding to our current best estimate of noncatastrophic impacts (i.e., temperaturesquared) significantly increases (almost doubling) when we address duplication bias and that the resulting estimates differ from Nordhaus (2013). Given that our results demonstrate that duplication bias can significantly affect the temperature–damage relationship, it is important to determine the relative importance of duplication bias in driving the difference between our results and those of Nordhaus (2013) relative to the other forms of bias discussed earlier—omitted variable bias and measurement error—and relative to obtaining a larger dataset and improving the estimation technique.
To address this issue, we run the Nordhaus (2013) model over five datasets and sets of variables to identify the relative impact of each adjustment made from the Nordhaus (2013) estimate [Specification (1) in Table 3] to the preferred estimate [Specification (5) in Table 3]. Each step captures the impact of one particular change that we introduced: (2) correcting data and improving the estimation technique, (3) introducing additional damage estimates, (4) accounting for omitted variable bias, and (5) accounting for duplication bias. Considering noncatastrophic impacts, we find that duplication bias and introducing new data have the most significant impact on the percent change in the coefficient corresponding to temperaturesquared, at 81 and 55%, respectively.^{Footnote 34}
If we make the assumption that the relationship captured in our metaanalysis represents a damage function (Tol 2009; Nordhaus 2013), we may be interested in understanding which factors—including source of bias—have the most influence on the SCC. Using DICE—the sole IAM to currently calibrate its damage function using metaanalysis and one of three models used to calculate the official U.S. SCC (Interagency Working Group on Social Cost of Carbon 2016)—we estimate the SCC and repeat the above analysis. If we consider only noncatastrophic impacts (as in the DICE2013R damage function), addressing the above biases increases the SCC (in 2015) by three to fourfold from Specification (1)–(5), depending on whether we account for the impact of climate change on economic productivity. If we consider total climate impacts (both noncatastrophic and catastrophic), addressing the above biases increases the SCC (in 2015) by four to fivefold depending on the treatment of productivity; see Fig. 3. When considering the percent change in the SCC from specification to specification, we find nearly identical results to the previous paragraph, indicating that duplication is one of the most significant sources of bias in previous estimates. Finally, we find that uncertainty over the SCC increases—as measured by the difference between the 5th and 95th percentile SCC estimates (accounting for noncatastrophic impacts only)—from Specification (1)–(5).
Unsurprisingly given the insignificance of duplicate publication bias found earlier when considering data corresponding to all temperature increases, we find that accounting for duplication bias has a similarly small effect on the temperature–damage relationship and the SCC when including hightemperature damage estimates; see Table B3 in Appendix B. However, the corresponding 2015 SCC estimates are still far above the original DICE2013R model, with an increase in the range of onehalf to fourfold when considering only noncatastrophic climate impacts and three to sixfold when considering total climate impacts, where the range is dependent on the inclusion of the impact of climate change on productivity. When we consider the climate impacts on productivity jointly with the noncatastrophic or total impacts of climate change, our final SCC estimates—Specification (5)—are almost identical, regardless of whether we account for hightemperature damage estimates, and considerably higher and more uncertain than the original DICE2013R estimate.
7 Sensitivity Analysis
We conducted additional sensitivity analyses (alternative functional forms, data and variable definitions, clustering scales, and estimators; treatment of outliers; and dropping estimates, studies, and estimation methodologies one by one) for the preferred specifications [i.e., regressions (4) and (8) in Table 2]. This section discusses notable results; see Appendix C in the Supplementary Material for an indepth discussion.
We use the Nordhaus (2013) damage functional form in our main text, despite some alternative damage functional forms (Weitzman 2012; Tol 2014, 2015b) that fit the data slightly better. These alternative functional forms increase the number of explanatory variables, running the risk of overfitting the model. Thus, they are less useful for damage predictions. Even so, we redo our primary analysis using the Tol (2014) functional form, and find strong evidence that duplication bias led to downward bias in the original Tol (2014) temperature–damage relationship; see Appendices C1 and C2.
Our primary estimates of climate impacts are quite robust to these sensitivity analyses, particularly for noncatastrophic impacts when excluding hightemperature estimates and total impacts in general. Total climate impacts tend to remain relatively constant as alternative assumptions shift impacts between the noncatastrophic and catastrophic components. A more restrictive definition of duplication that limits our choice of data to only one estimate per study (to exclude extrapolated estimates) shifts catastrophic impacts to noncatastrophic impacts (see Appendix C4) when including hightemperature estimates, while alternative estimators shift noncatastrophic impacts to catastrophic impacts (Appendix C5). While we find little evidence of nonnormal error distributions, regressions that are robust to outliers (including those due to nonnormal error distributions) cause small shifts from noncatastrophic to catastrophic impacts—such that total impacts remain relatively constant—as would be expected under a rightskewed distribution; see tables in Appendices C6 and C7.
Finally, earlier estimation methodologies (enumerative, CGE, crosssectional) predict a flatter temperature–damage relationship relative to more recent methodologies (science, panel regression) and surveys; see Appendix C8. As a consequence, older methodologies tend to reduce noncatastrophic impacts and increase catastrophic coefficients; newer methodologies tend to increase noncatastrophic impacts and reduce catastrophic coefficients. Dropping the currently debated panel methodology has no effect on the results (nor does dropping crosssectional studies); though, dropping all statistical estimates results in the positive impact of productivity becoming statistically significant. Like the newer methodologies, CS estimates imply a steeper temperature–damage relationship, despite capturing only climate impacts on nonmarket goods and services. Despite this result, including these CS estimates in the preferred specifications has little impact on key results; see Appendix C9. As theorized, we find evidence of CS estimates being significantly higher than willingness to pay (WTP) estimates.
In summary, considering the large number of alternatives tested and the fact that the overall effects of the sensitivity tests are small (mostly shifting impacts from noncatastrophic to catastrophic), the results should be seen as robust.
8 Discussion
Like Nordhaus (2013), we take the extra step of interpreting the temperature–damage relationship captured by the above metaanalysis as a climate damage function. To include these metaanalysis results in DICE2013R, we followed Nordhaus in multiplying the sum of the coefficients corresponding to noncatastrophic impacts (temperaturesquared and its potential interaction with productivity) by 1.25 to account for omitted noncatastrophic climate impacts.^{Footnote 35} Given that we use data including and excluding hightemperature damage estimates (i.e., above 4 \(^{\circ }\)C) and two sets of control variables when implementing our estimation strategy, it is important to clarify which of our estimates is the appropriate relationship to use in modifying the DICE2013R damage function.
When reestimating the DICE2013R damage function, it is appropriate on several grounds to exclude hightemperature estimates (i.e., above 4 \(^{\circ }\)C). First, hightemperature estimates are highly speculative in nature, given that uncertainty in climate damage estimates increases in temperature, yet including these five estimates (19% of estimates) significantly decreases the impact of temperaturesquared on noncatastrophic damages by half and shifts catastrophic impacts from statistically insignificance to significance at the 1% level (see Table 2); this is despite the inclusion of weights to partially mitigate this undue influence. In contrast to the outsized impact of these estimates, the majority (approximately twothirds) of the 2015 SCC estimate for DICE2013R correspond to impacts occurring this century; this result is relatively robust to the magnitude of the damage coefficient. Most hightemperature estimates are also potentially duplicates of damage estimates corresponding to temperature increases below 4 \(^{\circ }\)C, whose exclusion results in a noncatastrophic damage function more stable to the definition of duplication; see Appendix C4 in the Supplementary Material. Thus, excluding damage estimates for temperature changes above 4 \(^{\circ }\)C prevents hightemperature estimates from unduly influencing the estimated temperature–damage relationship relevant for the twentyfirst century, for which estimates for approximately 4 \(^{\circ }\)C or less are more germane.
Following Nordhaus (2013), we specify a damage function corresponding to noncatastrophic damages from Regression (4) in Table 2, excluding productivity, such that the quadratic damage function parameter is equivalent to the coefficient corresponding to temperaturesquared. Thus, while we control for catastrophic damages and productivity in the preferred regression specification, we exclude the corresponding estimated coefficients in the base specification of the damage function because of their mixed significance and volatility across the various specifications. Instead, given the debate over the impact of climate change on productivity and economic growth (Dell et al. 2012; Burke et al. 2015; Howard 2015), we recommend conducting an analysis of sensitivity to the inclusion of the productivity impact. Replacing the DICE2013R damage function with our estimates of noncatastrophic damages, we find a mean 2015 SCC estimate of $68 (2015 USD) with a 95% confidence interval from $16 to $132 using our base specification, and a mean of $83 with a range of −$2 to $206 when including productivity. To put this range in context, the higher end of this SCC range implies that the United Nation’s 2 \(^{\circ }\)C limit is potentially too high from a social welfare perspective. Despite these wide confidence intervals, which highlight the extreme risk of climate change, it should be noted that these ranges underestimate uncertainty by ignoring the uncertainty underlying the data estimates themselves (due to insufficient data).
One potential solution to addressing this risk is factoring in catastrophic damages. To account for catastrophic impacts, we would ideally run a Monte Carlo simulation whereby catastrophic impacts are captured by the variance of the damage coefficient (hence the importance of improved estimation of standard errors). However, given that we do not reject the normal distribution (nor could we know the “true” distribution even if we did) we choose a normal distribution (a symmetric, thintailed distribution) to model error terms. If this assumption fails (due to too little data to detect it or our inability to capture the uncertainty underlying most estimates), a Monte Carlo simulation may fail to truly highlight the underlying climate risks—as highlighted by Weitzman (2009)’s Dismal Theory—providing little more information than the 95th percent confidence interval reported earlier. Furthermore, given that DICE is often run as a deterministic model, we recommend also conducting a sensitivity analysis with respect to the inclusion of catastrophic damages. Replacing the DICE2013R damage function with our estimate of total damages (noncatastrophic and catastrophic), we find a mean 2015 SCC estimate of $97 (2015 USD) and $113 when we exclude and include productivity, respectively. Using these estimates for policy would put a higher premium on risk, and keeps the optimal temperature level at 2.1 \(^{\circ }\)C or lower.
Some have questioned whether this type of topdown analysis is appropriate, and thus question the validity of SCC estimates such as those derived above. For example, Tol (2009) argues that these studies should not be treated as timeseries data, and that any analysis attempting to estimate a damage function should be interpreted cautiously. Citing the Intergovernmental Panel on Climate Change (2014), JEP (2015) argues against this methodology because estimates use different estimation methods and models to capture overlapping sets of impacts. In this paper, we attempt to address these shortcomings by including methodological variables that capture differences in overlapping estimates and by accounting for different estimation methods and models. While this type of metaanalysis is fraught with challenges, as critics suggest, we believe that analysts should continue to work to improve these estimates because such analyses provide valuable information about the temperature–damage relationship; in fact, some analysts have already chosen to embrace the technique (Nordhaus 2013; National Academies of Sciences, Engineering, and Medicine 2017).^{Footnote 36} However, analysts should also be clear about the limitations of the methodology. Furthermore, as more data become available, we should expect metaanalysis estimates to improve as analysts are able to control for a more refined set of differences between estimates (including which sectors are included, such as agriculture, sealevel rise, etc.).
As of late, there has been a new wave of advanced statistical estimates of climate damages—including bottomup estimates (Schlenker and Roberts 2009; Hsiang et al. 2011) and global market impact estimates (e.g., Dell et al. 2012; Burke et al. 2015)—that some economists may argue make the topdown approach to estimating a damage function unnecessary. However, the above techniques are also key to the bottomup approach, given that most of these studies focus on particular sectors in developed nations only, and will require benefitcost transfer and metaanalyses methods—as developed in this paper—to assemble global climate damage estimates. For example, there are a multitude of regional agricultural studies on the costs of climate change, and metaanalyses are necessary to estimate regional and global agricultural damage functions.^{Footnote 37} In the meantime, as statistical methods and data improve, the topdown approach—metaanalysis and surveys—are necessary. Climate policy cannot wait while statistical studies are perfected. Instead, we should utilize the best available estimates of total climate damages, and should update these estimates as new data becomes available. We believe that topdown and bottomup approaches should converge over time, as methods improve and more data becomes available. Future work should aim to improve global climate damage estimates (the topdown approach) and regionsector climate damage estimates (a bottomup approach), and utilize metaanalyses to determine whether these approaches are converging in magnitude.
9 Conclusion
This paper contributes to the literature on IAMs and climate change more generally, by offering an improved estimation of the temperature–damage relationship. Using a larger dataset than previously assembled, this paper highlights several methodological improvements that address bias present in previous metaanalyses. Specifically, due to restricted analyses resulting from limited availability of data, previous metaanalyses of climate damages suffered from multiple sources of coefficient and standard error bias: duplication bias, dependence, heteroskedasticity, measurement error, and omitted variable bias. By expanding our dataset (to include additional published and grey literature estimates), including methodological variables, clustering standard errors, and correcting the specification of temperature (to account for different reference periods), we are able to address and test for these biases. While it turns out that publication bias and the measurement error in temperature were not very significant in most cases, duplication bias and omitted variable bias significantly impact the temperature–damage relationship. According to the preferred specification, noncatastrophic damages are likely between 7 and 8% of GDP for a 3 \(^{\circ }\)C increase (from the preindustrial period), compared to the 2.4% predicted by DICE2013R (Nordhaus 2013), and are between 9 and 10% when factoring in catastrophic risks.
We also demonstrate that decisions regarding the treatment of hightemperature estimates (exceeding 4 \(^{\circ }\)C) are critical in determining the temperature–damage relationship. When we include these hightemperature estimates, our range of noncatastrophic damage estimates shifts to 4–8% for a 3 \(^{\circ }\)C increase. The range of total damage estimates shifts to 7–11% for a 3 \(^{\circ }\)C increase when we include these hightemperature estimates. In presenting these findings, we highlight the need to carefully consider the relevant time period over which the temperature–damage relationship is identified.
We can interpret the temperature–damage relationship estimated above as a damage function (as done in DICE2013R). Using the preferred specification, we find that the 2015 SCC increases by approximately three to fourfold depending on the treatment of productivity. Applying a SCC of this magnitude globally implies that aggressive climate action that limits the maximum temperature increase to below 2.5 \(^{\circ }\)C is optimal. Factoring in catastrophic risks increases the SCC range and further restricts the optimal maximum temperature increase to 2.1 \(^{\circ }\)C or lower.
DICE2013R has yet to be integrated into the U.S. government’s analysis (stopping currently at DICE2010 (Interagency Working Group on Social Cost of Carbon 2016), which relies on the previously employed enumerative calibration strategy). This paper highlights how the U.S. government could reestimate the DICE2013R damage function to meet previously set government standards for metaanalyses (U.S. Environmental Protection Agency (EPA) 2006). Making this sole change would increase the resulting SCC of the Interagency Working Group on Social Cost of Carbon by between oneandahalf to twofold. While the Trump Administration disbanded the Interagency Work Group in 2017, U.S. agencies will still need to calculate the cost of greenhouse emissions and our above analysis still applies.
Notes
While recent papers investigate other potential forms of climate damage functions—for example, Dietz and Stern (2015) include direct effects of temperature change on capital stock and total factor productivity—for purposes of clarity we restrict ourselves to the dominant form as stated.
There are authors who find the difficulty of estimating damage from unprecedented climate change so daunting that they lean towards seeing it as fundamentally unknowable. Examples of this view include: Pindyck (2013)’s statement that “more than temperature change itself, economic impact may be in the realm of the ‘unknowable”’; Heal and Millner (2014)’s view that “we also have to recognize, however, that many of the most important empirical uncertainties, such as...the adaptability of future societies to an altered climate, are fundamentally unknowable”; and Wagner and Weitzman (2015)’s questioning of “how can we know the relevant probabilities of rare extreme scenarios that we have not previously observed and whose dynamics we understand only crudely at best? The true numbers are largely unknown and may simply be unknowable.” We greatly respect these positions, but naturally we still believe—as researchers conducting a metaanalysis—that it is worth trying to quantify despite the daunting challenges.
While duplication bias is not explicitly cited in Ackerman and Stanton (2010), they recognize that a similar selection procedure for Tol (2008)’s metaanalysis of SCC estimates may result in bias because approximately 50% of the estimates were drawn from his own papers, which tend toward the lower end of the range (Van den Bergh and Botzen 2015).
Mendelsohn et al. (2000b) produce two estimates using statistical and experimental methodologies, though we classify the latter as statistical when clustering due to likely dependence. See Supplementary Material.
Because some CGE models capture the impact of lost ecosystem services on GDP (e.g., Bosello and Parrado 2014), we conduct analysis of sensitivity to this assumption.
Unlike other CGE model estimates, Bosello and Parrado (2014) attempt to capture transitional costs.
The ongoing debate over whether the impacts of weather capture changes in the climate boils down to whether shortrun and longrun adaptation are similar and whether impacts intensify over time (Dell et al. 2014; Burke et al. 2015; Nordhaus 2015; Hsiang 2016). Just as we do not question the various identification strategies employed in crosssectional regressions to address omitted variable bias, we include panel estimates without questioning the method’s identification strategy. Instead, we analyze sensitivity to their exclusion in Appendix C in the Supplementary Material, and find that the exclusion of panel estimates has little effect on our primary results.
In the former methodology, Weitzman (2012) estimates impacts of 50 and 99% of global GDP for 6 and 12 \(^{\circ }\)C increases (Ackerman et al. 2012), respectively, based on the share of human habitat that will be inhospitable. While the 12 \(^{\circ }\)C estimate is based on a global limit from Sherwood and Huber (2010), the 6 \(^{\circ }\)C estimate internalizes regional limits, such as the Middle East becoming inhospitable by the end of the century under a Business As Usual scenario (Pal and Eltahir 2015). In the latter methodology, Nordhaus (2014) calibrates the DICE2013R’s damage function assuming that the 2 \(^{\circ }\)C limit is optimal under alternative constraints: a strict 2 \(^{\circ }\)C limit and an average increase of 2 \(^{\circ }\)C over the 2050–2250 period.
In general, global damage estimates become less accurate (i.e., higher variance) the farther out into the future that they predict (Van den Bergh and Botzen 2015). While there are no enumerative based estimates for above 3 \(^{\circ }\)C, an enumerative estimate is likely to become less accurate as the climate scenario for which it makes a prediction moves farther and farther from the studies that underlie it; this is also the case for the similarly calibrated CGE studies. Surveys rely on expert elicitation, which is likely to become more unreliable as experts are asked to make predictions for future scenarios farther away from current observations (i.e., time periods and warming levels). Statistical (crosssectional and panel) estimates are based on observed variation, and become less accurate (i.e., there is an increase in variance) the larger the outofsample prediction that is required. In crosssectional studies, the higher the temperature change predicted, the more nations enter unobserved temperature levels. Similar issues also arise in outofsample prediction using panel regression, where “average annual temperatures in a country are almost never more than 2 \(^{\circ }\)C from their longrun historical mean.” (Dell et al. 2014) Finally, while scientific estimates seem the most likely to be accurate for hightemperature increases, the predictions are only as accurate as the climate models upon which they are based (see disparity between Sherwood and Huber 2010; Pal and Eltahir 2015 at 6 \(^{\circ }\)C), the extent to which other limits to human adaptation occur earlier, and/or the extent to which human ingenuity is able to overcome the limitations of heat stress.
Some of these damage estimates corresponding to hightemperature increases are potential leverage points that may confound OLS regression and other low breaking point estimators (Verardi and Croux 2009; Alma 2011). See the Supplementary Material for a more thorough discussion of leverage points and their identification.
The normality assumption may not hold—causing inefficient coefficient estimates and biased standard errors—if errors terms are rightskewed due to more negative than positive climate surprises (Tol 2009). We estimate robust regressions partially to address this issue; see the Supplementary Material.
When measurement error is unobserved, some metaanalyses use the sample size of the study (n) to construct a weight \(\left( {\sqrt{n}} \right) \) because studies with larger sample sizes tend to be more accurate (Day 1999; Nelson and Kennedy 2009). In the case of climate change, the number of observations is unobserved for nonstatistical based estimates.
The search terms were: “percentage of GDP”, “climate change”, & global; “percent of GDP”, “climate change”, & global; “percent GDP”, “climate change”, & global; “% of GDP”, “climate change”, & global; “% GDP”, “climate change”, & global; “climate change” & “world output”; “Estimated impact of global warming on world output”; “climate change”, “economic impact”, & global; “climate change” & “global impact”.
When it was necessary to select only one estimate, we chose the estimate that utilizes the Business As Usual scenario, the most climate information available (e.g., temperature and precipitation changes), and GDP weights.
There are eight studies that present two potentially independent estimates using alternative temperatures or methodologies. We conduct sensitivity analysis to maintaining one data point from these studies instead of two.
While testing for publication bias would be ideal, it is not possible in this context because many estimates do not provide standard errors. Thus, the Egger and Begg test statistics for publication bias cannot be constructed.
Standards for metaanalysis recommend either studying comparable estimates only or controlling for the underlying differences using explanatory variables. We choose the former approach in this case. Given that the vast majority of damage estimates are willingness to pay (WTP) estimates, we analyze WTP to avoid climate damages and drop CS welfare measures (which combine how much climate losers are willing to accept for their worsening climate and how much climate winners are WTP for their improving climate). Given that CS is the appropriate welfare measure “assuming rights to the existing climate (Maddison and Rehdanz 2011)” and implies higher damages per temperature increase than WTP (hence, Tol (2015a) finds that Maddison and Rehdanz (2011) is an outlier), our study represents a lower bound estimate of the effects of climate on wellbeing. While it is tempting to choose the latter approach by including an indicator variable equal to one if the estimate measures CS, a confounding factor is that these three estimates are also the sole estimates to exclude market impacts of climate change; this makes the interpretation of the indicator variable difficult. Even so, we conduct sensitivity analysis to this approach and show that it has little impact on the final results; see Appendix C in the Supplementary Material.
With respect to dropping estimates based on already included estimates, we make exceptions for Hanemann (2008), as adjusted by Ackerman et al. (2012), Meyer and Cooper (1995), Roson and Van der Mensbrugghe (2012) and Bosello and Parrado (2014), where we believe that the authors used the original estimates (Nordhaus 2008; Fankhauser 1995; Tol 2002) as a starting point for their own estimates, such that the final estimates represent a significant departure from the original estimates. We only conduct sensitivity analysis to these two previous exceptions, given that ENVISAGE (Roson and Van der Mensbrugghe 2012) and ICES (Bosello and Parrado 2014) overlap with FUND only in minor ways.
We assume that (1) all sciencebased damage estimates account for catastrophic impacts, and (2) panel estimates of the temperature–damage relationship (Dell et al. 2012; Burke et al. 2015) capture the effects of climate change on GDP via productivity because they measure the effect of climate change on economic growth.
Following Tol (2009), we regress damages on Time, finding that damage estimates (insignificantly) increase over time. To check our results for robustness, we regress damages on time and time squared with and without controlling for temperature. In general, the time variables are insignificant, except when we include only the linear time and temperature terms; in this case, damage estimates significantly increase in magnitude over time.
In our dataset, this adjustment is captured using a 5year average of NASA’s global annual mean landocean temperature index relative to 1880 (http://data.giss.nasa.gov/gistemp/graphs_v3/Fig.A2.txt).
Of the crosssectional studies, the studies that analyze the GEcon dataset (e.g., Nordhaus 2006)—which consists of GDP and climate data at the subnational scale—include country fixed effects.
Furthermore, the indicator variable for productivity is also perfectly correlated with CGE and panel models, though not with statistical models in general.
The Cameron and Trivedi’s decomposition of IM test also tests for homoskedastic normal error terms. While we consistently fail to reject the null of normal kurtosis and skewness, we reject the null of a normal error distribution, except when we exclude hightemperature damage estimates and include the full set of explanatory variables (i.e., the preferred set of assumptions). Given that we fail to reject normal kurtosis and skewness, we assume that this is primarily driven by heteroskedasticity.
At the author scale, we reject the null hypothesis of independence at the 5 and 10% significance levels when we include or exclude hightemperature damage estimates, respectively. At the method scale, we reject the null hypothesis at the 5% significance level.
We utilize seemingly unrelated regression (SUR) to test for statistical difference in the climate damage impacts when we account for hightemperature estimates. In terms of statistical significance, only the coefficients corresponding to temperaturesquared and its interaction with the market indicator variables significantly differ between the inclusion and exclusion of hightemperature damage estimates, such that only noncatastrophic impacts differ, but total impacts do not.
Using SUR, we reject the null that the coefficients corresponding to temperature squared and its interactions are equal when we do and do not account for duplication at the 1% significance level when considering only damage estimates for lowtemperature increases. At the 5% significance level, the difference is also significant when we consider only the coefficient corresponding to noncatastrophic damages (i.e., temperature squared) and total damages (i.e., temperature squared and its interaction with the catastrophic indicator variable), regardless of whether we consider productivity jointly. When we include hightemperature estimates, we fail to reject the null of no duplication bias for noncatastrophic damage and total damages.
Including an indicator variable for grey literature and/or its interaction with temperaturesquared in all specifications in Table 2, we fail to reject the null of no publication bias. Of the twentyfour different specifications, we find only one instance where we reject the null of no publication bias at the 10% significance level.
We can check whether the order of adjustment matters. When we reverse the order of the last two adjustments, such that (4) is duplication bias and (5) is omitted variable bias, we slightly adjust the ranking. Instead of duplicate bias being the primary change, it is now of similar magnitude as new data with an impact of approximately 50%. Omitted variable bias goes from 10% to 34% to become the third most important adjustment.
Most of the underlying estimates systematically omit key climate impacts that could significantly increase climate damages, including sociallycontingent climate impacts (migration, social and political conflict, and violence), ocean acidification, etc. (Howard 2014; Revesz et al. 2014). Given that some of the climate damage estimates in this study are over two decades old, and considering the positive coefficient corresponding to Time, we should expect climate damage estimates to continue to increase over time as currently omitted climate impacts are captured in future damage estimates. Given this bias, an adjustment upward of the temperature–damage relationship—such as the 25% adjustment employed by Nordhaus (2013)—may be justifiable when using metaanalysis results to calibrate an IAM damage function.
The National Academies of Sciences, Engineering, and Medicine (2017) recognizes metaanalysis of global damage estimates as one of three methods that the Interagency Working Group on the Social Cost of Carbon can use to calibrate an IAM’s damage function, given the caveat that they must address the dependence of damage estimates.
Another alternative, laid out by Kopp et al. (2013), is to develop an infrastructure that uses statistical (for example, Bayesian) methods to update damage functions as new estimates become available.
Abbreviations
 BAU:

Business As Usual
 \(\hbox {CO}_{2}\) :

Carbon dioxide
 CS:

Compensating surplus
 CGE:

Computable general equilibrium
 DICE:

Dynamic Integrated ClimateEconomy model
 ENVISAGE:

Environmental Impact and Sustainability Applied General Equilibrium Model
 EPA:

Environmental Protection Agency of the U.S.
 FUND:

Framework for Uncertainty, Negotiation and Distribution
 GLS:

Generalized least squares
 IAMs:

Integrated assessment models
 IWG:

Interagency Working Group on Social Cost of Carbon
 IPCC:

Intergovernmental Panel on Climate Change
 ICES:

Intertemporal Computable Equilibrium System
 JEP:

Journal of Economic Perspectives editors
 NAS:

National Academies of Sciences, Engineering, and Medicine
 NOAA:

National Oceanic and Atmospheric Administration of the U.S.
 OLS:

Ordinary least squares
 PAGE:

Policy Analysis of the Greenhouse Effect
 SUR:

Seemingly unrelated regression
 SCC:

Social cost of carbon
 WLS:

Weighted least squares
 WTP:

Willingness to pay
References
Ackerman F, Stanton EA (2010) The social cost of carbon. A Report for the Economics for Equity and Environment Network. Economics for Equity and Environment Network, Cambridge. http://e3network.org/wpcontent/uploads/2015/04/Ackerman_Social_Cost_of_Carbon.pdf. Cited 21 Dec 2016
Ackerman F, Stanton EA, Bueno R (2012) CRED v. 1.4. Technical Report. Stockholm Environment Institute, Somerville
Aldy JE, Krupnick AJ, Newell RG et al (2010) Designing climate mitigation policy. J Econ Lit 48(4):903–934
Alma ÖG (2011) Comparison of robust regression methods in linear regression. Int J Contemp Math Sci 6(9):409–421
Bosello F, Eboli F, Pierfederici R (2012) Assessing the economic impacts of climate change. Review of Environment, Energy and Economics (Re3). http://re3.feem.it/getpage.aspx?id=4569. Cited 29 Dec 2014
Bosello F, Parrado R (2014) Climate change impacts and limited marketdriven adaptation. Review of Environment, Energy and Economics (Re3), forthcoming
Burke M, Hsiang SM, Miguel E (2015) Global nonlinear effect of temperature on economic production. Nature 527(7577):235–239
Cameron CA, Trivedi PK (2010) Microeconometrics using stata, revised edn. Stata Press, College Station
Challinor AJ, Watson J, Lobell DB et al (2014) A metaanalysis of crop yield under climate change and adaptation. Nat Clim Change 4(4):287–291
Choiniere CJ, Horowitz JK (2006) A production function approach to the GDPtemperature relationship. Preliminary version. https://scholar.google.com/. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.629.6492&rep=rep1&type=pdf. Cited 31 Oct 2014
Cooke RM (2013) Uncertainty analysis comes to integrated assessment models for climate change.. and conversely. Clim Change 117(3):467–479
Day B (1999) A metaanalysis of wagerisk estimates of the value of statistical life. CSERGE Working Papers, Centre for Social and Economic Research on the Global Environment, University College London, London
Davidson R, MacKinnon JG (1993) Estimation and inference in econometrics. Oxford University Press, New York
Dell M, Jones BF, Olken BA (2012) Temperature shocks and economic growth: evidence from the last half century. Am Econ J Macroecon 4(3):66–95
Dell M, Jones BF, Olken BA (2014) What do we learn from the weather? The new climateeconomy literature. J Econ Lit 52(3):740–798
Dietz S, Stern N (2015) Endogenous growth, convexity of damage and climate risk: how Nordhaus’ framework supports deep cuts in carbon emissions. Econ J 125(583):574–620
Eboli F, Parrado R, Roson R (2010) Climatechange feedback on economic growth: explorations with a dynamic general equilibrium model. Environ Dev Econ 15(05):515–533
Editors of Journal of Economic Perspectives (2015) Editorial note: correction to Richard S. Tol’s “The economic effects of climate change.” J Econ Perspect 29(1):217220
Fankhauser S (1995) Valuing climate change: the economics of the greenhouse. Earthscan, London
Gelman A (2014) A whole fleet of gremlins: looking more carefully at Richard Tol’s twicecorrected paper, “The economic effects of climate change”. Statistical Modeling, Causal Inference, and Social Science blog. http://andrewgelman.com/2014/05/27/wholefleetgremlinslookingcarefullyrichardtolstwicecorrectedpapereconomiceffectsclimatechange/. Cited 05 Oct 2016
Gøtzsche PC (1989) Multiple publication of reports of drug trials. Eur J Clin Pharmacol 36(5):429–432
Hanemann WM (2008) What is the economic cost of climate change? University of California, Berkeley. http://repositories.cdlib.org/are_ucb/1071/. Cited 09 Jan 2014
Havranek T, Irsova Z, Janda K et al (2015) Selective reporting and the social cost of carbon. Energy Econ 51:394–406
Heal G, Millner A (2014) Reflections: uncertainty and decision making in climate change economics. Rev Environ Econ Policy 8(1):120–137
Hoel M, Sterner T (2007) Discounting and relative prices. Clim Change 84(3):265–280
Houser T, Hsiang S, Kopp R et al (2014) American climate prospectus: economic risks in the United States. Rhodium Group. https://gspp.berkeley.edu/assets/uploads/research/pdf/American_Climate_Prospectus.pdf. Cited 29 Oct 2014
Howard PH (2014) Omitted damages: what’s missing from the social cost of carbon. The Cost of Carbon Pollution Project. http://costofcarbon.org/files/Omitted_Damages_Whats_Missing_From_the_Social_Cost_of_Carbon.pdf. Cited 13 Mar 2014
Howard P, Sylvan D (2015) Expert consensus on the economics of climate change. Institute for Policy Integrity, NYU School of Law, New York
Hsiang S (2016) Climate econometrics. Annu Rev Resour Econ 8:43–75
Hsiang SM, Burke M, Miguel E (2013) Quantifying the influence of climate on human conflict. Science 341(6151):1235367
Hsiang SM, Meng KC, Cane MA (2011) Civil conflicts are associated with the global climate. Nature 476(7361):438–441
Intergovernmental Panel on Climate Change (2007) Climate change 2007: the physical science basis. Contributions of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge
Intergovernmental Panel on Climate Change (2013) Climate change 2013: the physical science basis. Contributions of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge
Intergovernmental Panel on Climate Change (2014) Climate Change 2014: impacts, adaptation, and vulnerability. Part A: global and Sectoral Aspects. Contributions of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge
Interagency Working Group on Social Cost of Carbon (2016) Technical support document: technical update of the social cost of for regulatory impact analysis under Executive Order 12866. United States Government. https://www.whitehouse.gov/sites/default/files/omb/inforeg/scc_tsd_final_clean_8_26_16.pdf. Cited 15 Sept 2016
Kopp RE, Hsiang SM, Oppenheimer M (2013) Empirically calibrating damage functions and considering stochasticity when integrated assessment models are used as decision tools. Paper presented at Impacts World 2013—International Conference on Climate Change Effects, Postdam
Maddison D (2003) The amenity value of the climate: the household production function approach. Resour Energy Econ 25(2):155–175
Maddison D, Rehdanz K (2011) The impact of climate on life satisfaction. Ecol Econ 70(12):2437–2445
Mendelsohn R, Morrison W, Schlesinger ME et al (2000a) Countryspecific market impacts of climate change. Clim Change 45(3):553–569
Mendelsohn R, Schlesinger M, Williams L (2000b) Comparing impacts across climate models. Integr Assess 1(1):37–48
Meyer A, Cooper T (1995) A recalculation of the social costs of climate change. The Ecologist, Occasional Paper
National Academies of Sciences, Engineering, and Medicine (2017) Valuing climate damages: updatingestimation of the social cost of carbon dioxide. The National Academies Press, Washington. doi:10.17226/24651
Nelson JP, Kennedy PE (2009) The use (and abuse) of metaanalysis in environmental and natural resource economics: an assessment. Environ Resour Econ 42(3):345–377
Newbold SC, Marten AL (2014) The value of information for integrated assessment models of climate change. J Environ Econ Manag 68(1):111–123
Nordhaus WD (1994a) Managing the global commons: the economics of climate change. MIT Press, Cambridge
Nordhaus WD (1994b) Expert opinion on climate change. Am Sci 82(1):45–51
Nordhaus WD (2006) Geography and macroeconomics: new data and new findings. Proc Natl Acad Sci USA 103(10):3510–3517
Nordhaus WD (2008a) A question of balance: weighing the options on global warming policies. Yale University Press, New Haven
Nordhaus WD (2013) The climate casino: risk, uncertainty, and economics for a warming world. Yale University Press, New Haven
Nordhaus W (2014) Estimates of the social cost of carbon: concepts and results from the DICE2013R model and alternative approaches. J Assoc Environ Resour Econ 1(1/2):273–312
Nordhaus WD (2015) Q & A and damage functions: answers prepared by William Nordhaus, Yale University, for NRC Panel on the Social Cost of Carbon. http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_170062.pdf. Cited 21 Dec 2016
Nordhaus WD, Boyer J (2000) Warning the world: economic models of global warming. MIT Press, Cambridge
Nordhaus WD, Yang Z (1996) A regional dynamic generalequilibrium model of alternative climate change strategies. Am Econ Rev 86(4):741–765
Oppenheimer M, Little CM, Cooke RM (2016) Expert judgement and uncertainty quantification for climate change. Nat Clim Change 6(5):445–451
Pal JS, Eltahir EA (2015) Future temperature in southwest Asia projected to exceed a threshold for human adaptability. Nat Clim Change 6(2):197–200
Pindyck RS (2013) Climate change policy: what do the models tell us? J Econ Lit 51(3):860–872
Plambeck EL, Hope C (1996) PAGE95: an updated valuation of the impacts of global warming. Energy Policy 24(9):783–793
Rehdanz K, Maddison D (2005) Climate and happiness. Ecol Econ 52(1):111–125
Revesz RL, Howard PH, Arrow K et al (2014) Global warming: improve economic models of climate change. Nature 508(7495):173–175
Rhodes W (2012) Metaanalysis: an introduction using regression models. Eval Rev 36(1):24–71
Roson R, Mensbrugghe DVD (2012) Climate change and economic growth: impacts and interactions. Int J Sustain Econ 4(3):270–285
Schauer MJ (1995) Estimation of the greenhouse gas externality with uncertainty. Environ Resour Econ 5(1):71–82
Schlenker W, Roberts MJ (2009) Nonlinear temperature effects indicate severe damages to US crop yields under climate change. Proc Natl Acad Sci 106(37):15594–15598
Sherwood SC, Huber M (2010) An adaptability limit to climate change due to heat stress. Proc Natl Acad Sci 107(21):9552–9555
Stanley TD, Doucouliagos H, Giles M et al (2013) Metaanalysis of economics research reporting guidelines. J Econ Surv 27(2):390–394
Sterner T (2015) Economics: higher costs of climate change. Nature 527(7577):177–178
Tol RSJ (1995) The damage costs of climate change toward more comprehensive calculations. Environ Resour Econ 5(4):353–374
Tol RSJ (2002) Estimates of the damage costs of climate change—part 1: benchmark estimates. Environ Resour Econ 21(1):47–73
Tol RSJ (2008) The social cost of carbon: trends, outliers and catastrophes. Economics 2(25):1–22
Tol RSJ (2009) The economic effects of climate change. J Econ Perspect 23(2):29–51
Tol RSJ (2014) Correction and update: the economic effects of climate change. J Econ Perspect 28(2):221–226
Tol RSJ (2015a) Bootstraps for metaanalysis with an application to the impact of climate change. Comput Econ 46(2):287–303
Tol RSJ (2015b) Economic impacts of climate change. University of Sussex Working Paper Series No. 752015, University of Sussex Department of Economics, Falmer
Tramèr MR, Reynolds DJM, Moore RA et al (1997) Impact of covert duplicate publication on metaanalysis: a case study. Bmj 315(7109):635–640
U.S. Environmental Protection Agency (EPA) (2006) Report of the EPA Working Group on vsl metaanalysis, report EE0494. National Center for Environmental Economics, Washington. http://yosemite.epa.gov/ee/epa/eerm.nsf/vwAN/EE049401.pdf/$file/EE049401.pdf. Cited 21 Dec 2016
van den Bergh JCJM, Botzen WJW (2015) Monetary valuation of the social cost of \(\text{ CO }_{2}\) emissions: a critical survey. Ecol Econ 114:33–46
van den Bergh JCJM, Button K (1999) Metaanalysis, economic valuation and environmental economics. In: van den Bergh JCJM (ed) Handbook of environmental and resource economics. Edward Elgar, Cheltenham, p 796
Verardi V, Croux C (2009) Robust regression in Stata. Stata J 9(3):439–453
Wagner G, Weitzman ML (2015) Climate shock: the economic consequences of a hotter planet. Princeton University Press, Princeton
Weitzman ML (2009) On modeling and interpreting the economics of catastrophic climate change. Rev Econ Stat 91(1):1–19
Weitzman ML (2012) GHG targets as insurance against catastrophic climate damages. J Public Econ Theory 14(2):221–244
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Howard, P.H., Sterner, T. Few and Not So Far Between: A Metaanalysis of Climate Damage Estimates. Environ Resource Econ 68, 197–225 (2017). https://doi.org/10.1007/s106400170166z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s106400170166z
Keywords
 Climate change
 Damage function
 Duplication bias
 Metaanalysis
 Social cost of carbon