Causal identification using synthetic control methods
We use the Synthetic Control Method to generate a “synthetic UK” as a weighted average of other OECD, upper middle-, and high-income countries in our sample, or “donor pool.” Countries in the donor pool are selected through an algorithm so that the pre-CCP emissions trajectories of the UK and of the synthetic UK match each other as closely as possible. We then evaluate the causal effect of the UK’s Climate Change Programme by comparing the trajectory of emissions in the “synthetic UK” with the observed post-treatment emissions in the UK.
More formally, assume a sample of J + 1 countries where j = 1 corresponds to the treated UK, and \(J = \{2, \dots , J+1\}\) is our donor pool. The intervention (i.e., the passage of the CCP) occurs at T0 + 1 and so the pre-invervention time periods are indexed by \(t = 1, 2, \dots , T_{0}\) and the post-intervention time periods are indexed by \(t = T_{0}+1, T_{0}+2, \dots , T\). Let \(Y_{1t}^{C}\) represent the potential outcome under control for the UK, where j = 1 indexes the UK. These are the potential CO2 emissions in the UK if the CCP had not been passed. Let \(Y_{1t}^{T}\) represent the potential outcome under treatment; which are the potential CO2 emissions in the UK if the CCP had been passed. The causal impact of the CCP is the difference between the two, and so our estimand of interest is \(\alpha _{1t} = Y_{1t}^{T} - Y_{1t}^{C}\). However, \(Y_{1t}^{C}\) is unobserved.
Consider the following J × 1 vector \(\mathbf {W} = (w_{2}, \dots , w_{J+1})^{\text {\scriptsize T}}\) which contains the weights that reflect how much the j th candidate in the donor pool contributes to the synthetic counterfactual for the UK’s emissions trajectory. These weights are restricted to be non-negative and sum to 1, that is, wj ≥ 0 for \(j=2,\dots ,J+1\) and \({\sum }_{j=2}^{J+1} w_{j} = 1\). This restriction on the weights is imposed in order to avoid extrapolating when constructing the synthetic counterfactual (Abadie et al. 2010, 2015).
Let X1 be a K × 1 vector of the pre-treatment values of the K predictor variables of CO2 emissions in the UK. The K × J matrix X0 contains the corresponding values of the pre-treatment values of explanatory variables for the J control countries. In our case, the K = 11 attributes correspond to pre-treatment values of the outcome variable chopped up into discrete segments corresponding to CO2 per capita emissions in each pre-treatment time period, respectively. Using a specification which includes all pre-treatment lags of the outcome variable has been recommended as the benchmark specification, unless researchers have strong theoretical priors on how other covariates affect the outcome (Ferman et al. 2020).
The pre-intervention characteristics of the synthetic UK will be given by \(\mathbf {X_{1}^{\ast }} = \mathbf {X_{0}}\mathbf {W^{\ast }}\). The optimal W∗ should thus be chosen so as to minimize the distance ||X1 −X0W||, in order to construct a synthetic counterfactual that best approximates the treated unit with respect to pre-treatment outcome values. In practice, the SCM implementation seeks a W∗ that solves \(\underset {\mathbf {W^{\ast }}}{{\arg }} \ {\min \limits } \ \sqrt {(\mathbf {X_{1}}-\mathbf {X_{0}}\mathbf {W})^{\text {\scriptsize T}} \mathbf {V} (\mathbf {X_{1}}-\mathbf {X_{0}}\mathbf {W})}\). V is a K × K positive semi-definite, diagonal matrix of weights applied to the K variables that predict CO2 emissions. Therefore, the loss function is a scalar. The implementation of the SCM by its authors (Abadie and Gardeazabal 2003) allows for the choosing of a custom V weight matrix. This can be a fruitful approach if we possess a priori knowledge on the relative predictive power of different explanatory variables. However, in the absence of strong priors, we follow Abadie and Gardeazabal (2003) and Abadie et al. (2011) and adopt a data-driven approach whereby the matrix V is the one that minimizes the mean square prediction error (MSPE) of the pre-treatment outcome variable, i.e., such that the average squared discrepancies between the pre-treatment CO2 emissions of the UK and of the synthetic UK are minimized. A numerical optimization algorithm is used to solve for these optimal weights.Footnote 2
Finally, the observed emissions (pre- and post-treatment) of the UK are collected in a T × 1 matrix Y1. The CO2 emissions of the countries in the donor pool are recorded in a T × J matrix Y0. The emissions of the synthetic UK are simulated as \(\mathbf {Y_{1}}^{\ast } = \mathbf {Y_{0}}\mathbf {W}^{\ast }\). The estimated treatment effect is thus given by \(\hat {\alpha }_{1t} = Y_{1t} - {\sum }_{j=2}^{J+1} w_{jt}^{\ast } Y_{jt}\).
Causal identification is achieved using SCM under less restrictive conditions than difference-in-difference strategies. First, there can be no treatment spillover to other countries in the donor pool. Although the authors of the SCM approach do not explicitly refer to this assumption as such, this assumption is the stable unit treatment values assumption, or SUTVA, which states that “[t]he potential outcomes for any unit do not vary with the treatments assigned to other units, and, for each unit, there are no different forms or versions of each treatment level, which lead to different potential outcomes” (Imbens and Rubin 2015, p. 10). Second, to avoid interpolation bias, variables used to form the weights must be within the same support of the data for the treated unit and countries in the donor pool (Abadie et al. 2010, 2015). In other words, the variables used to form the weights must have values for the donor pool countries that are similar to those of the UK. This is because interpolation biases may be severe if the procedure interpolates across different regions with very different characteristics (Abadie et al. 2010).
In general, the UK during the early CCP era satisfies these conditions. The UK is the only country to be treated by the CCP in 2001, and is the only country in the sample that passed major climate legislation until the European Union launched its emissions trading scheme (EU ETS) in 2005. Our dependent variable is operationalized as CO2 emissions per capita, which ensures that the outcome variable across regions is broadly on the same order of magnitude and thus avoids interpolation bias. Moreover, alternative specifications provided in SI Section G also achieve a restriction of the data to a common support for all countries in the sample by employing a rescaled dependent variable (e.g., relative to a 1990 and a 2000 baseline, respectively). Running the synthetic control estimator on absolute CO2 emissions levels is not appropriate given the variance in emissions levels across countries.
Data sources and sample selection
To implement the synthetic control method, we use data on CO2 emissions and CO2 emissions per capita from the World Bank’s World Development Indicator (WDI) database, extracting indicators “EN.ATM.CO2E.KT” (CO2 emissions in kilotons) and “EN.ATM.CO2E.PC” (CO2 emissions per capita in metric tons), respectively. The CO2 emissions measured are those stemming from the burning of fossil fuels and the manufacture of cement. We impute some missing data for Germany, Kuwait, and Liechtenstein using alternate data sources. This procedure is described in the online SI Section A.
We define our donor pool as the 51 countries which were either OECD members or classified by the World Bank as upper middle–income or high-income countries at the time of treatment in 2001, that had a population greater than 250,000, and that did not have a carbon pricing policy in place. The Work Bank classifies countries into income categories according to GNI per capita in US$. In fiscal year 2001, the World Bank classified high-income (HIC) countries as those with GNI per capita above 9265 US$, and upper middle–income (UMC) countries as those with GNI per capita in the 2996 US$ to 9265 US$ range. In 2001, there were 47 high-income countries, 38 upper middle–income countries, and 30 OECD countries. Our donor pool is the union of those sets, minus countries for which data is missing or countries that were deemed “treated” in 2001, and minus countries with a very small population.
We determine whether countries in the sample were “treated” by building on the World Bank’s State and Trends of Carbon Pricing 2019 report (World Bank 2019), albeit with some modifications. Even though the World Bank report notes that Poland had passed a carbon tax in 1990, we do not consider it “treated” until 2005 (the start of the EU ETS) because the Polish tax was so small in scope and incidence that it cannot be considered a materially important carbon pricing policy. Indeed, the Polish carbon tax of 1990 was less than 1 US$ per ton CO2e and covered only 4% of the jurisdiction’s emissions (World Bank 2019).
Moreover, we consider the Netherlands to be “treated” in 2001, even though the World Bank report does not consider the Netherlands as having a carbon tax. However, the Netherlands introduced a tax on energy in 1996, which complemented a tax on fuel that came into force in 1992. Tax rates were set as a function of CO2 per energy content, and were estimated to be around NLG 30 per metric ton of CO2 (Hoerner and Bosquet 2001, p. 20).
The countries that were “treated” in 2001 were thus the following: Denmark (carbon pricing policy first passed in 1992), Estonia (2000), Finland (1990), Netherlands (1992), Norway (1991), Slovenia (1996), and Sweden (1992). These countries are excluded from the donor pool.
Specifications
In the main specification we report below, we construct this synthetic UK from a donor pool of countries that were either OECD, upper middle-, or high-income countries in 2001. We exclude small countries with a population less than 250,000 in 2001 since these may have different fundamental drivers of CO2 emissions than the UK. Not all countries in this donor pool contribute equally to this synthetic control. In our main specification, 8 countries make up the effective sample (see Fig. S1 in the SI) accounting for 88% of the weights, with the other countries having weights of less than 1%. In the SI’s Fig. S2, we also display the CO2 per capita emissions of the donor countries in the effective sample. In this specification, which generates the strongest pre-treatment fit and performs best according to diagnostics reported in the Findings section and in SI Section G, the counterfactual trend is estimated using a blend of 19% Poland, 19% Libya, 18% Bahamas, 16% Belgium, 6% Trinidad and Tobago, 5% Uruguay, 4% Luxembourg, and 1% Brunei. Here, the pre-treatment MSPE achieved with that donor pool was 1.24 × 10− 4. Figure S1 in the online SI displays the weights applied to each country in the donor pool.
The fact that surprising countries, such as the Bahamas and Libya are part of the top donors, while an intuitively similar country like France is at the bottom should not be cause for concern. Rather, it suggests that there were latent, unobserved forces driving British emissions, and that a weighted combination of these forces was found in the top donor countries. Specifically, the synthetic control approach estimates a latent factor model with a linear combination of time-varying and time-invariant confounds. Some combination of the unobserved factors responsible for driving British emissions was also present in donor countries, which are then re-weighted to create a credible control for the UK.
Instead, an advantage of this effective donor pool is that it rules out spatial spillover effects.Footnote 3 One of the assumptions required for causal identification is that the treatment affected the treated unit only and did not spillover to other control units (the SUTVA assumption). Since the UK’s untreated neighbors such as France and Germany are not part of the effective sample of countries used to generate the synthetic control, our results are not at risk of over-estimating the treatment effect of the CCP due to a violation of the SUTVA assumption.
As a robustness check, we also evaluate specifications generated by progressively smaller donor pools, again applying population filters: (1) on countries that were either OECD members or high income countries in 2001; and (2) on countries that were OECD members in 2001. The pre-treatment MSPE increases (indicating a poorer fit between the UK and the synthetic UK) as the donor pool decreases: from 5.24 × 10− 4 (donor pool consisting of 2001 OECD and HIC countries) to 2.13 × 10− 3 (donor pool consisting of 2001 OECD members). However, despite these specifications being slightly weaker from a SCM perspective, they still generate similar estimates of the effect of the UK policy (see section G in the SI). In this way, while we choose our specification in a principled way based on synthetic control method best practices, our results hold even for a range of donor pools that rely only on countries with substantively similar political and economic systems.
Generally, there are a multitude of observed and unobserved factors, both dynamic and constant in time, that drive British emissions in ways that are hard to specify a priori. Attempting to specify a functional form that would accurately reproduce the emissions trajectory of the UK is a difficult task. The advantage of the SCM is that it enables us to sidestep the need to enumerate all of the structural drivers of CO2 emissions. By contrast, we employ a non-parametric approach where we find the combination of (latent) drivers in donor countries that serve as an appropriate control by numerically minimizing the distance between the pre-treatment trends of the UK and the control.
The predictor variables used to construct a synthetic UK are the pre-treatment values of per capita CO2 emissions from 1990 to 2000, with no other covariates. Other covariates might be useful to improve the match between the UK’s pre-CPP emissions and its synthetic counterpart. In Section G2 of the SI, we show this was not the case, and therefore we report our estimates using pre-intervention values of the dependent variable only. Kaul et al. (2018) show theoretically that using all pre-treatment values of the outcome variable as separate predictors in the SCM algorithm leads to an optimization procedure that renders all other covariates irrelevant. We verify empirically that this is the case: specification 2 in our SI uses 4 covariates as predictors (GDP per capita, renewable energy consumption, fossil fuel energy consumption, and energy use per capita), in addition to the pre-treatment values of per capita CO2 emissions. The weights on the 4 covariates when constructing the synthetic UK are all 0.
We construct our synthetic UK on the basis of the lagged values of CO2 emissions per capita alone for three reasons. First, doing so leads to an optimal pre-treatment fit between the UK and its synthetic control. Since the goal of SCM is to create a credible counterfactual for the treated unit in the absence of treatment, a guiding heuristic is to choose the specification that minimizes the distance in potential outcomes pre-treatment. Second, this research design choice minimizes the risk of specification searching on the part of researchers. Ferman et al. (2018, 2020) suggest that despite the advantage of the transparency of the SCM, researchers have some latitude to engage in specification-searching. By restricting our choice set to specifications that only include pre-treatment values of the outcome variable, we tie our hands at the outset. Third, we do not have strong theoretical priors on the types of covariates that would capture most of the drivers of British CO2 emissions. While we may account for observable characteristics that correlate with the outcome, such as income per capita, this is by no means a guarantee that we would account for the unobservable characteristics that determine the pattern of emissions. Ferman et al. (2020) address this problem and recommend that in the case where researchers do not have strong theoretical priors on the covariates to use, a specification which uses all pre-treatment lags of the outcome variable should be used and reported as the benchmark specification. Nevertheless, as a robustness check, we also estimate the treatment effect using alternative specifications, which we report below and in further detail in online SI Section G.