Abstract
The limited microeconometric evidence on the efficacy of environmental Negotiated Agreements (NAs) is an obstacle to both their introduction and effective design. We help fill this gap by providing evidence on the impact of the second Climate Change Agreements (CCAs) on business electricity consumption and employment. The CCAs are NAs offering a reduction on the Climate Change Levy (CCL), an energy consumption tax, in exchange for commitments to improve energy efficiency. We use the novel changes-in-changes method to account for heterogeneity in treatment effects. Our results indicate that the second CCAs yielded improved outcomes compared to the counterfactual of full CCL with an average reduction of − 4.81% in electricity consumption. They also reveal the importance of allowing for heterogeneity, as the impact on electricity consumption at the identified deciles varied between − 9.33 and 12.54%. This is a marked difference from the first CCAs which were found to increase consumption. The heterogeneity in treatment response is corroborated when extending the study to two large industrial sectors in the sample and when studying firms selecting differing target reporting methods. Confirming the findings from earlier studies of the first scheme, our results indicate a non-statistically significant reduction in employment, about − 4.6% on average, for the second CCAs.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
The use of voluntary agreements in environmental regulation has garnered substantial interest from academics and policy makers alike. When enacting regulation policymakers need to balance private and public welfare; overly stringent regulation could harm businesses whereas lax regulation would not adequately accomplish society’s environmental goals (Pearce 2006). Drafting the optimal policy is made difficult by information asymmetries between regulators and firms that are costly to remove and not necessarily in the interest of firms to resolve (e.g. Montero 2000; Bellassen and Shishlov 2016). Voluntary agreements (VAs) intend to achieve environmental policy aims without mandating participation. Participants are oftentimes given flexibility in how emissions abatement is achieved, letting them choose the path that minimises compliance costs (Segerson and Miceli 1998). Negotiated agreements (NAs) have a similar structure, as firms take part voluntarily, but involve negotiation of targets between the government and industry stakeholders with potential mandatory regulation as an alternative (Bailey and Rupp 2006; Fleckinger and Glachant 2011). Despite the prevalence of NAs in modern environmental policy making, their track record in terms of efficacy is mixed. We provide evidence of their efficacy by assessing the impact of the second UK Climate Change Agreements (CCAs).
The CCAs were introduced in 2001 with the Climate Change Levy (CCL), an energy tax levied on most non-domestic, i.e., non-household, energy consumption.Footnote 1 In order to avoid disadvantaging energy intensive sectors against international competition the CCAs reduce the CCL rates in exchange for a commitment to meet negotiated energy efficiency targets. Participation in the scheme is voluntary and policy targets are negotiated between the government and sector bodies, which also monitor the performance of participating firms. The first CCAs ended in December 2010 and were followed by the second CCAs in April 2013, which are planned to continue until 2025. A thorough discussion of the CCAs/CCL package is provided in Sect. 2.
This paper contributes to the existing literature by building on the quantitative evaluation of energy and environmental policy in three ways. First, this work measures the impact of the currently unstudied second phase of the CCAsFootnote 2 on electricity consumption and employment, where we find evidence supporting the efficacy of negotiated agreements. Second, we use a matched data set containing facility-level electricity consumption based on meter-level data for the second CCAs; hitherto available studies of the CCAs are restricted to plant level data, that can contain a multitude of facilities, of the first phase.Footnote 3 Third, it conducts a quasi-experimental evaluation using the changes-in-changes (CIC) estimator developed in Athey and Imbens (2006), which estimates heterogeneous treatment effects non-parametrically across the quantiles of the outcome distribution. The CIC estimator has seen seldom prior application and, to the knowledge of the authors, none in evaluating energy policy.
Many structural features of the CCAs contribute to potential heterogeneity in its impact. Sector-wide efficiency targets were negotiated between the government and trade bodies, thus it is safe to assume that the stringency of different sector-wide agreements cannot be assumed identical. In a similar fashion, facilities in the policy are heterogeneous in size, which can affect feasible abatement options. Furthermore, participants are able to choose the type of targetsFootnote 4 used to assess compliance with the policy. Thus, understanding how different aspects of the policy affected participants is valuable in uncovering its real impact. The CIC estimator retains this nuance in the treatment effect by evaluating treatment effects across quantiles of the outcome distribution. Furthermore, the CIC requires only an outcome variable and a treatment indicator, which is beneficial here due to data matching reducing the observations available. These advantages of the method ultimately motivated the use of the CIC estimator.
We build on the statistical evaluation of the first phase of the CCAs conducted in Martin et al. (2014) by focusing on the heterogeneous impacts of the second CCAs. Our results indicate that the outcomes of the second CCAs have markedly improved compared to the first, as electricity consumption decreased by a statistically significant − 4.81% compared to a full CCL counterfactual. This result stands in strong contrast to the findings in Martin et al. (2014), which showed that the emissions and consumption targets in the first CCAs did not provide sufficiently strong incentives for firms to reduce electricity consumption [See also Ekins and Etheridge (2006)]. On the other hand, our results indicate no statistically significant impact of the second CCAs on average employment, mirroring the results in Martin et al. (2014). With regard to the heterogeneity of the impacts on electricity consumption across different sized participants, the estimated treatment effect ranges from a − 9.33% decrease to a 12.54% increase. In most deciles of the distribution, participation in the CCAs reduced or not significantly affected electricity consumption compared to the CCL. Therefore, the policy achieved similar policy outcomes to the levy without adding to the tax burden of participating firms. Similar results are found when estimating the what-if scenario of the CCL control group participating in the CCAs.Footnote 5 This implies that CCAs, or a similarly structured NA, could be an attractive alternative to price-based instruments when the dominant objectives are to encourage reductions in emissions or energy consumption while minimising additional costs for firms. This might be particularly relevant in those instances where opposition to price-based instruments is strong, for example, due to concerns related to international competitiveness.
We also expand the analysis to different policy target types and individual sectors. The treatment effect within target types is mixed regarding heterogeneity, while outcomes between target types differ strongly. Results of the two sectors studied, henceforth Sector A and Sector B,Footnote 6 provide an interesting contrast. Sector A, consisting of predominantly small and some large electricity consumers, mostly increased electricity consumption in the CCAs; however, some very large facilities decreased their consumption skewing the average impact of the policy downward to a 0.93% increase. Sector B, in comparison, are medium to large scale users that consistently decreased their electricity consumption in the CCAs by − 13.9% on average.
The remainder of the paper is structured as follows: Sect. 2 discusses the CCA/CCL policies, Sect. 3 frames the CCAs within the wider literature of VAs/NAs and provides select econometric evaluations of VAs/NAs, Sect. 4 elaborates on the changes-in-changes estimator, Sect. 5 describes the data and related matching procedures, Sect. 6 presents the results, and, lastly, Sect. 7 concludes the paper.
2 The Climate Change Levy (CCL) and the Climate Change Agreements (CCAs)
The Climate Change Levy (CCL) is a tax charged on energy consumed in industry, commerce, and the public sector introduced on April 1st, 2001.Footnote 7 The aim of the policy was improving businesses’ energy usage and stimulating investment in energy efficiency (DETR 2000) by imposing a per unit of energy or weight basis tax that applies to electricity, gas, and other solid fuels. Levy rates were fixed from 2001 to 2006 and adjusted with inflation in the following years. In 2012 and 2014, the period used in this study, electricity rates were 0.51 and 0.54 pence/kWh, respectively.
Fearing a reduction in the competitiveness of energy intensive industries, the government introduced the Climate Change Agreements (CCAs) alongside the CCL.Footnote 8 The CCAs are negotiated agreements between the government, sector bodies, and firms granting reduced CCL rates to participants when meeting energy efficiency targets. The first CCAs ended in December 2010 and were followed by the second CCAs, which are set to cover January 2013 to March 2025; the period between 2011 and 2012 served as a gap between the two policies that granted CCAs discounts without emissions or efficiency commitments. As the policy was revised between the two iterations, we will focus the discussion on the workings of the second CCAs and highlight the differences in Sect. 2.1.
Discount rates were adjusted throughout the policy; for example, the applicable rates on electricity in 2012 and 2014 were 65% and 90%, respectively. Instead of setting performance targets with individual firms, the Department for Business, Energy and Industrial Strategy (BEIS) negotiated targets with sector associations, which monitor and collect participating firms’ energy consumption and emissions. The negotiated targets were then included in an umbrella agreementFootnote 9 between the Environment Agency and the sector associations (AEAT 2004).
Firms taking part in the CCAs need to agree to meet established targets in underlying agreements with the sector associations. Performance in the policy is evaluated for each target unit, which can be a single or a group of facilities based on vicinity and production criteria, rather than the firm in its entirety. Targets are established as a percentage decrease from a base year, which for the second CCAs is in most cases 2008, and can be set in three ways: (1) absolute; (2) relative; (3) novem. Absolute targets are based on total energy usage or emissions, for example, measured in kWh or tCO2. Relative targets are set relative to some unit of output, e.g. kWh/\(\text {m}^2\) of fabric. Lastly, novem targets are applicable in cases where a target unit produces more than one product with different energy intensities or output units.Footnote 10 Performance against targets is evaluated in biennial target periods; failure to meet the goals can be remedied through accrued over-performance in earlier periodsFootnote 11 or a buy-out mechanism (EA 2014).Footnote 12 If neither option is chosen access to the rebate is revoked and regular CCL rates apply (EA 2017).
Eligibility to the CCAs changed throughout the policy. Initially, firms holding a permit for the Integrated Pollution Prevention and Control (IPPC) regulation, which was later merged with the Waste Management Licensing (WML) legislation and renamed Environmental Permitting Regulation (EPR), were allowed to join. The rules were expanded in 2005 to allow in sectors fulfilling energy intensity and trade metrics.Footnote 13 In contrast, production processes under the mineralogical and metallurgical processes category were made exempt from the CCL and thus the CCAs in 2014 to avoid distortions of competitiveness compared to other EU countries (HMRC 2015).
2.1 Changes from the First to Second CCAs
To illustrate why outcomes for the first and second CCAs could be different, we provide a brief overview of key changes implemented between the two. Scheme compliance was measured at the sector level during the first CCAs; access to the discount was granted collectively to the entire sector when sector targets were met, however, when targets were missed, DEFRA could request performance data of specific target units and revoke access to the CCAs rebate to individual firms. This implied that participants could still access the rebate without meeting their individual targets, which could incentivise shirking.Footnote 14 In contrast, the second CCAs evaluate performance solely at the target unit level. In effect, this curbs free-riding incentives by holding each target unit accountable for continued access to the rebate regardless of sector performance. The percentage rebate on the CCL for electricity also varied throughout the policy, starting at an 80% reduction in 2001 that subsequently changed to 65% and 90% in April 2011 and April 2013, respectively (BEIS 2020). Additionally, in the first CCAs sectors were allowed to choose a base year to measure improvements against, which lead to base years ranging between 1990 and 2008 at the tail-end of the first policy (DEFRA 2007). Although this afforded flexibility to participating sectors, it could have incentivised them to choose years that lessen the stringency of targets. The second CCAs homogenised the base year to 2008 in order to reduce administrative burden, which also likely affected relative target stringency by preventing firms strategically choosing base years.Footnote 15 Furthermore, the buy-out mechanism was simplified and uncoupled from the UK Emissions Trading Scheme allowance price, which was historically low ranging from 50 pence to \(\pounds\)4 per tonne of CO2 (DECC 2012).Footnote 16 Instead, prices were initially fixed at \(\pounds\)12 per tonne CO2 in the second CCAs and were subject to review in following target periods (EA 2015a). By introducing a constant cost that is substantially higher than historic prices, non-compliance became significantly more expensive. Ultimately, it seems plausible that information gathered and learning accrued during the first CCAs has contributed to reducing information asymmetries between policy makers and firms, with the resulting changes made to the policy leading to more ambitious outcomes.
3 Literature Review
Voluntary agreements (VAs) and negotiated agreements (NAs) are a broad category of policy instruments encompassing a multitude of configurations reflected in the academic literature (Bailey and Rupp 2006). Many theoretical models have been developed to understand how different flavours of the two might affect adoption rates, economic performance, and other policy outcomes (Khanna 2001), but evidence of their success is rather sparse.Footnote 17 In order to maintain focus on the quantitative evaluations of VAs/NAs, this section will feature two parts. First, it provides an overview of general criteria for a VA/NA to be successful and their applicability to the CCAs/CCL.Footnote 18 Second, it will briefly discuss the mixed reputation of VAs/NAs and findings from econometric evaluations of the CCAs.
3.1 Theoretical Context of the CCAs and CCL Package
From a theoretical perspective, VAs/NAs are an interesting form of environmental regulation since affected subjects can—at least on paper—choose whether or not to participate in the scheme. In practice, however, participation is often negotiated under duress of alternative mandatory policies, influencing whether participation was truly voluntary (See Goodin 1986; Segerson and Miceli 1998; Croci 2005). However, voluntary measures can be advantageous for all parties involved as they encourage a pro-active cooperative approach that reduces conflicts, offers greater flexibility to find cost-effective solutions tailored to specific conditions, and grants the ability to quickly meet environmental targets due to decreased negotiation and implementation lags (Segerson and Miceli 1998). When setting targets, the government needs to strike a balance between achieving policy aims and ensuring participation. Asymmetric information about abatement potential could result in overly stringent policy targets that discourage participation, whereas lax targets could lead to not achieving social environmental goals.Footnote 19
As there are many potential pitfalls that might render VAs/NAs ineffective, we draw on reviews of the literature to find common criteria that ensure that the policy meets its environmental objectives. First, participation incentives, both in the form of benefits and punishments, need to be sufficiently strong to encourage firms to engage with the policy. These can come in multiple forms, such as regulatory threats, subsidies, or establishing green credentials to policymakers and consumers (Segerson 2013). Regulatory threats in particular are considered to be effective if the likelihood of enacting legislation is high, i.e., the threat is credible (Segerson 2018). Second, monitoring of scheme performance needs to be transparent and verifiable. Without adequate monitoring of compliance it becomes difficult to determine when to grant and revoke access to privileges of the policy (Rezessy and Bertoldi 2011). Third, free-riding incentives need to be minimised and punishments need to be enforceable. In conjunction with the second criterion, free-riding becomes an attractive alternative if accurate measurements of individual performance are not obtainable. For example, a policy that only measures compliance at the sector level renders the danger of individual punishment moot and makes free-riding economically advantageous (de Vries et al. 2012).
How are the salient criteria explored in the aforementioned papers applicable to the CCAs and CCL package? The CCAs offered strong legislative participation incentives by granting a substantial discount on the CCL. Since the CCL and CCAs were introduced at the same time, there is no uncertainty about the regulatory threat. Thus missing targets in the CCAs becomes costly and unattractive. Additionally, the policy offered an opportunity to establish green credentials with consumers and policymakers. Surveys conducted among firms in the second CCAs indicate that reputation was a major motivator to participate, as 58% of respondents answered that demonstrating green credentials affected their decision, making it the third most frequently cited reason after a reduction in the CCL and the achievability of sector targets (BEIS 2020). Regarding the second and third criteria, while the first CCAs measured scheme compliance at the sectoral level,Footnote 20,Footnote 21 thus not effectively prohibiting free-riding (Dijkstra and Rübbelke 2013), this short-coming was remedied in the second iteration when overall targets were negotiated at the sector level, passed down to individual target units, and scheme compliance was evaluated at the target unit level.
3.2 Statistical Policy Evaluation of VAs/NAs and CCAs/CCL
The available econometric studies on VAs/NAs have covered diverse configurations and geographic locations, finding mixed results regarding their efficacy (Segerson 2018). A prominent example is the United States’ Environmental Protection Agency’s 33/50 program, the evaluation of which has led to vigorous academic debate. The program was a voluntary agreement that asked participants to reduce emissions of 17 toxic gases by 33% by 1992 and 50% by 1995. Whether the policy succeeded in reducing emissions is debated; historically, studies have found evidence attributing reductions to the policy (Bi and Khanna 2012; Khanna and Damon 1999; Innes and Sam 2008) as well as disputing them (Vidovic and Khanna 2012; Gamper-Rabindran 2006). Results from more recent studies are similarly inconclusive. Research conducted in Zhou et al. (2020), which accounts for potential information spill-over effects between participants and non-participants, shows that the policy was effective in reducing emissions as non-participants learned from participants. On the other hand, estimates from Hoang et al. (2018) indicate that the policy did not succeed in lowering general emissions, although firms shifted away from chemicals most toxic to human health.
While the overall literature evaluating impacts of VAs/NAs is mixed, a policy sporting a similar structure to the CCL/CCAs packages has seen success. In 1993 a CO\(_2\) tax was implemented in tandem with a possible rebate for energy intensive firms in Denmark. To qualify for the reduction, firms had to enter an agreement with the government to undertake negotiated energy saving activities and progress was evaluated by energy consultants. Results from Bjørner and Jensen (2002) indicate that the negotiated agreement was indeed successful in reducing carbon emissions, as energy consumption was found to substantially decrease. However, the authors caution that the sample size of eligible firms was rather small and related criteria, such as economic efficiency of the policy, were not considered making broader conclusions of the efficacy of NAs difficult. By using a large sample of firms covering diverse industries, this paper provides further evidence towards NAs successfully accomplishing their policy goals.
3.3 Econometric Evaluations of the CCAs and CCL
Focusing on the CCAs and CCL, two major econometric studies have investigated their effect, Adetutu and Stathopoulou (2020) and Martin et al. (2014). Adetutu and Stathopoulou (2020) studies the role of information asymmetry between regulators and firms in the first CCAs and CCL from 2001 to 2007. During target negotiations, the government is potentially at an information disadvantage regarding abatement opportunities available to specific firms. In such cases, firms might be able to negotiate lenient targets while claiming the CCAs rebate. The study first establishes the best-practices level of energy efficiency via stochastic frontier analysis and contrasts results to firm-specific energy efficiency levels. It then uses these results to evaluate how relative efficiency affects the received CCAs discount using pooled OLS, fixed effects, and system-GMM estimators.Footnote 22 Results indicate that the average firm in the CCAs could reduce energy expenditure by an additional 19% when matching best practices available in the sample and that firms with higher energy efficiency received lower CCAs discounts. Thus, it appears that participants in the first CCAs left substantial improvement potential unrealised.
Martin et al. (2014) compares the impact of the CCL against the CCAs over the 2001 to 2004 period at the plant level.Footnote 23 Before presenting the results, it is important to highlight that carbon taxes, i.e., the CCL, is the instrument of interest in Martin et al. (2014), leading to firms affected by the CCAs being used as the control group. In contrast, as our focus lies in the efficacy of VAs/NAs, we treat CCL participants as the control group. While this complicates the comparison of our results with those in Martin et al. (2014), the different framing is made explicit when presenting and contrasting outcomes between the two studies. Estimation of the model in Martin et al. (2014) is conducted via two-stage IV regression and results show that plants subject to the CCL reduced electricity consumption by 22.6% and energy share in gross output (EGO), defined as energy expenditures over gross output, by 17.2% when compared to participation in the first CCAs, corroborating results of macroeconomic evaluations of the policies in Ekins and Etheridge (2006). Results for employment and \(CO_2\) indicate that not participating in the CCAs leads to higher employment, although not statistically significant, and decreased emissions by between 9.6 and 22.6%. Overall, the findings indicate that the first CCAs delivered decreased environmental performance, that is, worsening energy efficiency and increasing emissions, while not delivering any statistically significant improvement in economic performance. The study posits that the poor performance of CCAs compared to the CCL may be related to low target stringency.
4 The Changes-in-Changes Estimator
The changes-in-changes (CIC) estimator used in this study was developed in Athey and Imbens (2006) as a generalisation of the popular differences-in-differences (DID) estimator. Instead of producing a singular summary statistic, such as the average treatment effect, the method preserves the heterogeneity in treatment response by estimating the treatment effects across the distribution of the outcome variable. Akin to the DID estimator, the CIC estimates the counterfactual distribution of the treated group in the absence of the policy post-treatment by combining the difference in the distributions of the control group before and after treatment, the time effect, with the distribution of the treated group before treatment. Furthermore, as the method estimates the treatment effect non-parametrically, no assumptions are made about the functional form of the equation generating the outcomes, Y. Outcomes without or prior to any treatment are posited to be generated by a shared function
while outcomes after treatment are generated by
with \(U_i\) being an index aggregating all individual characteristics relevant to the outcome variable and \(T_i\) being a treatment indicator for participant i. Given a realisation of U = u, the treatment effect, \(\tau (u)\), can be computed as the difference in outcomes
Thus, the estimation of the treatment effect hinges on the unobservable U. To help conceptualise U, a real-life example of such an index would be the EPC rating used in the United Kingdom to evaluate a building’s energy efficiency, which aggregates housing characteristics such as window glazing and water heating system into a singular score; however, U does not need to be directly observable for the estimation given the identifying assumptions discussed in detail in the subsequent Sect. 4.1.
By finding matching levels of the unobservable, the method is robust to self-selection in the quantile treatment effects, as U encapsulates factors that could affect the treatment outcomes. However, since there are no restrictions on the differences in the distribution of U between the treated and control group, it is important to bear in mind that the estimated average impact is a treatment effect for the treated. As an example, suppose that a treatment shifts the distribution of U asymmetrically, for example, by a large margin in the treatment group and by a small margin in the control group if it had been subjected to treatment. In that case, the estimated average treatment effect would only be valid for the treatment group and not for the control group.
4.1 Identification
The focus of this section is to present the three key assumptions of the estimator, how they enable identification of the treatment effect, and a way to test the assumptions in an intuitive manner. For a detailed discussion of the CIC methodology we refer to the original (Athey and Imbens 2006) paper.
The first assumption for the CIC is that outcomes before treatment, regardless of later treatment status, are generated by the same function \(h(U_i,T_i)\). This allows the comparison between the treated and control group before any treatment occurs.
The second assumption requires the function \(h(U_i,T_i)\) to be strictly monotonic. As mentioned in the previous section, the key to the estimation in the CIC is the identification of similar levels of U. Although finding this matching level of U is complicated by not necessarily being directly observable, under monotonicity the relative position of U, i.e., the ranking in the distribution, can be found. Instead of using the exact values of U, one can use quantiles and therefore the ranking of each U to find similar individuals throughout the treatment and control group.
The last assumption requires time invariance of U within groups. If the passing of time substantially reshuffles the relative position of outcomes inside a group it becomes impossible to trace the same level of U across time. For example, for a value of U, if it is at the 60th percentile in the control group in the pre-treatment time period, the same U needs to be at the 60th percentile of the control group in the post-treatment period.Footnote 24 Through this assumption it becomes possible to recover the time effect by comparing the pre-treatment outcomes and post-treatment outcomes of the control group at the same percentile. Thus, the three assumptions can be written more formally as
-
1.
Model: The outcome of an individual in the absence of treatment satisfies \(Y_i^N = h(U_i,T_i)\)
-
2.
Strict Monotonicity: the function h(u,t), where h: \(\mathbb {U}\times \{0,1\} \xrightarrow {} \mathbb {R}\) is strictly increasing in u for t\(\in \{0,1\}\)
-
3.
Time Invariance within Groups: \(U\perp T|G\)
When combining the three assumptions, it becomes possible to identify the treatment effect \(\tau _q\) for each quantile q via
where \(F_{Y,gt}\) is the cumulative density function (cdf) of the outcome variable Y with group membership g at time period t and \(F^{-1}_{Ygt}\) is the inverse of the cdf. The first part of the right-hand side of the equation, \(F^{-1}_{Y,11}(q)\), is the observed outcome of the treated (g = 1) after the treatment occurred (t = 1) at quantile q, while the second part, \(F^{-1}_{Y,01}(F_{Y,00}(F^{-1}_{Y,10}(q)))\), is the counterfactual outcome. The estimation of the counterfactual and treatment effect is straightforward when broken down into steps and tackled from the innermost to the outermost operation. To further illustrate the procedure, we provide a visual example in Fig. 1 for some arbitrary outcome variable and quantile q = 0.65, where blue curves are the outcomes of the treated, red curves the outcomes of the control, the green curve the counterfactual, solid curves the outcomes at t = 0, dashed curves the outcomes at t = 1, and black lines serve as visual guides.
For each quantile q, we first find the associated outcome value in the distribution of the treated at t = 0 (e.g. point A in Fig. 1). Since both groups’ outcomes are generated by the same equation h(U, T) before treatment (assumption 1), we can find the associated quantile of the outcome value in the control group at t = 0 (point B in Fig. 1) by locating the same U in the distribution of the control group before treatment. Then, via time invariance within groups (assumption 3), we recover the time effect from the control group by evaluating the same quantile in the distribution of the control group at t = 1 (point C in Fig. 1), which serves as the counterfactual value at q (point D).Footnote 25 By repeating this process for each q, the procedure allows us to trace out a counterfactual distribution by applying the appropriate time effect to each quantile in the outcome distribution of the treatment group at time 0. Lastly, the treatment effect is estimated by taking the difference in the observed outcome of the treated in t = 1 and the counterfactual at q (\(\tau\) in Fig. 1).Footnote 26
A consequence of the method is that the range of estimable values for the counterfactual distribution is determined by two factors: 1. the overlap between the distributions of the treated and control at time 0; 2. the support of \(Y_{01}\), \(\mathbb {Y_{01}}\). Any \(y_{10}\not \in \mathbb {Y_{00}}\) cannot be translated from the treatment group to the control group at t = 0, that is the second step of translating the outcome in the treatment group in Fig. 1 (moving from A to B) vertically to the control group is not possible as no reference value exists. Additionally, for any \(y\not \in \mathbb {Y_{01}}\), the counterfactual is not identified, due to the counterfactual distribution being limited by the range of values existing in the control group at t = 1. Therefore, the distributions of the treatment and control group do not affect estimation results, but the range of estimable quantiles.
In order to verify the assumptions we use the test introduced by Melly and Santangelo (2015), which evaluates the following criterion
for two time periods before treatment and all \(q \in (0,1)\) using a two-sample Kolmogorov–SmirnovFootnote 27 test. Similar to tests of the common trend assumption in DID models it uses periods prior to treatment, labelled t = − 1 and t = 0, to check whether model assumptions are satisfied. The intuition of this test is that, before treatment, the translation of outcomes from the treatment to the control group at quantile q should consistently remain similar in ranking when all outcomes are generated by h(U, T). If this is not the case, it would indicate that either outcomes are not generated by a shared h(U, T) or that the distribution of U is not independent of time within each group. We provide a visual representation of the test in Fig. 2 using the same colour and curve styles as in Fig. 1.
The left-hand side of the equation first evaluates the corresponding outcome of a quantile q at time period − 1 of the to-be-treated (A in Fig. 2). Then, it finds the quantile associated with the value, i.e., the translated ranking, among the control (C in Fig. 2) in the same period. The right-hand side does the same for another pre-treatment period 0 (B to D in Fig. 2). If these rankings between time 0 and − 1 vary, either the distribution of the unobservable, and by extension of the outcome variable, is unstable within groups over time or the outcomes are not generated by a shared function h(U, T). Referring to Fig. 2, if the vertical distance between C and D labelled ”discrepancy” were too big in the pre-treatment periods at any quantile, the KS-test would reject its null hypothesis of the two samples being drawn from the same distribution. Thus, given the assumptions of the method, a KS-test would fail to reject its null hypothesis.
5 Data
The data for this article was combined from a multitude of data sources provided by the Department for Business, Energy and Industrial Strategy (BEIS). Participants were identified using scheme data, while the control group was created using information declarers in the Carbon Reduction Commitment (CRC), which are defined in detail in Sect. 5.2. The two groups were matched to readings in BEIS’s meter dataset through text-based address matching. Both groups were matched to the Inter-Departmental Business Register (IDBR) to gain employment statistics and cross-referenced against major concurrent policies.
5.1 Treatment Group: CCA Participants
Information for the treatment group comes from scheme data collected by the Environment Agency (EA). The original dataset contains 14,366 facilities in 53 sectors that were in contact with the EA regarding participation in the second CCAs. Firms that were deemed non-eligible and those from 14 sectors related to mineralogical and metallurgical process are removed from the sample, the latter due to becoming exempted from the CCL on April 1st, 2014. The dataset used in this study includes 6849 facilities that entered the policy during the treatment year, 2013, and had an active CCA in the post treatment period, 2014. Table 1 shows the median, mean and standard deviation for electricity consumption and employment in five sectors, chosen to be large enough to be non-disclosive, and the total in 2012. The table illustrates the heterogeneity not only across but also within sector, indicated by the large standard deviations and discrepancy between the median and mean, corroborating our case to use the CIC estimator.
5.2 The Control Group: CRC Information Declarers
The control group consists of facilities of information declarers during phase 1 of the Carbon Reduction Commitment (CRC) scheme. The CRC, later renamed to CRC Energy Efficiency Scheme, was introduced on April 1st, 2010, and concluded on March 31st, 2019. The CRC aimed to drive energy efficiency and reduce carbon emissions in large energy users in the public and private sector that did not fall under EU ETS or CCA (EA 2015b). Eligibility to CRC phase 1 was determined by two factors: (1) The organisation had at least one settled half-hourly meter; and (2) total qualifying supplies in 2008 matched the qualification threshold (6000 MWh) or more (DEFRA 2007). Information declarers are firms satisfying only criterion (1), and as such are not affected by the CRC. However, the presence of half-hourly meters is indicative of high electricity usage.Footnote 28 Overall, information declarers are attractive as a control group since they are the largest electricity consumers that have neither been offered participation in the CCAs nor the CRC while being subject to full CCL rates. They offer substantial overlap in the outcome distributions for the CIC due to their sizeFootnote 29 and they are not in the sectors covered by the CCAs, which alleviates concerns about potential spillover effects. The CRC data provided by BEIS contains 11,225 facilities for information declarers.
5.3 Data Matching
Members of the treatment and control groups are matched to electricity meter locations through the Levenshtein ratio, which evaluates how many edit operations are required to change one word to another relative to their lengths (Levenshtein 1966).Footnote 30 To ensure the quality of the data, only facility-meter pairs within a narrow range of postcodes with Levenshtein ratios above 80% are accepted, resulting in an overall 43.21% match rate for the treatment and 48.75% for the control group. Using a similar algorithm, facilities in both groups are matched to local units,Footnote 31 the closest equivalent of facility level data in the Inter-Departmental Business Register (IDBR), to recover employment information. Match rates for employment are 60.39% in the treatment group and 55.54% in the control group.
5.4 Concurrent Policies
The second CCAs did not occur in a legislative vacuum and, as such, important concurrent policies exist. Besides the aforementioned CRC, two other policies are relevant to firms in the treatment and control group: the EU Emissions Trading Scheme (EU ETS) and Energy Savings Opportunity (ESOS). The EU ETS, introduced on January 1st, 2005, is a cap and trade system aiming to reduce \(\text {CO}_2\) emissions from facilities with large combustion equipment. Energy covered by EU ETS is generally not eligible for the CCAs discount. However, under very specific circumstances both policies can apply at a given facility.Footnote 32 ESOS is a mandatory assessment policy that came into force July 1st, 2014, which applies to large UK undertakings and their corporate groups.Footnote 33 Firms are tasked to identify assets and activities that constitute at least 90% of their energy consumption and undertake an energy audit including energy savings suggestions. However, these suggestions are non-binding and as such, improvements to energy efficiency are non-mandatory (EA and BEIS 2014).
To isolate the impact of the CCAs, this paper removes any facility in the treatment group participating in CRC phase 2, approximately 13% of the original sample, and EU ETS phase 3, approximately 2% of the original sample. Overlap with CRC phase 2 was checked using scheme lists provided by BEIS. EU ETS participant data was gathered by web scraping from the websiteFootnote 34 of the European Commission and matched to CCAs participants. Facilities in ESOS are not removed, due to the first compliance period, December 2015, being significantly beyond the treatment year, 2014, and only 0.15% of participating companies’ directors reviewing the recommendations contained in the policy audit in Q4 of 2014 (EA and BEIS 2018) and none before. After cleaning the data of erroneous readings, e.g., error codes, the following summary statistics for the sample used in the study are obtained (Tables 2, 3).
6 Results
Before discussing the results it is important to evaluate the estimable range for each variable and verify the CIC assumptions. As discussed in Sect. 4, the range of identifiable quantiles depends on the overlap in the distributions of the outcome variables between the treatment and control group. Within the sample of this study, electricity consumption is identified up to the 93rd percentile of the treated group and employment is fully identified. Additionally, we also find evidence that the distribution of unobservables, U, is time invariant within groups. We applied the test of Melly and Santangelo (2015) described in Sect. 4 on observations in 2011 and 2012, an interim period between the first and second CCAs that did not impose any policy targets while granting rebates (BEIS 2020). Without targets in place, there are fewer factors affecting facilities’ electricity consumption and employment, motivating our choice to verify estimator assumptions in those years. As the p values for the Kolmogorov–Smirnov test for electricity and employment are 0.12 and 0.06, respectively, we fail to reject the null hypothesis that the treatment and control groups are drawn from the same distribution when no treatment has occurred. Thus, the time invariance assumption seems to be verified in the pre-treatment period for electricity, while not as strongly supported for employment.
Going forward, the pre-treatment year is set to 2012 and the post-treatment year to 2014 to coincide with the end of the first CCAs and the end of the first target period of the second CCAs. To minimise excess description, we will use negative treatment effect as a shorthand to describe those instances when the observed level of the outcome variable in the treated group is smaller than the level in the counterfactual, that is, participation in the CCAs is associated with a smaller level than the counterfactual CCL outcome, whereas a positive treatment effect implies the opposite.
6.1 Electricity Consumption
Table 4 contains the treatment effect (\(\tau\)), standard error (SE) and significance level, outcome for the treated units (\(Y_{11}\)), counterfactual outcome (\(Y^N_{11}\)), and the percentage difference between the treatment and counterfactual of electricity consumption (% diff.) for each decile, q. The final row contains what we call the Identifiable Quantile Mean (IQM), which is the mean of the treatment effect within the identifiable range.
Overall, the identified deciles indicate that the CCAs delivered electricity consumption reductions greater or about equal to the CCL, as most deciles are negative and the majority not statistically significant. This conclusion is supported by the IQM, which shows an average reduction in electricity consumption of − 4.8% significant at the 10% level. Interestingly, the positive treatment effects are located at the lower and upper end of the presented deciles. These findings mirror the qualitative analysis of the CCAs conducted in BEIS (2020), which found that firms at the extremes of energy consumption were less receptive to the incentives provided by the CCAs. For small firms meeting targets in the CCAs may not have been cost-effective, while very large firms were making substantial efforts to improve energy efficiency regardless. Thus, the CCAs appear to be most efficacious in medium sized electricity consumers. Figure 3 contains plots for each quartile of the results.
Each facility participating in the CCAs faces two opposing economic incentives compared to the CCL baseline. On one hand, CCAs participation lowers the cost of electricity by reducing the applicable CCL rate and therefore incentivises higher consumption. On the other hand, CCAs impose energy efficiency targets aimed at making up for the lower financial incentive to reduce energy consumption. Despite the majority of firms (82%) choosing relative targets that do not require a decrease in absolute levels of consumption, the CCAs reduced electricity consumption similar to what would have been delivered by the CCL. Thus, the pressure to increase efficiency included in the targets appears to have delivered a decrease of similar levels for electricity consumption when compared to the CCL. As the aim of CCAs was to mitigate the impact of the CCL on energy intensive industry and to deliver efficiency improvements broadly equivalent to what would be achieved by the application of the full rate of the CCL (DECC 2012), the policy appears to have met its intended objective. These findings contrast with the evaluation of the first CCAs conducted in Martin et al. (2014), which found a statistically significant reduction in electricity consumption delivered by full CCL rates compared to CCAs. They concluded that the combination of targets and discount provided by the first CCAs were insufficient to outweigh the price incentives of the full levy. The same cannot be said for the second CCAs.
An almost 5% average reduction of electricity consumption among a subset of business users harbours another interesting possibility, that is, whether lowered electricity demand could reduce electricity prices.Footnote 35 We investigate this with a back of the envelope exercise by multiplying the IQM with the number of identified facilities and find that the estimated reduction is approximately 0.21 tWh, which is 0.22% of industrial and 0.07% of total electricity consumption in 2014 (BEIS 2022). It therefore appears unlikely that the second CCAs affected the electricity market price, even when considering that not all facilities in the policy were able to be matched.
6.2 Employment
Before discussing the estimates for employment in Table 5 and Fig. 4, it is important to mention that the quantiles of the CIC analysis are with respect to the variable being assessed, so that the facility at the 90th percentile of electricity consumption is not necessarily related to the facility at the same quantile of employment.Footnote 36
CCAs participation appears to have a limited effect on employment. Contrary to expectations, employment in CCAs participating firms is predominantly lower than in the counterfactual across quantiles with the exception occurring at the upper tail-end of the distribution where the impact of the CCAs is positive at some percentiles. The IQM is negative, but, like any other estimate of the treatment effect in Table 5, statistically not significant. The graphs in Fig. 4 support these findings as the counterfactual distribution is mostly to the right of the treated distribution, implying a lower demand for labour under the CCAs compared to what would have been delivered by the full CCL rates. Similar results were found in Martin et al. (2014) for the first CCAs, as full CCL taxation was found to increase demand for labour, although the impact was not statistically significant.
Our results indicate that firms responded differently to the CCAs depending on the level of electricity consumption, while the same is not as evident for employment. To further investigate factors that could affect the treatment response in electricity consumption and employment, we conduct two additional analyses. First, we study the effect of different target types and, second, we choose two sectors for within and across sector analysis.Footnote 37
6.3 Target Type Analysis
As many economic considerations may influence a firm’s choice of target type, such as expected demand or rate of technological change, facilities with different target types are likely to respond to the policy differently. Participants in the second CCAs were allowed to choose between three types of targets: 1. relative, 2. novem, and 3. absolute.Footnote 38 The majority of participants, 82% in target period 1 of the second CCAs, adopted relative targets, while 13% used novem, and 6% absolute targets.Footnote 39 The small number of facilities adopting absolute targetsFootnote 40 discourages the use of the CIC estimator within the sub-group, leading us to combine them with facilities choosing novem targets. While the joint group is still relatively small, the results provide an interesting contrast to the relative target group. Here it is important to reiterate that the percentiles in the tables below are not the same as those found in Table 4. Nevertheless, since relative targets constitute the majority of the sample, the percentiles in the overall and the relative target sample are fairly close. As the CIC requires overlap in the outcome distributions between the control and treatment group, we use the entirety of the CRC information declarer control group and verify the use via the pre-treatment KS-test for each subset of the data.
We also estimated the impact of the CCAs on employment within each sub-group, which closely resembled treatment effects in the overall sample, i.e., the policy resulted in a non-statistically significant reduction in employment. In a few deciles the treatment effect was significant at the 10% level, but due to the low significance and sporadic frequency our prior conclusion on the effect of the policy remains the same. This is particularly interesting from the sector analysis, as facilities in Sector A and B are highly heterogeneous in terms of their electricity consumption scale and capital intensity. Thus, there seems to be little evidence that the CCAs had a heterogeneous impact on employment regardless of the economic reality faced by facilities. For the sake of brevity, we included the estimation tables for employment in “Appendix CCA Employment Impacts by Targets and Sectors”. We do not report results of the 100th percentile due to data security concerns,Footnote 41 however, none of the identified results at the percentile were statistically significant.
6.4 Relative Targets
For relative targets, 2078 and 2097 facilities are in the sample for the pre- and post-treatment periods, respectively.Footnote 42 As the KS-test fails to reject the null hypothesis of identical distributions of the unobservable in the pre-treatment phase with a p value of 0.27, we proceed to the estimation using the CIC. Estimates of the treatment effect, identified up to the 94th percentile, are shown in Table 6 and Fig. 7 in the Appendix.
Our results for the units adopting relative targets very much resemble those of the overall sample. The 20th, 30th, and 90th percentile treatment effects are positive, implying higher electricity consumption under the CCAs than what would have been observed under the full CCL while the opposite occurs in all other cases. However, of the three, only the 20th percentile effect is statistically significant. This percentile, approximately 70,000 kWh, is situated between the 20th, 65,000 kWh, and 30th percentile, 100,000 kWh, in Table 4, corroborating the statistical significance observed in those cases. The 90th percentile in Table 6 is located between the 80th and 90th percentile in Table 4, which explains the small treatment effect reported in Table 6 due to proximity to the crossing-point between negative and positive treatment effects in the whole sample. This overall negative impact is confirmed by the IQM, which shows a reduction of − 5.7% significant at the 5% level.
6.5 Absolute and Novem Targets
Compared to the relative targets, much fewer facilities opted to use the other two target types.Footnote 43 Since the KS-test fails to reject the null hypothesis of similar distributions in the pre-treatment period with a p value of 0.93, we proceed with the estimation. Electricity consumption in the group is relatively long-tailed to the right, causing the maximum identifiable quantile of the treatment effect to be the 86th percentile. In contrast to the total and relative target samples, the facilities choosing absolute or novem targets have decreased their electricity consumption under CCAs compared to what would have been observed under full CCL taxation at all deciles. Standard errors of the estimates in Table 7 are generally quite large, leading to only two percentiles, the 20th and the 80th, having statistically significant treatment effects. Another consequence of the smaller sample size, see Fig. 8 in the Appendix, is that for high levels of electricity consumption both the observed and counterfactual distributions become less smooth. This results in the impact becoming more volatile at higher levels of electricity, confirming results observed throughout this study. Overall, based on the impact at the deciles and the value of the IQM, it appears that targets in the second CCAs for absolute and novem facilities were sufficiently stringent to lead to a decrease or no significant difference in energy consumption when compared to the CCL.
6.6 Sector Analysis
We now focus our attention on the impact of the policies on two CCAs sectors. Since the targets were negotiated for each sector by trade associations it is possible that target stringency differs across sectors. Additionally, electricity consumption within a CCAs sector spans a smaller range than in the whole sample, therefore allowing a more granular investigation into the impact of the policy. We study the treatment effect in two CCAs sectors that cover different ranges of the outcome distribution. To ensure data security for the CCAs participants, we opted to anonymise the names of the sectors. Sector A consists mostly of small-scale consumers, whose electricity consumption is below the median of the total sample, and few large-scale consumers. In comparison, facilities in Sector B are medium to large-scale consumers with their 10th percentile situated close to the median of the whole distribution. Therefore, our analysis illustrates the effect of the policy in different sectors but also across levels of electricity consumption.
Outcomes between the two sectors are markedly different, as the treatment effect in Sector A is mostly non-statistically significant, whereas firms in Sector B consistently decreased their electricity consumption compared to their counterfactual. Overall, the sector results corroborate the findings of increases in electricity consumption for facilities around the 20th to 30th percentile of the overall distribution and decreases for larger facilities.
6.7 Sector A
Sector A has a high number of treated units in the sample with 586 and 589 facilities in the pre- and post-treatment periods. Since the KS-test fails to reject the null hypothesis of similar distributions of the unobservable in the pre-treatment phase with a p value of 0.22, we proceed with the estimation. When looking at Table 8 there appears to be limited heterogeneity in the treatment response, since the effect is mostly positive with two exceptions at the 10th and 90th percentile of the distribution, neither of which is statistically significant. Two out of three statistically significant deciles, the 20th and 50th, largely coincide with the 20th and 30th percentiles in the total distribution (Table 4). The remaining statistically significant decile is the top-end of Sector A at the 100th percentile, which lies between the 70th and 80th percentiles in the overall distribution (4). Thus, from the results in Table 8 there seems to be some evidence that the CCAs increased electricity consumption in Sector A, but the average impact is rather small at a non-statistically significant 0.93% increase. This seems to be mainly driven by large facilities above the 95th percentile but below the 100th percentile, which show a negative treatment effect; due to the very long-tailed nature of electricity consumption in the sector, the larger facilities substantially impact the IQM. Thus, Sector A showcases the strength of the CIC estimator over more conventional methods, as the small average impact would lead to the conclusion that the effect of the CCAs is a minor increase in electricity consumption over the CCL in this sector, while a number of medium to large electricity consumers, relative to the overall sample, have decreased their consumption. Figure 5 corroborates these findings, as the counterfactual consumption is predominantly lower than the observed level except at the upper end of the distribution, where the treatment effect is more variable.
6.8 Sector B
Sector B offers a similarly large sample size with 554 and 559 facilities in the pre- and the post-treatment period, respectively. As the KS-test fails to reject the null hypothesis of similar distributions in the pre-treatment period with a p value of 0.39, we continue with the estimation. Facilities in the sector are medium- to large-scale electricity consumers and their treatment effects differ significantly from the impact in Sector A. First and foremost, all deciles show a negative treatment effect, implying lower levels of electricity consumption than what would have been observed in the full CCL counterfactual. The effect, which largely lies between the 40th and the 80th percentile in Table 4, is significant in 7 out of the 9 deciles studied. In percentage terms, the treatment effect is quite stable, perhaps a reflection that facilities are homogeneous in their production function regardless of size. In line with the decile treatment effects, the IQM in the sector is significant at the 1% level and indicates an approximately − 14% decrease in electricity consumption attributable to the CCAs. The graphs in Fig. 9 support these findings as the counterfactual consumption is to the right of the treated consumption by a large margin. One would therefore conclude that there seems to be strong evidence of the CCAs reducing electricity consumption in Sector B when compared to the alternative of full CCL (Fig. 6, Table 9).
7 Conclusion
In this paper, we contribute to the academic discussion on voluntary and negotiated agreements by conducting a quasi-experimental evaluation of the second Climate Change Agreements. The policy offers a discount on the Climate Change Levy, a tax on energy consumed by most non-domestic, i.e., non-household, users in exchange for energy efficiency improvements. Our work contributes to the existing literature from three points of view. First, we analyse the impact of the so-far unstudied second CCAs on electricity consumption and employment, finding evidence towards the efficacy of negotiated agreements. Second, we exploit facility level data,Footnote 44 while currently available studies of the first CCAs are restricted to plant level data, which can include multiple facilities grouped together. Third, we apply the novel changes-in-changes estimator developed in Athey and Imbens (2006), which, in contrast to the more common differences-in-differences method, allows for the evaluation of the policy impact across the quantiles of the distribution of outcomes. Therefore, we explicitly account for heterogeneity at different levels of the variable of interest.
Outcomes of the second CCAs provide a marked contrast to the first. The first CCAs were criticised for insufficiently stringent targets that lead to worse policy outcomes when compared to full taxation under the CCL; estimates in Martin et al. (2014) attribute a − 22.6% reduction in electricity consumption to the CCL versus the first CCAs. This shortcoming seems to have been addressed via changes made in the second CCAs, which have largely delivered, as intended, consumption reductions at least equivalent to what would have been achieved by the application of the full CCL rates. Electricity consumption tends to be lower than the counterfactual in six of nine deciles with the reduction ranging from − 1.5 to − 9.3%, while it was higher in the remaining three deciles, ranging from a 9.3 to 12.5% increase. This results in an average reduction of − 4.8%. On the other hand, our findings on employment effects mirror those for the first CCAs in Martin et al. (2014), as the impact is not statistically significant and negative with an average − 4.6% decrease among the identified treatment effects.
Our analysis uncovered a tendency of the CCAs to deliver increased electricity consumption for small consumers compared to a full CCL counterfactual. The opposite tends to occur in the case of large consumers. Target types seem to have some influence on this pattern. 82% of the sample adopted relative targets, which follow the overarching trends established for the entirety of the sample. However, facilities choosing the other two targets, novem and absolute, consistently decrease their electricity consumption compared to the counterfactual. The pattern above is also related to the impact in specific CCAs sectors. We focus on two large sectors that occupy different portions of the electricity consumption distribution. On one hand, Sector A consists primarily of small-scale electricity consumers, which tend to increase electricity consumption under the CCAs compared to the full CCL. Only some very large facilities in the sector reduced their electricity consumption, skewing the average impact within the group downward to a statistically non-significant 0.93% increase. On the other hand, facilities in Sector B tend to be medium- to large-scale consumers that consistently decreased electricity consumption in the CCAs compared to the CCL by on average − 14.9%.
This study delivers important evidence that the second CCAs, an example of negotiated agreements, have delivered reduction in electricity consumption largely comparable to what one would have obtained if the CCL, an energy tax, was applied at the full rate. Our work shows that consumption behaviour was systematically different between smaller and larger electricity consumers participating in the CCAs, which is further supported by the impact in two sectors with large numbers of facilities. As is the case in many empirical studies, results could perhaps be more insightful by improving the quality of data. Our facility level consumption data is novel, but having to match different datasets to obtain the sample used in this study is not ideal, due to the compounding impact that failure to match across the datasets has on available sample size. Requiring scheme participants to disclose associated meter and company reference numbers could substantially improve the analysis by making data matching obsolete. Without heavy sample attrition, integrating the effects of other covariatesFootnote 45 into the distributional analysis would be valuable to investigate how different economic factors contribute to the heterogeneity in treatment and might uncover interesting economic mechanisms. Possible variables of interest include investments, existing levels of capital, R &D expenditure, employment data by skill level/education, whether electricity is purchased centrally or facility by facility, characteristics of the demand for the main good produced, the market the firms operate in, strength of sector associations, and levels of international competition. Lastly, it would have also been interesting to assess the impact of the CCAs not only in terms of energy consumption, but also in terms of their impact on energy expenditure by studying how reductions in consumption interact with the effective electricity price paid by firms.
Notes
See Bassi et al. (2013) for a thorough discussion of the policy landscape in the United Kingdom.
The three types of targets (relative, novem, and absolute) are discussed in detail in Sect. 2.
See Appendix “Section Hypothetical Impact of CCAs on CRC Information Declarers”.
For data security reasons we are unable to disclose the sector names.
Only a handful of energy uses are exempt from CCL, such as energy used to propel a train for freight or passenger transport (HMRC 2015).
See Varma (2003) for a discussion of the concern regarding the introduction of the CCL.
Umbrella agreements set eligible processes and energy consumption target for the entire sector and are informed by a host of performance and market criteria, such as expected growth rates, technological advancements, market structure, and so forth.
A target unit satisfies its novem target if the base year production basket evaluated at current energy efficiency is sufficiently lower than total energy consumption in the base year (EA 2018). By evaluating policy performance using base year production baskets, novem targets avoid penalising increases in energy usage due to changing market demand. As an illustrative example, an abattoir produces 10 tons of frozen and and 10 tons of fresh meats in the base year at 1 energy units per ton (eu/t) and 10 eu/t, respectively, so that total base year energy consumption is 110 eu. In the policy year, the facility produces 5 tons of fresh and 15 tons of frozen meats at 1.2 eu/t and 8 eu/t so that total energy consumption is 126 eu, implying that energy consumption has increased compared to the base year. However, when evaluating the consumption in the policy year at base year production mix, total energy consumption is 1.2 eu/t x 10t + 8 eu/t x 10t = 92 eu. This shows that overall energy efficiency increased and that the higher energy consumption is due to a shift in production basket.
For the period ending in 2014, scheme performance data provided by BEIS showed that only 0.45% of facilities used over-accrued performance to cover some or the entirety of their shortfall.
This mechanism was introduced during the second target period of the first CCAs (AEAT 2005). The cost per ton of \(\text {CO}_2\) was \(\pounds\)12 in 2014.
Sectors are allowed to join the CCAs by satisfying either of two criteria. First, if their energy intensity, defined as energy expenditure divided by turnover, is above 10%. Second, if their energy intensity is above 3% and their sector trade intensity, defined as imports/(net imports + domestic supply) is above 50% (DECC 2008).
It is difficult to conclusively determine which CCAs had tighter targets, as they were informed by market conditions at the time and negotiated. However, removing the option to choose base year should, ceteris paribus, lead to equal, if 2008 was the optimal choice to minimise compliance cost, or tighter targets, if any other year was best.
See DEFRA (2006) for prices from 2002 to 2006 in the UK ETS.
DEFRA could deny access to the CCAs rebate to specific non-compliant target units when overall sector targets were not met. However, it was possible for firms to shirk and still enjoy the benefits of the rebate as long as other firms abated sufficiently.
See Bailey and Rupp (2006) for an overview of the role sector bodies played throughout the implementation of the first CCAs through stakeholder surveys.
The data for the study contains manufacturing firms in the UK and is a combination of the Quarterly Fuels Inquiry, Annual Respondents Database, and Quarterly Capital Expenditure Survey.
The dataset used in the study is obtained by matching the Annual Respondents Database, the Quarterly Fuels Inquiry, CCAs participant data, and European Pollution Register. In approximately 30% of the observations plants consist of multiple facilities (local units in the IDBR). Grouping multiple facilities into a single reporting unit is done in order to limit data reporting overheads for large firms.
It is noteworthy here that an observation does not need to have the same value of U over time. The only requirement is that U, regardless of which observation contains that U, remains in the same rank pre- and post-treatment within the group.
This step shows the DID heritage of the method, as the time effect is established from the behaviour of the control group.
Variance estimation for the quantile effects can be conducted using the consistent estimators proposed in Athey and Imbens (2006).
Half-hourly meters are mandatory when average peak electricity consumption over the 3 highest months is above 100 kW over a 12 month period; they can also be voluntarily installed (EA 2013).
It is noteworthy here that the CIC estimator does not rely on similarity of specific moments between the treatment and control group, but rather the overlap of the outcome variables as discussed in detail in Sect. 4.
Defined in Council of the European Union (1993) as ”The local unit is an enterprise or part thereof (e. g. a workshop, factory, warehouse, office, mine or depot) situated in a geographically identified place. At or from this place economic activity is carried out for which—save for certain exceptions - one or more persons work (even if only part-time) for one and the same enterprise.”
If 70% or more of energy used in a facility is consumed in processes eligible for CCAs, all of the site’s energy consumption becomes eligible for CCAs, some of which could simultaneously be covered by the EU ETS. In this scenario, a production process covered by EU ETS can also claim the CCAs discount. See EA (2018) for more detail.
From EA and BEIS (2014): ”A large undertaking is any UK company that either employs at least 250 people, or has an annual turnover in excess of 50 million euro (\(\pounds\)44,845,000), and an annual balance sheet total in excess of 43 million euro (\(\pounds\)38,566,700), or an overseas company with a UK registered establishment which has 250 or more UK employees (paying income tax in the UK)”
https://ec.europa.eu/clima/ets/nap.do?languageCode=en &nap.registryCodeArray=GB &periodCode=-1 &search=Search ¤tSortSettings=
The electricity wholesale market in Great Britain at the time of the study was governed by the British Electricity Trading and Transmission Arrangements (BETTA), which unified electricity markets between England, Wales, and Scotland in 2005. This singular market structure allows consumers and generators in the three countries to competitively buy and sell electricity, which determines the national wholesale price. A more detailed discussion of the UK electricity markets can be found in Liu et al. (2022).
We investigated the possibility of there being a clear relationship between the levels of employment and electricity consumption through a fixed effects model regressing employment on electricity consumption, but coefficients were non-statistically significant and small.
We also conduct a counterfactual analysis investigating what if the CCAs were applied to facilities under the CCL instead in Appendix “Hypothetical Impact of CCAs on CRC Information Declarers”, that is we evaluate the treatment effect on the untreated.
Refer to section 2 for a detailed description of each target type.
This allocation of target types is not surprising. Relative and novem targets are less affected by the impact of fluctuations in demand on electricity consumption compared to absolute targets. When demand for outputs increases, absolute targets become restrictive, whereas decreases in demand make them more lenient.
In the sample, only 49 observations use absolute targets in the pre- and post-treatment period.
Employment data is more readily available when compared to electricity readings, which could be used to identify sectors.
The sample size can differ between the pre- and post-treatment period when meter readings for a matched site were not available for both years.
Our sample contains 49 facilities with absolute and 125 with novem targets in the pre-treatment period. Facilities with absolute and novem targets in the post-treatment period are 49 and 131, respectively.
Access to facility level electricity consumption and employment data used in the study was provided by the Department for Business, Energy, and Industrial Strategy and matched to the treated and control group using string-matching methods.
Abbreviations
- BEIS:
-
Department for Business, Energy & Industrial Strategy
- CCAs:
-
Climate Change Agreements
- CCL:
-
Climate Change Levy
- CIC:
-
Changes-in-Changes
- CRC:
-
Carbon Reduction Commitment
- EA:
-
Environment Agency
- EGO:
-
Energy share in gross output
- EPA:
-
Environmental Protection Agency
- ESOS:
-
Energy Savings Opportunity
- EU ETS:
-
European Union Emissions Trading System
- IDBR:
-
Inter-Deparmental Business Register
- IPPC:
-
Integrated Pollution Prevention and Control
- IQM:
-
Identifiable Quantile Mean
- KS-test:
-
Kolmogorov–Smirnov test
- NAs:
-
Negotiated Agreements
- VAs:
-
Voluntary Agreemetns
- WML:
-
Waste Management Licensing
References
Adetutu MO, Stathopoulou E (2020) Information asymmetry in voluntary environmental agreements: theory and evidence from UK climate change agreements. Oxford Economic Papers
AEAT (2004) Climate change agreements—results of the first target period assessment. Technical Report 01838/1, AEA Technology
AEAT (2005) Climate change agreements—results of the second target period assessment. Technical Report AEAT/ENV/R/2025, AEA Technology
Athey S, Imbens GW (2006) Identification and inference in nonlinear difference-in-differences models. Econometrica 74(2):431–497
Bailey I, Rupp S (2006) The evolving role of trade associations in negotiated environmental agreements: the case of United Kingdom climate change agreements. Bus Strategy Environ 15(1):40–54
Bassi S, Dechezlepretre A, Frankhauser S (2013) Climate change policies and the uk business sector: overview, impacts and suggestions for reform. Technical report, Centre for Climate Change Economics and Policy
BEIS (2020) Evaluation of the second climate change agreements scheme—synthesis report. Technical Report 2020/014, Department for Business, Energy, and Industrial Strategy
BEIS (2022) Historical electricity data: 1920 to 2021. Technical report, Department for Business, Energy & Industrial Strategy. https://www.gov.uk/government/statistical-data-sets/historical-electricity-data. Accessed 25 Dec 2022
Bellassen V, Shishlov I (2016) Pricing monitoring uncertainty in climate policy. Environ Resource Econ 68:949–974
Bi X, Khanna M (2012) Reassessment of the impact of the EPA’s voluntary 33/50 program on toxic releases. Land Econ 88(2):341–361
Bjørner TB, Jensen HH (2002) Energy taxes, voluntary agreements and investment subsidies-a micro-panel analysis of the effect on danish industrial companies’ energy demand. Resource Energy Econ 24:229–249
Cornelis E (2019) History and prospect of voluntary agreements on industrial energy efficiency in Europe. Energy Policy 132:567–582
Council of the European Union (1993) Council regulation (EEC) no 696/93 of 15 march 1993 on the statistical units for the observation and analysis of the production system in the community. Technical Report 696/93, Organization for Security and Co-operation in Europe
Croci E (2005) The handbook of environmental voluntary agreements: design, implementation and evaluation issues. Springer, Dordrecht
Dawson NL, Segerson K (2008) Voluntary agreements with industries: participation incentives with industry-wide targets. Land Econ 84(1):97–114
de Vries FP, Nentjes A, Odam N (2012) Voluntary environmental agreements: lessons on effectiveness, efficiency and spillover potential. Int Rev Environ Resource Econ 6(2):119–152
DECC (2008) Energy intensive eligibility criteria - guidance for sector associations and participants. Technical Report CCA-B02, Department of Energy and Climate Change. https://webarchive.nationalarchives.gov.uk/20121217175626/http://www.decc.gov.uk/assets/decc/what%20we%20do/global%20climate%20change%20and%20energy/tackling%20climate%20change/ccas/ccas_guidance/cca-b02.pdf. Accessed 16 Feb 2021
DECC (2012) Proposals on the future of climate change agreements. Technical Report DECC0040, Department of Energy and Climate Change
DEFRA (2006) Appraisal of years 1–4 of the UK emissions trading scheme. Technical report, Department for Environment, Food and Rural Affairs. https://webarchive.nationalarchives.gov.uk/ukgwa/20130822084033/http://www.defra.gov.uk/Environment/climatechange/trading/uk/pdf/ukets1-4yr-appraisal.pdf. Accessed 23 May 2022
DEFRA (2007) Consultation on the recommendations of the climate change simplification project. Technical report, Department for Environment, Food and Rural Affairs. https://webarchive.nationalarchives.gov.uk/20080520180141/http://www.defra.gov.uk/corporate/consult/cc-instruments/index.html. Accessed 16 Feb 2021
DETR (2000) Climate change—the uk programme. Technical Report Cm 4913, Department of the Environment, Transport and the Regions. https://web.archive.org/web/20060512120000/. http://www.defra.gov.uk/environment/climatechange/uk/ukccp/2000/pdf/section1.pdf. Accessed 16 Feb 2021
Dijkstra BR, Rübbelke DT (2013) Group rewards and individual sanctions in environmental policy. Resource Energy Econ 35(1):38–59
EA (2013) Crc energy efficiency scheme guidance for participants in phase 1 (2010-2011 to 2013-2014). Technical report, Environment Agency. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/298073/LIT_6794_83c7b8.pdf. Accessed 16 Feb 2021
EA (2014) Managing your climate change agreement (CCA). Technical report, Environment Agency. https://www.gov.uk/guidance/managing-your-climate-change-agreement-cca. Accessed 16 Feb 2021
EA (2015a) Climate change agreements: biennial progress report 2013 and 2014. Technical report, Environment Agency. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/479679/Biennial_progress_report_2013_and_2014.pdf. Accessed 18 May 2022
EA (2015b) CRC energy efficiency scheme guidance for participants in phase 2(2014-2015 to 2018-2019)
EA (2017) Climate change agreements: biennial progress report 2015 and 2016. Technical report, Environment Agency. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/661666/Biennial_progress_report_2015_and_2016.pdf. Accessed 16 Feb 2022
EA (2018) Climate change agreements operations manual version 8. Technical report, Environment Agency. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/772721/Climate_Change_Agreements_Operations_Manual.pdf. Accessed 16 Feb 2021
EA and BEIS (2014) Guidance—ESOS. Technical report, Environment Agency and Department for Business, Energy, and Strategy. https://www.gov.uk/guidance/energy-savings-opportunity-scheme-esos. Accessed 16 Feb 2021
EA and BEIS (2018) Energy savings opportunity scheme 2018 publication final. Technical report, Environment Agency and Department for Business, Energy, and Strategy. https://environment.data.gov.uk/portalstg/home/item.html?id=5393998a108f42fdafd90340039cbce9. Accessed 4 May 2022
Ekins P, Etheridge B (2006) The environmental and economic impacts of the UK climate change agreements. Energy Policy 34(15):2071–2086
Fleckinger P, Glachant M (2011) Negotiating a voluntary agreement when firms self-regulate. J Environ Econ Manag 62:41–52
Gamper-Rabindran S (2006) Did the EPA’s voluntary industrial toxics program reduce emissions? a GIS analysis of distributional impacts and by-media analysis of substitution. J Environ Econ Manag 52(1):391–410
Goodin RE (1986) The principle of voluntary agreement. Public Adm 64(4):435–444
HMRC (2015) Excise notice ccl1/3: Climate change levy—reliefs and special treatments for taxable commodities. Technical report, HM Revenues & Customs. https://www.gov.uk/government/publications/excise-notice-ccl13-climate-change-levy-reliefs-and-special-treatments-for-taxable-commodities/excise-notice-ccl13-climate-change-levy-reliefs-and-special-treatments-for-taxable-commodities. Accessed 16 Feb 2021
Hoang PC, McGuire W, Prakash A (2018) Reducing toxic chemical pollution in response to multiple information signals: the 33/50 voluntary program and toxicity disclosures. Ecol Econ 146:193–202
Innes R, Sam AG (2008) Voluntary pollution reductions and the enforcement of environmental law: an empirical study of the 33/50 program. J Law Econ 51(2):271–296
Khanna M (2001) Non-mandatory approaches to environmental protection. J Econ Surv 15(3):291–324
Khanna M, Damon LA (1999) EPA’s voluntary 33/50 program: impact on toxic releases and economic performance of firms. J Environ Econ Manag 37:1–25
Kolmogorov A (1933) Sulla determinazione empirica di una legge di distribuzione. Giornale dell’Instituto Italiano degli Attuar 4:83–91
Levenshtein V (1966) Binary codes capable of correcting deletions, insertions and reversals. Soviet Phys Doklady 10:707
Liu J, Wang J, Cardinal J (2022) Evolution and reform of UK electricity market. Renew Sustain Energy Rev 161:112317
Martin R, de Preux LB, Wagner UJ (2014) The impact of a carbon tax on manufacturing: evidence from microdata. J Public Econ 117:1–14
Melly B, Santangelo G (2015) The changes-in-changes model with covariates—preliminary. Unpublished
Montero JP (2000) Optimal design of a phase-in emissions trading program. J Public Econ 75:273–291
Navarro G (2001) A guided tour to approximate string matching. ACM Comput Surv 33(1):31–88
Pearce D (2006) The political economy of an energy tax: the United Kingdom’s climate change levy. Energy Econ 28:149–158
Rezessy S, Bertoldi P (2011) Voluntary agreements in the field of energy efficiency and emission reduction: review and analysis of experiences in the European union. Energy Policy 39(11):7121–7129. Asian Energy Security
Richard S, L’Ecuyer P (2011) Computing the two-sided Kolmogorov–Smirnov distribution. J Stat Softw
Segerson K (2013) Voluntary approaches to environmental protection and resource management. Annu Rev Resource Econ 5:161–180
Segerson K (2018) Voluntary pollution control under threat of regulation. Int Rev Environ Resource Econ 11(2):145–192
Segerson K, Miceli TJ (1998) Voluntary environmental agreements: good or bad news for environmental protection? J Environ Econ Manag 36:109–130
Smirnov N (1939) Estimate of deviation between empirical distribution functions in two independent samples. Bull Moscow Univ 2:3–16
Varma A (2003) UK’s climate change levy: cost effectiveness, competitiveness and environmental impacts. Energy Policy 31:51–61
Vidovic M, Khanna N (2012) Is voluntary pollution abatement in the absence of a carrot or stick effective? evidence from facility participation in the epa’s 33/50 program. Environ Resource Econ 52(3):369–393
Wagner RA, Fischer MJ (1974) The string-to-string correction problem. J ACM 21(1):168–173
Zhou R, Bi X, Segerson K (2020) Evaluating voluntary environmental programs with spillover effects. J Assoc Environ Resource Econom 7(1):145–180
Acknowledgements
We would like to thank Mary Anderson (CAG Consultants), Laura Bell, Laura Edwards (both from the Department for Business, Energy and Industrial Strategy—BEIS), and the three anonymous reviewers for their thoughtful and valuable contributions towards this paper. Kentaro Mayr is grateful for the funding provided by the EPSRC (EP/R513143/1). Paolo Agnolucci would like to acknowledge funding from the UK Energy Research Centre (EP/S029575/1).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
1.1 CCA Employment Impacts by Targets and Sectors
The subsequent tables contain the impact of the CCAs on employment for the target type and sector analysis conducted in Sects. 6.3 and 6.6, respectively. Tables contain information on each decile (q), estimates of the impact measured in number of employees (\(\tau\)), the standard error (SE), headcounts under treatment (\(Y_{11}\)), headcounts under the counterfactual (\(Y^N_{11}\)), and percentage difference between the two cases (% diff.). The results within the four sub-groups studied here did not significantly diverge from the employment results found in Sect. 6.2, leading us to include them in the appendix (Figs. 7, 8).
1.2 Relative Targets
For the relative target group the entire counterfactual distribution is estimable and the p value of the pre-treatment KS-test is 0.06. We continue the estimation with the caveat that model assumptions in this subgroup are not satisfied at the highest (10%) level. Outcomes for the group in Table 10 only show minor differences to the overall results shown in Table 5, that is the treatment effect is predominantly negative and statistically non-significant. Only two deciles do not follow this trend, the 30th and 70th, which are negative and significant at the 10% level. As the results mostly mirror the outcomes in the main estimation, they seem to confirm that participation in the CCAs did not affect employment differently than the CCL.
1.3 Novem and Absolute Targets
The entire distribution of the counterfactual is estimable and the p value of the pre-treatment KS-test is 0.99 in the novem and absolute targets group. We thus proceed with the estimation for which the results can be found in Table 11. The impact of the CCAs in the group strongly resemble the overall outcomes as none of the estimated treatment effects are significant. A key difference is that at three deciles of the distribution, the 10th, 20th, and 60th percentile, the treatment effect is positive while it was negative in similar sized facilities in the overall distribution (Table 5). However, as the results are not statistically significant the findings corroborate that the CCAs did not lead to different employment outcomes when compared to the CCL.
1.4 Sector A
In Sector A the entire counterfactual distribution is estimable and the p value for the pre-treatment KS-test is 0.99, leading us to proceed with the estimation. Matching outcomes of the overall sample found in Tables 5, 12 shows that the treatment effect in Sector A is non-statistically significant and negative across the deciles studied. This result is interesting as Sector A covers a wide range of small- to large-scale electricity users, further indicating that the CCAs did not significantly affect employment when compared to the CCL.
1.5 Sector B
Lastly, for Sector B, the counterfactual distribution is identified from the 1st to the 99th percentile and the p value of the pre-treatment KS-test is 0.12, allowing us to proceed with the estimation. Outcomes found in Table 13, strongly resemble the results of the overall distribution in Table 5, as the majority of results are negative and statistically non-significant. Only the 60th decile is negative and significant at the 10% percent level. In contrast to Sector A, Sector B consists of a more homogeneous group of medium- to large-scale electricity users. Thus, both sectors appear to confirm that, overall, the CCAs did not affect employment significantly when compared to the CCL even in light of different sectoral features.
1.6 Hypothetical Impact of CCAs on CRC Information Declarers
While the CCAs were designed as an alternative to the CCL with the objective of encouraging similar performance without increasing the tax burden of energy intensive firms, it is interesting to see how firms with lower energy intensity would have fared in the policy. Therefore, as an additional robustness check, we estimate the treatment effect on the original control group, the CRC information declarers, for both electricity and employment. In this case, the CCAs participants become the control group and CRC information declarers paying full CCL rates are now the treatment group. Therefore, the counterfactual in this section is the hypothetical outcome if the CRC information declarers had received the CCAs instead. Since the treatment effect is estimated as the observed value of the treated minus the counterfactual and the roles have reversed, so has the interpretation of results. A positive treatment effect implies that the counterfactual, i.e., the hypothetical CCAs outcome, is smaller than the observed CCL outcome of CRC Information Declarers. Overall, this analysis confirms our conclusion that the CCAs performed similarly to the CCL with regard to electricity consumption and employment.
1.7 Electricity Impact
The p value on the pre-treatment KS-test is 0.23, thus giving us confidence in the time invariance assumption, and the entire distribution of electricity consumption is estimable. Results in Table 14 confirm the initial findings of the CCAs performing about as well as the CCL in reducing electricity consumption, as most treatment effects are not statistically significant. The only exception here is the 10th percentile showing a higher electricity consumption under the CCL than the CCAs, which is significant at the 1% level. In the case of the distribution of CCAs firms (see Table 4), electricity consumers of a similar size also showed consumption higher under the CCL compared to the CCAs. This provides some evidence that very small electricity consumers, at approximately the 10th percentile of the overall distribution, may react more strongly to efficiency targets than price incentives under the CCL.
1.8 Employment Impact
The distribution of employment is identified from the 1st to the 99th percentile and the p value on the pre-treatment KS-test is 0.99. Results for this scenario shown in Table 15 confirm that the CCAs do not appear to have an overall statistically significant impact on employment. One exception to this occurs at the 90th percentile, at which employment is reduced by 7.61% under the CCAs significant at the 5% level. The treatment effect of similar sized firms in the original estimation (see Table 5) showcase an effect of similar magnitude, however, the results were not significant.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Mayr, K.F., Agnolucci, P. Heterogeneous Impacts in Voluntary Agreements: A Changes-in-Changes Approach to the UK Climate Change Agreements. Environ Resource Econ 86, 345–379 (2023). https://doi.org/10.1007/s10640-023-00797-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10640-023-00797-z
Keywords
- Voluntary agreements
- Environmental tax
- Environmental subsidies
- Energy
- Changes-in-changes
- Climate Change Agreements