Introduction

A major policy target for many developed countries is to reduce energy demand in every sector of the economy. Particularly, it is envisaged that lower consumption levels in buildings through increased energy efficiency ease dependence on energy imports and improve the trade balance of energy-importing countries (Umbach 2010). Lower energy demand is also recognised to be a major linchpin in the effort to minimise environmental impacts such as climate change associated with energy use and exhaustion of energy resources. For these reasons, policymakers across developed economies have gradually shifted their attention to energy efficiency efforts. In the UK, energy consumption used for space heating contributes to approximately 26 % of final energy consumption and 25 % of UK’s greenhouse gas emissions (IEA 2012).

The energy-saving obligation has become the principal policy for reducing heating energy demand in UK homes. This policy instrument was introduced in the mid-1990s, around the time of energy market liberalisation and has evolved to become the second most important climate policy mechanism after the EU Emissions Trading Scheme (Rosenow 2012). The saving targets are set for the major energy suppliers by the Department of Energy and Climate Change (DECC) and are administered and enforced by the energy regulator, the Office of Gas and Electricity Markets (OFGEM). The targets are then achieved at the customer end of operations. The focus has therefore been on substantial energy efficiency measures in buildings such as cavity wall insulation, loft insulation and installation of condensing boiler. In the past few years, these measures were predominately delivered via major refurbishment schemes for the existing dwelling stock, notably the most recent version of the energy suppliers’ obligations, the Carbon Emission Reduction Target (CERT) and the Community Energy Saving Programme (CESP). Both schemes were running between 2008 and 2012. Amongst many other targets, the CERT’s foremost objective was to legally oblige energy companies to cut down carbon emissions by 293 million lifetime tonnes through energy efficiency measures in domestic dwellings across the UK. Professional loft insulation was the most common retrofit measure installed in homes as part of this scheme, with approximately 3.9 million dwellings undergoing installations (Office of Gas and Electricity Markets (OFGEM) 2013). Privately installed loft insulation was second on the list of measures in the scheme, with nearly 2.8 million homes undergoing such retrofit. Cavity wall insulation was the third most common measure installed, with over 2.5 million properties benefitting (OFGEM 2013). A large number of the properties targeted also installed a new efficient boiler via the scheme. All these figures exclude carryovers from CERT’s predecessors, the Energy Efficiency Commitment (EEC) and the Energy Efficiency Standard of Performance (EESoP). The CERT came to an end in December 2012 with energy companies having achieved 296.9 million lifetime tonnes of carbon dioxide savings. In addition to CERT, during the same period, the CESP was running. This complementary scheme required gas and electricity suppliers to reduce carbon emission by approximately 19 million lifetime tonnes of carbon dioxide through area-based schemes. This was to primarily be achieved by delivering energy efficiency measures to households in targeted low-income neighbourhoods defined by the Index of Multiple Deprivation in the UK (OFGEM 2013). CESP came to an end in December 2012 with energy companies having achieved a reduction of 16.3 million lifetime tonnes of carbon dioxide, almost 85 % of the overall target. Yet, the above cited emission reductions achieved by CERT and CESP are defined in terms of lifetime savings achieved by the measures promoted via the obligations and do not necessarily entail reductions in final energy use. Along with this measurement challenge, the energy-saving obligations under CERT and CESP were also based on self-reporting as OFGEM were unable to check each individual case due to the administrative challenges of such verification efforts. This gives room for inaccurate self-reporting, although the potential fine of up to 10 % of global turnover may have deterred such conduct to a certain extent (Rosenow 2012).

The present study analyses the extent to which household gas and total energy consumptions are reduced by installing energy efficiency measures through the CERT and/or the CESP. Natural gas is the most widely used heating fuel in the UK, and it is also the cheapest option available to households. Although not all UK homes are on the gas network, all dwellings in the sample are connected to the gas network and have access to grid gas. Likewise, a large majority of UK’s households use a combination of gas and electricity to heat their homes. These two main sources of domestic energy use covering space heating, lighting and hot water are, therefore, examined in this paper. Electricity consumption alone is not analysed here due to its sensitivity to usage of individual electrical appliances. Furthermore, we focus on the impact of the three main measures mainly delivered by CERT and CESP, cavity wall insulation, loft insulation and condensing boiler. Empirical estimations are made by comparing the observed changes in gas and total energy consumptions of pre- and post-installation for a sample of properties undergoing efficiency measure(s) relative to the changes experienced by a comparable control group of properties with no record of improvements under the CERT and CESP. This analysis provides one of the first empirical estimations of the link between energy efficiency efforts and actual metered energy consumption in the UK’s residential housing market. The key data source used in this study is the NEED. NEED is a government-administered data framework enabling researchers to understand and explain energy consumption in buildings and to identify potential energy efficiency improvements from a national targeting perspective.

Background

Research on the impact of energy efficiency measures on energy consumption is largely ex ante and prospective rather than based on actually measured ex post consumption (see for example, Siller et al. 2007; Cheng and Steemers 2011; Skea 2012; Nord and Sjøthun 2014). One possible complication with ex ante analyses is that they often overestimate the true impact of energy efficiency improvements as circumstances of individual installations may vary and other factors including households’ characteristics and their behaviour may change. Comparability of the result is also limited due to large variation in approaches, parameter choices and definitions of both the dependent and independent variables (see Sorrell et al. 2009 for a more in-depth discussion). Similarly, some studies use simple before-after comparisons to evaluate the cost-effectiveness of energy efficiency measures. For instance, Cohen et al. (1991) found loft insulation and wall insulation to be cost-effective, with annual consumption savings ranging between 12 and 21 %. However, in the absence of a comparison group or controls for confounding variables, counterfactual analysis is not undertaken in these studies and results are prone to bias.

An alternative approach to estimating future energy savings is through an econometric analysis of the ex post effectiveness of energy efficiency measures. Energy billing or metre reading data and a number of dwelling and household characteristics are required to study the drivers of residential energy consumption (Ma et al. 2012; Swan and Ugursal 2009). One of the first ex post analyses investigating the relationship between energy efficiency retrofits and energy consumption was conducted by Hong et al. (2006), which examined the impact of cavity wall insulation, loft insulation and energy efficient heating system installed on domestic space heating fuel consumption. By monitoring a sample of English dwellings before and after installation of energy efficiency measures, the authors report that cavity wall and loft insulation reduce heating demand by 10 % in centrally heated properties and 17 % in non-centrally heated dwellings. This is supported by Scheer et al. (2013) evaluating the energy savings realised by households participating in a government-sponsored residential retrofit scheme in Ireland. Their study uses an ex post billing analysis to examine the change in gas consumption for a sample of Irish households pre- and post-scheme participation relative to the change in gas consumption for a control group. An average reduction of about 3664 kWh or 21 % following installation of energy efficiency measures is reported. When compared to an ex ante estimation of energy savings, a shortfall of approximately 36 % between technical potential and ex post measured savings is found.

In one of the studies closely related to the research presented here, Wyatt (2013) uses annualised electricity and gas consumption data as well as information on installed energy efficiency improvements obtained from the Homes Energy Efficiency Database, a subset of the National Energy Data-Framework used in this study. The author concludes that realised gas reductions for cavity wall insulation and installation of a new condensing boiler are broadly equivalent with median savings in the range of 13.5–19.5 %. The effect of installing loft insulation is found to be somewhat lower between 8.4 and 12.2 % in the year following installation. Using the same data source, Hamilton et al. (2013) also report that energy savings are achieved following the installation of loft and cavity insulations as well as the installation of double glazing and a new efficient boiler in the dwelling. This is consistent with new evidence from Ireland in which households without wall insulation are found to consume significantly more gas compared to those who have wall insulation (Harold and Lyons 2015). The scope of these studies is limited by the data sets used as they were unable to conduct matching on key explanatory variables, making the results sensitive to latent differences in the characteristics of treatment and control groups. More importantly, these studies focus on the impact of installing a single energy efficiency measure, disregarding the impact of installing a combination of different energy efficiency measures.

In this paper, we exploit a new data source made available through the NEED in order to estimate changes in actual gas and total energy consumption when dwellings are retrofitted with one or more of the following energy efficiency measures: cavity wall insulation, loft insulation and a new efficient boiler. A sub-sample of NEED has been used previously by DECC (2014) to produce a preliminary report investigating the impact of energy efficiency measures on energy consumption. However, the report only considers the impact on gas consumption and excludes flats from the sample. A rather generic matching procedure is also used as opposed to the one used in this study. The analytical differences between the DECC report and the study presented here are further elaborated in the “Method” section.

Method

A quasi-experimental approach is the most appropriate estimation method to quantify the causal effect of a treatment on an outcome variable (Meyer 1995). In order to estimate the impact of energy efficiency measures on gas and total energy consumptions, we compare the performance of a sample of dwellings pre- and post-treatment relative to the performance of some control group pre- and post-treatment. Properties receiving energy efficiency measure(s) in the treatment year represent the treated group, and the non-upgrading properties in the sample make up the control group. In essence, variation in gas and energy consumptions is explained across time and groups. This research design is superior to most other estimation methods used in previous studies. For instance, the ex ante analysis of energy savings relies on treated properties to serve as their own controls and the simple before-after evaluation does not account for macroeconomic influences. A difference-in-difference analysis, on the other hand, controls for external factors affecting both the sample and the control group between periods by using trends in the control as the baseline. In this study, we also apply a coarsened exact matching (CEM) procedure to control for the confounding influence of pre-treatment control variables in the data. The idea is that by creating a better distributional balance between the treated and control groups, a simple difference in means on the matched data or a difference-in-difference approach can be used to estimate the causal effect with greater reliability than previous studies on this topic. For instance, in the DECC report (2014), a simple matching algorithm seeking to restrict aspects of the variance and hence the matched sample size is applied, leaving the impact of much bigger problems of statistical bias and model dependency uncertain and to ad hoc ex post diagnosis. Equally worrying, properties of a simple matching algorithm only hold on average across samples and rest on strong unverifiable assumptions about the data generation process; see Iacus et al. (2011) for a discussion. CEM, on the other hand, yields a relatively larger matched sample size, is less model-dependent and does not require assumptions about the data generation process. For these reasons, we adopt a matched difference-in-difference approach to estimate the ex post effectiveness of energy supplier obligation programs (CERT and CESP) implemented in the UK by comparing changes in gas and total energy consumptions of dwellings retrofitted relative to the changes experienced by an analogous non-retrofitted group of dwellings. The change in average consumption for non-retrofitted properties over the same period represents the counterfactual, i.e. changes in gas and energy consumptions unobserved for the treated but would have been observed in the absence of the retrofit (Scheer et al. 2013; Frondel and Schmidt 2005). This proposed methodology requires data measured at two or more time periods in order to estimate the expected difference in the outcome variable between the treated and the control groups. That is, the difference would still exist if neither group experienced energy efficiency improvement(s). In this respect, time trend factors such as changes in weather conditions, energy prices and socio-economic characteristics over time as well as behavioural factors directly or indirectly affecting energy consumption are controlled for and assumed to affect both groups in the same way. This would reduce potential bias in post-treatment period including trends or inherent differences between the treated and the control groups (Wooldridge 2007). Studies using robust matching procedure such as CEM are better able to produce a consistent unbiased estimator relative to studies applying a difference-in-difference estimation on an unmatched data sample as in the case of Hamilton et al. (2013) and Wyatt (2013).

We present the following identifying equation for the difference-in-difference estimation:

$$ {y}_{it} = {\beta}_0+{\beta}_1{T}_{it} + {\beta}_2{A}_{it} + {\beta}_3{T}_{it}{A}_{it}+{\epsilon}_{it} $$
(1)

y it is the outcome variable of interest, i.e. mean annual gas or total energy consumption for dwelling i in time t. \( {T}_{it} \) is equal to unity if dwelling i belongs to the group of dwellings that will eventually be treated (retrofitted), and thus, β 1 captures the differences between the treated dwellings and the ones in the control group prior to the energy efficiency treatment. A it is equal to unity in the period after treatment has occurred, and hence, β 2 captures aggregate factors that would cause changes in gas or energy consumption in the absence of the treatment. The interaction term β 3 is the coefficient of interest and is equal to unity for treated dwellings after the intervention. That is, it identifies the causal effect of the treatment, the impact of energy efficiency measure(s) on gas or energy consumption. β 3 can be obtained by directly estimating Eq. (1) or simply by calculating the change in average gas or energy consumption for the treated dwellings pre- and post-treatment less the change in average gas or energy consumption for dwellings in the control group pre- and post-treatment as shown in Eq. (2).

$$ {\widehat{\beta}}_3 = {\left({\overline{y}}_{A = \mathrm{p}\mathrm{r}\mathrm{e}} - {\overline{y}}_{A=\mathrm{p}\mathrm{ost}}\right)}_{\mathrm{treatment}}\ \hbox{--} \kern0.5em {\left({\overline{y}}_{A=\mathrm{p}\mathrm{r}\mathrm{e}} - {\overline{y}}_{A=\mathrm{p}\mathrm{ost}}\right)}_{\mathrm{control}} $$
(2)

In this study, the first method of directly estimating the three parameters of interest (β 1, β 2, and β 3) is used. The crucial identifying assumption in Eq. (1) is that β 3 is equal to zero in the absence of the treatment at the time of measurement. That is, without any record of energy efficiency measures installed, the coefficient identifying the causal impact on gas or energy consumption would be zero. Statistically, the zero conditional mean of errors, E(ε it |T it ) = 0, is required. This assumption is most plausible when the non-participating control group is very similar to the treatment group (Meyer 1995). As discussed in earlier paragraphs, this can be achieved by applying a CEM algorithm, ensuring that the empirical distributions of the covariates in the groups are more similar. Matches are made on the basis of key explanatory variables including property type, size and age as well as the region in the UK the dwelling is located in. In essence, statistical twins are created by pairing treated dwelling with recorded energy efficiency installations with comparable non-upgrading dwellings in the control group to isolate the causal effect of energy efficiency measure. The results may still be open to an omitted variable bias in the form of idiosyncratic differences in socio-economic characteristics of households as well as a selection bias in receiving retrofit through CERT and CESP. Yet, the matching procedure applied and the robustness of the difference-in-difference estimation are likely to considerably reduce risks of such biases.

Data

As discussed in the previous section, if the treated and the control groups are analogous across the covariates, the matched difference-in-difference estimation achieves high internal validity, an instrument of reflecting the extent to which a causality effect is justified. For this reason, a comprehensive data framework which tracks energy efficiency features, physical building characteristics and actual energy consumption of dwellings both with and without recorded installations of energy efficiency measures is required. An important building block for compiling such databases is the recently released UK’s NEED which comprises of annualised electricity and gas consumption metre readings obtained from Xoserve, a company managing financial transactions between the gas transporters, as well as readings obtained from groups of independent gas transporters. These metre readings are then weather-corrected using historic and forecasts from the Met Office and are aligned with information on the specific location of the metres. Annual data between 2008 and 2012 is used in this study. More importantly, NEED draws together information on energy efficiency measures installed in dwellings from the Homes Energy Efficiency Database (HEED) and information on physical property characteristics obtained from the Valuation Office Agency (VOA). HEED is a national database which tracks the energy efficiency characteristics of the UK’s housing stock and records the uptake of energy efficiency measures across the country. VOA, on the other hand, is responsible for allocating all properties in the UK to the appropriate Council Tax band. It maintains a property database covering information on the age of a dwelling, property type and floor area. The address information in each of these data sources is used to combine them into one comprehensive database by assigning a unique property reference number to each record.

The unmatched sample in this study is made up of approximately three million properties created using random sampling of residential properties in England and Wales. This sample contains valid records defined as properties with electricity consumption value between 100 and 25,000 kWh and gas consumption value between 3000 and 50,000 kWh. Electricity and gas consumptions of the majority of households in the UK fall within these thresholds. Outliers are excluded to minimise the risks of including invalid consumption data or records from non-domestic properties. Similar to the DECC study (2014), estimated metre readings and dwellings with implausible changes in gas and electricity consumptions are also excluded from the sample. Implausible changes are defined as annual changes in gas or electricity consumption of over 50 % as such deviations are likely to be due to data errors or factors other than the installation of energy efficiency measures. For instance, a large change in gas or total energy consumption is likely to be observed if there is a change in occupants or their circumstances which we cannot control for given the data available. The final integrated sample is used to set out the matched difference-in-difference methodology, that is, estimating the changes in gas and total energy consumptions following observed installation of energy efficiency measures in the dwelling relative to a comparable control group of properties. Consistent with previous ex post analysis (Hamilton et al. 2013; Wyatt 2013), a retrospective treated-control method is adopted by assigning dwellings to the treatment group if they had undergone one or more energy efficiency improvements in 2011 and to the control group if they had no record of installation under the CERT and/ or CESP. However, dwellings in the control group may include those with insulations installed by the homeowner themselves or dwellings which had their installation done when built. A separate control group is created for each measure and all criteria applied to the treated group are also applied to the control group, except the installation of the energy efficiency measure being analysed. To control for confounding factors of energy consumption, dwellings with recorded energy efficiency installations are matched to comparable dwellings in the sample on the basis of property type, size and age as well as region. The key goal of this procedure is to ensure a better balance between the treated and control groups in the final sample. Table 1 shows the distribution of categorical matching variables used for the analysis of installing a new boiler on gas consumption for three groups, the treatment group, the full control group and the matched control group. The full control group shows some differences to the treatment group; for example, a larger proportion of dwellings in this sample are located in London. Post-matching, the empirical distributions of the covariates in treatment and in the matched control group are identical. The exact matching procedure applied means that controlling for the covariates in the difference-in-difference (DID) regression is not necessary.

Table 1 Distribution of categorical matching variables used for the analysis of a new boiler installation (all rows sum to 100 %)
Full size table

Table 2, on the other hand, displays summary statistics for all groupings in the matched sample. It gives the breakdown of median consumption, standard deviation and the rate of change between the pre- and post-upgrade time periods for the treated dwellings known to have installed one or more energy efficiency measures in 2011 (intervention year) and for the ones in the control group with no record of energy efficiency measures installed through CERT and/or CESP. In model 1, median gas and total energy consumption are shown to be higher in 2010 (pre-intervention) relative to 2012 (post-intervention) for both the treated and control groups. A closer inspection of Table 2 reveals that the treated group had higher gas and total energy consumptions in 2010 but that the fall in gas and energy consumptions after the intervention (installation of cavity wall insulation) is larger for treated dwellings relative to the dwellings in the control group. A similar trend is observed in the other models of Table 1 as clearly shown by the large difference in the rate of change of the treated group and the control group. This could perhaps be explained by that dwellings receiving the treatment were systematically selected due to their low levels of energy efficiency and hence high energy consumption. This is likely to be the case because CERT and CESP were designed to target hard-to-treat homes with potentially high baseline consumption to begin with and as a result large saving potential post-treatment. The difference-in-difference method applied in this study is robust to such selection bias, provided that the self-assignment choice does not determine different time courses of change independent of treatment.

Table 2 Descriptive statistics of gas and total energy use in thousand kWh/year for sub-groups
Full size table

In addition to matching observations on the basis of building characteristics and region, the energy performance rating (EPC) of dwellings is controlled for in the regression analysis. This rating is obtained through an assessment using the UK Government’s Standard Assessment Procedure (SAP) which produces annual estimates of energy consumption per unit of floor area, CO2 emissions per unit of floor area and an environmental impact index rating. The estimated energy consumption ratings are then used to assign bands from A (highly efficient) to G (very low efficiency). All dwellings that are sold or rented require an EPC, although exemptions can be obtained, for example, for listed buildings. The EPC assessment is conducted by a certified assessor who inspects the dwelling and its characteristics. Crucially, the rating is not based on actual energy consumption but rather on the intrinsic energy efficiency of the dwelling as surveyed by the assessor and it is valid for up to 10 years. For our analysis, we can thus assume that the EPC band represents the baseline characteristics of a dwelling prior to the energy efficiency retrofit. In some cases, the EPC may reflect the state of the property post-retrofit but we assume that the majority of property owners will use the existing valid EPC in a subsequent transaction. As shown in Fig.  1 , gas and total energy consumptions prior to the energy efficiency upgrade appear to vary with the EPC ratings. With increasing ratings, there is a corresponding decrease in the gas and total energy consumptions, except for properties with the least energy efficiency rating (G).

Fig. 1
figure 1

Average gas and total energy consumptions in kWh by EPC rating for dwellings in the sample prior to energy efficiency upgrade

Full size image

Results

The DID regression results for the changes in gas and total energy consumptions are reported in Table 3. The most effective single energy efficiency measure is shown to be cavity wall insulation with an observed change of 10.5 % in gas consumption and 8.0 % in total energy consumption in the year after installation. This is followed by the impact of installing a new boiler, found to reduce annual gas consumption by 4.1 % and total energy consumption by 5.2 %, respectively. Amongst the three energy efficiency measures considered, loft insulation is the least effective retrofit measure when installed on its own with an estimated reduction in gas consumption of 3.1 % and in total energy consumption of 1.8 % in the year following installation. Comparing bundles of energy efficiency upgrades, the largest reduction in gas consumption is achieved by combining cavity wall insulation with a new efficient boiler; an average annual reduction of about 13.3 % in the year following installations is observed. This is followed by a 12.5 % reduction in annual gas consumption for dwellings with both cavity wall insulation and loft insulation. Next, dwellings with all energy efficiency measures considered are observed to have 11.9 % lower gas consumption in the year following installations. The least effective combination in reducing gas consumption is achieved by a combination of loft insulation and a new efficient boiler, with a 7.7 % reduction in gas consumption. Interestingly, this reduction is lower than the one observed for dwellings with cavity wall insulation alone. Turning to total energy consumption, installations of cavity wall insulation and a new efficient boiler during the same year are found to reduce total energy consumption by 13.5 %. This is followed by installing all three measures in the same year, found to reduce total energy consumption by approximately 10.5 %. Combining cavity wall insulation with loft insulation is found to reduce energy consumption by 7.2 %. Lastly, loft insulation and a new efficient boiler are found to reduce energy consumption by 5.2 % in the year following installation. Consistent with the results for gas consumption, savings realised under a combination of cavity wall insulation and loft insulation as well as loft insulation and a new efficient boiler are lower than energy savings realised by installing cavity wall insulation on its own.

Table 3 Overview of difference-in-difference estimation results
Full size table

In the next step, we re-estimate the difference-in-difference regression models as shown above but including the EPC ratings of dwellings as control variables. The treatment effects of the different measures are broadly similar to the ones reported in Table 3. However, in line with expectation, annual gas and total energy consumptions are found to vary with the EPC ratings of dwellings. Dwellings with excellent to good EPC ratings (A/B or C) are found to have lower gas and total energy consumptions relative to average EPC-rated dwellings (D), whilst below-average E- and F-rated dwellings are marked by higher consumption. There are two findings that deviate from this pattern. Firstly, although some of the coefficients of A/B rating are insignificant, it is shown that C-rated dwellings appear to consume less gas than A/B-rated dwellings. In model 6, C-rated dwellings are also found to have lower total energy consumption than A/B-rated dwellings. Secondly, households inhabiting dwellings in the lowest energy efficiency band (G) appear to consume less total energy than expected since their coefficients are significantly below the average reference category. This is also the case for gas consumption in model 4, where the coefficient of G-rated dwellings is significantly below the one for average D-rated dwellings.

Discussion

The results presented in this study suggest that energy savings are statistically observed for dwellings with energy efficiency measures installed through the CERT and the CESP schemes. Considering the impact of installing a single energy efficiency measure, results reported in this study confirm the findings of previous analyses of ex post effectiveness of energy efficiency measures in the UK (Hong et al. 2006; Hamilton et al. 2013; Wyatt 2013; DECC 2014). However, an interesting and divergent insight is found when more than one energy efficiency installation is undertaken. Particularly, we find mostly sub-additive gains from combining different energy efficiency measures. Combining two measures (cavity wall insulation and a new efficient boiler) is found to be more effective than installing all three energy efficiency measures considered. Additionally, installing cavity wall insulation on its own is found to be more effective than combining loft insulation and a new efficient boiler. These findings are less straightforward and could be due to several factors. A possible explanation is the existence of the “pre-bound effect” discussed by Sunikka-Blank and Galvin (2012) to explain the gap between measured and calculated household energy use. The pre-bound effect refers to the situation where energy consumption prior to energy efficiency installation is lower than the calculated figure for pre-installation consumption, based on the building’s physical characteristics. This can lead to overestimation of the amount of energy saved when combining two or more energy efficiency measures, as householders cannot save energy that was not already being consumed (Rosenow and Galvin 2013). The NEED data framework used in this study does not allow for estimation of the pre-bound effect. However, as shown in Table 4, households inhabiting dwellings with the lowest energy performance rating (G) prior to the energy efficiency installations are found to consume, on average, less gas than households inhabiting average EPC-rated dwellings (D). This may imply a pre-bound effect in that households living in poorly insulated homes in the sample had low energy consumption prior to the energy efficiency installations and hence installing; for example, three different energy efficiency measures may not have led to large energy savings for them. The objectives of the CERT and the CESP also imply that the least energy efficient dwellings in the sample were also likely to have received several energy efficiency installations in order to raise their energy efficiency to an acceptable level. On technical grounds, factors other than the pre-bound effect could explain why some measures are sub-additive. For instance, there could be inconsistencies between different measures if the power of the boiler is not sized according to the level of insulation or if there is a variation in insulation levels. Chidiac et al. (2011) found in their empirical study that combining multiple energy retrofit measures is not as beneficial as the sum of individual measures and conclude that an in-depth simulation of how the individual measures will interact with each other is required which is beyond the scope of our paper. Whilst this is not conclusive, some coefficients of C-rated dwellings were also found to have lower gas consumption relative to A/B-rated dwellings.

Table 4 Overview of difference-in-difference estimation results adjusted for EPC ratings
Full size table

A data framework without flaws is a rarity; the NEED used in this study is based on a rich, well-structured and reliable data of good quality, enabling a robust estimation of ex post effectiveness of energy efficiency measures. An important caveat is that some measures not supported by CERT or CESP could well have been installed by the control group which could explain the higher effectiveness of some of the measures. For instance, a DIY loft insulation undertaken by the control group or a replacement of a broken boiler without support from the schemes could explain the higher savings achieved by installing cavity wall insulation alone in comparison to a combination of cavity wall insulation and loft insulation. Loft insulation could also reflect a broader remodelling of the attic leading to additional energy use. Jack et al. (2011) show empirically that loft conversions and extensions increase the energy consumption of a property in England by an average of 16 %. Hence, whenever loft insulation is combined with cavity wall insulation or a new boiler or with both, it is of no effect or counter-productive due to possible (unobserved) loft conversions into heated living space. More importantly, despite the fact that some of the variations in gas and total energy consumptions appear to be explained by building characteristics contained within NEED, there are other factors to consider such as performance of heating systems, building construction and appliances. Other important factors such as household composition, income profiles and age are implicitly controlled for by the use of the difference-in-difference estimation in this study and add robustness to the results.

Conclusions

This study presents an econometric analysis of the ex post effectiveness of energy supplier obligation programs (CERT and CESP) implemented in the UK. Access to a new comprehensive data source made available through the NEED allowed us to adopt a matched difference-in-difference estimation. Descriptive statistics reveal that gas and energy consumptions exhibit considerable variation across dwelling types, age and size, justifying their inclusion in the matching procedure prior to the estimation. The results suggest that a drop in gas and energy consumptions is statistically observed following installation of different energy efficiency measures. Particularly, cavity wall insulation, loft insulation and installation of a new efficient boiler are all found to reduce gas and total energy consumptions when installed separately or in different combination packages. The single most effective energy efficiency measure when installed alone is found to be cavity wall insulation, reducing annual gas consumption by 10.5 % and annual total energy consumption by 8 % relative to a comparable control group of dwellings with no record of considered energy efficiency measures. Next, reductions of 4.1 % in gas and 5.2 % in total energy consumption are observed following installation of a new efficient boiler on its own. Loft insulation is found to be the least effective energy efficiency measure when installed alone with a 3.1 % reduction in gas and a 1.8 % reduction in total energy consumption in the year following installation. Amongst different combinations of these measures, dwellings retrofitted with both cavity wall insulation and a new efficient boiler experience the largest reductions in annual gas and energy consumptions of 13.3 and 13.5 %, respectively. This is followed by a mean annual reduction of 10.5 or 11.9 % in gas and total energy consumptions for dwellings with all three energy efficiency measures installed in the same year. Lastly, the least effective combination is loft insulation and a new efficient boiler with 7.7 % reduction in gas consumption and 5.2 % reduction in total energy consumption in the year following installations. Nevertheless, the impact of a combining cavity wall insulation and loft insulation is inconclusive. A mean reduction in gas consumption of 12.5 % is found, making it the second most effective combination, and a reduction in total energy consumption of 7 %, making it the second least effective combination. Again, a full energy audit of retrofitted buildings would be required to elucidate the interplay between gas and other energy (particularly electricity) consumptions in retrofitted buildings.

There are several implications for energy efficiency policy in the UK arising from this analysis. Firstly, the analysis shows that a tangible impact of energy efficiency upgrades such as insulation and boilers exists. To the extent that these measures have also been targeted through government incentive schemes, it appears that they have been effective. The hypothetical energy savings translate into realised observed savings despite several intervening factors such as the rebound effect and other behavioural and socio-economic factors. Secondly, incentive schemes may be reconsidered to take into account that certain combinations of measures yield better results than others and could be prioritised if found to be indeed more cost-effective. Finally, government-backed green loan schemes should also take into account the empirical findings on realised savings (as opposed to hypothetical savings) to lower the risk of adverse and unexpected outcomes regarding payback periods of energy efficiency investments.

This study has presented some first evidence on the average effects of energy efficiency upgrades on subsequent energy consumption of households. Follow-up research is required to arrive at a better understanding of the dynamic effects around these upgrades. In particular, time series and panel data analysis may be able to show whether the reductions in energy consumption found in this analysis are immediate and stable or whether they increase or decrease over time. For example, behavioural adaptation to the upgraded property and its technical systems for heating etc. may require learning and experience so that the full impact of the upgrade on energy consumption patterns may only be realised with a relatively long time lag. A dynamic analysis would also enable a more in-depth study of the presence and magnitude of the rebound and pre-bound effects described above.