1 Introduction

As global greenhouse gas emission constraints are becoming more stringent, substantial reductions must be achieved in all economic sectors. The residential sector consumes a large share of total energy. Residential users account for on average 18.4% of the total fuel consumption in OECD countries in 2017 (IEA 2019) and 25.7% in the EU (EC 2018). The energy efficiency gap, i.e. the difference between actual and optimal energy use (Jaffe and Stavins 1994), remains high in this sectorFootnote 1 due to reasons such as inattention of residents (Palmer and Walls 2015), the landlord-tenant dilemma (Allcott and Greenstone 2012) and high non-monetary costs of energy efficiency investments (Fowlie et al. 2015).Footnote 2

In attempts to close the gap, governments offer a variety of publicly funded financial incentives, including direct subsidies and other instruments, to encourage households to conduct conservation works that improve energy efficiency and thereby also help to reduce GHG emissions.Footnote 3 Evaluating the cost-effectiveness of these government policies is essential to gain insights on which support schemes achieve the highest energy reduction for each euro of public money spent. This matters for fiscal policy, especially in times of constrained public budgets. The involved redistribution between higher and lower income groups also matters. To be able to meaningfully compare instruments in an ex-post evaluation, researchers need data on different instruments that were implemented under the same administration during the same period.

In this paper, we assess the performance of four types of financial schemes used to support residential energy conservation works (retrofits) that were all implemented in France between 2014 and 2016: a grant scheme for low-income households, a reduction of the value-added tax (VAT), an income tax credit and the White Certificates. The latter are certificates that French energy suppliers have to collect to a certain amount in exchange for helping energy consumers to reduce their energy consumption. Energy suppliers can then trade the certificates among each other to achieve the energy reductions set by the government. We use the TREMI2017 survey data produced by the French Environment and Energy Agency (ADEME). The dataset covers close to 14,000 observations at household-level and includes information on works related to energy conservation and the adoption of specific financial support schemes.Footnote 4

To address potential endogeneity and self-selection bias, we employ double-robust inverse probability weighting (IPW) estimators.Footnote 5 IPW uses the propensity score to compute weights based on the inverse probability of treatment. The procedure assigns high weights to observations in the control and treated group that are most comparable. The double-robust feature additionally employs predictive information of the covariates on the outcomes. As such, the estimator is robust to incorrect specification of either the propensity score or the outcome model and is consistent if only one of the models is correctly specified. We estimate the effect of adopting each scheme on the funding acquired, the private and total investment by households, and the reduction of households’ energy expenses. For each scheme, we use these estimates to evaluate the cost-effectiveness of public funds to reduce energy expenses, the ability of the schemes to induce private investment (leverage effect) and their capacity to increase total investment (additionality). We compare the four schemes along these criteria. We also analyse the distributional effects of the schemes based on the reduction in energy expenses and the funding received for low- versus high-income households.

We find that the White Certificates scheme is most cost-effective, followed by the VAT reduction and the grant scheme. The income tax credit is the least cost-effective. The VAT reduction triggers most additional private investment into conservation works, followed by the income tax credit and the White Certificates scheme. The grant scheme for low-income households does not trigger any additional private investment. The VAT reduction also has the highest capability to increase total investment, followed by the income tax credit and the White Certificates. The grant scheme induces least additional total investment. For each of the four schemes, households in the upper and lower half of the income distribution equally benefit in terms of funding received and reduction in energy expenses.

Earlier studies set in the US context have diverse findings. While Walsh (1989) does not find any effect of a change in the income tax credit on the propensity to invest, Hassett and Metcalf (1995) find that a 10 percentage point decrease in the value-added tax on energy investment increases the propensity to invest by 24%. Estimates for the cost-effectiveness of financial support schemes to increase residential energy efficiency range between US$3.9 cent and US$47.9 cent per kWh in a heat pump rebate program in Maryland (Alberini and Towe 2015; Alberini et al. 2016). Few studies exist in the European context. Alberini and Bigano (2015) find an Italian income tax credit program to have no effect on the propensity to replace heating equipment. Blaise and Glachant (2019) assess the average impact of conservation works on energy savings in France without differentiating between support schemes.

The novelty of our work in comparison with the existing literature is fourfold. First, we present a comprehensive comparison of the performance of four different financial support schemes, while previous econometric studies have commonly focused on one or two schemes. The comparison becomes possible due to the availability of data on energy efficiency works and the use of four financial schemes that were all implemented during the same time period in the same country. Second, we assess the performance of four schemes after adoption while most previous studies analyze the effect of a scheme on the uptake of conservation works. The empirical evidence on the performance after adoption is rare. Third, we compare four schemes according to the criteria of cost-effectiveness, redistribution, additionality of investment and leverage for private investment, whereas earlier studies usually focus on one dimension only, which prevents them from uncovering trade-offs between different performance dimensions. Finally, we provide new insights on the EU perspective. Previous studies that focus on the US context may not carry external validity for the EU context since they are placed in a different institutional and cultural setting. Further empirical evidence on EU member states matters.

2 Literature Review

The empirical literature on financial support schemes is much focused on the US but, in the recent years, more studies on the European context were published. Numerous studies evaluate tax credits for conservation works (e.g. Alberini and Towe 2015; Hassett and Metcalf 1995; Nauleau 2014; Risch 2020; Walsh 1989); fewer studies evaluate other types of schemes, such as grants, interest-free loans and rebates (Amstalden et al. 2007; Eryzhenskiy et al. 2022; Fowlie et al. 2018). Empirical studies which compare the relative performance of schemes are scarce (e.g. Zhao et al. 2012).

Policy evaluation of financial support schemes is conducted at the extensive and intensive margin; at both margins schemes can affect households’ investment behavior. First, households take the binary decision to invest into energy conservation. Schemes can affect the households’ propensity to invest, operating at the extensive margin. Second, households decide how much to invest. Here, schemes can affect the size of households’ investment, operating at the intensive margin.

Previous literature focuses on the first stage decision, i.e. whether schemes encourage the uptake of conservation works. There are fewer studies that analyze the second stage decision, to what extent schemes increase the amount invested into conservation works.

Two seminal papers study the effect of financial support schemes on the propensity to invest; they find contradicting evidence. Walsh (1989) assesses the effect of a change in the income tax credit rate on take-up of energy conservation works, exploiting variations between US state income tax credit rates. It is found that a change in the income tax credit rate does not affect the propensity to invest. Hassett and Metcalf (1995) assess how reduced VAT tax rates on energy investment affect the take-up of conservation works in various US states, and find that a 10 percentage point decrease in the tax increases the propensity to invest by 24%.

More recent studies have likewise found ambiguous evidence. Zhao et al. (2012) compare the impact of income tax credits and interest-free loans on the propensity to invest, based on a household survey in Florida, US. They find tax credits to be more attractive than interest-free loans. The interest-free loan does not increase the propensity to invest, but the tax credit increases the rate of investing households by 12%. Alberini and Bigano (2015) find an Italian income tax credit program to have no effect on the propensity to replace heating equipment. Grösche and Vance (2008) estimate the proportion of inframarginal households that would even have invested in the absence of a financial support scheme by the German government. Around 50% of households are found to be inframarginal adopters, no hidden costs of adoption assumed. Rivers and Shiell (2016) assess the proportion of inframarginal households that adopt a natural gas furnace replacement scheme in Canada. They find that around 50% of adopters would have replaced their gas furnace even in absence of the scheme. Boomhower and Davis (2014) measure inframarginal participation in an appliance replacement program in Mexico. They estimate that at least 65% of the participating households are inframarginal and would have invested into an energy-efficient appliance even without subsidy. In the French context, Nauleau (2014) investigate the effect of an income tax credit schemeFootnote 6 and find a small positive effect on investment at the extensive margin for insulation measures, however at a free-riding proportion of 40 to 85%. Risch (2020) finds the same tax credit scheme to increase the impact at the extensive margin only by 1.1 percentage points, while the increase at the intensive margin is economically more significant at 22%. Eryzhenskiy et al. (2022) evaluate the effect of a zero-interest green loan program on the extensive and intensive margin investment. They find only a small effect of 3 to 4 percentage points at the extensive margin but the effect fades out in the third year after the start of the program. At the intensive margin, the amount invested increases by 9%.

The cost-effectiveness of financial support schemes has been evaluated for different contexts and schemes. Gillingham et al. (2018) compare the cost-effectiveness of financial schemes from different studies on energy efficiency subsidies. Reduction costs per kWh lie in a range between US$3.9 cent and US$47.9 cent in a heat pump rebate program in Maryland (Alberini and Towe 2015; Alberini et al. 2016). Blaise and Glachant (2019) find that conservation works in France reduce the energy bill on average by only 0.64%. They express concerns about the effectiveness of financial support schemes that aim to save energy.Footnote 7 Using a cost-benefit analysis, Giraudet et al. (2012) indicate that the White Certificates scheme is a cost-effective and economically efficient instrument.

Financial support schemes provided by the government reallocate taxpayers’ money to recipients; distributional effects of the reallocation depend on a scheme’s design and target group. Both these features affect which socio-economic strata adopts a scheme. For instance, high-income households benefit most from income tax credits since these can deduct costs of conservation from higher tax liabilities (Neveu and Sherlock 2016). Lower-income households preferably use grants and rebates when they are pessimistic about paying back loans. Marketing and implementation conditions can play a crucial role in determining the socio-economic background of recipients (Hoicka et al. 2014). For instance, demanding requirements of paperwork to apply for financial support schemes can be a barrier to less-educated households, and the design of information campaigns can determine to which groups a scheme reaches out (Walsh 1989). Empirical studies find recipients of financial support schemes to be a homogenous group with regressive effects on the income distribution. Using survey data, Allcott, Knittel, and Taubinsky (2015) show that the majority of households benefitting from conservation subsidies are wealthy environmentalist homeowners. Rivers and Shiell (2016) find likewise that recipients of a gas furnace replacement scheme in Canada are in large parts middle- and high-income households.

3 Institutional Background

In France, the 2005 law setting the orientations of the national energy policyFootnote 8 introduced the White Certificates to oblige energy suppliers help consumers to lower their energy consumption. This followed the 2002 EU Directive on energy performance of buildings (EU 2002), which introduced minimum energy performance requirements for buildings and energy performance certificates. In 2009, the Grenelle I law (RF 2009) set the target to renovate 800,000 social housing units in order to halve their energy consumption by 2020, to encourage the construction of low-energy consumption buildings, and aim for a 38% reduction in energy consumption in old buildings by 2020. The Grenelle II law of 2010 (RF 2010) introduced the objective to reduce energy consumption in new buildings by a factor of five by 2012. The 2010 Amendment of the 2002 EU Directive (EU 2010) stated the objective to have all new building nearly zero-energy after 2020 (after 2018 for new building occupied and owned by public authorities). In this context, the Housing Energy Renovation Plan (“Plan de Rénovation Energétique de l’Habitat”, PREH) of 2013 (RF 2013) introduced the objective to renovate 500,000 housing units per year by 2017Footnote 9 (including 380,000 privately owned units) and to reduce energy consumption in the housing sector by 38% by 2020. To do so, national public aids were developed, which are presented below.Footnote 10

In France, households can benefit from a variety of financial support schemes for renovation works that aim to improve the energy efficiency of private dwellings. In the following, we discuss four types of at national level available schemes on which we focus our analysis: an income tax credit, a reduction of the VAT, a grant scheme and the White Certificates scheme. Further information on the financial support schemes can be found in RF and ADEME (2020).

The income tax credit (“Credit d’Impôt pour la Transition Energétique”, CITE) that is offered by the French government applies to expenditures on home insulation or equipment that reduce energy consumption (see non-exhaustive list of examples in appendix). The installation costs of the works are also eligible. The work must be carried out by a company that does all the work or subcontracts some of it to another company. For most of the work, the company must be certified. The scheme allows deductions from the income tax of up to 30% of the invoice sum.Footnote 11 The maximum amount of expenses that could be considered was € 8,000 for a single person and € 16,000 for a couple.Footnote 12 Only home owners can use this tax credit scheme.

Another financial instrument funded by the government is the reduction of the value-added tax (VAT) to 5.5%. Every household is eligible to the VAT reduction as long as it conducts renovation works that target the energy efficiency of their dwellings. The VAT reduction applies to the installation, maintenance and supply of materials, appliances and equipment mentioned in a nationally defined listFootnote 13 (see list in appendix) under the condition that they are compliant with technical characteristics and minimum performance criteria defined by law.Footnote 14 To benefit from this reduction, households have to submit their invoice three months at the latest after the invoicing. The granted amount directly depends on the amount of private investment into energy efficiency works, funding being a fixed proportion of the latter; the higher the private investment, the higher the funding received from the VAT reduction is.

The French housing agency (“Agence Nationale pour l’Amélioration de l’Habitat”, ANAH)Footnote 15 provides the program “Habiter Mieux” to help households by supplying a direct grant of between 35% and 60% of renovation work net expenses. The maximum amount that can be allocated is € 10,000 on top of which a bonusFootnote 16 can be added if the energy efficiency improvement obtained is at least 25%. The grant aims at supporting households with low-income levels that live in dwellings of 15 years of age or older. The eligibility depends on the joint annual income, the number of persons in the household and whether the dwelling is located in the Paris region (Île-de-France) or not (Anah 2016). The lower the household income and the higher the number of persons in the household, the higher is the maximum amount granted by ANAH. The survey data however suggests that the eligibility criteria are not strictly enforced. In the data, we see that even some households that report incomes above the eligibility thresholds of the ANAH grant receive funding from this scheme (see Fig. 2 in the appendix for a detailed overview).

Finally, since 2006, French energy suppliers have been obliged to collect a certain volume of “Certificats d’Economie d’Energie” (CEEs), which correspond to the so-called “White Certificates”. In exchange for assisting energy consumers to lower their energy consumption, energy suppliers receive certificates which they can trade among each other to cover the energy reductions they are required to achieve by the government. To accomplish the energy savings and to earn certificates, energy suppliers offer programs which inform households about energy consumption and savings, and financially support them to conduct conservation works. To be funded, the works have to be done by certified operators. Energy suppliers are proactive to detect consumers for whom the energy saving potential is the highest. They contact households individually, inform them about the programs they offer and encourage them to renovate their dwellings. This scheme does not only support energy efficiency of residential buildings but also of industrial and public buildings. By design, funds provided for energy conservation measures stem from the firms obtaining White Certificates, not from the public sector. The amount of funding received by households from this scheme depends on the specific program. The programs differ widely in their scope and the type of works that they support.

A non-negligible share of households in our sample adopted more than one scheme. The VAT reduction has the highest absolute number of adopters in the sample and is frequently adopted in combination with one of the other three schemes. The VAT reduction seems to be a scheme “for everyone”, possibly due to a simple application process and being pointed out to households by professionals conducting the works. The grant scheme is predominantly adopted by households with lower average income due to its eligibility criteria. Households that adopt more schemes in parallel tend to be households with higher incomes and older family heads. A larger number of schemes adopted in parallel also correlates with a higher amount of acquired funding and higher private investment by households. Households that adopt more than one scheme in parallel remain in our study sample. We control for parallel adoption by including national, regional and local scheme variables in the specification IV of the treatment model (see Sect. 4).

4 Material and Methods

4.1 Data

We use a dataset from the TREMI survey (“Travaux de Rénovation Energétique dans les Maisons Individuelles”), conducted by KANTAR PUBLIC/TNS SOFRES, Énergies Demain and Pouget Consultants for ADEME, the French Agency for the energy transition, in spring 2017. The survey targets energy conservation works that French households conducted between 2014 and 2016. The household sample was randomly drawn from the data base of the national statistics agency (Institut National de la Statistique et des Etudes Economiques, INSEE). The questionnaire was approved by TNS SOFRES. The resulting dataset has a cross-sectional structure at the household level and contains 44,921 observations, whereof 14,081 households conducted renovation works.

The survey offers a rich set of covariates, including information on the works conducted and the types of schemes used, as well as household and housing characteristics which can be used as controls. In the following, we present the variables that we employ for our analysis. Detailed summary statistics are reported in appendix A. Household characteristics include the age of the household’s reference person, the number of persons living in the household (household size), the annual household net income, the profession of the household’s reference person, the region where the household lives as well as the size of the agglomeration. The housing characteristics include the type of housing - 29,253 households in single-family houses, of which 9,964 conducted works, and 15,481 in apartments, of which 3,990 conducted works - the construction date, the status of ownership (owner versus renter) and living space of dwellings.

The data also reports the funding that households acquired from the schemes, the total amount invested including public money and private investment, and the reduction of energy expenses observed in the household. In total, the dataset includes 32,876 individual renovation works. Most of the works are related to doors and windows (22.5%), walls (e.g. insulation, 18.6%), the roof (17.8%) and the heating system (17.6%). Other types of works relate to warm water (7.4%), the floors (9.3%) and ventilation (5.4%). A variable specifies the year in which all works in the household were finalized. Apart from the national schemes, households also used regional and local schemes that are only available in specific regions.Footnote 17 Given the low rate of adoption of regional aids in the survey data, we only analyze the performance of national schemes. The nationally available schemes on which the analysis focuses include the grant scheme by ANAH, adopted by 7.9% of the households that conducted retrofits, the VAT reduction to 5.5%, adopted by 41.2% of the households, the income tax credit (“Crédit d’Impôt à la Transition Énergétique”, CITE), adopted by 9.8% of households, and the White Certificates scheme, adopted by 7.8% of households. Due to the design of the survey not all households were asked which schemes they adopted. This restricts our sample for the estimations to 7,939 observations. Households report the total amount of funding received, the amount invested into conservation works, and whether they observed a reduction of their energy expenses after conducting the works. In addition, households were asked what triggered the uptake of works and what motivated the works (see detailed statistics in appendix).

Figure 1 provides an overview of the distribution of work types across schemes. The share of each specific work type out of all works funded by a specific scheme is not uniform across the schemes. In our regressions, we include the types of work implemented by the households to control for different shares of work types funded across the four national schemes.

Fig. 1
figure 1

Distribution of work types across national schemes. Note This figure shows which work types were financed by the adoption of each national scheme. To create this figure, we used the sample of households that adopted only one scheme and that only reported one work type

To check the representativeness of the survey, we compare the proportion of each type of housing in the survey and in the French population, the income distribution, the number of persons living in the household as well as the share of home owners. In the survey sample, 65% of households live in single-family houses, whereas the proportion is 57% for the French population (INSEE 2017). According to TREMI, 64.7% of households in the survey are home owners, compared to 57.9% of the entire population (INSEE 2017). The distribution of the number of persons living in the households differs between TREMI and the French population (see detailed statistics in appendix A). The income distribution in the survey and the French population are comparable (see detailed statistics in appendix A).

4.2 Methodology

Our analysis seeks to identify the causal effect of the adoption of each scheme on the four outcomes of interest, i.e. the reduction of energy expenses, private and total investment, and the funding received, based on observables. A naïve approach could try to estimate the impact of the adoption of each scheme on each of the outcome variables of interest (e.g. the funding received or the reduction in energy expenses) controlling for the household and housing characteristics as well as the work type. However, there is a risk of self-selection bias as households did not randomly select themselves into adoption of the schemes.

When self-selection bias is likely, treatment is not randomly assigned but correlated with important characteristics of the households. We use Inverse Probability Weighting (IPW) pioneered by Robins and Rotnitzky (1995) in a double-robust estimator (Robins et al. 1994) to account for self-selection into adoption of the four national support schemes we analyze. We use this method to estimate the effect of the adoption of each scheme on the four outcomes based on observables.Footnote 18

IPW is related to matching techniques in that it invokes the same conditional independence assumption that selection into treatment is based on observed characteristics and can be modelled as independent of confounders. The method corrects for self-selection if the causes of selection can be explained by observables available to the researchers. As matching, IPW estimates a propensity score to find the most credible counterfactuals for the treated observations in the control group. IPW makes use of the propensity score to compute weights based on the inverse probability of treatment, constructing a pseudo-population with equally large control and treated groups that are balanced on the potential confounders. If the distribution of observables is the same in the treated and control groups, the relationship between observables and treatment is removed so that treated and control households are exchangeable. Weights are constructed to give higher weights to observations in the control and treated group which are most alike, and, therefore represent the most credible counterfactuals for one another: observations in the treated group with low probability to be treated, as well as control observations with high probability to be treated. The IPW method has virtues that make it a prudent alternative to matching methods. Unlike nearest-neighbor matching that restricts the control group to one or few observations per treated observation, IPW sustains the sample size and does not discard information. The IPW estimator has been found to perform best in finite sample applications in a variety of treatment effect estimators (Busso et al. 2014), though small sample properties are poor when propensity scores get close to zero or one (Glynn and Quinn 2010).

After estimation of the fitted values \(p_i\) from the propensity score model, observations in the sample are weighted using the following weights \(w_i\):

$$\begin{aligned} w_i=\frac{T}{p_i}+\frac{1-T}{1-p_i} \end{aligned}$$

with \(T \in \{0,1\}\) indicating the treatment status, so that, for \(T=1, w_i= \frac{1}{p_i}\) and, for \(T=0, w_i=\frac{1}{1-p_i}\). Outcomes are weighted using corresponding weights to obtain the difference in means of weighted outcomes, so that the treatment effect for the simple IPW estimator is

$$\begin{aligned} \begin{aligned} \beta _{IPW}&=n^{-1}\sum ^n_{i=1}Y_i\frac{T_i}{p_i}-n^{-1}\sum ^n_{i=1}Y_i\frac{(1-T_i)}{(1-p_i)} \\&=n^{-1}\sum ^n_{i=1}\left( Y_i\frac{T_i}{p_i}-Y_i\frac{(1-T_i)}{(1-p_i)}\right) \end{aligned} \end{aligned}$$

where \(Y_i\) is the outcome for observation i.

For the IPW to be able to recover the causal effect of scheme adoption, two assumptions need to hold. First, the propensity score model needs to include all relevant confounders that could be associated with both the self-selection of households and the outcomes of interest so that, after controlling for observables, the potential outcomes are independent of the treatment status. This assumption is not testable but we argue that we include a wide range of variables which are related to the adoption decision and could potentially affect the outcome variables, for instance, the income, the number of persons in the household and the construction date of the dwelling, parameters which determine eligibility and the amount of funding granted. We moreover employ “softer” information on attitudes and the life situation of the households by integrating the motivation and trigger for the renovation works, components which could also be related to both the adoption decision and the outcomes of interest.

Second, common support needs to hold so that for each level of each confounder, there is a positive probability for being in both the treated and control groups. We test this assumption empirically by comparing the density distribution of propensity scores for the treated and the controls for each treatment model specification for each scheme (see Figure 3 in the Appendix). We find the assumption to hold relatively well for all schemes and all specifications: the propensity score distributions of the treated and control groups overlap well.

We also run overidentification tests for each treatment model specification for each scheme to check whether the IPW-adjusted means of each covariate are the same in the treated and the control group. We cannot reject the H0 of equal means in treatment and control group for all specifications, except for the VAT reduction treatment pscore specifications II and IV (p < 0.01).Footnote 19

An extension to the \(\beta _{IPW}\) estimator is the so-called double-robust estimator that adds a separate outcome model for both the treated and control groups (Robins et al. 1994). The double-robust estimator utilizes information of the covariates on the probability of treatment as in the simple \(\beta _{IPW}\) estimator, and in addition it employs predictive information on the outcome variables from the covariates (Glynn and Quinn 2010). Taking advantage of both these features, the double-robust estimator is robust to incorrect specification of the propensity score model or the outcome model. If either one of the two models is correctly specified, the double-robust estimator is consistent (Scharfstein et al. 1999). The treatment model is correctly specified if the propensity score indicates the true probability of treatment given all confounders. And the outcome models are correctly specified if all relevant confounders are included as covariates in the regression. The double-robust IPW estimator is then:

$$\begin{aligned} \begin{aligned} \beta _{double-robust~IPW}&= n^{-1}\sum ^n_{i=1}\left( Y_i\frac{T_i}{p_i}-\frac{(T_i-p_i)}{p_i}~m_1(x_i)\right) \\&\quad -n^{-1}\sum ^n_{i=1}\left( Y_i\frac{(1-T_i)}{(1-p_i)}+\frac{(T_i-p_i)}{(1-p_i)}~m_0(x_i)\right) \end{aligned} \end{aligned}$$

with \(m_0(x_i)=Y(T=0,X_i)\) and \(m_1(x_i)=Y(T=1,X_i)\). The outcome models \(m_0\) and \(m_1\) estimate predicted outcomes for the control and treated groups respectively. The adjustment terms added to each outcome model have two convenient properties. First, when propensity scores are correctly specified, the adjustment term has expectation 0 over the sum of observations i. Second, the adjustment term stabilizes the estimator when estimated propensity scores are close to 1 or 0, curing one undesirable property of the simple IPW estimator (see Glynn and Quinn 2010, for formal proof).

In our survey data, self-selection of households into scheme adoption is likely. Households that use a scheme for conducting conservation works potentially differ in important characteristics from households that do not use a scheme. We use the double-robust IPW estimator to recover the causal effect of scheme adoption on the four outcomes of interest. Our dataset offers a wide range of observed variables, covering household and housing characteristics, information on the conservation work conducted and other schemes used in parallel. Assuming conditional independence, we can model the probability of adopting a scheme based on observed variables.

We specify the treatment model by including all variables that potentially affect the decision to adopt scheme A. We employ logit models to estimate four variants of the propensity score for household \(i\) to adopt scheme \(A\):

$$\begin{aligned}&~P(Scheme_A~adoption=1|H_i)=\beta _0+\beta _1H_i \end{aligned}$$
(I)
$$\begin{aligned}&~P(Scheme_A~adoption=1|H_i,W_i)=\beta _0+\beta _1H_i+\beta _2W_i \end{aligned}$$
(II)
$$\begin{aligned}&~P(Scheme_A~adoption=1|H_i,W_i, MT_i)=\beta _0+\beta _1H_i+\beta _2W_i + \beta _3MT_i \end{aligned}$$
(III)
$$\begin{aligned}&~P(Scheme_A~adoption=1|H_i,W_i, S_i)=\beta _0+\beta _1H_i+\beta _2W_i + \beta _3S_i \end{aligned}$$
(IV)

where H is a vector of household and housing characteristics. W is a vector including dummies for each type of work conducted and the variable specifying the year when the works were finalized. MT is a vector of dummies indicating the motivation for and the trigger of the works, and S is a set of dummies for the adoption of national, regional and local schemes. We specify the variables included in each vector in appendix B.

The choice of variables included in models (I) to (IV) is guided by going from the most parsimonious specification with exogenous variables determined strictly before the adoption of schemes and the implementation of renovation works to richer specifications that additionally take into account the motivations and triggers for the works as reported by the households and parallel scheme adoption. The basic specification (I) of the treatment model includes all important household and housing characteristics that can affect the decision of households to adopt scheme A. Specification (II) adds the types of work conducted and specification (III) adds the motivations and the triggers for conducting the works. Specification (IV) accounts for potential systematic patterns in the parallel adoption of national, regional and local schemes that may affect the outcome variables. We thereby test if the estimation results are robust to different specifications of the propensity score and whether the simultaneity of adoption of different schemes biases the results. We find the estimates to be robust across the four schemes and the four outcome variables.

We specify the outcome models to entail a comprehensive set of exogenous covariates controlling for household and housing characteristics, and the types of work conducted.Footnote 20 We define the outcome model for the treated \(m_0\) and the outcome model for the controls \(m_1\):

$$\begin{aligned} \begin{aligned} m_0(T=0,H_i,W_i)&=\alpha _1+\alpha _2~H_i+\alpha _3~W_i+\epsilon _i \\ m_1(T=1,H_i,W_ii)&=\alpha _1+\alpha _2~H_i+\alpha _3~W_i+\epsilon _i \\ \end{aligned} \end{aligned}$$

For each of the four outcome variables (funding received, the reduction in energy expenses, and private and total investment) the outcome models are estimated with each of the four treatment model specifications in the double-robust IPW estimator, resulting in four estimates for each outcome for each scheme. We bootstrap standard errors (using 50 repetitions) as recommended in the literature (Austin 2016; Bodory et al. 2020; Huber 2013). We adjust the significance level of the coefficients of our main specifications in Tables 1, 2, 3 and 4 and Tables 8 and 9 for multiple hypothesis testing, using the sharpened False Discovery Rate q-values approach.

5 Results and Discussion

In Sect. 5.1, we present the estimation results of the impact of scheme adoption on the acquired funding, the private and total investment and the reduction of energy expenses. In Sect. 5.2, we use the estimates to construct and compute metrics of the cost-effectiveness, leverage effect for private investment, additionality and redistribution. This allows us to compare the schemes’ performance along different dimensions and discuss the advantages and drawbacks of each of them.

5.1 Funding Acquired, Private and Total Investment, Reduction of Energy Expenses

This section presents the estimation results of the impact of scheme adoption on the reduction of energy expenses, the private and total amount invested and the acquired funding. The use of the double-robust IPW estimator addresses potential endogeneity concerns and corrects the self-selection bias. For each scheme, the analysis compares households that conducted conservation works using funding from the scheme with households that did not use this scheme.

Funding

We estimate the effect of adopting each scheme on the acquired funding. The acquired funding is defined as the amount of funding that household i receives from all adopted schemes in €. The treatment effect of scheme adoption on the amount of funding received is estimated in a separate regression for each scheme. In Table 1, we present a compiled overview of the estimations of the coefficient of interest for all regressions that we conducted (four specifications estimated for each of the four scheme types, as explained in sect. 4.2).Footnote 21

Table 1 Impact of scheme adoption on funding

The effect of adoption is positive and significant for all schemes and across all specifications. The effect of the grant scheme is between € 2,505 and € 2,777. In comparison with the three other schemes, it provides the largest amount of funding as it provides a full grant. Adoption of the income tax credit induces funding between € 987 and € 1,100, and the White Certificates scheme induces funding between € 836 and € 1,199, both in a lower range than the funding provided by the grant scheme. The amount of funding received from the income tax credit depends on the total amount invested as it determines the deductions from the taxable income and whether the marginal rate for the household is reduced due to the deduction. The funding received from the White Certificates scheme comes in the form of small grants, e.g. for equipment and installation. The VAT reduction provides the smallest amount of funding: the estimated coefficient is between € 567 and € 671. The funding received from the VAT reduction is a direct percentage of the total investment. An important difference between the White Certificates scheme and the other ones is that the White Certificates scheme uses mostly private rather than public funds. We discuss this further in the assessment of the cost-effectiveness below.

Private Investment

We estimate the treatment effect of adopting each scheme on private investment. This quantifies the increase in private investment induced by the scheme adoption. We apply a log transformation to the outcome variable private investment to take into account observations corresponding to very large investments in the sample. The estimated coefficients can then be interpreted as percentage increases in private investment due to scheme adoption. The treatment effect of each scheme is estimated in a separate regression. We compile the results and report them in Table 2.

Table 2 Impact of scheme adoption on private investment

The effect of adopting the VAT reduction, the income tax credit and the White Certificates scheme is positive and significant across all specifications. Adopting the VAT reduction increases private investment by between 36.6 and 39.2%.Footnote 22 The effect of the other schemes is smaller. Adopting the income tax credit induces between 20.8 and 28.7% additional private investment and the White Certificates scheme induces between 17.0% and 20.6% . For the grant scheme, the effect of adoption is not significant from 0 whatever the specification. A possible explanation could be that the ANAH scheme targets lower-income households that do not necessarily have the capacity to increase their investment much.

Total Investment

The total investment variable is the sum of the funding acquired and the additional private investment. We estimate the treatment effect of adopting each scheme on total investment. This estimation quantifies the impact of adoption on the combined increases in private investment and funding. While the impact on private investment informs about the leverage effect of the scheme to induce additional private investment, the effect on total investment indicates whether the scheme adoption increases the total amount invested or whether the funding induces a windfall gain to the recipient.

We apply a log transformation to the total investment variable to take into account observations corresponding to very large total investments in the sample. The estimated coefficients can then be interpreted as percentage increases in the total amount invested. The treatment effect of each scheme is estimated in a separate regression. We compile the results and report them in Table 3.

Table 3 Impact of scheme adoption on total investment

The effect of adoption on total investment is positive and significant for all schemes and robust across specifications. The largest increase in total investment is induced by the adoption of the VAT reduction, followed by the grant scheme. The White Certificates scheme induces less additional investment, and the income tax credit provides the lowest increase in investment. Adopting the VAT reduction increases total investment by 48.7 to 52.8%, a large increase that seems to be driven by a high increase in private investment while the induced funding for the scheme is low. The grant scheme increases total investment by between 41.8 and 49.5%, explained by the highest amount of induced funding while private investment does not increase with its adoption. Using the White Certificates increases total investment by 39.5 to 45.9%. The income tax credit increases total investment by between 35.4 and 44.2%.Footnote 23 None of the four schemes analyzed appears to induce pure windfall gains to recipient households as all schemes increase the total amount invested.

Reduction of Energy Expenses

We estimate the impact of adopting each scheme on the reduction of energy expenses. We recode the categorical variable that provides a qualitative measure of the monetary reduction in energy expenses into a binary variable: the outcome is set to 1 for households that report a significant reduction in energy expenses after the renovation works and to 0 for households that report “a bit”, “not so much” or “not at all” for the reduction in energy expenses (summary statistics for the variable are shown in Table 5). We estimate a linear probability model that provides a coefficient for the increase in the rate of households that report a significant reduction in energy expenses after renovation works due to the adoption of each scheme.Footnote 24 The treatment effect of each scheme is estimated in a separate regression. We compile the results and report them in Table 4.

Table 4 Impact of scheme adoption on the reduction of energy expenses

The effect of adoption is positive and significant for the four schemes across all specifications. Coefficients for the grant scheme are the highest, ranging between 0.137 and 0.195, corresponding to an increase of between 14 and 20 percentage points in the rate of households that experience a significant reduction in their energy expenses after renovation works. Lower-income households that are targeted by the grant scheme tend to live in smaller dwellings so that the same investment may improve energy efficiency more than in larger dwellings. The White Certificates scheme induces a more moderate increase of 9 percentage points. The increase induced by the income tax credit is 5 percentage points lower, and the VAT reduction has the smallest effect on the reduction rate, between 2 and 4 percentage points.Footnote 25

Our estimates indicate which schemes induce most funding, private and total investment as well as which schemes induce a significant reduction in energy expenses. We can however not directly compare the performance of the schemes using these results. We need to account for the facts that the schemes involve different amounts of funding, that they induce different amounts of additional private investment and that they are adopted by different groups of households. For instance, the same increase in the rate of households that significantly reduce their energy expenses may be driven by a lower amount of funding for one scheme than for another, so that the cost-effectiveness differs widely. Also, additional private investment induced by each scheme should be related to the amount of funding provided by the scheme to be able to compare the leverage effect per unit of received funding. We hence construct four metrics by means of which we compare the schemes according to their cost-effectiveness, their leverage effect for private investment, the additionality of investment via the induced funding and the redistribution between higher- and lower-income households.

5.2 Cost-Effectiveness, Leverage Effect, Additionality and Redistribution

We construct metrics of the cost-effectiveness of funds, the ability of the received funding to trigger additional private investment, the additionality of investment via the induced funding, and the redistribution involved by each scheme.

Cost-Effectiveness of Funds

We compute a metric to characterize, for each scheme, the cost-effectiveness of the funding received to reduce the energy expenses. It takes into account the funding received by the households and the increase in the rate of households that experience a significant reduction in energy expenses induced by the scheme.Footnote 26 We define the cost-effectiveness \(\Phi\) of the received funds to reduce energy expenses under scheme \(A\) by normalizing the estimates for the reduction of energy expenses with the estimates for the funding acquired:

$$\begin{aligned} \Phi =\frac{{\hat{\beta }}_A^{~energy~expenses~reduction}}{{\hat{\beta }}_A^{~funding~received}}*10\,000 \end{aligned}$$

This allows us to compare the four schemes according to their relative capacity to reduce energy expenses per €   of funding received.Footnote 27 The larger the ratio, the more cost-effective the received funds are in promoting energy conservation. The computation results are presented in Table 5.

Table 5 Cost-effectiveness of received funds

We find the White Certificates scheme to be the most cost-effective to reduce energy expenses: the average amount of funding received via this scheme is moderate, but the increase in the rate of households that reduce their energy expenses significantly is substantial. The income tax credit ranges below the White Certificates scheme due to the smaller increase in the reduction in energy expenses but higher funding that the schemes induces. The cost-effectiveness ranges associated with the grant scheme and VAT reduction overlap with both the lower bound of the range for the White Certificates and the upper bound of the range for the income tax credit. With a high reduction in energy expenses but also high funding for the grant scheme and a low reduction in energy expenses but also low funding, both schemes’ cost-effectiveness is moderate.Footnote 28

Despite criticisms of the White Certificates,Footnote 29 we find this scheme to be most cost effective.Footnote 30 Our result may be explained by the incentive energy suppliers have to target households whose dwellings have a high energy saving potential. Energy suppliers indeed have to purchase certificates if they do not comply with the obligation that they themselves have to save a specific amount of energy. They are hence proactive to inform customers about potential energy savings for their dwelling. For instance, some energy suppliers send customers a thermal image of their house to let them visualize the deficiencies in their dwelling’s thermal insulation. Similar to a small audit, this approach may specifically encourage a group of customers that otherwise would not be attentive to the energy saving potential for their dwelling and that have not yet invested in energy efficiency of their dwellings.Footnote 31 In addition, not only is the funding spent targeted to lower cost energy savings opportunities, but also it uses mostly private funds. This reinforces even more the economic efficiency argument in favor of this scheme.

Leverage Effect as Ability to Induce Private Investment

To know which scheme induces the strongest increase in additional private investment per euro of funding, we divide the estimates for private investment by the estimates for the acquired funding. The larger this metric \(\Lambda\), the higher the relative increase in private investment the scheme A is able to induce per euro of funding.

$$\begin{aligned} \Lambda _A=\frac{{\hat{\beta }}_A^{~private~investment}}{{\hat{\beta }}_A^{~funding~received}}*10\,000 \end{aligned}$$

The results of our computations are presented in Table 6. We find that the leverage effect is highest for the VAT reduction. It induces the strongest increase in additional private investment per euro of funding, as the average funding received is the lowest in comparison with the other schemes and the increase in private investment is the strongest. The leverage effect for the income tax credit is lower, since this scheme only induces moderate amounts of additional private investment but higher funding. The White Certificates scheme has an even lower leverage effect as the relative increase in private investment is smaller while the funding is moderate. The grant scheme does not induce significant amounts of additional private investment; its leverage effect is 0.

Table 6 Leverage effect for private investment

Additionality of Investment

We characterize additionality by the relative increase in the amount invested beyond what would have been invested without adoption of a given scheme. The metric we employ is the estimates of the increase in total investment over the estimates of funding acquired for a scheme A as presented below. The larger this ratio \(\Gamma\), the more capable the scheme is to induce additional investment for each €  of funding:

$$\begin{aligned} \Gamma _A=\frac{{\hat{\beta }}_A^{~total~investment}}{{\hat{\beta }}_A^{~funding~received}}*10\,000 \end{aligned}$$

The results of this computation are displayed in Table 7. Additional total investment per €  of funding is the highest one for the VAT reduction: this scheme induces only small amounts of funding but induces the highest increase in total investment jointly with the grant scheme. The VAT reduction is followed by the income tax credit on par with the White Certificates.Footnote 32 The grant scheme induces least additional total investment per euro of funding: Its adoption increases total investment substantially, but the scheme also induces the largest amount of funding.

Table 7 Additionality of investment

Redistribution

Studies have found substantial heterogeneity in energy efficiency gaps across households so that schemes targeted at specific households (for instance, low-income households or households that have not yet participated in another program) can potentially generate larger welfare gains than general schemes (Allcott and Greenstone 2012). To assess whether the effect of the four schemes is heterogeneous across income groups, we compare the funding received and the reduction in energy expenses induced by each scheme for lower- and higher-income households separately. We divide our sample at the income median and compare redistribution to the households above and below this median. As a robustness check, we also compare the 25% richest and the 25% poorest households. To compute the welfare impact of each scheme, we need to consider not only the funding received but also the reduction in energy expenses induced by the schemes by income groups.Footnote 33 We first look at the funding received by income groups.

Using the double-robust IPW estimator as for the estimations on the whole sample in section 5.1, we estimate the effect of scheme adoption on funding received for the split samples of households with a net income below the median, €30,700, (lower-income households), and households with a net income equal or higher than €30,700 (higher-income households) for each of the four schemes.Footnote 34 The results for each of the subsamples of lower- and higher-income households are reported in Table 8. On average, the grant, income tax credit and White Certificates scheme induce slightly less funding to lower-income households than to higher-income households, while the VAT reduction induces slightly more funding to lower-income households. The absolute difference is largest for the income tax credit and smallest for the White Certificates scheme. However, we find that the estimated coefficients for the lower- and higher-income households do not significantly differ from each other for neither of the four schemes.Footnote 35 This finding also holds true when comparing the funding received by the 25% poorest and richest households (see Appendix Table 25).

Table 8 Funding received by the lower- and higher-income households

Additionally, we estimate the reduction in energy expenses induced by each scheme separately for the lower- and higher-income households (see Table 9).Footnote 36 We find that the increase in the rate of households that report a significant reduction in energy expenses is higher for the higher-income households for the grant scheme - by about 40%, and the income tax credit - by more than double. For the White Certificates scheme, the increase in the rate is very similar for higher- and lower-income households. The VAT reduction is the only scheme for which the rate of significant reduction increases much more in the lower-income households - by about four times as much - and the increase is not significant for the higher-income households. We check the significance of the difference between lower- and higher-income households as we did for the coefficients on the funding received. Again, we find the difference between the coefficients to be statistically insignificant at conventional levels. This finding also holds true when comparing the reduction in energy expenses for the 25% richest and poorest households (see Appendix Table 26).

Table 9 Reduction in energy expenses by lower- and higher-income households

We conclude that all four schemes are neither progressive nor regressive in the sense that the welfare effect taking into account the funding received and the reduction in energy expenses does not differ significantly between higher- and lower-income households.

Trade-Offs Between Schemes

The metrics we defined to characterize the cost-effectiveness, additionality, as well as the leverage and redistribution effects of each scheme are interdependent and trade-offs arise between these policy targets. No scheme is found to perform best along all dimensions. The White Certificates scheme is the most cost-effective and, in addition, it uses mostly private rather than public funds. However, it is average in terms of additionality and leverage. The VAT reduction ranks average in terms of cost-effectiveness but performs best in additionality and leverage. The income tax credit ranks low in terms of cost-effectiveness and performs average for leverage and additionality. The grant scheme performs average for cost-effectiveness but it does poorly in terms of leverage and additionality. Noticeably, none of the four schemes distributes significantly more funding to either lower- or higher-income households and none significantly increases the rate of poorer households with a significant reduction in energy expenses more than the corresponding rate for richer ones. This means none of the schemes can be classified as either progressive or regressive. In summary, if the policy focus is on cost-effectiveness and redistribution, we would recommend the White Certificates scheme as it allows households to achieve more energy savings for a given amount of funding received, in addition to using private rather than public funds.

6 Conclusion

The residential sector plays a pivotal part in efforts to reduce energy consumption and GHG emissions. At present, national governments offer a diverse spectrum of financial support schemes to encourage energy conservation works in the residential sector. Given the current public budget constraints, in particular following the COVID pandemic, the cost-effectiveness and the redistribution involved by these schemes need to be assessed.

We compare the performance of the four following types of financial support schemes available in France: a VAT reduction, an income tax credit, a grant scheme and the White Certificates scheme. To account for self-selection into adoption of financial support schemes and address potential endogeneity concerns, our analysis employs double-robust IPW estimators.Footnote 37 We use the TREMI2017 survey data from the French Environment and Energy Agency (ADEME). This dataset covers close to 14,000 households that conduct energy conservation works and reports information on the financial support schemes they potentially used. We estimate the effect of the adoption of each aid on the reduction of energy expenses, the amount invested and the acquired funding. We use these estimates to assess the cost-effectiveness, additionality, redistribution and ability to trigger private investment.

We find that the cost-effectiveness of public funds received is the highest with the White Certificates. This might be due to the active approach by energy suppliers (for example via nudging) to target households for whom the energy saving potential is the highest. Energy providers are indeed incentivized to fund high-impact works because they have an obligation to save a specific amount of energy. The cost-effectiveness is more moderate for the grant scheme and the VAT reduction. The income tax credit is least cost-effective. These other schemes require more proactivity from households than the White Certificates. An additional advantage of the White Certificates with regards to public budget constraints is that it uses mostly private rather than public funds.

Additional private investment is triggered most via the VAT reduction, followed by the income tax credit and the White Certificates scheme, whereas the grant scheme does not encourage additional private investment. All four national schemes increase the total amount invested by households so that none of the schemes induces pure windfall gains. Total investment increases most with the VAT reduction, and moderately with the income tax credit and the White Certificates. The grant scheme induces least additional investment. Concerning redistribution, households in the upper and lower half of the income distribution equally benefit from each of the four schemes in terms of funding received and the reduction in energy expenses.

To conclude, if the criteria for selecting a scheme are focused on cost-effectiveness and redistribution, the White Certificates seem to be the scheme policymakers should develop, potentially taking into account improvement suggestions made to address criticisms.Footnote 38 The White Certificates are indeed found to be more cost-effective without being detrimental in terms of redistribution.