The Effects of Home Energy Efficiency Upgrades on Social Housing Tenants: Evidence from Ireland

This research examines the impact of a home energy efficiency upgrade programme on social housing tenants. Employing a quasi-experimental approach we examine a range of self-reported and objectively measured outcomes, including metered gas consumption, for a control and upgrade group, before and after the upgrade. We draw our sample from a large home energy efficiency programme in Ireland, The SEAI Better Energy Communities Scheme, which provides funding for whole communities to upgrade the efficiency of their dwellings. Dwellings are selected for upgrade based on need, allowing us to control for observable dwelling characteristics correlated with selection into the trial. The upgrades undertaken are extensive relative to the average home energy improvement in Ireland, with many dwellings receiving a number of measures, such as cavity wall, external wall and attic insulation; new boilers; replacement doors and windows; CFL lighting and heating controls. Households report improvements across a range of outcomes associated with heating-related deprivation and comfort in the home. Panel regression models examine the elasticity of gas demand with respect to the thermal efficiency of the dwellings. Overall, we find that use of natural gas falls much less than 1:1 for each increment to thermal efficiency of the home. For the average household in this study, about half of a marginal increase in thermal efficiency is reflected in reduced gas demand. *Corresponding Author: daire.mccoy@esri.ie


Introduction
Many governments subsidise residential energy efficiency upgrades for vulnerable households. The objectives motivating these policies include helping reduce carbon emissions from domestic heating, improving public health and assisting segments of the population that suffer from poverty and deprivation. In this paper we examine the effects of one channel of support to such households: funding upgrades to social housing.
The socio-economic characteristics of social housing residents, who often rent rather than owning dwellings and receive relatively low incomes, mean they are less likely to invest in energy saving measures than the general population. Improving the thermal efficiency and reducing carbon emissions of such residences is likely to require some intervention by the state. Because social housing residents do not generally invest in dwelling upgrades in the absence of intervention, measures directed at such households are less likely than other energy efficiency subsidies to generate deadweight losses.
Access to social housing is usually means tested, so these households are likely to suffer from high rates of poverty and deprivation, as well as family structures associated with socioeconomic vulnerability such as single parenthood or job tenures such as unemployment. To the extent that they lead to lower energy bills, upgrades may help advance anti-poverty and related distributional objectives.
Groups that might be particularly vulnerable to temperature-related health problems, including the elderly, the very young and people with disabilities (World Health Organization, 1987;Liddell and Morris, 2010) , also tend to be concentrated among social housing tenants. In some jurisdictions, public health objectives are identified as an important reason for supporting dwelling upgrades.
Although it is well understood that support for upgrades to energy efficiency in social housing can help address objectives in all of these policy domains, less is known about which policy domain is likely to benefit most from an upgrade programme. Households that receive upgrades make choices that affect the distribution of benefits between climate policy and poverty alleviation goals (i.e. reduction in energy use and bills) and improving health and well-being (i.e. increasing thermal comfort). Put simply, a household that receives an efficiency upgrade may save money and reduce carbon emissions, or it may take advantage of the lower marginal cost of energy services by consuming more energy and becoming more comfortable and perhaps healthier. The latter response is one element of the rebound effect described in the energy policy literature (which we discuss further in the next section).
Both energy-saving and comfort-increasing responses could be welfare-improving if market failures had previously led to sub-optimal levels of investment in efficiency, so mixtures of both responses (probably typical in practice) may also yield net welfare gains. However, policymakers also care about how the benefits are shared among policy objectives. This is most obvious in the case of climate policy, where interim goals often take the form of target levels of abatement for a set of jurisdictions, sectors or activities. If a measure like upgrades for social housing delivers smaller reductions than anticipated, additional measures will need to be taken to compensate if overall targets are to be met.
In this paper we study responses to residential energy efficiency upgrades using microdata from a sample of social housing tenants in Ireland who use natural gas for heating. We ask whether affected households perceive 1 benefits from the upgrades and how their demand for natural gas changes after their dwellings are improved.
In particular, we test whether there seems to be a reduction in gas demand proportionate to the efficiency improvement or whether usage patterns are consistent with a significant rebound effect.
The research was carried out in collaboration with Respond! Housing Association, a charity providing housing services and related supports to vulnerable households across Ireland. Respond! was allocated public funding for residential energy efficiency upgrades and helped the research team to carry out surveys of tenants before and after upgrades were installed. Billing information on gas and electricity use was also collected and Respond! provided physical information on the dwellings. In addition to the households who received upgrades, information was collected on a control sample of similar households whose dwellings were not upgraded during the period.
Importantly, dwellings were selected for the upgrades on the basis of need. Households did not self-select into the trial. This means the occupants of upgraded dwellings are very similar to the occupants of dwellings not upgraded. The differences between these groups are observable and related to the characteristics of their dwellings, primarily thermal efficiency. This allows us to control for these differences in our econometric models.
We also use fixed-effects panel estimations giving us further robustness against any heterogeneity which we do not observe.
As expected, we find that beneficiaries report significant benefits from efficiency upgrades and fewer report difficulty affording adequate heating and paying their bills afterwards. Some report changes in their use of energy services. Overall, we find that use of natural gas falls much less than 1:1 for each increment to thermal efficiency of the home. For the average household in this study, about half of a marginal increase in thermal efficiency is reflected in reduced gas demand. Vulnerable groups should probably exhibit higher rebound than the general population, reflecting their relatively high income elasticities and preferences for additional thermal comfort. However, our estimate of the gap between efficiency upgrades and energy savings may also be affected by imperfections in the proxy used to represent each dwellings energy efficiency.
The rest of the paper is organised as follows: Section 2 details other research which relates to home energy efficiency, in particular with regard to socially vulnerable groups. Section 3 contains a discussion of the data available and the econometric modelling approach used in this paper. This is followed by the discussion of the results in Section 4, while Section 5 draws some final remarks and policy implications of this research.

Related literature
Researchers have devoted considerable attention to defining and measuring rebound effects, which Sorrell et al. (2009) define as "any increase in energy service consumption [that] will reduce the 'energy savings' achieved by the energy efficiency upgrade". They also highlight three overlapping concepts of rebound effects, including shortfall (difference between engineering-estimated energy consumption savings and actual consumption), temperature take-back (reduction in energy savings associated with change in mean internal temperature after energy retrofit) and behavioural change (reduction in estimated energy savings associated with the change in heating controls or other user-related behaviour).
Empirical estimates of rebound vary widely, partly because authors may be focusing on different mechanisms or ways of measuring the effects, but also because the strength of the effect may depend upon the socioeconomic and policy context. For example, Sorrell et al. (2009) cite nine econometric studies and twelve quasi-experimental studies of rebound in household heating and found that while average long run direct rebound effects are probably lower than 30%, individual studies report effects ranging from 0 to 100%.
Similar results are set out in Sanders and Phillipson (2006), who also survey a set of energy efficiency studies and highlight the confusion surrounding the definition of 'rebound effect' and the common discrepancy between predicted energy savings (from engineering-based models) and actual energy savings. They find a typical shortfall (which they term the 'reduction factor') of about 50% between the predicted savings and actual savings, with the comfort factor (the portion of the reduction factor associated with temperature take-back) roughly 15% of the entire reduction factor.
Looking specifically at Ireland, from which we draw the data for this study, Scheer et al. (2013) study the rebound effect of an energy efficiency upgrade on household gas consumption. Using ex-post examination of billing data, they find a shortfall of approximately 36%, estimated as the difference between the predicted engineering model-based consumption change and the actual change. Although these findings are not necessarily applicable to the broader population, respondents did report other benefits of the energy efficiency upgrade, ranging from improved well-being, home comfort and an increase in the perceived value of their home. The authors acknowledge that selection bias is a potential problem when using data from an upgrade programme in which beneficiaries have to opt-in and were required to contribute to the cost of the energy efficiency upgrade.
Research looking specifically at low income households has found they tend to take a higher share of the benefits from efficiency upgrades in the form of improved thermal comfort, or temperature take-back, than better-off households do. Milne and Boardman (2000) review 13 studies that examined fuel consumption before and after energy efficiency upgrades for homes designated as being in fuel poverty. They find a comfort factor of 30-50%.
They also show that the comfort factor is a function of mean internal temperature and that houses with lower initial mean internal temperature (often those with lower incomes) are more likely to have higher comfort factors.
This finding is consistent with the idea that households seek to achieve a target profile of internal temperatures and that as temperatures move towards this profile, the comfort factor for additional upgrades decreases. If this is so, households who have difficulty paying their heating bills may behave differently from those without income constraints that bind as tightly and may have very different rebound effects from the average household. Milne and Boardman (2000) find that for low income households the lower average internal temperatures result in up to half of the predicted energy saving being achieved, with the other half devoted to increased comfort in the house. Other research shows that rebound is inversely related to household income in Australia (Murray, 2013) and the United States (Thomas and Azevedo, 2013).
Some studies have even suggested that rebound can be larger than 100%, i.e. some types of households use more energy after an efficiency upgrade than before. Heyman et al. (2011) conduct a randomised controlled trial to examine the effect of energy efficiency upgrades on households in fuel poverty. They find that treatment homes (who receive a retrofit one year before the control group) tend to increase their energy consumption. However, the authors acknowledge that the results may have been subject to bias due to sample attrition over the four year period of surveying. The research period involved four surveys over four years, with upgrades in either the 3 third (treatment) or fourth (control) year. Ultimately, Heyman et al. (2011) favour retrofit programmes as they "generate modest but long-lasting fuel efficiency gains which translate into increased room temperatures rather than financial savings, a sign of the importance which people with limited resources place on staying warm" (Heyman et al., 2011). Hong et al. (2006) also find that households can increase their fuel consumption after an upgrade.
In general, the literature measuring the direct rebound effect in home heating is sparse. Research is constrained by the difficulty in obtaining reliable data. We agree with Sanders and Phillipson (2006), Sorrell et al. (2009) and others that a randomised controlled trial is the preferred method, with fewer sources of error than experiments without control groups or econometric estimates using household survey data. However, given the cost of installing upgrades in addition to carrying out the research itself, it is seldom possible for researchers to obtain the funding to do a large scale randomised controlled trial.
In this study, we take a quasi-experimental approach dictated by the nature of the data we could obtain. Sorrell et al. (2009) note how the estimation of a direct rebound effect through a quasi-experimental approach requires a counterfactual satisfying two necessary conditions. First, one needs data of the energy consumption that would occur without the energy efficiency upgrade. This may be approximated by including a control group in the analysis, as in the early example of a randomised controlled trial on the effects of efficiency upgrades carried out by Hirst et al. (1985). Second, one requires an estimate of the energy consumption that would have occurred following an energy efficiency upgrade but with no behavioural change. For example, Hong et al. (2006) focuses on the comparison of theoretical predicted energy savings from engineering-based models with actual energy savings in a study of participants of the UK Warm Front scheme and finds large differences. Dowson et al. (2012) studies the thermal performance of the UK housing stock after a government scheme targeted at improving energy efficiency. They find that the estimated energy savings of retrofits as part of the forthcoming policy may "only be half as effective as anticipated due to a lack of monitoring, poor quality installation and the increased use of heating following refurbishment" (Dowson et al., 2012).
By selecting a control group of households who have broadly similar characteristics to our treatment group but which did not receive an efficiency upgrade in the sample period we hope to address the first requirement. We use a proxy for each residences thermal efficiency informed by engineering-based models to try to address the second.

Research design
This research adopts a quasi-experimental approach by studying the effects of domestic energy efficiency upgrades on a sample of social housing tenants in Ireland. The study aims to identify trends in residential heating energy demand for two groups of social housing tenants -those who received a home energy efficiency upgrade (the upgrade or treatment group) and those who did not (the control group). In 2011, 9% of private dwellings in Ireland were rented from a local authority or voluntary body (CSO, 2011).
The Sustainable Energy Authority of Ireland's (SEAI) Better Energy Communities Scheme provides funding for community-based home energy efficiency improvements. Community groups submit a proposal to SEAI to upgrade their housing stock. SEAI evaluate the bid and decide which dwellings to upgrade. SEAI then co-fund the upgrade with the relevant community group. This scheme was launched in 2013, and by 2016 had supported over 12,000 homes, community, private and public buildings in receiving energy efficiency upgrades.
In 2014 Respond! Housing Association received approval from SEAI to undertake home energy improvements in a number of its housing estates throughout Ireland. Households did not self-select into the trial but were chosen by Respond! and SEAI based on the characteristics of the dwellings. Dwellings were identified by Respond! Housing Association as being in need of energy efficiency improvements. SEAI allocated support on the basis of an application made by Respond! Amarach research was contracted to undertake a door-to-door pre-upgrade survey, prepared by the authors, on the households in June-July 2014. Households were asked a range of questions related to factors such as family composition, income, self-reported fuel poverty and heating-related problems. Houses in the control group also completed surveys to allow before and after comparisons between groups. As part of the survey the household manger was asked to sign a data access agreement, allowing the authors access to their electricity and gas consumption over a three year period. At the time of completing the survey, households in the upgrade group would have been known about the proposed upgrades in the upcoming months.
The upgrades were completed over the following few months, and a post-upgrade survey was completed in October-November of the following year. The time-line for this project is displayed in Figure 1.
The final sample contains 260 households who completed both waves of the survey, 164 of which received home energy efficiency upgrades, 96 of which did not. This total sample contains two subgroups: a group which consists of 210 households with signed electricity billing access agreements; another group of 100 households who provided signed gas billing access agreements.
The focus of this paper is on how the upgrades affected home heating and other related factors. Given this, we focus on the total sample of 260 homes, and the gas sub-sample of 100 homes. The sample of dwellings for which we have details of electricity use will be the focus of future research.

Data
As discussed in the previous section respondents in both the upgrade and control groups completed pre-and post-upgrade surveys. Respondents also signed a waiver allowing the authors access to their gas and electricity consumption. ESB Networks provided metered electricity consumption data and Gas Networks Ireland provided metered gas consumption data. Data on the dwellings, including dwelling characteristics, location, and information on the type of upgrade and when they were completed were obtained from Respond! Housing Association.
Weather data was downloaded from The Irish Meteorological Service, Met Eireann's website.

Dependent variable: Gas use
Gas consumption was provided to the research team by Gas Networks Ireland, the Irish gas network operator.
Households signed a waiver allowing access to their gas consumption. In most cases the period covered was from Jan 2013 -Dec 2015, a three year period (n=65). In some cases we could only get access to a two year billing period, from Dec 2014 -Dec 2015 (n=35). Also, some houses had new gas boilers installed as a replacement for their previous heating system (n=8). For these dwellings we don't have consumption data prior to their upgrade, however we include them in the analysis as an unbalanced panel.
The initial cleaning and smoothing of the raw gas data is described in the annexes to this paper. In addition to the smoothing described, we also run sensitivity checks using a three-period moving average for consumption.
We use a bimonthly billing cycle, with 6 billing periods in each calendar year.

Thermal efficiency of dwellings: BERpred
Our proxy for the thermal efficiency of each dwelling in the sample is based on its Building Energy Rating (BER). Established by the Sustainable Energy Authority of Ireland, this engineering-based metric is based on a bottom-up model of factors affecting thermal efficiency. The model predicts the average energy required to heat each dwelling to a specified standard given its physical characteristics. The BER values relating to the residences in this study were provided to us by Respond! Housing Association.
We take the raw BER score, which is in units of kWh/square meter/year, and scale it to match the units in our dependent variable by multiplying it by the area of each dwelling and dividing by six (to convert it to a two-monthly basis, matching the gas billing period). This figure, which we refer to as BERpred, gives the average gas demand expected in a given billing period for a particular residence assuming the residence uses only gas for heating. If a residence received an efficiency upgrade during the sample period, we reflected this by reducing the BERpred accordingly. Figure 2 compares the BERpred values for our sample to actual average consumption. Consumption values are lower than predicted levels in almost all cases. Moreover, the gap tends to widen at higher level of predicted energy consumption (i.e. for lower efficiency houses). Particularly given that these households are social housing tenants, it is likely that limited income constrains energy consumption. Some households may not be able to afford to maintain the level of thermal comfort assumed by engineering-based models, particularly those living in dwellings that are inefficient and thus relatively expensive to heat. Use of secondary fuels may also help explain the divergence for some households. Actual average gas use Expected gas use based on BER

Weather conditions
Daily weather data is taken from Met Eireann website 1 . Daily data was downloaded for a number of weather stations located around the country. These were then assigned to the nearest housing estate in our data, using GIS software.
The variables we include are sunlight hours, rainfall (mm), windspeed (knots) and heating degree days. Sunlight hours is a measure of the duration of sunshine in a day. Daily rainfall is measured in millimetres; the daily mean windspeed is measured in knots (equivalent to 1.852km/hr). Heating degree days is calculated from the average daily outside air temperature. It is defined relative to a base temperature of 15.5 degrees celsius, above which it is assumed a building needs no heating. If the average daily temperature is 1 degree below this, it is referred to as one heating degree day.
As our gas billing cycle is bimonthly, we aggregate the weather data to reflect the average conditions in a given period.

Socioeconomic characteristics
Earlier in the paper we noted that socioeconomic characteristics can have a significant effect upon a household's energy use and response to energy efficiency upgrades. To control for such effects we include a range of socioeconomic variables in our models. In this sub-section we list them and briefly explain their expected effects on residential heating demand.
Income should have a positive effect on energy usage, as shown in previous research (Brounen et al., 2012).
However given the limited degree of cross-sectional variation in income in our data we may not observe a statistically significant effect.
Older households have been found to spend a significant proportion of their income on space heating, and demand is also found to increase with age (Liao and Chang, 2002). In a large randomised-controlled gas smartmetering trial Harold et al. (2015) find that households with older chief economic supporters (CES) consume more gas and younger households consume less relative to a reference category of 36 to 45 years. However, the magnitude and significance of these effects are reduced once dwelling characteristics relating to energy efficiency are included. This reflects the propensity of older people to live in lower quality dwellings, on average, than their younger counterparts.
The number of occupants is positively correlated with gas consumption (Harold et al., 2015), however scale economies have also been observed, and each additional person decreases the per-capita consumption by 26% (Brounen et al., 2012).
Only 15% of our total sample are in full-time employment (18% for the gas sample). A large proportion (61% of total, 57% of gas sample) describe their employment status as unemployed, retired, suffering from illness or disability or home duties. One might expect these groups to use more energy, on average, than others spending more time outside the home. This also highlights the importance of being able to heat the house properly for these vulnerable groups.
Fuel Allowance is a case payment under the Irish National Fuel Scheme to help with the cost of home heating during the winter months. It is paid to people who are dependent on long-term social welfare payments and who are unable to provide for their own heating needs. 60% of households in our gas sample receive this benefit. It is unclear whether receipt of this benefit will increase gas consumption. On the one hand it might enable income constrained households to more adequately heat their homes, increasing consumption. On the other hand, these households are likely to be more vulnerable to poverty and deprivation generally (Watson and Maitre, 2015), and additional cash may be used to meet other needs. We compare our sample across gender, education and income with the population of social housing inhab-itants from the Central Statistics Office (CSO) Household Budget Survey in Table A2. We observe a higher proportion of female respondents in our sample; both groups have very similar levels of education; levels of income are lower on average in our sample, with a much higher proportion earning A C20,000 or less per annum.
It most be noted that the most recent HBS for which we could obtain data was conducted in 2009/2010, our survey was conducted in 2014. Table F1 in the Annex displays the results of a binary regression model examining the probability of being in the upgrade group for the gas dwellings. These results indicate that males and unemployed CES are more likely to be in the upgrade group. No other socioeconomic coefficients are significant.
Our average family features between two and three people with a chief economic supporter who is over 55 years of age, with a leaving certificate (upper secondary) level of education, and who is unemployed.

Dwelling characteristics
Structural dwelling characteristics have been found to influence space heating demands more so than factors related to occupancy and the socioeconomic characteristics of inhabitants (Brounen et al., 2012).
Semi-detached homes account for 72% of dwellings in our gas sample. The remainder are bungalows, apartments and terraced homes. We aggregate all other groups in the analysis and use semi-detached as the reference category. Harold et al. (2015) found that relative to households living in semi-detached dwellings, those living in apartments used less gas, while those in detached homes and bungalows used more. We should expect consumption to increase with the size of the dwelling. This is measured as the number of rooms in each dwelling.
Pre-payment meters can help households to reduce energy consumption (Faruqui et al., 2010). Many incomeconstrained households will opt for a pre-paid meter to help with managing energy bills. A concern some have is that these households might be under-heating their home due to inability to top up the meter. In our gas sample 37% of households have pre-paid meters installed. We control for meter type in our estimations in order to determine if the meter type has an impact on energy consumption.
The typical dwelling type in our sample is a three bedroom semi detached house with PVC windows and three occupants (including the respondent). Details on the dwellings can be found in Table C1 in the annex.
As per Table F1, dwellings in the upgrade group have lower energy efficiency pre-upgrade and are more likely to be semi-detached.  Table 2 highlights the combinations of upgrades undertaken and the percentage of treatment households who received them. We observe that most households receive multiple energy efficiency upgrades, with the most common combination being attic insulation, heat boiler, heating controls, CFL light bulbs and cavity wall insulation (23%).  control and upgrade group, both prior to and after the upgrade. The control group is on average more energy efficient prior to upgrade. The upgrades significantly improved the energy efficiency of the dwellings, shifting the distribution of BERs to the left of the control group. A similar pattern is observed for the sub-sample of dwellings for which we were able to obtain metered gas consumption data.

Consumption of other fuels
This section focuses on the sub-set of gas connected households, who had a metered gas connection both pre-and post-upgrade (n=86). In both surveys respondents were asked a number questions relating to their purchasing of solid and other liquid fuels, excluding metered gas. These questions enquired about the quantity and costs of their most recent purchase, the frequency of purchasing and the approximate amount purchased in the past 12 calendar months. Table 3 illustrates the proportion of gas-connected households consuming any non-zero amount of a range of other fuels. It shows that households with metered gas are consuming a range of other fuels, particularly coal. This reduces somewhat after the upgrade but a cetrain proportion continue to use other fuels along with gas. From this table it is difficult to determine the extent to which households are using these other fuels as primary or secondary heating sources. Figures 3 and 4 display the self-reported annual spending on all other fuels before and after the upgrade for dwellings with gas boilers. This data is likely subject to measurement error as it is self-reported by households, but still gives a sense of the potential magnitude of expenditure on other fuels. Many households are reporting spending a significant proportion of their total annual fuel expenditure on other fuels.  Table 4 displays percentiles of the distribution of other fuel spending for metered gas dwellings, before and after the energy upgrade. Some of these dwellings had replacement gas boilers installed (n=8). Of the households surveyed, the bottom quartile do not report any other fuel spend either before or after the upgrade. However, the top quartile in the pre-upgrade survey report spending a significant amount on other fuels. This reduces considerably after the upgrade but a small number persist with heavy usage of other fuels. This has implications for climate policy as it suggests the effect of upgrades to more efficient gas and oil boilers to reduce emissions, may be less than expected if some households continue to use coal and other dirty fuels.
We create a dummy variable for those households that report solid fuel expenditures pre and post to control for the fact that their gas consumption will not reflect their total fuel consumption. However, this will still lead to a certain degree of error in our models as it is difficult to determine the extent to which households are substituting gas for other fuels.

Time controls and interactions
We add period fixed effects to control for any unobserved seasonal trends not picked up by the weather variables.
These variables are created as dummies for each period. We also interact them with the BER variable, allowing us to observe how actual usage departs from predicted usage at different times of the year.

Analytical methods
To explore how energy efficiency upgrades affected the households in our sample, we first carry out descriptive analysis of a range of self-assessed outcomes, including the presence of housing quality problems and a subjective indicator of fuel poverty (going without heating or difficulty paying utility bills).
We then model the gas use of the subset of households in our sample that use natural gas. Panel regressions are estimated to measure how observable factors affect the use of gas by households. Our main interest is in isolating the effect of our energy efficiency proxy, BERpred, from confounding factors. Because the data are longitudinal, we can also include random and fixed effects to control for household-specific effects that do not vary over time. Seasonal factors not captured in our weather variables are addressed using dummy variables for each billing period.
We estimate the gas consumption of each household in each period (Y it ) is a function of the energy efficiency of the dwelling (BERpred it ), other dwelling characteristics (D i ), socioeconomic characteristics of household members (X i ), weather (W it ), and time dummy variables for each period (ρ t ).
Examining the elasticity of gas consumption with respect to the predicted energy consumption will allow us to determine the percentage actual energy change for each percentage predicted energy change, based on the BER of the dwelling.

Occupant's satisfaction with upgrade
After the upgrade was conducted, households were asked a range of questions relating to their level of satisfaction with the upgrade undertaken. The questions covered a range of topics, such as overall satisfaction, perception of warmth, improved awareness of energy usage, behavioural change as a result of the upgrade.
From Figure 5 it is clear that households were broadly satisfied with the upgrades, agreed that their homes felt warmer, agreed that their homes are now more pleasant places to spend time in, and didn't find the upgrade overly disruptive. Most respondents did not find the new system more difficult to operate. The level of disruption expected is widely cited as a factor which makes households less likely to engage in retrofits (when they have a choice). Given that most of these households received deep retrofits and many are likely to be at home a lot, it is encouraging that they did not generally find the upgrades to be disruptive.
Awareness of energy use seems to have increased following the upgrades, and most households agreed that post-upgrade they heat more rooms when their heating is on, but that they also use less heat due to improved control. The answers to the other questions on behavioural change were less consistent across the households surveyed. Households varied in their responses to questions about whether they had changed how often the heating is on, the time spent in the home and the likelihood of going to bed due to cold. Time spent in the home is likely to be significantly affected by socioeconomic factors other than thermal comfort. Frequency of heating system use may interact with other aspects of use; e.g. whether one is heating a single room or using central heating.

Energy affordability and self-reported heating problems
In addition to improving the energy efficiency of dwellings, one of the objectives of providing home energy retrofits is to alleviate fuel poverty. The relevant questions are in Annex G. Figure 6 shows the change in proportion of households who reported having to go without heating on a cold day over the previous 12 months, due to a lack of money. There is little difference in the control group's response before and after (about 25%), but the upgrade group report a large and statistically significant reduction in the proportion of households reporting that they went without heating through lack of money (39% to 14%).
16 Figure 6: Proportion of households who went without heating through lack of money 0 . Proportion of households Figure 7 shows the proportion of households who missed a heating or other utility bill before and after the upgrade. Initially, there is a larger proportion of control group households reporting difficulty paying utility bills in the year prior to the pre-upgrade survey (42% and 38% for the control and treatment groups respectively). In the post upgrade survey period, we observe a reduction in the proportion of all households reporting difficulty paying utility bills. Although there is a noticeable fall among the control group (42% to 26%), there is a larger change for the treatment group (38% to 18%). The reduction in reported difficulty paying utility bills for both the treatment and control groups indicate that all households in the study are able to better pay their utility bills, which may reflect improved economic circumstances over the course of the trial.
Another aim of the upgrades is to improve the quality of accommodation for tenants. Table 5 highlights the proportion of households who report heating-related problems in their home and the total number of problems reported.
In both survey periods, respondents were asked to report the presence of issues which would indicate an inadequately heated home. The most frequently reported issue is draughts, mentioned by over 70% of control households and 87% of treatment households. The results of this are presented in Figure 8 to Figure 11 with the control group in the left hand panel and the upgrade group results on the right in each case. For each heating problem, the change in the proportion of households (split out by treatment) is reflected in the difference between the pre-upgrade and post-upgrade bars. There is a general trend of both control and upgrade groups reporting an improvement across a range of issues. However the upgrade houses had more heating problems prior to upgrade than the control group, they recorded a much bigger improvement in all cases, and these improvements were generally statistically significant, unlike the improvements recorded by the control group. Inability to pay util 95% CI

Proportion of households
Having said that, we did not expect that the control group would report an improvement across all of these measures, even if not statistically significant in most cases. We are unsure as to why this might be the case. The pre-upgrade survey was conducted in Jun-July 2014. The post-upgrade survey in Oct-Nov 2015. It is possible that the timing difference exerted an unobserved effect on both groups, as external weather conditions would have been different in each period. It is also possible that improved economic conditions more generally allowed both groups to heat their home more adequately in the post-upgrade period. Another explanation is that we are witnessing a form of "Hawthorne effect", in which the responses of both groups are altered because they are being studied. This issue was raised with the housing association and they were not aware of any other external factors which might have contributed to it.
This observation highlights the importance of having a control group in a study such as this, as there may be unobserved general trends affecting both groups that would otherwise be missed by the researchers.

Analysis of gas consumption
This section will provide insights from the reduced sample of our data featuring 100 gas connected households who completed surveys before and after the upgrade period. A unique feature of this dataset is the availability of high resolution gas usage data which occupants consented to allowing the research group to obtain. The billing data (mostly) covers a three year period (January 2012-December 2015) before and after the period of the home retrofit upgrade scheme 34 Figure 12 displays the average consumption for the upgrade and control group, for a three-year period during which the upgrades were undertaken. It would appear that the upgrade dwellings consumed more gas on average than the control dwellings prior to the upgrade, and that this difference was reduced post-upgrade.

Results of regression analysis
In this section we show the results of our regression models. Three models are shown in Table 6. Model 1 is an OLS random effects model that includes a set of socio-demographic characteristics that did not change over the sample period, as well as the energy efficiency proxy variable BERpred, weather and time dummy variables. This model is tested down to exclude collectively insignificant variables (P=0.37), and the resulting parsimonious random effects model is shown as Model 2. Model 3 includes all the time-varying controls, and it is estimated using fixed effects. All the models are shown with robust standard errors, because a likelihood ratio test indicated the presence of heteroscedasticity (P=0.00). As expected, there is a statistically significant positive association between BERpred, the predicted heating requirement based on the Building Energy Rating, and households' actual gas use in each billing period. This implies that households who received efficiency upgrades tended to have lower gas use afterwards, all other things equal. The interaction terms between bimonthly time dummies and BERpred indicate that the efficiency effect was concentrated in billing periods 1, 2 and 6, which roughly equate to the autumn and winter months when most gas was consumed.
The number of heating degree days in a billing period increase gas demand, also as expected. Other weather variables are not significant after we include time dummies and time interactions with BERpred. Among these time dummies, the one denoting the first billing period of the year (January/February) indicates significantly lower average demand compared to the sixth (November/December). This may reflect weather variations during these specific years not fully captured by our set of weather parameters, or it may indicate that households consumed less thermal comfort at the coldest time of year than a linear relationship would have predicted. Rationing due to income constraints could help explain this, but we do not have a large or diverse enough sample to test this idea.
The socioeconomic controls are generally not significant. The sample used in this study has less variation across these dimensions than the national population, because social housing tenants are selected at least in part on observable characteristics. Some variables have the expected signs, e.g. number of rooms or low income status, so their lack of statistical significance may be due to the limited sample size.
The positive marginal effect of thermal efficiency is consistent across the three models. As a test of robustness we estimated a log-log version (logging the dependent variable and BERpred), a version with a three period moving average of gas demand in place of the smoothed gas demand series used in the models above, and a variant of Model 2 omitting the interactions between BERpred and time dummies. These checks yield similar estimates of the BERpred relationship to the main models. A Sargan-Hansen test (P=0.0024) suggests that the fixed effects model (Model 3) is preferred to those with random effects. Table 7 below shows the elasticity of demand with respect to BERpred as estimated in the models we estimated. The elasticities resulting from these models imply that improving the thermal efficiency of an average residence by 1 kWh reduced its gas use by about half that amount. incomes should be expected to take some of the benefits of improved energy efficiency in the form of increased thermal comfort, with the remainder feeding through into lower heating bills and carbon emissions.

Conclusions
A sizeable majority of tenants were satisfied with their efficiency upgrades and reported that their dwellings felt warmer and were more pleasant places to spend time after being upgraded. Few said they faced disruption due to the upgrades or difficulties using new heating systems. Respondents also generally felt more aware of their energy use after the upgrade, and small majorities reported that they now heat more rooms when their heating is on and use less heat due to improved control. Questions on changes in how often the heating is on, the time spent in the home and the likelihood of going to bed due to cold attracted more heterogeneous responses, probably because these behaviours depend on other aspects of lifestyle and preferences regardless of their being an upgrade.
We observed a statistically significant improvement in the self-reported proportion of households who went without heating through lack of money, which is sometimes used as a subjective indicator of fuel poverty. This improvement stands in contrast to the experience of our control group, who reported a lower level of difficulties but did not see an improvement during the study period.
Several other indicators of deprivation or housing quality showed improvement for both upgrade and control households, including the incidence of mould and draughts and the broader ability to pay utility bills. Conditions seem to have been improving generally for the social housing tenants surveyed during this period. There is some evidence that these changes were larger and more statistically significant for upgraded households, but the difference with the control group is not large.
Focusing on the sub-sample of gas-using households, our econometric results support the findings in international research that lower income households exhibit relatively high levels of shortfall (a measure of rebound) when their energy efficiency is upgraded. This is likely accompanied by relatively high temperature take-back, 26 though we could not test this directly. The estimated level of shortfall in this study is higher than that found previously for Ireland in Scheer et al. (2013), who examined a programme giving grants to households that can afford to make part of the investment themselves. One caveat is that we were not able to measure use of secondary fuels as accurately as natural gas consumption. Though many households reduced their use of secondary fuels, self-reported purchases of coal and other fuels were surprisingly high in the sample.
An inverse relationship between rebound and income might be taken to imply that environmental policy will be more effective when it is focused on better-off households (e.g. Thomas and Azevedo (2013)). However, this is true only in the narrow sense that upgrades to such households will be more effective at reducing energy use and carbon emissions. Total welfare gains from upgrades may well be as high or higher for upgrades to low income households, depending upon one's distributional preferences and on the value of the benefits associated with higher dwelling temperatures.
Of particular interest is the heavy usage of solid fuels in addition to gas central heating in our sub-sample.
Our households did not self-select into this trial, and a certain reluctance to switch heating source was noted by the housing association. This has clear implications for carbon reduction in the domestic sector. Certainly, publicly funded home energy upgrade programmes must take account of behavioural factors when upgrading heating systems.
The effect of energy efficiency upgrades on social housing tenants remains an important topic for research because this group includes many vulnerable people whose housing quality is directly amenable to policy intervention. It is important to understand the full range of effects, not just on energy use and carbon emissions but on deprivation and health outcomes. Health effects tend to take more time to emerge in a measurable way than the other benefits of upgrades, which suggests that data collection may have to take place over a longer period to measure them reliably. Another possibility is to track in-home temperatures before and after upgrades, as has been done in some studies internationally, and then to infer likely health benefits from improved temperature profiles. The falling cost of sensor technology may make this a more practical proposition for large scale studies than it was before.
Studies of social housing tenants also offer methodological advantages compared with field experiments involving other groups, not least because they give rise to less risk of self-selection bias. Although there may be sample selection involved, it is more likely to be on the basis of observable characteristics than would normally be the case for programmes where participants opt in.

Annex A Metered gas data
This section aims to provide an overview of the process involved in preparing the metered gas usage data for analysis. Metered gas usage data was provided by Gas Networks Ireland. Signed access agreements were obtained from 100 households in our sample who were connected to the gas network 5 . For most dwellings (n=65) we have access to 3 years of data (2013)(2014)(2015), for a subset we could only get access to 2 years of data (2014)(2015).
The raw gas features household level interactions between the household and the gas network operator. Each household is identified by their GPRN (Gas Point Registration Number), a unique code for each mains gas meter.
The use of GPRN codes also serves to preserve the anonymity of individuals in our data. The raw data features the date of interaction, the read type and the consumption recorded. The read type provides information on whether a customer received an estimated read of consumption, an actual read of consumption or a customer moved dwelling or switched retail provider. Figure 14 provides a graphic representation of the aggregate gas consumption of each household for each day during the sample period.
Cleaning requires a number of steps. First, some observations feature negative consumption values. These values represent corrections of household readings which were inaccurate due to measurement error (typically from overestimates of previous consumption, occasionally from incorrect meter readings). The corrections serve to balance out the consumption for each household and to ensure the accuracy of the aggregate level of consumption for each household. Any observations with extremely large negative readings (less than -8,000kWh) which could not be allocated to previous periods were removed from the sample (n = 2).

30
Another potential issue is the concern that some of the residents moved house over the course of our analysis. Based on the gas reading data, we can identify 26 unique households which have changed account holder during the sample period (identified by the red dot in Figure 14 on the date a change was registered). It is also possible that the account holder may have changed between adults in the same household, perhaps to avail of an promotional offer. Of these households, 25 reported in the post-upgrade survey that they are the same respondent as the pre-upgrade survey. Consequently we do not omit these households from our analysis.
At this point, the data are aggregated to a bimonthly billing cycle. Our sample includes both households on post-pay meters and households on pre-pay meters. Post-pay meters generally have a reading every two months, pre-pay meters are usually read weekly. In order to compare consumption for both groups as a panel, we need to allocate the reading to the period in which consumption occurred, and align them for both groups. An issue we encountered when doing this was a mis-match in billing periods both within and between meter type. For example, the majority of the post-pay meters have a billing cycle that runs bimonthly with January-March being the first period in each year, but for a large minority the cycle starts a month later from February-April. A small number of other households have a different cycle again. To resolve this, we use the median bill period, i.e. the bimonthly cycle beginning in January as our main unit of analysis. All other readings are aligned to this billing cycle based on the date of the reading, and the proportion of consumption that falls into each period. For example, the reading of 17,000 kWh on 03 April 2013 in Table A1 which falls roughly in the middle of period 3 is split between periods 2 and 3 using the ratio of 52:48 as 52% of the days this reading covers relate to period 2 and 48% relate to period 3. Gas consumption in kWh Given our cycle is a weighted average of two period consumption, we have to drop both the beginning and the end period from our sample (periods 1, 7 and 19), for both the two and three year gas time-series.
Prepaid meters reflect consumption that occurred over the previous week. We allocate this consumption to the mid-point of the current and previous reading, then aggregate these by bimonthly cycle to generate comparable gas consumption data over each period for both meter types.

31
At this point any observations with zero gas consumption for any given period are removed (n = 3), and a dwelling with a G-rated BER is removed (n = 1).
Finally, any remaining negative readings are smoothed across previous periods. In cases where the absolute value of the negative reading exceeds the positive value of the reading proceeding it, it is allocated across all proceeding periods whose sum is less than the absolute value of the negative reading. For example, the negative reading in period 4 of Table A2 is allocated to periods 2,3 and 4, but not period 1. A three period moving average is also used as a robustness check.  This is an engineering-based metric, based on a bottom-up model of factors affecting thermal efficiency. Each label A1-G corresponds to a predicted energy demand of the dwelling, as displayed in Table D1.