We now discuss and justify some of our assumptions, and analyse the stability and robustness of our method and results.
Nightly setback
Often, households have a timer-driven heating system which lets the temperature drop to a certain setback temperature at night in order to save energy. One could argue that for our analysis a baseline in which the temperature is decreased during the night makes more sense than the always-on baseline. However, if we used a baseline with a night-time setback temperature, we could also use this setback in the occupancy-based strategies, which then consequently would also use less energy (because at night a home is typically occupied). For the setting with a nightly setback the savings are even higher (6.64 MWh on average for the oracle strategy compared to 4.83 MWh over the 75 weeks period). This is due to the possibility of obtaining schedules with very long periods of absence, e.g. when the dwelling is unoccupied the whole day, it does not have to be heated above the setback temperature for the previous night and that day. This effect is naturally even stronger for reactive schedules (8.41 MWh instead of 5.48 MWh energy savings).
Heating simulation model
For the heating simulation, we use a standardised (ISO 13790), but relatively simple 5R1C household heating model and additionally make some assumptions on the building, such as the insulation characteristics, based on the year of construction. There are more sophisticated and exact heating models available, which however require more detailed information on each household and building. If this information is available, the 5R1C model could be replaced by the more complex model without defying the purpose of our method. For large-scale evaluations of several thousand households, such as the CER dataset, this information is typically not available and hence more sophisticated models are no viable option.
The simple model used in our evaluation might induce modelling errors in the estimated heating energy. However, for each household the error will most likely persist in all the strategies and have a similar value. Since we are only interested in the difference between the strategies, the errors will largely counterbalance each other, the resulting error in the savings estimate thus being mitigated.
Sensitivity to the occupancy estimation
As we perform a post-analysis of a household’s energy consumption, employing occupancy detection is sufficient for our calculations. In a real-world setting, this also applies to the reactive strategy, as no future occupancy information is needed. However, to be able to employ the oracle strategy in practice, occupancy prediction is required, which is a more challenging problem. Approaches for neither of the estimation paradigms are perfect. Prediction algorithms additionally face the fact that humans sometimes behave inconsistently and not “according to plan”, e.g. spontaneously deciding to skip their weekly sports training. Research has shown that detection and prediction can be performed with reasonably high accuracy (e.g. for detection: on average 83% (Becker and Kleiminger 2018), 82% (Kleiminger et al. 2013), 73% (Chen et al. 2013), e.g. for prediction: 85% when using GPS data (Kleiminger et al. 2014a)). Other systems not based on electricity consumption, such as Tado (2018), which uses the location of the inhabitant’s smartphone from which the return time can be estimated, may be even more accurate. Such a system requires an explicit opt-in, and is thus too cumbersome, slow, and costly to be used in a first screening of the households. Once a household has opted for a smart heating solution, changing the occupancy estimation from one based on smart meters to a system tracking the inhabitants’ smartphones would allow a highly accurate prediction, thus enabling both the savings and the comfort of the oracle strategy.
Errors in the detection or prediction may impair the savings potential when the false positive rate is high, i.e. the dwelling is heated when nobody is at home. The comfort may suffer from false-negatives, i.e. the dwelling is not heated or the temperature is not yet high enough when the home is in fact occupied. However, in a real-world deployment there are several possibilities for technical measures to counteract this comfort loss, e.g. an “override” button inside the home or a smartphone app to overrule the automatic heating control. The discussion of these means is out of the scope of this paper.
As our simulation and as such our savings estimation depend on the output of the occupancy detection, we might be facing second-order errors in the savings estimation due to errors in the occupancy detection. Since we have no occupancy ground truth for the CER dataset, we cannot directly validate our occupancy detection results. We acknowledge that potentially there are errors in the detection, but the question is how strongly the savings results react to errors in the occupancy detection: If the detection makes only a few more errors, are the savings affected only a little, too, or possibly a lot? To address this question, we simulate artificial households: one “new” and one “old” building with a floor area of 149 m2 each, the mean in the CER dataset. We vary the occupancy pattern to examine how the savings are influenced by the changes. For a specific duration of continuous absence we create artificial schedules which all have an average occupancy of 75% (the average in the CER dataset), but different absence patterns. For example, for a period of absence of two hours, we set the first four 30 min slots to unoccupied and the following twelve slots to occupied, then the next four to unoccupied again and so on. Figure 7 shows how the relative savings increase as the duration of absence increases. This is because for longer absences the dwelling has to be pre-heated less often. The curves show that small changes only have a small impact and thus few errors in the occupancy detection will only have a minor influence on the results. The interdependence of energy savings, discomfort due to prediction errors, and occupancy estimation performance is explored in greater detail in (Gluck et al. 2017).
Sensitivity to the thermostat settings
Another interesting point is to examine how the savings depend on the temperature settings. Our simulation has two temperature parameters, the comfort temperature, which is the target to be reached when the dwelling is occupied, and the setback temperature, the value to which the temperature is allowed to drop when the dwelling is unoccupied. A low setback temperature is important to benefit from longer absences (in particular for old houses with poor insulation). While the overall savings for a setback temperature of 10°C are 9.2% for the oracle strategy (cf. Section “Savings potential evaluation”), they drop to 7.6% if we let the setback temperature increase to 15°C. Similarly, the comfort temperature has a significant influence on how much energy is consumed for heating. Applying an occupancy-based heating strategy, the absolute savings will be higher when the comfort temperature is increased due to saving the greater amount of energy required for heating to higher temperatures. The question is how strongly this affects the relative savings, i.e. the ratio of estimated absolute savings and absolute consumption for the “always-on” baseline strategy. To explore this, we run simulations for two artificial but typical schedules, “employed singles” and “family”, varying the comfort temperature. In the “employed singles” schedule, the dwelling is unoccupied from 9 a.m. to 6 p.m. from Monday to Friday, and from 8 p.m. to 11 p.m. on Fridays and Saturdays. In the “family” schedule, the dwelling is unoccupied from 9 a.m. to 2 p.m. Monday to Friday. Additionally, for each schedule we simulate a “new” and an “old” dwelling, i.e. we obtain four artificial households. The comfort temperature is varied from 18°C to 25°C in steps of a quarter of a degree. The range corresponds to advice on temperature settings for households published by the Sustainable Energy Authority of Ireland (2018) and the German Federal Environmental Office (Umweltbundesamt 2017). The results are depicted in Fig. 8. It shows that the relative savings only slightly increase when increasing the comfort temperature. This effect is strongest for the “employed singles” setting with an “old” dwelling and employing the reactive strategy – however the increase is still less than two percent points over the full range. For the “family” setting the relative savings are nearly constant. As usual, the savings are smaller for the oracle strategy than for the reactive strategy, but also the increase in savings is less. This is due to a contrary effect for the oracle strategy: the higher the comfort temperature, the earlier the household has to be preheated in periods of absence.
Additionally, we ran the simulation for the whole dataset again for the extremes of the examined comfort temperature range, which are marked as red circles in Fig. 8. The average relative savings for all households at comfort temperatures of 18°C and 25°C were 8.69% and 9.94%, respectively. The values show little deviation from the savings at a comfort temperature of 20°C (9.24%, cf. Table 3) which we used for evaluation. Overall, we find that the relative savings results for the chosen comfort temperature of 20°C are also valid for other reasonable temperature settings.
Sensitivity to the heating power
In our analysis, we determined the maximum power the heating system of a dwelling is able to deliver (the so-called design heat load) according to the European standard EN 12831. One can expect, however, that in practice a particular heating system deviates in one way or the other from that standard. For occupancy-based heating regimes, the available heating power is indeed an important aspect to consider. We perform similar simulations as in the previous subsection, using the same artificial households. Instead of altering the comfort temperature (which is set to its default value of 20°C here), we scale the design heat load by a scalar, the heating power factor. We vary it in a reasonable range from 0.75 to 1.5. For these values, the total energy consumed for the always-on strategy (our baseline and the denominator in the calculation of the relative savings) is almost constant. A heating power factor of one results in our default design heat load value. Figure 9 shows the results for two of the artificial households with both heating strategies. For the other household types, the conclusions are similar. The higher the design heat load, the shorter the period a dwelling has to be preheated before the arrival of the inhabitants when employing the oracle strategy. Therefore, the savings are higher with a more powerful heating system. For the reactive strategy the opposite is the case, however. The reactive strategy only heats the dwelling upon arrival, however then it will try to heat it up as quickly as possible with all the heating power available, if necessary, as its primary concern is to minimise the comfort loss of the inhabitants. That means, with a higher heating power, the comfort will be higher, but also the amount of energy consumed. Overall, the gap between the oracle and the reactive strategy shrinks with an increasing design heat load.