Background

Maintaining a balance between the production and consumption of electric energy is crucial for a stable grid and for maximizing the use of sustainable energy sources. This balance is usually achieved by carefully controlling energy production, and residential customers are typically not involved in the process. However, in recent years, various approaches that consider the role of the consumer, such as dynamic pricing signals and information interfaces, have been proposed, introduced and researched. Nevertheless, there are still many unresolved questions regarding the optimal implementation of such systems and how they influence user experience over longer-term use. There are particularly few studies available on DSM systems in the context of rented residential homes, as such concepts have mostly been tested and trialed with owner-occupants.

In this paper, we report the design considerations and empirical findings on the long-term user experience and usage patterns of a system that aims to involve residential end users in grid balancing by encouraging them to adjust their energy consumption based on price signals and active information. Residential users can access a web platform to view the predicted savings potential, which is calculated using a statistical model based on historical consumption patterns and calendar features. Users receive active prompts in the form of text messages when the potential for savings is particularly high. In the following sections, we first review related work on influencing energy consumption (sections "Demand side management", "Energy feedback", "Persuasive technologies"). We then describe the system that we developed in our research project, along with its usage context and the trial method (section "Methods"). Subsequently, we present the results of our study (section "Results"), followed by a detailed discussion of their implications. Furthermore, we provide suggestions for future research in this domain (section "Discussion").

Demand-side management

The main objective of demand-side management is to control energy consumption to prevent overloading during peak times and shift demand to periods of high renewable energy generation. Flexibility can be achieved either through manual control by users or via an automated system control approach.

Studies have shown that automatic load shifting can save energy and has a higher response rate than manual demand response [1, 2], but issues related to trust, perceived control, and ease of use need to be better addressed to improve the acceptance of automated solutions [3, 4]. In this context, the concept of a “social-license-to-automate” has recently been introduced to address these issues [5].

Regarding manual load shifting, price-based strategies have a long history in electricity. The three main approaches are time of use (TOU), critical peak pricing (CPP), and real-time pricing (RTP). TOU uses predetermined price levels and their temporal occurrence, with peak prices usually ranging from two to four times higher than non-peak prices. Research is ongoing regarding optimizing the reduction of peak energy consumption [1, 6, 7]. CPP is a time-variable tariff where the electricity price deviates from the basic time-variable structure at preannounced times. Usually, only a single event price level is used, which is commonly announced the day before. Peak rates typically are six to eight times higher than those during low demand periods. As a result, load reductions have consistently been achieved [1, 8,9,10]. RTP is a dynamic tariff that corresponds to the current prices at the electricity exchange, the pricing was determined at the time of consumption or shortly before. Pilot studies showed that small differences in price levels resulted in only modest load reductions, highlighting the importance of a wide price spread [8, 11]. However, it is not rational to arbitrarily increase price spreads since higher prices do not equally result in a proportionate increase in load shifting by end-users [12].

To maximize their effectiveness and impact on energy consumer behavior, variable tariffs should be complemented with a range of supplementary measures such as comprehensive and easily understandable feedback, actionable recommendations and reminders, and supportive technology [13,14,15].

Energy feedback

Energy feedback is a commonly used approach to influence consumers’ energy behavior. The use of information interfaces provides users feedback on their consumption patterns, allowing them to reflect on and potentially alter their consumption behavior. Interfaces can provide feedback on past consumption (indirect feedback), current consumption in real time (direct feedback), and predicted future consumption patterns. Additionally, they can offer disaggregated feedback on the consumption of specific devices, appliances, and systems.

Indirect feedback has been shown to lead to energy savings of 5–10% [16,17,18,19,20,21,22,23]. The effectiveness of this approach can be influenced by various factors, including the timeliness [16] and duration [17, 19] of the feedback, expert consulting [20], and comparison to oneself or others [16, 19].

Direct feedback It has been argued that direct energy feedback can enhance the effectiveness of energy feedback [18], and research has demonstrated that such systems can lead to significant energy savings [16, 18, 24]. According to meta-studies, direct feedback can result in a reduction of approximately 6–10% [25, 26]. However, in a more realistic context, this reduction drops to 3–5% [27].

Predictive feedback Predictions of behavior and consumption, as well as their associated consequences such as costs, have been used in energy feedback applications to influencing behavior [28,29,30,31,32]. To guide users’ consumption behavior, various designs have been proposed to utilize this predictive information [33,34,35].

Studies have suggested that providing energy feedback in a disaggregated form can enhance its efficiency. However, a meta-review by Kelly and Knottenbelt [36] found that disaggregation may not be required to achieve significant savings. They conclude that “disaggregated electricity feedback may help a motivated subgroup of the population to save more energy, but fine-grained disaggregation may not be necessary to achieve these energy savings” (page 1).

Persuasive technologies

Another approach to influencing consumers' consumption patterns in conjunction with energy feedback is the use of persuasive technologies. To effectively use persuasive technologies, system designers should consider various design principles and strategies [37, 38]. For example, Oinas-Kukkonen and Harjumaa [39] outline 28 design principles (e.g., tailored information, personalized content, cooperation, and competition) that have been utilized and evaluated in research on persuasive technologies (e.g., [40,41,42,43]).

Numerous studies have demonstrated that personalized persuasive technologies can be more effective in influencing user behavior [44,45,46]. Personalization can be achieved in various ways, such as matching persuasive strategies to individual users' personality traits [44, 47, 48] or by addressing gender differences [49, 50].

Timing is an important factor in the design of persuasive technologies, as it can significantly impact the likelihood of behavior change [37, 51]. Recommendations aimed at influencing user behavior should be provided at an appropriate time, and the number and frequency of messages should be managed carefully to avoid causing irritation or leading to acceptance issues [52]. Consequently, systems that understand the context of users and are able to predict the actionability of interventions are desired.

Methods

The objective of this research project was to explore the potential of optimizing electricity demand in apartment buildings connected to the grid and its impact on user experience. To meet shifting demand needs, consumption forecasts were calculated, and price signals were adjusted accordingly. Additionally, an optimized energy feedback system was implemented consisting of several elements to encourage behavior change.

Context description and recruitment of participants

The study was conducted in St. Cugat, a Catalan municipality north of Barcelona, and involved a building block with 44 eligible apartments. The study focused on an apartment building that is part of a social housing complex in the area. The availability and access to real-time consumption data of a whole building with multiple tenants was the key factor in selecting that building for indirect incentivized demand response services. At the time of the project, that social housing building was the only one available by the public promoter of Sant Cugat del Vallés (Spain). The residents of the building were generally low income, and there was a relatively high fluctuation of residents. Heat and hot water, which were provided through solar thermal panels coupled with a gas boiler, were paid for through the social housing program and were not affected by the pricing signals and not part of the trial. When interpreting the results of this study, it is important to consider the specific characteristics of our study participants and acknowledge that the insights provided in this paper are applicable to this particular type of housing and residents.

System description

After reviewing available concepts and approaches and considering practical factors, we developed a personalized energy feedback system that aimed to increase awareness, engage participants, and encourage them to shift their consumption toward demand-based patterns. Figure 1 provides an overview of the system’s key concepts and components.

Fig. 1
figure 1

System overview

Full size image

The main features of the energy feedback system were: (1) consumption analysis and prediction; (2) the calculation of the savings potential; (3) opportunity-driven active messages, and (4) the visualization of energy consumption forecast and saving potential.

For the consumption analysis and prediction, we used a hybrid clustering and classification methodology. This approach enabled us to estimate the probability of the most common daily load curves for each household. We then estimated the users’ individual cost savings potential based on the predicted probability of the day-ahead load curve patterns and the day-ahead energy prices (derived from the hourly day-ahead-market prices).

The details of the algorithms used are described in detail in Appendix.

Based on these calculations, we sent SMS messages to trial participants who met the defined criteria (especially the configurable threshold on savings potential). This filtering step was included to improve the meaningfulness of the messages and the chances that people act upon them. The messages included a direct link to personalized web pages (html) for each individual household. The messages were sent in Catalan, and the English translation (by the authors) is provided in the next paragraph. Based on feedback from residents, the message content was slightly modified in the second phase to better address their needs.

Version 1: “Do you want to save < Savings Potential in Percent of tomorrow's electricity expenses > ? Please use your electric devices between < Start time > and < End time >. You usually have a peak power around < Peak time >. If you need more information, please click on the link: < Link >”

Version 2: “Hello!, tomorrow, Tuesday < Date >, you have a potential saving of < Savings Potential in Percent > on your electricity bill. Please do not hesitate to visit the website, now and whenever you want:” < Link >

To provide participants with easy access to their predicted consumption pattern and savings potential, we implemented a platform within the existing web portal of the participating energy provider. The platform allowed participants to view a comparison of their predicted consumption pattern with their optimized consumption curve, as well as the related savings potential, and energy prices for the following day. The first version of the platform is depicted in Fig. 2.

Fig. 2
figure 2

User interface of the energy feedback system. Left: predicted energy saving potential; middle: energy saving potential over time; right: personalization settings for determining the savings potential threshold for sending notifications

Full size image

Following feedback and suggestions from trial participants, the main screen was modified in the second version by adding the electricity price and the real energy consumed. The adapted screen is shown in Fig. 3.

Fig. 3
figure 3

User interface of the energy feedback system (second version including energy price and real energy consumed)

Full size image

Procedure

The system was tested in an extensive field test between November 2019 and October 2020. Following an initial survey on general attitude and expectations, tenants were invited to two information events about the project and were provided with the possibility to change their contracts to profit from saving opportunities. The actual trial was then conducted in two phases. After the first phase of the trial, the message selection strategy was modified to address issues identified in the interim feedback received from the participants (Fig. 4).

Fig. 4
figure 4

Timeline showing the main events and activities during the trial

Full size image

The first phase of the trial took place from November 2019 through May 2020. It started with an email notifying users that the trial had commenced and that the service was free with the goal to costs help lower electricity costs. Based on the analysis of the consumption patterns of a participant, text messages were sent that indicated the savings potential accompanied by a link to the platform where users could gain additional insights. The savings potential varied between 10 and 50%. Initially, only users who had a savings potential higher than a specified threshold received messages. This threshold was set to 20% by default. This rather high threshold was chosen due to the low absolute consumption and low energy prices (at the time of the study). This default threshold could be adjusted individually by the participants themselves. After complaints from participants that they did not receive any messages, the possibility of changing this setting was communicated more actively to the end users.

In mid-January, an evaluation survey was sent out to gather the participants initial impressions of the project, including their load shifting experiences, suggestions for improvement, and any issues encountered during their participation. The feedback was collected in open-text format, and 11 responses were received. Based on the feedback and insights gathered from data, such as the frequency of messages sent and platform queries, the messages and forecasts provided were adapted to resolve any technical issues, such as text messages not being received. Furthermore, to increase the number of messages sent to the users, the savings potential threshold was set to 10%. These changes were implemented between May and June 2020, with the trial resuming in August and concluding in October of the same year. An evaluation survey was conducted in September. Participants were asked to provide feedback on the platform itself, the newly introduced features and any barriers encountered while using it. They were also asked to share their thoughts on the text message-based alert system used for prompting any changes in their consumption behavior and any barriers experienced while attempting to shift their loads. Additionally, the survey included questions related to the participants’ overall feedback on the project, including questions on the willingness to allow automated control of loads experiencing the required efforts involved with manual load shifting.

Out of the 46 apartment inhabitants (from 38 households), 34 inhabitants were fully involved in the trial. A total of 54.3% were women, and the mean age of participants was 27 years, ranging from 18 to 35 years old. The size of the apartments varied between 42 square meters (31 apartments) and 56 square meters (3 apartments). The annual household net income was below 23,000 € for the single-occupant small apartments and below 26,000€ for the bigger apartments suitable for 2 persons. Approximately three-quarters of the participants had a university degree (21.7% bachelor’s degree, 52.2% master’s degree). A total of 4.4% had completed high school or the equivalent, and 21.7% had completed professional training. Most participants in St. Cugat were employed, either full time (65.2%), part time (10.9%), or self-employed (13%). A total of 8.7% of the participants were not employed, and 2.2% mainly cared for their households. Regarding the costs of energy, the average bill of the participating households was approximately 25€/month. Therefore, the 20% saving threshold used for triggering messages would generate an absolute saving of approximately 0.15 €/day. This rather small absolute amount was also the reason for setting the default threshold rather high in terms of percentage (20%). During the experiment, tenants were able to adjust the threshold according to their preference.

Results

System actions and usage

The subsequent sections present the results and findings of the analysis of the personalized incentivization system's actual usage and related energy consumption during the period of November 2019 to September 2020. As previously mentioned, text messages were sent to the users throughout the trial phase. Figure 5 provides an overview of the numbers of alerts sent to the users during the trial and offers insights into the power saving potential specified in the messages.

Fig. 5
figure 5

Time distribution of alerts sent throughout the evaluation period. Each bar represents a week, and colors indicate the share of the saving potential. The gaps between May and June 2020 correspond to the system maintenance period

Full size image

We can see that text messages were continuously sent with the exception of two brief gaps caused by technical issues that were promptly resolved. Additionally, there was a system maintenance phase during May and June 2020. The initial grey bars in the graph depict the alerts sent during the first two weeks of the trial period, which did not include information on savings potential. This feature was added only afterwards. Overall, we can see that a) in the second phase (after the maintenance and the adjustment of the default minimum savings potential to trigger a message), the number of sent alerts increased and b) the sent SMS were dominated by saving potentials below 20%.

A closer look at the relationship between messages sent and web platform queries may help to determine whether text messages can prompt users to obtain additional insights on how to achieve energy savings and reduce costs. Figure 6 provides a detailed depiction on the messages sent and web portal visits recorded for each user during the trial period.

Fig. 6
figure 6

Messages (green) and web portal visits (orange) per household ID. Log data for access to the website are only available from February 2020 onward

Full size image

In Fig. 6, we can observe that the number of messages received by users varied, with approximately 40% (15 users) receiving messages at a relatively high frequency. During the second phase of the trial, we can see that the procedure was modified, resulting in the fact that all participants receive alerts more frequently. We can also see that the number of users who accessed the system on a regular basis was rather low and that some initially active users decreased their access in the second phase.

User feedback

Intermediate evaluation

The evaluation survey using open-ended questions conducted during the first part of the trial was completed by 11 participants. Overall, both the SMS messages and web portal queries were experienced as useful, and shifting activities were reported regarding the washing of dishes and clothes. The only direct criticism regarding the platform was that some users found the suggested time slots for shifting to be either unavailable or undesirable. Several participants also reported issues with receiving text messages, and there was a general interest in receiving more alerts with savings opportunities. Additionally, several participants expressed doubts about the accuracy of the recorded consumption patterns.

The results, therefore, indicated that a low frequency of delivered messages is problematic. This can lead to lower involvement of users and reduced likelihood of taking up opportunities when they arise. Furthermore, the possibility of flexible consumption in the form of specified time slots during which shifting is possible would further enhance the usefulness of the platform and SMS messages. Consequently, issues with receiving messages were investigated and resolved, and the default minimum savings potential was lowered to increase the number of messages received.

Concluding evaluation

The concluding evaluation survey was completed by 26 participants, 62.2% of whom were female. Participants had overall ages ranging from 26 to 39 years (average 32 years).

Out of the participants who responded to the feedback survey, 42.3% stated that they used the provided web platform and opt-out messaging. Of those who used the platform, usefulness was perceived more positively (36.4% rated it at 4 or 5 with 5 = very much) than negatively (0%), but the majority (63.6%) were neutral regarding this question (see Fig. 7).

Fig. 7
figure 7

Summary of user feedback on the overall usefulness of the system as well as the usefulness of specific system aspects. Data were collected during the final survey using Likert-scale statements

Full size image

When asked about the usefulness of specific features, participants particularly favored the possibility of comparing real and optimal consumption curves (rated 4 or 5 out of 5 by 90.1%) and the possibility of displaying achieved savings (72.2%). The response to the visualization of historic day-by-day consumption and day-long hourly price information was not as strong but still clearly positive (4 or 5 ratings were 54.6% for both questions). When asked directly if they accessed the platform regardless of the text messages received, slightly more than half (54.6%) of the participants who were using the platform stated that they did so 1–2 times a week. This self-assessment of platform usage appears to be significantly inflated when compared to the website log data. Regarding the overall perception of the platform, participants were positive in general. However, they noted that the platform could have supported them better with more active engagement and/or consistent visibility, such as through an in-home display.

The text alert frequency was perceived as “just right” by the majority (65.4%), with 19.2% perceiving it as too frequent and 3.9% as not frequent enough. Several users (11.5%) stated that they did not receive any text message. More than two-thirds of the participants (69.6%) found the text messages to be useful or very useful (rated 4 or 5 out of 5). In open feedback, participants mentioned that having the rate included in the text messages would have been useful.

Regarding shifting efforts undertaken by the participants, among those who were aware of the optimal consumption curve, only 23.1% stated that they made a true effort to shift their consumption. The most frequently shifted activity was running the washing machine (52.4% shifted this often or very often), while both cooking and housework were shifted never or rarely (by 52.4% and 42.9%, respectively; shifting of housework was stated by 23.8%) (see Fig. 8 for more details). Charging computers and mobile devices, as well as using a hair dryer were mentioned as additional activities that were shifted. The use of programming to achieve shifting was most often made with washing machines (42.9%), followed by dryers (14.3%). Dishwashers were rarely programmed for this purpose (4.8%). The majority of the participants (61.9%) stated that their shifting activities stayed the same throughout the runtime of the project, but some changes were reported (28.6% reported an increase, 9.5% reported a decrease).

Fig. 8
figure 8

Frequencies of shifting activities. Colors indicate the frequencies from “(almost) always” to “no answer”

Full size image

With regard to reasons for not shifting, 8.3% of participants reported being unsure about what to shift, 20.8% found it to be too impractical, 25% stated that they forgot about it, and 8.3% did not see enough benefits. The main additional reason stated was incompatibility with working hours, and some participants expressed a preference for automated processes in this context.

When asked about their willingness to automate different systems and appliances to align with optimized consumption curves, participants were most open to automation of heating and cooling systems, followed by appliances and water heating (see Fig. 9). Not surprisingly, participants felt most at ease with automation that they could control directly. This preference was strongest in the context of heating and cooling systems, and significantly diminished for other systems and appliances. There were minimal differences between participants' preferences for automation with consent per process and automation with the ability to monitor and interrupt. In most cases, there was only a slight preference for direct consent.

Fig. 9
figure 9

Preferences for automated control of different household functions. Colors indicate different levels of control by the household

Full size image

When asked about their interest in accessing information on saving opportunities after the project’s conclusion, 53.9% of participants expressed interest. As a final feedback, participants noted that they appreciated the overall idea and the daily information updates. However, many faced challenges in finding opportunities due to constraints such as busy work schedules and other demands that pushed the project to the background.

Discussion

Overall, the assessment of individual feedback suggests that our trial participants found the concept of tailored and proactive energy savings information to be valuable and useful. However, only limited use was made of the platform, and only a small subset of users regularly engaged with the platform. These results are similar to findings summarized by Gyamfi et al. [53], who report that a significant proportion of households do not respond to price signals. In contrast to our expectation, the tailored provision of energy savings potential did not seem to greatly increase the users’ willingness to engage in energy shifting behavior in comparison to untailored approaches of other studies.

Based on the survey responses and informal communications with project partners we conclude that the main reasons for participants for not engaging were a lack of awareness regarding savings potential, a perception of limited usefulness, and the absence of ongoing and motivating activities that support the users’ engagement. These findings are similar to factors identified in prior research [2] and support the notion of using automation to manage demand response [54].

This limit in usefulness was mostly due to schedule constraints, as the optimized consumption curve tended to clash with the times during which participants were able to actively shift their consumption. This problem has also been identified by Stelmach et al. [55], and future research should specifically address possibilities to mitigate this factor. If incentivized shifting is used, personal schedules of end-users need to be considered. This can be achieved by providing system settings that allow users to specify time periods during which shifting is feasible.

End users also need to be engaged on a more continuous base in order to keep the program more present and supports the breaking of habits. Also, access to the information supporting shifting needs to be made more immediate, e.g., through an app. Our system has already improved on traditional energy information systems by identifying especially useful savings opportunities, but further work is needed to better address users’ situational needs and circumstances. We think that methods to automatically detect users’ current context [56, 57] might be a promising addition to improve users’ engagement with demand response.

The forecasting methodology for predicting the day-ahead baseline, based on clustering and classification techniques, provides good accuracy compared with ANNs, which is the most common methodology for domestic individual forecasting. In future work, we want to include the consideration of the end-user schedule for the next day, as we expect this to boost the prediction accuracy.

Direct load control has, depending on the affected loads, a much higher likelihood of avoiding issues with personal schedules and puts less strain on end users, as it does not require active involvement. It does, however, require sufficient trust of end-users to grant a “social license to automate” [5]. Based on our results, end users might therefore be more likely to participate in DLC if they have previously attempted to actively shift their consumption. A promising approach based on these conclusions might therefore be to start with indirect load control with the possibility to state schedule constraints but offer participants a switch to DLC during workday hours (or during the week/overall if preferred by the user) with a fixed incentive as part of the bill tied to this decision.

Regarding limitations of the study, it is worth noting that the revenue potential was relatively small due to the fact that in our setup shifting did not affect heating or hot water costs. Therefore, under such circumstances, shifting efforts may need to be incentivized more strongly or framed differently, for instance by emphasizing successful community-wide CO2 emission savings. It should also be borne in mind that the selection of households was determined by the availability of data from the public real estate development company of Sant Cugat del Vallés (Spain). This building was the only one with the data requirements that were defined for the use case. Nevertheless, to improve user acceptance and energy flexibility in absolute and relative terms, it would be interesting to apply the same type of services to users with more electrification in their houses, mainly heat pumps or electric radiators for heating or cooling and electric water heaters. One of the problems that has been noted for the limited use of our system is the low potential economic savings in absolute daily terms (only approximately 0.2–0.3 €/day on average). In the case of large electricity consumers, being able to vary thermostatic signals throughout the day and having a higher baseload consumption would improve the savings potential in both relative and absolute terms; therefore, there should be greater interest in the implementation of demand-side management by messaging to end users.

Future research, should explore whether and how flexibility potential specifications can be integrated in manual shifting incentivization programs while maintaining attractive revenue streams for all participating stakeholders, and also identify the necessary system components, such as communal batteries or electric storage heaters, to ensure this integration is feasible. Furthermore, it would be valuable to investigate whether prior experience with manual shifting genuinely enhances willingness to participate in automated demand-side management.

Conclusions

In this paper, we presented the results of our work on studying the user experience and acceptance of tailored messages in the context of influencing energy consumption patterns. We found that participants in general were interested in the concept and that more than two-thirds found the text messages notifying them of shifting opportunities to be useful. Despite the positive perception of the system the actual usage was limited, highlighting the need for future research to explore ways to increase participant engagement. Regarding system features, the possibility to compare real and optimal consumption and the display of achieved savings were most important for the users, but the display of prices and historic consumption was also noted as relevant. Our study showed that the use of washing machines, dishwashers, and dryers was primarily shifted emphasizing the need for demand response systems to address the specific requirements related to these activities. Several users noted that stronger support in automating the shifting of activities would be desirable in this regard. Concerning barriers to behavior adoption, impracticality, forgetting and limited benefits could be identified as important factors hindering system use and behavioral adoption.

Several promising approaches have been identified that could help mitigate these factors, such as offering attractive incentives or utilizing automation to provide tailored, simple, and convenient control of appliances. Future research should explore ways to ensure revenue for stakeholders in manual shifting incentivization programs, identify required system components, and examine how prior experience with manual shifting can be leveraged to increase willingness for automated demand-side management. To extend the applicability of the findings of this study, future work should also explore to what extent the results of our work can be transferred to other, broader user groups (such as homeowners). For this it should also be investigated which design elements can be directly adopted, and which design elements would require modifications. In summary, our study results led us to believe that tailored messages represent a good opportunity to influence energy consumption behavior. Yet, to ensure user convenience and sufficient impact, a combination with automated control approaches should also be pursued.