1 Introduction

Peaks in electricity demand bring about significantly negative environmental and economic impacts. This is because if a vast number of users is consuming electricity at the same time, suppliers have to activate power plants with higher greenhouse gas emissions and higher system costs. In Europe, the residential sector is responsible for about one third of overall electricity demand and up to 60% of peak demand (Barton et al. 2013). During peak demand, electricity prices in wholesale markets can experience fluctuations with peak prices up to nine times higher than off-peak prices (Torriti 2015).

In a decarbonised future, peaks are likely to remain an issue, where capacity margins are slim particularly in seasons with issues, where the problem is exacerbated by weather (e.g., winter in the UK and summer in parts of the USA). However, even more frequent issues of demand and supply balancing are likely to occur on a daily basis due to the high penetration of intermittent renewable sources, on the one hand, and increasing implementation of electric heating and transport, on the other hand. The need to change demand on a relatively short notice is becoming known in the energy demand literature as the ‘flexibility problem’, which reflects the fact that flexibility of demand can be defined in different ways (Grunewald and Diakonova 2018). To understand, where flexibility may originate from, a fundamental step consists of examining the timing of energy demand and whether there is intrinsic flexibility in what people do. A subsequent step is to understand if time when consumers may offer flexibility overlap with system needs, i.e., times of the day when electricity spot prices are higher due to higher costs of generation and high demand. The ambition of this paper is to provide an innovative approach to measuring flexibility which combines system needs—signalled through wholesale pricing—and intrinsic flexibility in the rhythms of everyday life. Showing at high time granularity how the intrinsic flexibility index correlates with electricity prices is critical, because the time slots when electricity prices are high correspond to time slots when the electricity demand is high. When the intrinsic flexibility index correlates with the price of electricity, demand-side interventions are more likely to flex demand. Conversely, at times when there is no correlation and, for instance, the wholesale price is high, whereas the intrinsic flexibility index is low, the system’s need to flex is high (to reduce prices), but interventions are less likely to be successful at flexing demand.

This paper presents an analysis of the 2014–2015 Office for National Statistics National Time Use Survey with a view to derive an intrinsic flexibility index (i.e., the ability to modify routines and practices associated with household energy demand) based on: how synchronised activities are within the family and with the rest of the country; how many activities requiring electricity we share with others; and how fragmented days are in terms of number and duration of electricity-related activities. The dual aim of this paper is to apply the intrinsic flexibility index and to analyse the extent to which the intrinsic flexibility index is correlated with spot prices in the UK wholesale electricity market. The first aim addresses an important gap in research on flexibility of electricity demand. The literature points to the importance of flexibility on the one hand (Carbon Trust and Imperial College 2016) and the central role of people’s practices in providing flexibility on the other hand (Blue et al. 2020). Introducing and implementing an intrinsic flexibility index is an attempt to quantitatively synthesise what we know about practice ordering for the purposes of demand flexibility. The second aim is more operational and can be framed as an exemplification of the flexibility index to understand how its implementation relates to electricity price variations. This is with a view to explore the potential synchronous and a-synchronous characteristics of available flexibility combined with price fluctuations in the spot market.

The results of this paper are likely to contribute to work on demand-side flexibility. After this introduction, the paper briefly reviews key concepts around price elasticity and flexibility (Sect. 2); describes the methodology and data (Sect. 3); presents findings on the intrinsic flexibility index estimates and on the analysis of settlement prices and intrinsic flexibility index discusses the potential applications of the intrinsic flexibility index (Sect. 4); and concludes (Sect. 5).

2 Price elasticity and flexibility

2.1 Measuring intrinsic flexibility

An approach to measuring demand-side flexibility consists of investigating what people do at peak time and how much flexibility is associated with what people do. Information about the timing of people’s activities can reveal something around the flexibility of people’s patterns in peak demand. This could be referred to as intrinsic flexibility, i.e., the ability modify routines and practices associated with household electricity demand.

The reason for focusing on intrinsic flexibility is that whilst the volume of electricity demand relates to many factors (e.g., weather, type of appliances, types of building), patterns throughout the day are a direct reflection of people’s activities. A simple example derives from the substantial difference between residential electricity load curves for weekdays and weekends. During the same season the weather can be equal at the weekend compared with the weekday. Everything else remains the same between a day of the week and the weekend: building, appliances, fuel substitution, price of energy and appliance control, and even the moment of the day in which sunlight is present or absent. Changes between weekday and weekend are triggered by people’s activities. Surveys as well as diary data can shed light on different characteristics of residential loads (Sokol et al. 2017; Buttitta et al. 2016; Yamaguchi et al. 2019).

Routines and practices are complex and extremely varied both across sociodemographic groups and within the same groups. There are common features to routines which have been investigated and at times catalogued in different disciplines, including social science and medicine (Jensen et al. 1983) and household labour (Coltrane 2000). The closest interpretations of routines and activities in relation to peak energy demand come from social practice theory. According to this conceptual approach, people do not consume energy per se, but they consume or demand the services that energy provides, e.g., laundry services, heating services, etc. (Wilhite 2013). Individual behaviours and social practices are supported by the consumption of relevant energy services. Material arrangements, including energy services, only have meaning within, and in relation to, the practices in which they are enfolded, and through which they are reproduced (Schatzki 2010). Changes to the timing of energy demand could only be triggered by non-discretionary factors, such as practices, levels of occupancy, location, number of occupants, type of tenancy and dwelling type (Shove 2004; Warde 2005).

Peak demand originates from a high level of synchronisation of activities happening in different households. High synchronisation can be seen as a multitude of people carrying out activities at the same time. When synchronised activities are linked with appliances and devices which require electricity, then these become of interest for peak demand purposes. Rush hours, hot spots and experiences of time squeeze have been seen as temporal manifestations of relations between practices (Pantzar and Shove 2010). If synchronisation with the rest of the population measures the societal obstacles and opportunities associated with shifting electricity demand, internal synchronisation, e.g., within the same family, plays an equally important role in determining intrinsic flexibility. Hotspots, synchronisation and shared practices are concepts which are used in this paper to operationalise intrinsic flexibility.

A high level of synchronisation with the rest of the population reveals how social practices converge at peak time and are difficult to shift. A lower level of synchronisation implies lower societal constraints contributes and will bring about higher flexibility. High levels of synchronisation are indicative of a time of the day in which there is more hurriedness and higher societal constraints to move activities in time (Southerton and Tomlinson 2005; Torriti 2013). This assumption is consistent with recent empirical work, which shows that practices specifically tied to socially conventional times constrain their temporal flexibility (Friis and Christensen 2016). Practices performed during periods of high synchronisation with others (e.g., food and entertainment) are considered to play an important role in shaping and maintaining social bonds between members of a household (Smale et al. 2017). Two explanations are presented for the inflexibility of eating practices. First, bodily needs and temporality seem to be more strictly defined when it comes to eating). Second, the timing of food is a matter of (often complex) coordination between household members (Higginson et al. 2014). Hence, higher synchronisation of activities during socially constrained are considered here as leading to lower flexibility.

The concept of hotspots and squeezing time implies that it is more difficult to shift the timing of activities when these are numerous in a short amount of time (Southerton 2003). Hotspots are thus characterised by the compression of certain tasks into the perceived time frames so that time can be ‘saved’ for other practices. In a way, hotspots are a response to the perceived ‘time squeeze’ which results from the felt need to follow the institutional and social rhythms. Hence, a high variation in terms of activities carried out during hotspot periods makes it more difficult to move activities throughout the day, bringing about lower flexibility. Conversely, a higher time spent on one’s own may imply that there is lower simultaneity of loads and within-household synchronisation, making it less difficult to move shared activities in time, hence contributing to higher flexibility.

From social practice theories, concepts of societal synchronisation, variation of activities and internal synchronisation constitute an analytical framework for exploring how the timing of people’s activities affects the price elasticity of residential electricity demand (Torriti 2017). The next section will operationalise these concepts into component indices which together form the intrinsic flexibility index. The latter consists of an attempt to quantify the impact of the time of the day and people’s activities on the timing of residential electricity demand. The intrinsic flexibility in what people do at different times of the day has an effect on how likely they are to respond to changes in prices.

3 Price elasticity and time of use tariffs

In the energy economics literature, traditionally electricity demand has been seen as relatively inelastic to price changes in the short term. Elasticity of demand, which is generally understood in energy economics as a way to measure the relationship between price and quantity of energy demanded in a given time period. Underpinning the concept of elasticity is the idea that price is the main factor influencing changes in energy demand. This is because demand curves are plotted as functions of price and quantity. The formula for elasticity of energy demand calculated between periods 1 and 2 is

$$ \frac{{\left( {Q_{1} {-} \, Q_{2} } \right)/\left( {Q_{1} + \, Q_{2} } \right)}}{{\left( {P_{1} {-} \, P_{2} } \right)/\left( {P_{1} + \, P_{2} } \right)}}, $$
(1)

where Q is the quantity of energy demand and P is the price of energy. If the formula generates a number > 1, the demand is elastic. Measurements of elasticity measured vary according to different temporal scales. Long-term elasticity is generally over a decade, short term is over a year and extremely short is within the day. The latter is of particular interest to those interested in understanding how effective intervention due to price (for instance through dynamic pricing, like critical peak pricing or time of use tariffs) can be.

A logical step which follows on from price elastic demand is substitution. If prices for a fuel go up significantly, substitute fuels will be sought after (Ikpe and Torriti 2018). This type of substitution is to some extent possible in transport and much less in buildings. Cars are purchased with a higher frequency than homes. Besides, the infrastructure of fuel provision in and around the home makes it difficult to switch between gas and electricity. The evidence on which studies on elasticity draw varies.

For example, reviews of peak reductions in response to time of use pricing have shown peak reductions of over 50%, with median reductions around 30% for critical peak pricing. Examples of such reviews include Newsham and Bowker (2010); Faruqui and Sergici (2013); and Frontier Economics, Sustainability First (2012). Some reviews of the literature have attempted to seize the considerable body of work produced in this area and shed some light on the potential reasons why price elasticities vary greatly across different studies. Appendix C of a US Government report on ‘Benefits of demand response in electricity markets and recommendations for achieving them’ (Qdr 2006) reviews early studies and classifies them by type of elasticity measurement: elasticity of substitution (i.e., how consumers substitute one good for another, or goods in different time periods for one another, when relative prices change) and own-price elasticity (i.e., the measure of how customers adjust to increases in the price of electricity by adjusting the consumption of other goods). Both present issues in terms of data and measurements. For instance, Alberini and Filippini (2011) review earlier elasticity studies, which are typically based on annual or monthly data in the USA, and find endogeneity problems and potential mismeasurements of energy prices.

Examples of disparity emerge when comparing empirical studies. For example, Silk and Joutz (1997) find that a 1% increase in electricity prices reduces electricity consumption by 0.62%, whereas more recent studies show that elasticity of electricity demand ranges from as low as − 0.06 (Blazquez et al. 2013) to as high as − 1.25 (Krishnamurthy and Kriström 2013). Studies seeking to explain differences in price elasticities find that customers are willing to curtail or forego load which they consider “discretionary.” For instance, Goldman et al. (2005) find statistically significant differences in customer price response at different prices.

In general, it is understood that elasticity of electricity demand is low (Lijesen 2007). So, for intra-day price elasticity (or extreme short run price elasticity) there is evidence that residential end-users do not postpone energy-related activities to take advantage of a profitable off-peak tariff.

Dynamic tariffs are designed to provide price signals to consumers, so that they reduce consumption when the tariff is higher and shift it to non-peak times of the day. Empirical work on dynamic electricity tariffs has been increasing in recent years and shows that price is often an insufficient levy to ‘shift’ peaks and, hence, improve the flexibility of electricity demand.

For instance, a paper by Buryk et al. (2015) shows that the engagement of end-users in dynamic tariffs depends mainly on the extent to which everyday life and household routines can be reconciled with the new tariffs. Hence, price in isolation does not explain changes in electricity demand over time. This suggests that alternative approaches to the traditional view of price elasticity are needed to understand what affects changes in electricity demand besides price.

4 Data and methodology

4.1 Data

The paper makes use of three types of data -namely time use, electricity spot prices and electricity load curves. Time use data reveal what people are doing at different times of the day with a level of granularity of 10-min intervals. The most recent nationally representative UK time use survey—the 2014–2015 National Time Use Survey—was accessed from a publicly available dataset (UK data archives, available at https://discover.ukdataservice.ac.uk/catalogue?sn=8128) (Gershuny and Sullivan 2017). The data was collected between April 2014 and December 2015 using a nationally representative sample of the British population using a multi-stage stratified probability sampling. The time use diaries provide information about what individuals are doing during one weekday and one weekend day and when during 24-h periods. In total the National Time Use Survey comprises 270 individual activity codes that the respondents could choose from to describe their activities. Whilst people can report not only primary, but also secondary, tertiary etc. activities, the majority of diaries comprise only primary activities (Anderson and Torriti 2018). Spot price data was accessed from a publicly available dataset (APX, Spot Market, available at: https://www.apxgroup.com/trading-clearing/spot-market) and for the same year electricity load profiles with a time resolution of 30 min were collected from a dateset by the UK Transmission System Operator (available from: https://www.nationalgrid.com/uk/electricity/market-operations-and-data/data-explorer).

4.2 Methodological approach

This paper defines the intrinsic flexibility index based on three component indices developed by Torriti et al. (2015): (i) societal synchronisation index; (ii) internal synchronisation index; and (iii) variation index. The three component indices are un-weighted. This means that consistently high values for one component index will influence the flexibility index more than consistently low values for another component index. Differences between demographics were tested statistically to check for significant differences.

Torriti et al. (2015) developed a similar methodology and applied it to a smaller sample urban population making use of an additional component index based on mobility as their data contained GPS information for each respondent. The methodology is innovative in its application to the nationally representative ONS National Time Use Survey. Also, mobility information is not available for the ONS National Time Use Survey and is consequently not measured in the methodology of this paper.

Hence, specific components of the intrinsic flexibility index consist of: (i) how synchronised activities are within the household and with the rest of the country; (ii) how many activities requiring electricity are shared with others; and (iii) how fragmented days are in terms of number and duration of electricity-related activities. Active occupancy is introduced as a Boolean variable which assumes a value of 0 for periods of occupancy and a value of 1 for unoccupied periods. The values are simply derived from a variable in the Time Use survey which reveals the location of the respondent and removing sleeping time. The same variable (WhereAt) has been used in other analyses which derive active occupancy levels (Torriti 2012). It is assumed that flexibility can only occur either during occupied periods or up to one hour after leaving the household and from one hour before returning home. This is because it is assumed that devices could also be left running, but also that some level of remote automation might occur thanks to controllers and mobile phone apps currently entering the residential energy management market.

4.3 Component indices

The societal synchronisation index is estimated as the difference between 1 and the standardised Shannon’s H (Gnansounou 2008), which can be defined as follows:

$${H}_{\mathrm{t}}=-\sum_{\mathrm{i}=1}^{\uplambda }{\upgamma }_{\mathrm{tiln}({\upgamma }_{\mathrm{ti}})}$$
(2)

where λ is the number of different states, i (i.e., activity codes considered), t is the time of interest (i.e., 10-min time slot) and γti is the proportion of all individuals that are in state i relative to the total number of individuals. Ht equals 0 when all individuals are in the same state and ln(λ) when individuals are evenly distributed among the λ states. The higher Ht, the lower the homogeneity of state distribution at t. Conversely, a low entropy index means that all the individuals are in the same state (e.g., watching TV) at the same time. The index can be standardised as a percentage value relative to the maximum possible value (ln(λ)), thus allowing to compare data in different activity coding schemes. The index is not affected by the ratio of observations/λ (issue of sparseness), as it is not affected by the number of activities not performed at time t, since by definition γti = 0. A higher societal synchronisation index implies higher societal constraints (i.e., lower flexibility). Hence, formally:

$$ {\text{Societal synchronisation index }} = \, 1 - \, H_{{\text{t}}} /\ln \left( y \right). $$
(3)

The internal synchronisation index is measured as the average proportion of respondents in a given demographic group who were with others during specific time periods. From the dataset it is possible to derive with whom respondents were at different times of the day. A higher number of shared activities with others imply that there is higher simultaneity of loads and within-the-household synchronisation. A higher level in the internal synchronisation index does not involve any assumption in terms of either increase or decrease in energy consumption. Instead, what is assumed is that during periods of high internal synchronisation it becomes more difficult to move shared activities in time (i.e., lower flexibility). Single-person households are typically associated with an internal synchronisation index equal to 0 (since Ht = 1):

$$ {\text{Internal synchronisation index}} = 1 - \, Hw_{{\text{t}}} /\ln \left( y \right) , $$
(4)

where Hwt is the standardised Shannon’s H and y is the average proportion of respondents who were with at least another person at the time t.

The variation index for time use data was developed initially by Vrotsou and Forsell (2011) and applied by Torriti et al. (2015) as a measure of consistency or dispersion of activities over time. It is estimated as the average number of unique activities carried out by each respondent divided by the total possible number of activities in a specified period of time for weekdays. Formally

$$\mathrm{Variation\,index}=\frac{\frac{\sum_{i}^{N}{x}_{i}}{N} }{\stackrel{-}{N}},$$
(5)

where xi is the number of unique activities carried out by a respondent i, N is the total number of activities carried out by respondent i and \(\bar{N}=38\) is the total number of possible activities. The total number of possible activities is a product of grouping activity codes by similarity (e.g., “watching sports on TV” or “watching films on DVD” grouped as “watching TV”) and whether activity is likely to be directly linked with electricity consumption. A similar approach is used by Torriti and Yunusov (2020). The rationale for how the variation index interacts with the intrinsic flexibility index is that a higher number of activities distributed through the day make it more difficult to move activities to different times of the day (i.e., lower flexibility). As a simple intuitive example, someone working for longer hours from home with fewer activities over longer periods of time will have higher flexibility in terms of what they are doing (e.g., running the dishwasher), than someone who is only at home for a couple of hours and performing several activities at the same time.

The flexibility index for the time period ti and population N is derived as follows:

$$ {\text{Flexibility}}\left( {t_{i} , \, N} \right) = \left[ {\left( {1 - {\text{internal synchronisation index}}} \right) + \left( {1 - {\text{societal synchronisation index}}} \right) + \left( {1 - {\text{variation index}}} \right)} \right]/3. $$
(6)

The three component indices are un-weighted. This means that consistently high values for one component index will influence the intrinsic flexibility index more than consistently low values for another component index. This is because this paper does not aim to understand individuals’ flexibility, but to explore flexibility in relation to time. Unlike Torriti et al. (2015), where the index was estimated for different socio-demographic groups (gender and with/without children), the index here is calculated for the whole population. This is because the overall aim is to identify periods of the day when flexibility of demand is high (or low) independently from electricity prices. The comparison and correlation analysis between the flexibility index and wholesale electricity market prices is explained in the next section.

4.4 Comparison and correlation

The analysis presents an example of intrinsic flexibility index application by deriving how much change occurs in APX pricing during the day at the same time as the intrinsic flexibility index varies.

Price variations in APX price are indexed using 00:00–00:30 as the reference half hourly period. Thus, the variation in APX prices is expressed as an index the effect of time of day on the APX price (i.e., time of day index).

The comparison between time use intrinsic flexibility index and time of day APX pricing data consists of correlation analysis taking one statistical observation as one household and computing average rate or duration of use by day of experiment. The first tool to compare daily load curves will be the normalized variation factor similarly to the application in Capasso et al. (1994) as

$$\mathrm{Normalized\,variation\,factor }=\frac{\sum_{i}^{n}({D}_{\mathrm{Flex}}\left(i\right)- {D}_{\mathrm{ToD}}\left(i\right){)}^{2}}{n((\frac{1}{n})\sum_{i}^{n}{D}_{\mathrm{ToD}} (i){)}^{2}},$$
(7)

where \({D}_{\mathrm{Flex}}\left(i\right)\) is the intrinsic flexibility index (i.e., the value defined by Eq. 6) at the time i and \({D}_{\mathrm{ToD}}\left(i\right)\) is the time of day index. This shows the spread of the distribution of the intrinsic flexibility index and correspondingly its standard deviation is compared with the spread of the distribution of the time of day APX pricing variable. Variance analysis (ANOVA) is also performed to assess the differences between the mean frequencies of revealed (i.e., time use) and measured (time of day) indices; and to estimate differences between average durations by day. Difference in intrinsic flexibility index can be assessed by time interval.

Upon the condition that the normalized variation factor tests positively, the intrinsic flexibility index and indexed time of day time series data are subjected to autocorrelation. Time use data are available for the same days of 2014 as APX pricing data. The stochastic process associated with time of the day pricing of electricity is inevitably correlated to the timing of electricity demand. Hence, the autocorrelation function is estimated from the time series as follows:

$$R\left(s,t\right)=\frac{E\left({X}_{\mathrm{t}}-{\mu }_{\mathrm{t}}\right)\left({X}_{\mathrm{s}}-{\mu }_{\mathrm{s}}\right)}{{\sigma }_{\mathrm{t}}{\sigma }_{\mathrm{s}}},$$
(8)

where σ is the variance, μ is the mean and E stands for ‘expected value’, \({X}_{\mathrm{t}}\) is the intrinsic flexibility index value and \({X}_{\mathrm{s}}\) is the indexed variation in APX prices. If the function R is well-defined, its value must lie in the range [− 1, 1], with 1 indicating perfect correlation and − 1 indicating perfect anti-correlation. Correlation is also calculated between the intrinsic flexibility index and electricity demand load curves indexed to the highest peak during the whole year.

5 Intrinsic flexibility index

5.1 Synchronised activities

Figure 1 shows how synchronised people are at different times of the day. Sleeping was excluded, because it has negligible impact on electricity consumption, and therefore, the synchronisation of sleeping is not of interest to this analysis.

Fig. 1
figure 1

Societal synchronisation index (weekdays)

Full size image

Synchronisation is higher in the morning than evening peaks, due to more predictable sequence of activities which take place as respondents get ready to go to work under greater conditions of time squeeze. During the day synchronisation is relatively high, because people work similar hours in the middle of the day. As work phases out, TV watching ensues, driving synchronization upwards. However, in the evening peak the lowest level of synchronization is reached, meaning that the concept of hotspot is associated with several and diverse activities.

5.2 Internal synchronisation

Figure 2 shows findings for the internal synchronisation index, i.e., the times of the day in which people share activities with others. During weekdays, early morning, evening and night time are often spent with partner/spouse and children. The rest of the day is predominantly spent with work colleagues and/or by oneself. The shared activities index demonstrates how a substantially great percentage of respondents were on their own throughout the day. A high proportion of people were on its own between 10 am and 3.50 pm. The period with the highest level of shared activities was between 7 pm and 9.50 pm. At this time people are significantly less likely to be on their own with a peak of over 80% shared activities index at 7.50 pm. For instance, 52% stated that the first person they were with at this time was their partner or spouse.

Fig. 2
figure 2

Internal synchronisation index and average number of people-shared activities (weekdays)

Full size image

5.3 Variation

More activities are performed during the morning peak (7–10 am) compared to the evening peak (4–7 pm), irrespective of whether respondents are working or not. The average number of different activities performed is highest during these peaks, and lowest in the middle of the day. Accordingly, the variation index is higher for the morning peak as people change activities frequently in a limited period of time. It is lower during the central part of the day as activities tend to be prolonged (e.g., working) and increases again in the evening due to dinner preparation and leisure. The average number of activities performed is higher during the late evening period (7–10 pm) (Fig. 3).

Fig. 3
figure 3

Variation index (weekdays)

Full size image

5.4 Intrinsic flexibility

Figure 4 shows that the intrinsic flexibility index varies significantly depending on time of the day. Periods of the day, where intrinsic flexibility index values are high could correspond to higher availability to shift activities.

Fig. 4
figure 4

Intrinsic flexibility index (weekdays)

Full size image

With regards to morning peaks, the analysis shows that variation of activities is very high with high levels of synchronisation. At this time of the day more different activities are performed and synchronisation is higher than during evening peaks. Occupancy is very high before 8 AM and several activities are shared with others implying a very low level of potential flexibility at this time of day.

Turning to evening peaks, the lowest level of intrinsic flexibility is reached at 7 PM. This time of the day combines high variation of activities and a high internal synchronisation level (possibly due to the fact that people share eating-related practices), despite the fact that overall synchronisation is not very high. It can be observed that the intrinsic flexibility index is significantly influenced by the internal synchronisation index. During the daytime when electricity prices are high, the societal synchronisation index is also high (that is, inflexible). On the other hand, the internal synchronisation index is low (that is, flexible) during the day.

5.5 Analysis of settlement prices and intrinsic flexibility index

Settlement systems are processes by which suppliers’ contracted positions are matched with their customers’ consumption. Ofgem, the UK energy regulator, is targeting elective half hourly settlement for domestic and smaller non-domestic consumers in early 2017, with the potential for mandatory half hourly settlement to be introduced in 2018. This is to realise the benefits of more accessible (and more accurate) information relating to household consumption, improved competition from easier switching, and potentially more cost reflective pricing through time of use tariffs. Some work has been developed on the effects of increases of Demand Side Response and the price effects on Balancing Markets (Bradley et al. 2013; Cardoso et al. 2020). However, flexible demand in the residential sector is likely to affect spot markets through changes in central clearing and settlement.

This section makes use of time of the day price of electricity in the wholesale market for 2014–2015 to derive how much change occurs in APX pricing during the day. We derive the times of the day in which the intrinsic flexibility index varies in correspondence to changes in price.

Flexible generation has strong influence on the spot market price and, conversely, inflexible generation has a weak impact. This can be explained by the fact that the spot market reflects last minute adjustments to parties’ position prior to gate closure. Therefore, the spot market not only shows more volatility but also higher prices than would normally occur in trades well ahead of delivery, such as a forward bilateral contract taking place a year in advance. Thus, one might expect that the APX market prices to be inflated by flexible generation, whereas increases in baseload generation would tend to have weak effects on prices.

Figure 5 shows an example of the effect of time of day on the APX price excluding generation costs, i.e., the impact of time of day on the APX price (as derived from \({\mathrm{D}}_{\mathrm{ToD}}(i)\) in Eq. 7). The 3 February 2014 APX (2014) price was extracted using 00:00–00:30 as the reference half hourly period, where the value of the reference price which is taken as zero is £18.13 per MWh. Since the prices are determined by demand and supply, they are likely to reflect to a certain extent demand behaviour, relating to aspects other than the type of generation on the system. There is a premium cost of electricity present at peak and morning between 09:00 to 13:00. Conversely the early hours of the morning and early afternoon are periods, whereby the impact of time of the day is lower.

Fig. 5
figure 5

Impact of time of day on APX price

Full size image

Variations in APX prices were converted into a time of day index following the procedure explained in the Sect. 3. Findings are presented in Table 1 which shows the R2 values for time use data and indexed APX data split in terms of time of the day and day of the week. For time use data time of the day and day of the week have higher ANOVA values than APX data. Time of the day has higher R2 values than day of the week.

Table 1 ANOVA results for time use data and APX data (indexed)
Full size table

Since the normalized variation factor tests positively, the autocorrelation function is estimated for the time series of settlement prices with a view to assess the extent to which the two functions (i.e., intrinsic flexibility index and indexed variations in APX prices) autocorrelate. Figure 6 shows the months for which the two functions are autocorrelated. The sample autocorrelation captures the general form of the theoretical autocorrelation, even though the two sequences do not agree in detail. Since both functions represent well-defined time series, their values lie in the range between − 1 and 1. Values equal to 1 indicating perfect correlation and − 1 indicates perfect anti-correlation. The autocorrelation is associated with a 95% confidence band and the autocorrelation at lag 1 is significant. The autocorrelation plot together with run sequence of the differenced data suggest that the data are stationary. Negative autocorrelation corresponds to the month of January only. This implies that to high variation in prices does not correspond potential flexibility of demand.

Fig. 6
figure 6

Autocorrelation between intrinsic flexibility index and indexed variations in APX prices in 2014

Full size image

5.6 Application of the intrinsic flexibility index to assess the impacts of time of use tariffs

Variations in APX prices in correspondence with variations in intrinsic flexibility show the analytical potential of the index presented in this paper when it comes to pricing and tariff approaches. The literature on short-term elasticity of price takes the relationship between price and demand as a given without much consideration over how elasticity changes depending on the time of the day. Elasticity measures the extent to which people respond to price signals regardless of whether at different times of the day there is more or less availability to shift electricity consumption. Responding to short-term variations in price is related to time availability. The intrinsic flexibility index is a tool which quantifies this important measure. Whilst price elasticity provides a measurement of the outcome of short-term variations in price, the intrinsic flexibility index appraises how likely it is that (groups of) households may respond to price signals at given times of the day.

This separation between intrinsic flexibility index and short-term price elasticity does not imply any merit order or indeed call for a separation of the two. Quite the opposite, it is argued here that the intrinsic flexibility index may add explanatory power to well-established elasticity estimates and improve the accuracy of tariff setting, particularly when it comes to dynamic pricing. For example, formal analytical tools to assess the impacts of time of use (ToU) tariffs involve.

$$ \ln \;E_{{{\text{it}}}} = \alpha_{i} + W_{{{\text{it}}}} \gamma + {\text{peak}}_{{\text{t}}} \cdot \beta_{1} + Day1_{t} \cdot \beta_{2} + Day2_{t} \cdot \beta_{3} + (Peak_{t} \times Treat_{it} \times POST_{t} ) \cdot \delta_{1} + ({\text{Day}}1_{t} \times Treat_{it} \times POST_{t} ) \cdot \delta_{2} + ({\text{Day}}2_{t} \times Treat_{it} \times POST_{t} ) \cdot \delta_{3} + ({\text{Night}}_{t} \times Treat_{it} \times POST_{t} ) \cdot \delta_{4} + \varepsilon_{{{\text{it}}}}, $$
(9)

where E denotes electricity consumption by household i in period t and αi is a household-specific fixed effect. Vector W contains day of the week dummies, month and year dummies, and weather, the effects of which are assumed to be similar across control and treatment subjects. Peak, Night, Day1 and Day2 are dummies denoting the ToU bands, and the effect of ToU pricing is captured by the δ coefficients (since Treat denotes whether a household receives the ToU treatment, and POST denotes whether a trial is in progress). The omitted category in the equation above is the time of the day usage before ToU starts, which is assumed to be common (on average) across control and treatment households. Similar regression equations to (9) are presented, for example in Miller and Alberini (2016). However, work by Buryk et al. (2015) shows that the variable which counts the most toandards the effectiveness of ToU and Real Time Pricing options consists of how easy it was to shift the timing of electricity consumption based on people’s structure of the day. The intrinsic flexibility index can thus be integrated into the equation above to include the value of the intrinsic flexibility index at the time i. On the one hand, the intrinsic flexibility index is likely to have strong explanatory power when assessing the impacts of dynamic tariffs on the timing of demand. This means that the intrinsic flexibility index can partly explain trends and variations in price elasticity. On the other hand, it could find a field of application in tariff structuring for suppliers. For instance, charging peak pricing end-users who are not even home at peak time of the day under static ToU will produce an undesirable zero-sum effect. In this paper, the intrinsic flexibility index is estimated for the whole population—based on the nationally representative sample of the UK National Time Use Survey—to provide a comparison with wholesale electricity prices. However, energy suppliers and aggregators might be interested in different applications of the intrinsic flexibility index. Provided time use data of the type utilised in this paper (or proxies) are available to energy suppliers and aggregators, the index can facilitate the selection of which consumers are more likely to respond to changes in price (i.e., those consumers with higher price elasticity). Consequently, retailers may set varying peak/off-peak ratios for different household groups based on their intrinsic flexibility index. In the absence of time use data and proxies, a default option for market applications of the intrinsic flexibility index may consist of defining flexibility ‘cohorts’ in terms of household attributes. A challenge for researchers will consist not only of understanding the variation which exists in short-term responses to ToU tariffs, but also (longer term) structuring effects that ToU tariffs could have, i.e., the extent to which these could become one of the societal factors influencing synchronisation.

Findings from the intrinsic flexibility index show that this ranges from 30 to 45% during the day. The current patterns of everyday life seem to imply that the intrinsic flexibility in demand can assumed to be rather low and constant during the day. The index does not cover periods of the day in which occupants are asleep (especially during the night). Remotely automated devices, including smart heat pumps and electric vehicles (Torriti 2020) might provide levels of flexibility (and subsequent changes to system prices) which are not assessed as part of this paper. At the same time, the results indicate that there might be groups for whom there is potentially higher flexibility—for example, people working from home. People working from home may be associated with the highest level of flexibility, because their practices are associated with high active occupancy, long duration of a small set of activities mainly not shared with others and low synchronisation with the rest of the population.

This paper does not distinguish between groups of end-users or flexibility properties of practices. For instance, findings on the intrinsic flexibility index are not divided by socio-demographic groups. Bottom-up clustering of users based on specific may suggest that standard socio-demographic classifications are not necessarily of much use, although other analysis seems to indicate that older people spend longer periods at home doing similar activities (Torriti and Santiago 2019), hence affecting variation index, and that people with children have higher degrees of internal synchronisation than those without children. Equally, no assumptions are made around how effective intervention (e.g., through price and technology) might be on different practices. On these distinct effects, there is a growing literature showing that lighting, heating, cooking, eating and leisure activities are less flexible than domestic cleaning (Smale et al. 2017). In a Swedish study, practices which were regularly shifted from peak to off-peak hours included dishwashing and laundry (Öhrlund et al. 2019). For instance, the societal synchronization index measures the extent to which individuals are in the same state (e.g., watching TV) at the same time. However, 80% of the population watching TV at the same time is treated equally to 80% of the population drying clothes at the same time.

The composite nature of the intrinsic flexibility index spreads the dependability of the index. At the same time, the un-weighing of the three component indices is likely to have consequences on the outcomes of the intrinsic flexibility index. Whilst underpinned by a very simple methodology, the intrinsic flexibility index requires availability of time use data, hence raising questions over how replicable this approach is and how expensive the data collection might be taking into account different geographies and socio-demographics. Time use data are seldom collected in a statistically meaningful way—typically national offices of statistics carry out national time use surveys every decade. Collecting time use diary in addition to electricity metered data might not be cost effective. However, ICT technologies for the collection of time use data are rapidly developing and include mobile applications which automatically deduce information on people’s activities at different times of the day based on sensors, WiFi, internet mobile and social media use and GPS data.

6 Conclusions

Currently most consumers pay for the amount of electricity they use regardless of when they consume it. For generators providing electricity is more expensive at peak times and even though wholesale prices tend to be higher during these periods, there is widespread consensus that flexible demand can help overcome issues of demand and supply balancing.

At the residential level, the main economic approach to increase the flexibility of electricity demand consists of introducing dynamic tariffs to which end-users are supposed to respond by shifting consumption to different times of the day (Torriti et al. 2011). In research terms, how people respond to short term changes in price is quantitatively measured through price elasticity metrics. These tend to rely on average values and typically disregard time of the day effects. However, people’s activities are ordered according to rhythms and are not designed around electricity consumption. Because people carry out distinct activities at different times of the day, their ability to respond to changes in electricity prices will depend on the time of the day. In this context, we set two main aims at the beginning of this paper. First, quantifying which levels of flexibility might be available in the residential sector involves the implementation of an intrinsic flexibility index based on people’s activities. Second, the intrinsic flexibility index can be usefully compared and analysed against indexed wholesale electricity prices.

This paper presented an analysis of the 2014–2015 Office for National Statistics National Time Use Survey and derived the intrinsic flexibility index, which was operationalised based on how synchronised activities are within the family and with the rest of the country people live in; how many activities requiring electricity people share with others; and how fragmented days are in terms of number and duration of electricity-related activities. Findings showed how spot prices and intrinsic flexibility to shift activities vary throughout the day. Some reflections were also drawn on the application of this research to work on price elasticity as the paper illustrates how introducing a dummy variable based on the intrinsic flexibility index may—at least in principle—improve the accuracy of price elasticity predictions.

The first key contribution of this paper was to introduce the intrinsic flexibility index, which consists of an attempt to quantify the impact of the time of the day and people’s activities on the timing of residential electricity demand. The intrinsic flexibility in what people do at different times of the day has an effect on how likely they are to respond to changes in prices. The paper rests on an increasing volume of empirical literature applying time use data for the purpose of modelling load profiles (Torriti 2014) and theoretical work on the role of social practices in relation to peak electricity demand (Walker 2014). At the same time, the paper suggests methodological and analytical innovations (in the form of time use-based indices) which could find specific applications in the realm of price elasticity estimates. To this end, this paper suggests that the intrinsic flexibility index may hypothetically contribute to the analysis of short-term price elasticity for residential electricity demand. Ultimately, the most significant electricity demand challenges in future are expected to be around heating, cooling and transport. These are not addressed in the paper, which focuses solely on electricity demand in the home. However, the flexibility challenge is one which is likely to affect all sectors of demand. Hence, it is likely that flexibility issues around the integration of electric vehicles and electric heat pumps, for example, will need to be addressed at a whole system level, including residential electricity demand. Indeed, some flexibility might take place through fuel substitution (Torriti and Grünewald 2014; Ugursal and Fung 1996). For example, fuel substitution take place between gas and electricity for heating and cooking, where both sources are available.

The second key contribution was a comparison of residential consumers’ potential flexibility periods with changes in spot prices in the UK wholesale electricity market. It is derived that the intrinsic flexibility index and indexed APX prices are correlated. This correlation indicates that there are times of the day in which demand side interventions are needed to reduce the wholesale price of electricity and might temporarily reduce electricity demand thanks to an intrinsic predisposition as revealed from time use activities. Autocorrelation was also performed due to the independence of electricity generation prices from demand. This, combined with the institutional effort in the UK to maintain high capacity margins, may explain why there is low and even negative autocorrelation between intrinsic flexibility index and indexed APX prices in wintertime.