On Distributed Scheduling of Flexible Demand and Nash Equilibria in the Electricity Market

This paper presents a novel game theory approach for large-scale deployment of price-responsive electrical appliances. In the proposed distributed control scheme, each appli-anceindependentlyschedulesitspowerconsumptiononthebasisofabroadcastdemand/pricesignal,aimingtocompleteitstaskatminimumcost.Theconﬂictinginteractionsoftheappli-ances,competingforpowerconsumptionatthecheapesthoursoftheday,aremodelled through a differential game with a continuum of players, and efﬁcient deployment of ﬂexible demand is characterized as a Nash equilibrium. A novel approach is adopted to derive necessary and sufﬁcient equilibrium conditions: intrinsic properties of the problem (price monotonicity, unidirectionality of power transfers) are exploited to perform an equilibrium study based on sublevel sets of the considered demand proﬁles. As a result, it is possible to determine for which penetration levels of ﬂexible demand, types of appliances and inﬂexible demand proﬁles it is possible to achieve an equilibrium. Such stable conﬁguration is achieved through the broadcast of a single demand/price signal and does not require iterated exchange of information between devices and coordinator. In addition, the global optimality of the equilibrium is proved, necessary conditions for Pareto optimality are derived, and a preliminary analysis of devices with partial time availability is carried out. The performance of the proposed control strategy is evaluated in simulation, considering realistic future scenarios of the UK power system with large penetration of ﬂexible demand.


Introduction
The increasing number of flexible loads in the power system, such as "smart" appliances and electric vehicles, will give customers the possibility to partially schedule their power consumption and have an active role in the management of the network. The impact and potential advantages of this development have been widely investigated [1,27,29], showing how private customers could reduce the cost of their electricity bill with a minimum impact on their comfort. At the same time, the power system would be able to achieve an improved reliability, lower energy prices and a more efficient utilization of the existing infrastructure.
The study of flexible demand integration in the electricity market considers real-time pricing tariffs, as described, for example, in [30]: a price signal is broadcast to the devices which independently determine their power consumption and operate during the hours of the day with lower electricity prices. Real-time pricing schemes are considered a promising solution to crucial issues emerging in power systems, such as increased variability and uncertainty from renewable generation and modified consumption patterns from electrification of trans-portation and heating. These elements have been discussed and brought to the attention of regulators by a substantial number of papers and reports, such as [3,14,15]. As a result, after several experimental testbeds, real-time pricing is starting to be implemented in areas with large penetration of renewables and electric vehicles [4] and further diffusion is expected in the near future. In this context, the presented analysis establishes whether real-time pricing can lead to stable market configurations or additional control actions are required to ensure safe and reliable system operation.
In the implementation of real-time pricing tariffs, it is necessary to consider the global effect of the appliances power scheduling on aggregate demand and electricity prices. For example, if all devices operate when energy is expected to be cheaper, demand will increase at those times, leading to higher prices and therefore suboptimality of their initial operation strategies. Centralized approaches tackle this problem by considering a global optimization which is solved by the market operator on the basis of the data provided by generators and consumers, as proposed in [25,28]. Given the complexity of this problem for high number of appliances, privacy concerns and the traditional tendency of customers to have full control of their energy consumption, distributed mechanisms have also been analysed. The feasibility of this kind of strategies is investigated in [31] which considers the interactions between the system operator and the appliances as a closed-loop model and studies its stability. A wide array of different approaches have been proposed for the distributed management of flexible demand, including stochastic optimization [6] adaptive strategies [11] and Lagrangian relaxation [23].
Game theory has also been extensively applied to the problem of flexible demand deployment, as it naturally captures the conflicting interactions between the single devices that compete for power consumption at the cheapest hours of the day. For example, [19] characterizes the power consumption of the flexible loads as their best response to certain loads and tariffs. This concept is extended in [21], which proposes participation incentives, and in [26], which considers a larger time horizon in order to guarantee fairness for the devices and achieve better global results. Many iterative price-based schemes have been proposed for distributed coordination of price-responsive devices, in particular for large fleets of electric vehicles. For instance, [18] calculates, by taking into account the mean behaviour of the population, a charge profile of the vehicles which corresponds to a Nash equilibrium and is also globally optimal when the agents are identical. A similar approach is presented in [13], where an iterative procedure converges to a stable solution by penalizing variations of the vehicles' strategies from the previous iterations.
This paper also adopts a game theory framework and extends the preliminary study in [8], presenting more rigorous theoretical proofs and additional results on equilibrium optimality and partial flexibility of the appliances. A fully distributed scheme is considered, modelling the appliances as competing players of a differential game. Each appliance independently schedules its power consumption in order to complete its assigned task at minimum cost, on the basis of a demand/price signal broadcast by some central entity. Efficient integration of the devices in the electricity market is characterized as a Nash equilibrium, with all appliances having no unilateral interest in changing their operation strategy, formulated on the basis of the broadcast signal, when the energy price of the resulting aggregate demand profile is considered. To simplify the equilibrium analysis and the design of the broadcast signal, the appliances population is described as a continuum, assuming that the impact of the single device on total demand and electricity price is negligible and only the global behaviour of the devices population needs to be considered in such sense. Similar approaches have been previously applied in power systems contexts, studying large-scale market interactions [16], frequency control provision by thermostatic loads [2], coordination of electric vehicles [7] and energy arbitrage with micro-storage devices [9]. The main novelty of the present work is that the dynamic behaviour of the appliances is implicitly modelled by exploiting intrinsic properties of the problem (monotonicity of electricity price and unidirectionality of power exchanges). Consequently, the equilibrium analysis does not involve partial differential equations and can be performed by focusing on the sublevel sets of the different demand profiles. This allows to derive necessary and sufficient equilibrium conditions, determining in which scenarios a fully distributed control strategy can successfully coordinate the appliances. Such conditions take into account the types of flexible devices, their penetration in the system and the impact of the other inflexible loads in the grid.
With respect to previous works on the subject [13,18], the proposed coordination strategy does not require additional quadratic terms in the cost function of the appliances. Moreover, it achieves a stable configuration through the broadcast of a single demand/price signal, avoiding iterated broadcasts by the system operator and repeated power updates by the flexible loads.
To achieve these results, a power density of task durations is calculated from the distribution of the devices parameter (task time and required energy), describing the valley-filling capability of the population. Similarly, the inflexible demand profile is characterized by the negotiable valley capacity, a function that is related to the measure of its different sublevel sets and quantifies the amount of flexible demand that can be allocated while preserving an equilibrium. By comparing these two functions, it is possible to verify whether an equilibrium exists and show that, if this is the case, it can be achieved by simply broadcasting the price of inflexible demand. The analysis not only determines the scenarios in which a fully distributed control strategy can be successfully implemented, but it also provides the theoretical tools required to design additional control actions in the other cases. For instance, the presented theoretical study is extended in [10], showing that a stable solution can always be obtained by introducing a time-varying constraint on the power rate of the devices. In the second part of this paper, the properties of the Nash equilibrium are analysed, showing that it is always socially efficient and deriving conditions for its Pareto optimality. A preliminary analysis on devices with partial flexibility is also carried out, and the proposed distributed scheme is evaluated in simulations, considering a future scenario of the UK power grid with large penetration of flexible appliances.
The rest of the paper is structured as follows: Sect. 2 models the flexible appliances and the electricity market, while Sect. 3 describes the coordination problem as a differential game, introducing the distributed scheme proposed for appliances coordination. The theoretical analysis on the sublevel sets of the demand profiles is performed in Sect. 4, calculating the power scheduling of the devices as their best response to a broadcast demand/price signal. The necessary and sufficient conditions for equilibrium are derived in Sect. 5, evaluating its optimality properties in Sect. 6. After the qualitative discussions of Sect. 7 and a preliminary analysis of devices with partial time availability in Sect. 8, simulative results are presented in Sect. 9.

Modelling of Flexible Demand and Electricity Market
To describe the distributed control strategy proposed in the present work, we first analyse the involved components of the power system, clarifying the main assumptions and modelling choices required to incorporate them in a unified framework.

Flexible Price-Responsive Appliances
We consider electrical appliances that complete an assigned task by scheduling their power consumption over a certain time horizon [0, T ]. Assumption 1 All flexible devices fulfil the following properties: -Total flexibility: they can operate and consume power at any time t ∈ [0, T ].
-Interruptibility: they can arbitrarily stop and resume their power consumption.
Under Assumption 1, each device can be fully described by two quantities: the total energy E required to complete its task and the minimum task time τ (achieved operating at rated power P = E/τ ). The appliances are assumed to be price-responsive and to behave rationally. Each appliance, having received from the system operator a certain demand profile D(·) (or the corresponding energy price (D(·)), schedules its power consumption u in order to complete its task at minimum cost: We denote by u(·) = u * D (·, τ, E) the solution of (1) when the demand signal D is broadcast and the parameters τ and E are considered. Existence and uniqueness of u * D will be discussed in subsequent sections.
For a global description of flexible demand, we preliminarily denote by T and E the sets of distinct values within the appliances population for the time and energy parameters τ and E, respectively. The whole set of flexible appliances can be characterized by the unnormalized distribution m(τ, E), where The properties of the appliances population are abstracted by the associated energy density f : T → R + : where f (τ ) quantifies the total energy required by appliances with parameter τ to complete their tasks. This means that its support corresponds to the set T of distinct minimum task times τ of the appliances population. An additional hypothesis, related to the number of devices and their parameters, is introduced: The energy density f is a well-defined and bounded function.
This means that in our study the number of flexible appliances is sufficiently high and their time parameters τ are adequately diversified to be described as a continuum. For example, if all devices had equal τ =τ , the density f would correspond to a Dirac delta centred inτ , thus violating Assumption 2. To simplify our analysis, without loss of generality, we consider the following expression for the support of f : It is now possible to calculate the total variation of power demand D f,D which is introduced by the appliances. In this context, we remind that u * D (·, τ, E) represents the scheduled power consumption of a single device with parameters τ and E when D is broadcast. To calculate D f,D , it is sufficient to take the weighted integral of u * D over the energy and time parameters:

Electricity Market and Inflexible Loads
The electricity market has been abstracted by the monotonically increasing function : [0, +∞) → [0, +∞) which associates, to a certain value of aggregate power demand D a (t), the corresponding energy price p(t) = (D a (t)). The aggregate demand D a,D (t) (resulting from the broadcast of the signal D) is given by two different components. These are the inflexible demand D i , caused by the other loads in the system, and the contribution D f,D of the flexible appliances, defined in (4): The inflexible demand D i (assumed to be known a priori) is considered price-inelastic, i.e. the power consumption profile of each inflexible load is fixed and it is not impacted by electricity prices.

Distributed Control and Nash Equilibria
The proposed scheme for the management and coordination of the appliances consists of four distinct steps, associated in Fig. 1 with the corresponding interactions between the different entities: -Step 1 Ahead of the considered time interval, the system operator receives information on the flexible devices population (energy density f ) and on the inflexible loads (inflexible demand D i ). -Step 2 Using the data collected at the previous step, the system operator broadcasts to the appliances a demand signal D (or the equivalent price (D)). Note that the proposed control scheme is distributed, as the power consumption of the appliances is not determined by a central entity, but it is instead independently chosen by each device on the basis of a broadcast demand/price signal.
The problem of designing the broadcast signal D is approached in a game theory context, considering a differential game with the following elements: -Players The individual flexible appliances, whose state is described by the variable x(t).
Such quantity corresponds to the total energy absorbed in the time interval [0, t]. If u(t) denotes the power consumed by a single device/player at time t, it holds: Since the total energy required for task completion by the individual device equals E, the following terminal state constraint needs to be considered: -Strategies Each appliance schedules a feasible power profile u : [0, T ] → R + which allows completion of its assigned task. For an appliance with parameters E and τ , it must hold: The power scheduling is performed before the considered time interval [0, T ], and each device determines u(·) by solving (1) on the basis of the broadcast profile D and its parameters τ and E, setting u(·) = u * D (·, τ, E). -Objective function Minimization of the energy cost C sustained by the individual device to complete its task: The cost C is the integral over time of the scheduled power u(t) of the single device, multiplied by the electricity price (D a,D (t)). The latter term depends, through D f,D in (5), on the aggregate strategies of the flexible appliances.
In the chosen modelling framework, with large populations of small agents, it is assumed that the impact of the single player on the global quantities of the system is negligible. Therefore, the strategy u of the individual player does not significantly impact the aggregate demand D a,D and corresponding electricity price (D a,D ), which are only affected by the global behaviour of the players population (through the flexible demand D f,D ). This is similar to mean field games [17], where the interactions between infinitesimal players are characterized by a unique global quantity, the so-called mean field.
Our main objective is to design the broadcast demand profile D in order to induce a Nash equilibrium for the game presented above.

Definition 1
For a certain broadcast profile D, a Nash equilibrium is achieved if each appliance has no unilateral interest in changing its scheduled power profile u(·) = u * D (·, τ, E) when the resulting aggregate demand (and corresponding electricity price) is considered. Equivalently, the following must hold for all τ ∈ T and E ∈ E : As previously mentioned, the total demand D a,D and corresponding price (D a,D ) are only impacted by the global behaviour of the agents. Therefore, the Nash equilibrium can be expressed as a fixed point: from (6), the power scheduling u * D of the whole population is optimal for a certain profile of aggregate demand D a,D and, at the same time, it induces that very same profile of demand (through (5)).
The Nash equilibrium is chosen as a control objective as it corresponds to an efficient operation of the power system. In fact, it ensures low energy costs for the appliances, avoids "rebound peaks" (described, for example, in [20]) and achieves consistent valley-filling, with lower generation costs and reduced stress on the grid infrastructure. These properties are formalized in Sect. 6, where it is shown that the presented notion of Nash equilibrium is always socially efficient.

Analysis on Sublevel Sets of Demand Profiles
In order to determine Nash equilibria conditions for the population of flexible appliances (provided in the next section), a compact representation of the power scheduling performed by the individual devices and by the whole population is preliminarily derived. Such analysis is conducted on the sublevel sets of the different demand profiles, considering the following quantity Q D : where μ denotes the Lebesgue measure.
The function Q D (d) returns, for a certain demand value d, the amount of time "spent" by the signal D at lower values of demand (equivalently price). In the context of our study, the following hypothesis is introduced: Such assumption is not very restrictive and typically holds for real profiles of inflexible demand which will be considered in the next section to derive the equilibrium conditions. Having established in Assumption 3 that the considered broadcast profile D has no level sets of positive measure, we can infer from (7) that Q D is continuous, strictly monotone   increasing, has image equal to [0, T ] and takes the following values at the endpoints of its domain: For a better understanding of the relationship between the time variable t and the corresponding measure Q D (D(t)), some comparisons are performed next. Examples of broadcast demand profiles are shown in Fig. 2, while Fig. 3 contains the corresponding Q D (D(t)). The blue curve D 1 in Fig. 2 is a typical 24 h UK demand profile as recorded by [22], and the corresponding function Q D 1 in Fig. 3 shows the same monotonicity trends with values in the interval [0, 24]. The red curve D 2 is an example of monotonically increasing profile to which, by definition, corresponds a linear function Q D 2 (D 2 (t)) = t. The last considered curve D 3 violates Assumption 3 and is constant over an interval of positive measure. This introduces discontinuities in the measure function Q D 3 , as shown in Fig. 3.

Power Scheduling in the Measure Variable
To describe the behaviour of the flexible appliances in response to the broadcast demand signal D, it is useful to replace the time variable t with the measure q = Q D (D(t)). We denote the quantities in the new variable with a bar accent.

Definition 3
For a broadcast profile D fulfilling Assumption 3, the correspondingD : Note thatD(q) represents the demand value which determines a sublevel set of D with measure q or, alternatively, such that Q D (D(q)) = q. The correspondence with the equivalent function in time is straightforward to derive: Given the definition of Q D provided in (7),D(q) can alternatively be interpreted as the unnormalized quantile function of the broadcast signal. It is straightforward to extend the same notation to a specific class of functions ρ.

Definition 4
Consider a signal D fulfilling Assumption 3 and a function ρ : [0, T ] → R verifying the following condition: Define asρ : [0, T ] → R the unique function which fulfils the following: The functionρ(q) corresponds to ρ(t) evaluated at any time t for which Q D (D(t)) = q and is always well defined from (11). As in the previous case, to better understand the proposed change of variable, we provide a graphical representation of the considered quantities as functions of the measure q. For each demand profile over time, shown in Fig. 2, the corresponding profile as a function of the measure q is presented in Fig. 4. For example, if one considers D 1 (t), the resulting demand profile in the q variable is given byD 1 . Analogous relationships hold between D 2 (t) andD 2 (q) and between D 3 (t) andD 3 (q). Note thatD 1 (q) corresponds to an increasing "reordering" of the values of D 1 (t), whileD 2 (q) is equal to D 2 (t) since the latter is increasing in time. For the profile D 3 , which violates Assumption 3, there exists a subset of [0, T ] on whichD 3 (q) is not defined. We now demonstrate a crucial relationship between functions in the time variable t and the corresponding quantities in the measure variable q = Q D (D(t)). (11) and the correspondingρ introduced in Definition 4, the following equalities hold:

Lemma 1 Under Assumption 3, for any integrable function
Proof See "Appendix A". Applying Lemma 1 with ρ(t) = (D(t))u(t), the cost minimization (1) solved by each device to schedule its power consumption can alternatively be considered in the variable q: Proposition 1 For a broadcast profile D that fulfils Assumption 3, the unique solution u * (q, τ, E) of the cost minimization problem (13), for a device with parameters τ and E, has the following expression: Proof It is straightforward to verify the feasibility ofū * for problem (13) as it fulfils both the integral and inequality constraints. To show that it is also the unique optimizer of (13) (up to congruence in the 1 norm), consider that the integral of a feasibleū is fixed and equal to E. Sinceū * is a bang-bang control and the price function is strictly monotone increasing, it is sufficient to note that, as a result of the monotonicity of Q D from Assumption 3, it holds: Notation-wise, it is not necessary to express with the subscript D the dependency ofū * from the broadcast profile as such relationship is implicit through the change of variable q = Q D (D(t)).

Characterization of Flexible and Aggregate Demand
Having calculated the scheduled powerū * , it is possible to derive the total demand variation D f introduced by the appliances population (as a function of the measure q) when D is broadcast. Taking the weighted integral ofū * over the parameters τ ∈ T and E ∈ E yields: The equivalent expressions of the scheduled power profile u * D and resulting flexible demand D f,D in the original time variable can be derived from Definition 4: Given (14), the scheduled power consumption u * D can alternatively be expressed as: where the sublevel set S D (τ ) is defined as: Note that, as a result of Proposition 1 and Lemma 1, the power profile u * D is optimal for the energy cost minimization problem (1) in the original time variable.
The corresponding aggregate demand profile D a,D as a function of time t can also be calculated: In the particular case in which D = D i (viz. inflexible demand is broadcast), the aggregate demand depends exclusively on the current broadcast value of demand d = D i (t). Introducing the function K (d) and reminding that supp( f ) = T = [q min , q max ] from (3), we have:

Necessary and Sufficient Conditions for Equilibrium
Having characterized the power demand of the single appliances and of the whole population in response to a broadcast profile D, we can now design such signal in order to achieve a Nash equilibrium in the electricity market, as specified in Definition 1. The equilibrium analysis will compare two different functions in the measure variable q = Q D (D(t)) which are related to the chosen broadcast profile D and the energy density f of the flexible appliances population. The first function, named negotiable valley capacity, quantifies the amount of flexible demand that can be greedily allocated by the appliances while preserving the existence of a Nash equilibrium and is defined as follows: This definition is independent of the price of energy , and it will serve in the subsequent analysis as an indicator of flexible power density allowed in the valleys of the inflexible demand as a function of the measure q.
The second considered function is named power density of task durations and describes how a certain population of appliances will allocate their power consumption, based on the distribution m of their parameters τ and E: Definition 6 For a population of flexible appliances with parameter distribution m and energy density f , we define the power density of task durations as the function f : [q min , q max ] → [0, +∞) given below: q is a density function quantifying the total rated power of the appliances with parameter τ = q. Therefore, given q 1 , dτ is equal to the total rated power of the appliances with τ ∈ [q 1 , q 2 ]. It will be argued in the rest of this section that the existence of an equilibrium can be verified by comparing the values of f and D i on the interval supp( f ) = T = [q min , q max ].

Theorem 1 Consider a continuous profile of inflexible demand D i fulfilling Assumption 3.
A Nash equilibrium is achieved in the system for D = D i (as specified in Definition 1) if and only if: Proof See "Appendix B".
Theorem 1 states that, if the power density f is lesser or equal than the negotiable valley capacity D i on the set T , a Nash equilibrium can be achieved through a single demand/price broadcast, communicating the signal D = D i (or equivalently the associated price (D i )) to the flexible loads. Each device will schedule its power consumption at constant maximum rate within the sublevel set of D i of appropriate measure, viz. corresponding to its own minimum time parameter τ .
We want to understand next if alternative profiles D, different from D i , could be broadcast and induce an equilibrium when condition (22) is violated. To this end, let us regard two broadcast profiles as equivalent if they induce (for almost all times) the same scheduling of flexible demand. The equivalence class D i of the signal D i can be characterized as in the definition below.
From expression (16) of the flexible demandD f , reminding that q = Q D (D(t)) and supp( f ) = [q min , q max ], condition (23a) is equivalent to impose that time intervals for which the flexible demand equals its maximum value τ dτ when D b and D i are broadcast coincide up to sets of zero measure. Similarly, condition (23b) ensures that the intervals for which flexible demand is equal to zero when D b and D i are broadcast differ for sets of zero measure. Finally, condition (23c) imposes equality of the flexible demand at all other time instants. It is now possible to verify that an equilibrium can be achieved only with broadcast signals D that belong to D i . (6) is satisfied for a broadcast profile D = D b fulfilling Assumption 3 if and only if D b ∈ D i and inequality (22) holds.

Theorem 2 The Nash equilibrium condition
Proof See "Appendix C". Theorems 1 and 2 provide (respectively) sufficient and necessary conditions for the existence of a Nash equilibrium as specified in Definition 1. Any broadcast profile in the set D i will induce an equilibrium for the resulting aggregate demand if and only if the inequality (22) between negotiable valley capacity D i and power density of task durations f holds. If this is not the case, a Nash equilibrium cannot be achieved with a broadcast profile D which fulfils Assumption 3.

Optimality Properties of Nash Equilibrium
This section derives fundamental properties of the Nash equilibrium, analytically proving its social efficiency and providing sufficient conditions for its Pareto optimality. To this end, it is necessary to adopt a different notation so as to accommodate the more general case in which the power u(·) is not scheduled by each device as the best response u * D (·, τ, E) to a broadcast signal D. The power profile u cannot be defined as a function of τ and E since, in general, appliances with equal parameters could have different power profiles. For an alternative notation, the set of devices is denoted by V , while ν ∈ V represents the single device with minimum task time τ (ν) and total energy E(ν). Denoting by u(·, ν) the scheduled power of the device ν, the set U of feasible power profiles can be defined as follows: ) Under the current notation, the power scheduling u * (t, ν), obtained when D i is broadcast to the devices and corresponding to a Nash equilibrium when (22) holds, is straightforward to derive from (18): The flexible demand profiles obtained with the application of an arbitrary u ∈ U and of the Nash equilibrium inducing u * are denoted by D f and D * f , respectively, and have the following expression: To evaluate the optimality properties of the Nash equilibrium, the following functional J is considered: where g(D i , d) is strictly convex with respect to d. In order to perform the subsequent analysis in the measure variable q, the following hypothesis is introduced: Assumption 4 Consider the following subsetŪ ⊂ U of feasible power schedules: (28) For any functional J as defined in (27) and u 1 ∈ U \Ū (with associated flexible demand D f,1 ), there exists u 2 ∈Ū (and flexible demand D f,2 ) such that: In other words, if the functional J is evaluated over the feasible profiles of flexible demand, its minimizer d = D f is a function that fulfils (11) for ρ = D f and D = D i and therefore is well defined in the variable q = Q D i (D i (t)). This assumption can be easily verified with standard optimal control techniques if one relaxes the constraints and directly minimizes J in (27) over the positive functions d with a given total integral. The following optimality result can now be provided: Theorem 3 Consider an arbitrary power scheduling u ∈ U and u * as defined in (25). Let D f and D * f in (26) denote the corresponding profiles of flexible demand and consider the following functional J C : with g C strictly convex. If (6) is fulfilled for D = D i and u * corresponds to a Nash equilibrium, it holds: Proof See "Appendix D".
It follows from Theorem 3 that, if a Nash equilibrium exists, this is also optimal for a convex functional of aggregate demand. Note that this formulation is adopted in similar works to represent social welfare (a discrete-time version is presented in [13]) and can quantify, for example, the total generation costs or the flattening of the demand profile introduced by the flexible loads. We can therefore conclude that the proposed notion of Nash equilibrium is always socially efficient.
The Pareto optimality of the Nash equilibrium has also been analysed. In particular, we are interested in determining if there exists a feasible power schedule u ∈ U \{u * } which allows to reduce the cost sustained by at least one device, without increasing the costs of other appliances. The energy costs C(ν) and C * (ν) sustained by the individual device ν when u and u * are applied are straightforward to derive: In this context, the following result is provided: Dyn Games Appl Theorem 4 Assume that u * in (25) corresponds to a Nash equilibrium, denoting byD * f the equivalent in the variable q of D * f in (26). If the price function is strictly convex and the following inequality is fulfilled forD * a =D i +D * f : then u * is Pareto optimal and there exists no feasible u ∈ U such that the following holds for the energy costs C in (31): The application of Theorem 4 is particularly interesting when an affine price function (d) = a + b · d is considered. In this case, condition (32) corresponds to the following inequality between the negotiable valley capacity D i and the power density of task durations f :

Discussion
In this section we qualitatively discuss the equilibrium properties and the implementation of the proposed control strategy, highlighting its main novel elements and discussing its limitations and future developments.

Nash Equilibrium Properties
Necessary and sufficient conditions for equilibrium have been derived in Theorems 1 and 2, analytically characterizing the penetration levels of flexible demand and the devices parameters for which a Nash equilibrium can be obtained. Given the quantities D i and f , describing, respectively, the inflexible and flexible demand, an equilibrium can be induced if and only if the inequality (22) is fulfilled. Note that such condition can alternatively be expressed as monotonicity of the aggregate demand profile in the measure variable q. In fact, by comparing Definition 6 and (16), it can be seen that f in (22) is the derivative of the flexible demand in the variable q with changed sign. For D = D i , we have thatD i (q) = Q −1 D i (q) from (10) and the term D i , as specified in Definition 5, represents the derivative of the inflexible demand with respect to q. The equilibrium condition (22) can then be rewritten as: As discussed in detail later, this formulation constitutes a key insight for the development of an extended analysis that relaxes some of the initial assumptions and provides more general results.
Following the derivation of the necessary and sufficient conditions for the existence of a Nash equilibrium, it has been shown in Sect. 6 that such equilibrium is always socially efficient, determining in Theorem 4 the conditions that also ensure Pareto optimality. In this context, it is worth mentioning that the Nash equilibrium, when it exists, is also unique. Its uniqueness under Assumption 3 follows from Theorem 2, since an equilibrium can only be achieved by broadcasting a demand profile D b ∈ D i and all profiles in the signal class D i induce by definition the same power scheduling and flexible demand.

Distributed Implementation
Having discussed the properties of the Nash equilibrium, it is worth emphasizing some key aspects of the distributed control scheme that induces it:  (4). As for the latter case, the parameters value τ and E of the individual load have negligible impact on the energy density f . Therefore, the individual appliances have no interest in providing false parameters values, as they would not be able to modify the energy density f , which is used by the system operator to determine the broadcast demand profile D. This property does not hold in general if a nonnegligible fraction of the total devices create a coalition, agreeing to purposefully collaborate in providing false information and distorting the density f for their own advantage.
-One-step resolution There is no iterative exchange of information between the appliances and the system operator: a single demand profile D (or equivalent price (D)) is broadcast to the appliances, which independently perform their power scheduling. We wish to emphasize that such choice does not restrict the equilibrium results. As established in Theorem 2, if the inequality (22) is not fulfilled there exists no broadcast power profile D b associated with a Nash equilibrium. Therefore, also alternative methods that consider an iterative power scheduling of the devices, on the basis of updated demand/price signals, would not ensure better results, as there is simply no stable configuration to which they could converge.

Theoretical Assumptions: Restrictions and Possible Solutions
Finally, we wish to discuss in more detail the assumptions adopted in our study, evaluating their impact and proposing possible approaches for a more comprehensive analysis. From Assumption 1, only noninterruptible appliances with complete time availability can be controlled. Such hypothesis, albeit restrictive, can nevertheless accommodate realistic scenarios (e.g. charge of electric vehicles during night-time) and has been considered in other works on the subject, such as [18,24]. It is also worth mentioning that the power profile u(·) = u * D (·, τ, E), scheduled by each device in response to the received demand signal D, is bang-bang. As a result, the proposed coordination strategy not only applies to devices that can continuously modulate their power consumption but it can also be extended to ON-OFF appliances. The possibility of considering devices with partial flexibility is preliminarily explored in Sect. 8, designing a distributed coordination of the flexible loads which parameterizes the broadcast signal as a "single valley". The proposed technique could be extended, in future works, to consider a more general structure of the broadcast signals (for example com-posed by multiple valleys), ensuring appliances coordination and equilibrium under more general conditions. The inclusion of noninterruptible appliances in the current modelling framework is also being evaluated, through a two-step power scheduling. For example, one could first coordinate these additional loads (whose power consumption is more difficult to characterize in a compact manner) and then consider the resulting aggregate demand profile as D i in a second power scheduling of the interruptible appliances, using the techniques presented in this work.
Regarding Assumption 2, it is assumed that the energy density f is a well-defined and bounded function. For this to be the case, the number of considered flexible appliances needs to be infinite. In practical contexts, a finite number of flexible loads can be described with an acceptable degree of approximation through a bounded function f if the considered population is extremely large. In this respect, we wish to emphasize that the proposed control strategy has been designed for future large-scale applications on national power systems, expecting millions of flexible loads. For a faithful description of flexible demand through a bounded function f , it is also necessary that the time parameter τ is sufficiently diversified across the considered loads. Such property is generally verified by populations of heterogeneous loads or by homogeneous devices that need to perform different tasks. Note that Assumption 2 simplifies the analysis, allowing to introduce f in Definition 6 and derive the equilibrium condition (22). A more complex study should be able to obtain similar results without Assumption 2, using the differential equilibrium condition (35) as a starting point and demonstrating that, in the present case, an equilibrium can only be achieved if the profile of aggregate demandD a (discontinuous and nondifferentiable) is nondecreasing.
Finally, the results presented so far assume that the broadcast demand signal D fulfils Assumption 3 and does not have any level set of positive measure. If such assumption is removed, the best response of each appliance is, in general, not unique. For example, if the broadcast demand is constant with D(t) = d ∀t ∈ [0, T ], then all feasible power schedules are also optimal. Moreover, the definitions ofū * andD f provided in the previous section are no longer valid and the presented equilibrium conditions do not apply. An ongoing work suggests that existence and shape of a Nash equilibrium can still be determined under this relaxed conditions. In order to achieve this result, though, it is necessary to centrally coordinate some appliances on time intervals characterized by constant broadcast signal D. Alternatively, we show in [10] that a Nash equilibrium can be obtained by introducing a proportional constraint on the power rate of the appliances, limiting the flexible demand at critical time instants.

Appliances with Partial Flexibility
As established in Assumption 1, so far we have considered appliances that can schedule their power consumption at any time instant within the considered interval [0, T ]. Such formulation can be expanded to account for devices with stricter time constraints. In particular, we will consider the case of appliances that must perform their tasks at t ≥ κ where κ is the availability parameter of the individual device. This corresponds to an additional constraint on the power scheduling u: Remark 1 The analysis in this section can be extended to appliances that also specify a final availability time, obtaining similar results. For a more compact presentation, in this paper the simpler case with only the initial availability time κ is presented.
An additional variable must now be considered in the unnormalized distribution m of the appliances parameters: (τ, κ, E) dE dκ dτ denotes the number of devices for which E 1 ≤ E ≤ E 2 , t 1 ≤ τ ≤ t 2 and also κ 1 ≤ κ ≤ κ 2 . Similarly, the function f (τ, κ) in this case quantifies the total amount of energy required by devices with time parameter τ and availability parameter κ: (37)

Assumption 5
The energy density f has compact support:

Assumption 6
The availability time-window of each appliance allows task completion. In other words, all appliances are able to complete their task by operating at rated power in their availability window [κ, T ]: After a preliminary equilibrium analysis in Sect. 8.1, we describe the considered class of broadcast signals, providing expressions for the corresponding scheduled power consumption and flexible demand, in Sect. 8.2. The conditions for a Nash equilibrium are derived in Sect. 8.3, while Sect. 8.4 contains a design method for the broadcast profile.

Preliminary Equilibrium Analysis
The appliances schedule their power consumption in order to complete their task at minimum energy cost, on the basis of a broadcast demand profile D that fulfils Assumption 3. The additional availability constraint (36) is now included in the power scheduling problem through the indicator function, denoted by 1 x . The optimization problem solved by the single device with parameters τ , κ and E becomes: In order to calculate the solution u * D , it is useful to provide a preliminary result: (19), for any (τ, κ) ∈ supp( f ) there exists λ ∈ [0, T ], that we denote by (τ, κ), such that the following holds:

Proposition 2 Given the sublevel set S D defined in
Proof Under Assumption 3 for the broadcast D, the term μ ( [κ, T ] ∩ S D (y) ) in (39) is continuous and nondecreasing with respect to y. Furthermore, we have: Since τ ≤ T − κ for all (τ, κ) ∈ supp( f ) from Assumption 6, we can conclude that (τ, κ) exists as specified in the claim.
Note that S D ( (τ, κ)) returns the set of time instants t with the lowest values of broadcast demand D(t) (and lowest price (D(t))) whose intersection with [κ, T ] has measure τ . Hence, the solution to (38) for a device with parameters (τ, κ, E) has the following expression: The resulting profile of aggregate demand is straightforward to derive: where T , K and E denote, respectively, the set of distinct parameters τ , κ and E in the appliances population. As in the previous analysis, a Nash equilibrium is achieved if the following holds for all (τ, κ) ∈ supp( f ) and E ∈ E : In the case of appliances with partial flexibility, the concepts of negotiable valley capacity and power density of task durations are no longer applicable. In fact, given the additional constraints on the initial time of power consumption, condition (11) This in turn implies that there exists (τ, κ) such that: D ( (τ, κ)) .
which proves condition (43) and concludes the proof.

Description of Broadcast Signal and Power Scheduling
We restrict our study to a specific class D m of broadcast signals D : [0, T ] → [0, T ] that can be characterized as "single valleys". This allows to provide compact expressions for the scheduled power consumption of the devices and ultimately derive sufficient conditions for a Nash equilibrium.
Furthermore, for some T 0 ∈ [0, T ], at q = 0 and q = T we have: Definition 8 Any demand signal D which belongs to the class D m has the following expression: for some functions θ L and θ R that fulfil (46) and (47).
The profiles D ∈ D m can be visualized as valleys, with one decreasing profile θ −1 L in the interval [0, T 0 ] and an increasing one (θ −1 R ) on [T 0 , T ]. Note that D ∈ D m is properly defined by (48) since the functions θ L and θ R are invertible and the union of their images is equal to [0, T ]. Moreover, it holds Im(θ L ) ∩ Im(θ R ) = T 0 with θ −1 L (T 0 ) = θ −1 R (T 0 ) = 0. Given (48), the following relationship holds for the broadcast profile D ∈ D m evaluated at θ L (q) ∈ [0, T 0 ] and θ R (q) ∈ [T 0 , T ]: The power scheduling of the flexible appliances can now be characterized by the following result: Proposition 4 Given D ∈ D m and the corresponding functions θ L and θ R , the power consumption of the single device with parameters (τ, κ, E) is scheduled (at maximum feasible rate E/τ ) during a compact intervalS D (τ, κ): Proof For the first case in (50), it holds [θ L (τ ), θ R (τ )] ⊆ [κ, T ]. Furthermore, considering (46) and (48), the following properties are satisfied: For the second case in (50), since D is monotonic increasing for t > T 0 , it is sufficient to show the following: Since κ > θ L (τ ), we have κ + τ > θ L (τ ) + τ = θ R (τ ) > T 0 . For the second inequality in (52) two different cases have to be analysed: if t > T 0 then (49), it follows: which proves (52) and concludes the proof.
From Proposition 4, the power profileũ * D scheduled by appliances with parameters (τ, κ, E) can be written as: The resulting flexible demand D f,D (t) is straightforward to derive:

Proposition 5
The following expressions of D f,D as a function of θ L and θ R hold: Proof To prove (56a), consider definition (50) ofS D and note that no appliance completes its task for t < T 0 . This means that at time θ L (q) ≤ T 0 the flexible demand can be determined by taking into account only the starting time of power consumption. For any appliance with initial availability κ < θ L (q), two cases must be considered: if the task duration at rated power τ ∈ (θ −1 L (κ), T ] ⊂ [q, T ], then it holds κ > θ L (τ ) and from (50) the starting time is equal to κ ≤ θ L (q). If instead τ ∈ [0, θ −1 L (κ)], the starting time of power consumption is equal to θ L (τ ) and only devices with task time τ ≥ q must be accounted for in the computation of flexible demand. This proves expression (56a). To verify that also (56b) holds, note that the considered values of the parameter κ are lesser or equal than θ R (q) and, from (50), the same holds for the starting time. The integration bounds in τ when κ ≤ θ L (q) can be determined similarly to the previous case, considering the final time of power consumption. If θ L (q) < κ ≤ θ R (q), the only possible case to analyse in (50) is κ > θ L (τ ), when the interval of power consumption scheduled by the devices equals [κ, κ + τ ]. To account for the appliances operating at t = θ R (q), it is sufficient to impose κ + τ ≥ θ R (q) and therefore τ ≥ θ R (q) − κ.
Having calculated D f,D , the resulting aggregate demand D a,D can be easily derived:

Sufficient Conditions for Nash Equilibrium
The main equilibrium result for appliances with partial flexibility is now presented: Theorem 5 A Nash equilibrium is induced in the system and condition (43) is satisfied for D ∈ D m if, for the corresponding functions θ L , θ R and profile of aggregate demand D a,D , it holds: Proof See "Appendix F".
Proposition 3 and Theorem 5 determine whether an equilibrium exists by verifying that the optimal power profile of the devices, defined in (41), is identical when the broadcast D or the resulting aggregate demand D a,D are considered. Such conditions are only sufficient since the optimal power profile is unique and equal to (41) only if the broadcast signal D satisfies Assumption 3. If the same holds for the resulting aggregate demand D a,D , the equilibrium conditions become also necessary.

Synthesis Technique
Following the results of Theorem 5, it is possible to provide a constructive technique which verifies, for a given D i and f , if there exists a profile D ∈ D m which satisfies (58) and allows to calculate it numerically. To do so, it is useful to derive expressions for the derivatives with respect to the variable q of the different demand components: By replacing (59) in the equality of (58), it is possible to calculate θ L (q) and θ R (q) as the solution of the following dynamical system: If the solution of (60) fulfils the inequalities in (46) and (58) for some T 0 ∈ [0, T ], the corresponding D ∈ D m defined according to (48) induces an equilibrium. To determine T 0 , it is useful to consider the equality in (58), evaluated at θ L (q) = θ R (q) = T 0 :

Simulation Results
The equilibrium conditions presented in the previous section are now tested in simulations. A typical 24 h UK demand profile, as recorded by [22], is considered for D i (blue trace in Fig. 2), with a time discretization step t = 0.01 h. If one denotes by D k i the value of inflexible demand at t = k · t, the function Q D i D k i is approximated as S k · t where S k is defined as follows: In the first case study we consider a population of flexible appliances for which the equilibrium condition (22) in Theorem 1 is satisfied for the given D i . The total energy required by the devices amounts to 55 GWh and the corresponding f is a truncated Gaussian with mean equal to 8.2 h. This choice can represent heterogeneous devices that have different power ratings, but it can also model scenarios with only one type of device (with equal P = E/τ ) where each appliance needs to perform tasks that require different amounts of energy to be completed. The distribution m introduced at the beginning of Sect. 2, in the latter case, would have the following expression: where δ denotes the Dirac delta andm is the unnormalized distribution of the parameter E (total required energy). We can derive from (2) the relationship between f andm: The values of f (q) and D i (q) in the considered scenario are shown in Fig. 5. From Theorem 1, since the power density of task durations f (red) is always lesser or equal than the negotiable valley capacity D i (blue), an equilibrium is achieved by broadcasting to the appliances the profile of inflexible demand D i . The resulting demand components as functions of the measure q are shown in Fig. 6: given thatD i (q) = Q −1 D i (q) when D = D i , we expect the inflexible demand to be monotone increasing in the variable q. Conversely, the flexible demandD f (q), defined in (16), is always decreasing since it is equal to the integral of a positive function over the interval [q, T ]. As discussed in Sect. 7.1, since (22) is satisfied, the sumD a (q) of the two demand components will be nondecreasing. The Since such intervals are also characterized by the lowest values of aggregate demand D a,D i , the considered devices have no interest in changing their scheduled power consumption, implying that a Nash equilibrium is achieved. A different case study is now simulated, considering a population of flexible devices whose task duration profile f does not satisfy the equilibrium condition presented in Theorem 1. In particular, the function f is defined as the sum of two truncated Gaussians with mean equal to 4 h and 8 h. This choice could model, for example, two distinct types of appliances (with different rated power P = E/τ ), considering that all devices have similar values of the parameter E. The total energy required by the appliances, as in the previous scenario, amounts to 55 GWh. A graphical representation of the equilibrium condition in Theorem 1 is provided in Fig. 8: it is straightforward to verify that an equilibrium cannot be achieved in the present case since f (q) > D i (q) in the interval which goes approximatively from q = 2 h to q = 5 h. This can also be seen from the demand profiles shown in Fig. 9, obtained when D = D i is broadcast to the devices. In this case the shaded areas, which represent the scheduled interval of power consumption of devices with τ equal to 2 h, 4 h and 8 h, do not correspond to the lowest values of aggregate demand. For example, the appliances with τ = 2 h could reduce their total cost by shifting part of their power consumption to the small valley of aggregate demand which appears around t = 6 h. Therefore, the operation strategy formulated by the devices on the basis of the broadcast signal D = D i is not optimal for the resulting price signal induced by the aggregate demand and an equilibrium is not achieved.

Appliances with Partial Flexibility
Coordination of flexible appliances with partial time availability has also been simulated, applying the technique presented in Sect. 8.4. The proposed example considers a parabolic profile for the inflexible demand (with T = 12 h) and an appliances population that requires 10 GWh of total energy. The distribution of the parameter κ has been modelled with a truncated Gaussian with mean equal to 5 h, while the values of minimum task time τ are in the range [2 h, 5 h]. The resulting function f (τ, κ) is shown in Fig. 10. The initial condition T 0 for θ L and θ R has been calculated using (61) and is equal to 4.21 h. The equations in (60) have been integrated, and the broadcast profile D has been obtained with (48). The corresponding cumulative distribution Q D is compared in Fig. 11 with the ones of the inflexible and aggregate demand. The comparison of the demand profiles is presented in Fig. 12. As expected, the measure Q D a,D of the aggregate demand is equal to the function D (and to the corresponding Q D ), and therefore, an equilibrium is achieved according to Proposition 3. Note also that D has a minimum at time T 0 = 4.21 h, while the minimum in D i is achieved at t = 6 h. This is due to the time constraint κ: the earlier power scheduling of devices with lower values of κ (during a time interval which is not characterized by the lowest values of inflexible demand) is balanced by the other appliances which are constrained to operate at later times.

Conclusions
This paper proposes a novel fully distributed control scheme for coordination of flexible appliances and their efficient integration in the electricity market. The appliances population is approximated as a continuum, modelling the conflicting interactions between the individual devices through a differential game. An efficient deployment of flexible demand, which avoids rebound effects and loss of diversity, is characterized as a Nash equilibrium. Through the analysis of the sublevel sets of the different demand profiles, necessary and sufficient equilibrium conditions are derived, determining the penetration levels and types of priceresponsive appliances for which a suitable coordination can be induced. The optimality properties of the Nash equilibrium are also studied, analytically proving its social efficiency and deriving sufficient conditions for its Pareto optimality. A preliminary analysis of devices with partial flexibility has also been conducted, providing equilibrium conditions and a design technique for the broadcast demand when an additional constraint is introduced on the initial time of power consumption. Finally, the performance of the proposed control scheme has been evaluated in simulations.
verified if the measuresQ * D μ and μ are equal and the following holds for any measurable set X : Such condition is initially verified for a closed interval X = [0, x R ] with x R ≤ T . In this case, the setQ −1 D (X ) has the following expression: Since Q D (d) is a strictly monotone increasing function with image equal to [0, T ], there existsD such that Q D (D) = x R and we have: With similar steps, it is possible to verify that (65) is satisfied and The equivalence of the measures can be extended to any measurable set X by applying the Vitali covering theorem [5], which guarantees that for any set X there exists an at most countable set of disjoint closed intervals I j such that μ X \ ∪ j I j = 0. This means that (65) always holds and the lemma statement is verified.

Appendix B: Proof of Theorem 1
We remind that u * D in (18) is optimal for (1) since the following inequality holds (as a result of (19) and monotonicity of Q D ) for all τ ∈ T : To see this, consider that the feasible controls for (1) have fixed integral and u * D is bang-bang (equal to rated power on S D ). This means that, when u * D is applied, each device cannot shift part of its power consumption to time instants characterized by lower demand/price. For the same reasons, u * D is the solution of the minimization problem in (6) (when the aggregate profile D a,D is considered) if and only if a similar inequality holds for all τ ∈ T : (the "only if" direction being a consequence of continuity of the aggregate demand if Assumption 3 is fulfilled). Given expression (19) of the set S D (τ ) for D = D i , the optimality condition (69) on D a,D i (equivalent to (6)) can alternatively be written, for all τ ∈ T , as: The function K , defined in (21), returns the aggregate demand when the inflexible profile D i is broadcast. A more general expression which accounts for all values of τ can also be provided: In order to define the set D C , consider that the comparisons in (69) and (70) are performed between the demand values at some instant t 1 , included in the set S D i (τ ), and some other t 2 not included in it. Since the inequality must hold for all τ ∈ T = [q min , q max ], if one denotes by d min and d max the minimum and maximum values of inflexible demand, it is possible to provide the following definition: The sets D min (and D max ) correspond to pairs of demand values at which all devices are consuming power (respectively, not consuming). From expression (14) of the scheduled power in the variable q, considering that all values of the parameter τ are in the interval [q min , q max ] from (3), in the present case with D = D i we have: Note that the function K (d) in (21) is monotonically increasing in the intervals [d min , Q −1 D i (q min )] and [Q −1 D i (q max ), d max ], which represent the sets of demand values d for which Q D i (d) < q min and Q D i (d) > q max , respectively. Therefore condition (71), equivalent to (69), corresponds to K being monotonically increasing on D D i = Dividing both terms of the inequality by Q D i (d) and letting q denote Q D i (d) yields a condition equivalent to (22), concluding the theorem proof: Considering that s 0 l(q) dq ≤ 0 and assuming for the sake of simplicity that the number of connected components of S − is finite and equal to n, it is possible to partition S + and S − in n subsets (S 1 + , . . . , S n + ) and (S 1 − , . . . , S n − ): such that, for all i ∈ {1, . . . , n}, it holds: Such partition can be performed by choosing S i − as the connected components of S − . We denote now as q 1 + ≤ q 2 + ≤ · · · ≤ q n + ≤ q n+1 The corresponding S i + is defined as q i + , q i+1 + × R ∩ S + . Introducing the function V C (q, x) = g C (D i (q) +D * f (q) + x) and recalling thatD f =D * f + l, the functional J C (D f ) in (93) can be rewritten as: where the third and fourth equality follow from (98) and the set partition in (99), respectively. Note that the quantity ∂ ∂ x V C (q, x) = ∂ ∂ x g C (D i (q)+D * f (q)+ x) is monotone increasing with respect to x (convexity of g C ) and with respect to q, since from (35) the aggregate demand D i +D * f is monotone increasing at equilibrium. Given (100b)-(100c) and the mentioned monotonicity properties of ∂ V C ∂ x , the following inequality holds for all i = 1, . . . , n: From (100a) and monotonicity of the integral, it follows: As a result, each element of the sum in the last term of (101) is nonnegative, implying that (93) holds and therefore concluding the proof.
Equality a holds by definition of D ∈ D m , while b and c are a result of the monotonicity properties of the functions θ L and θ R (and consequentially of the inverse θ −1 L and θ −1 R ). The last equality d corresponds to the equation in (46) evaluated at q = D(t).