1 Introduction

With the high penetration of renewable generation and the increasing demand, the uncertainties in power systems are gradually increasing [1]. Therefore, more controllable devices are required to accommodate the uncertainties [2]. It is insufficient to rely only on the controllable generation resources in power system. Although electric vehicles [3] and storage [4] act as controllable resources, their excessive investment cost makes them difficult to obtain on a large-scale [5,6,7]. The more direct, effective, and economical way may be to guide customers to consume electricity rationally and efficiently [8, 9]. Thus, the role of demand response (DR) becomes more and more prominent [10].

Customers’ behaviors will affect units’ dispatch strategy, and enhancing the interaction between an energy supplier and its customers can eliminate the energy imbalances in power systems [11]. Analyzing the response behavior of different types of consumers is the basis for designing effective DR strategies to match the conventional generation dispatch. Meanwhile, designing reasonable market mechanisms, including appropriate compensation scheme for customers providing DR, is essential to ensure that DR is available to assist the safe and economic operation of power system.

An energy supplier designs a DR compensation scheme in order to acquire reliable resources that maybe scheduled in a timely manner, helping to cope with an emergency shortage of generation. Although the mechanism can be better designed using optimization methods [12, 13], what counts more is whether it is designed reasonably and executed accurately by the customers. A reasonable scheme can only be developed if sufficient information can be obtained, but in practice there exists an informational asymmetry between the energy supplier and its customers, since a customer’s private information (e.g. outage cost and preference parameters) is unknown to the energy supplier and other customers. In general, this information is difficult to obtain accurately from commercial customers due to their confidentiality requirements for strategic purposes [14]. Moreover, some customers wishing to maximize their own benefit may lie to energy suppliers [15, 16]. These factors can lead to low efficiency of the DR mechanism.

To solve this problem, [17] proposes mechanism design theory, and it encourages the participants to disclose their true value of outage cost, willingness and other private information by combining incentive compatibility and informational efficiency. Reference [18] introduces a continuous customer type parameter into the customer cost function, and applies mechanism design principles to formulate a model interruptible load management (ILM) contract with incentive compatibility, which is suitable for different types of customers. Reference [19] presents an improved approach to estimate the customer type and the customer outage cost function by using historical demand side management (DSM) data, and formulates the ILM contract model with a discrete customer type parameter. All show that the critical issue is how to give effective financial compensation to customers to encourage helpful energy consumption behavior and alleviate stress on the power grid through load curtailment. Mechanism design seems a promising way to solve the problem.

In China, an electricity market on the power generation side was created by separating power plants from the power network. Price competition is still not perfect in this market, and dispatch operations and electricity sales are still being integrated. Including reasonable DR mechanisms into the system dispatch model is an effective approach to improve economic efficiency and increase system security. In [1, 20, 21] time-of-use (TOU) pricing and interruptible load (IL) have been integrated into the conventional generation dispatch model and they can reduce cost and improve the accommodation of renewable energy. In [22, 23] the authors discuss the impact of price changes on demand elasticity and propose that unit commitment (UC) can be determined by minimum cost rather than optimal power flow (OPF) in the generation dispatch model. In [24], the author analyzes the impact of price-based DR on market clearing and locational marginal prices (LMPs) in a power system using network-constrained unit commitment (NCUC).

However, these studies are all based on the following assumptions: 1) demand is treated as a fixed value at each time (inelastic demand); 2) customers’ benefit functions are constant or linear functions of price, and their preferences are expressed solely in the form of constraints; 3) the integrated generation–consumption dispatch problem is converted from a social welfare maximization problem to an equivalent generation cost minimization problem. The market price is determined only by the marginal cost of generators. Additionally, it is considered that this price equals the consumers’ marginal utility. These assumptions help to quantify the model, however, they may not reflect the impacts of the demand side on market price setting, which may cause price spikes, and they fail to explain the actual DR behavior and reflect customers’ true utility. It is important that market pricing is co-determined from a common decision between the supply side and the demand side. Thus, DR mechanisms that do not consider users’ utility would fail to incentivize the initiative of customers.

The novel contribution of this paper, based on the electricity market in China [25], is an integrated generation–consumption dispatch model considering DR behavior based on mechanism design principles. Firstly, in light of the day-ahead load forecast data, we can deduce the utility function of different customers. The utility function is used to describe consumers’ preferences and energy consumption behaviors. Secondly, by subtracting units’ operational cost from consumers’ total utility, a new integrated generation–consumption dispatch model is formulated with decentralized demand participation. The new model improves the interaction of suppliers and their customers, which can benefit both of them. Thirdly, based on the above UC results, taking into account the behaviors of users and security threats of the system, an effective compensation mechanism is designed to ensure customers select a compensation scheme according to their true willingness to curtail of load. Finally, a new dispatch model is formed by incorporating the DR compensation mechanism into the generation–consumption dispatch model, which co-optimizes DR and generation–consumption dispatch decisions. The new model can realize maximum social welfare with Pareto optimality for which the customers’ marginal utility is equal to the marginal cost of generators.

The remainder of the paper is organized as follows. Section 2 describes the customer behavior and the mathematical model of the proposed compensation mechanism. Section 3 proposes the integrated generation–consumption dispatch model considering the demand utility function and the DR compensation mechanism integration. Section 4 gives the detailed method to obtain optimal solutions using the model. Section 5 demonstrates and discusses numerical results obtained for a six-bus system and the IEEE 118-bus system. Finally, Section 6 concludes this work. Appendix A gives a simple example to illustrate a derivation required for the analysis \(\theta\). Appendix B presents some details about the mathematical formulation of compensation mechanism.

2 Customer behavior and compensation mechanism

2.1 Customer behavior

Influenced by the factors such as the time, environment, national policies, etc., the electricity demand of each customer is different, therefore their response to certain incentives differs. According to the principle of micro-economics, a quadratic utility function is adopted to represent customers’ preferences and consumption behaviors [26]:

$$U(\alpha_{jt} ,D_{jt} ) = \left\{ {\begin{array}{*{20}c} {\alpha_{jt} D_{jt} - \frac{1}{{2\beta_{jt} D_{jt}^{2} {\kern 1pt} }}} & {0 \le D_{jt} \le \frac{{\alpha_{jt} }}{{\beta_{jt} }}} \\ {\frac{{\alpha_{jt}^{2} }}{{2\beta_{jt} }}} & {D_{jt} \ge \frac{{\alpha_{jt} }}{{\beta_{jt} }}} \\ \end{array} } \right.$$
(1)

The marginal utility is then

$$\begin{aligned} MU(\alpha_{jt} ,D_{jt} ) = & \frac{{\partial U(\alpha_{jt} ,D_{jt} )}}{{\partial D_{jt} }} \\ \, = & \left\{ {\begin{array}{*{20}c} { - \beta_{jt} D_{jt} + \alpha_{jt} } & {0 \le D_{jt} \le \frac{{\alpha_{jt} }}{{\beta_{jt} }}} \\ 0 & {D_{jt} \ge \frac{{\alpha_{jt} }}{{\beta_{jt} }}} \\ \end{array} } \right. \\ \end{aligned}$$
(2)

In (1) and (2), \(D_{jt}\) is the electricity demand of customer \(j\) in period \(t\); \(\alpha_{jt}\) is a parameter characterizing the demand preference of customer \(j\) in period \(t\); \(\beta_{jt}\) is the saturation point of the utility of customer \(j\) in period \(t\). The greater is \(\alpha_{jt}\), the greater is the customer’s saturation electricity demand. The greater is \(\beta_{jt}\), the lower is the customer’s saturation electricity demand [27]. It can be seen that the utility function is concave. The customer’s marginal utility increases as the demand increases before reaching the saturation point, and is assumed to be constant afterwards. Assuming the customer is rational, his or her demand will naturally maximize individual utility. Consequently, \(\bar{D}\) reflects the customer’s demand of maximum utility. At this optimal point, the marginal utility is zero.

$$\frac{{\alpha_{jt} }}{{\beta_{jt} }} = \bar{D}_{j}$$
(3)

Therefore, (1) can be replaced by:

$$U(\alpha_{jt} ,D_{jt} ) = \left\{ {\begin{array}{*{20}l} {\alpha_{jt} D_{jt} - \frac{1}{{2\beta_{jt} D_{jt}^{2} }}} & {0 \le D_{jt} \le \bar{D}_{j} } \\ {\frac{{\alpha_{jt}^{2} }}{{2\beta_{jt} }}} & {D_{jt} \ge \bar{D}_{j} } \\ \end{array} } \right.$$
(4)

Figure 1 illustrates the impacts of marginal utility on customers’ energy consumption behaviors. A part of the demand is necessary for the customers to maintain their basic productivity, and this part is perfectly inelastic and is defined as the taking load. Therefore, the responsive load part has a maximum range. To accurately describe the market participation behavior of customers and to better reflect the potential for customers to participate in DR, we define the ratio DRLjt of the DR amount and the original demand \(D_{jt}^{\text{o}}\) of customer \(j\) in period \(t\) as the DR level of the customer. Hence, the DRLjt is in the range \([0,1]\). DRLjt = 0 represents the case in which customer \(j\) in period \(t\) is not participating in DR.

Fig. 1
figure 1

Impact of marginal utility on DR level

Full size image
$$DRL_{jt} = \frac{{D_{jt}^{\text{o}} - D_{jt}^{\text{a}} }}{{D_{jt}^{\text{o}} }}$$
(5)

where \(D_{jt}^{\text{a}}\) is the actual load demand of customer \(j\) in period \(t\) after the DR participation.

As shown in Fig. 1, the marginal utility of customer j in period t during the peak periods is \(MU_{jt}^{\text{H}}\). If the demand amount is reduced to \(D_{jt}^{\text{a}}\) as the marginal utility decreases, the market will also fall to a new equilibrium \(E^{\prime}\). The unmet partial load during the peak periods will shift to the off-peak periods because their relative marginal utility increases during the off-peak periods \(MU_{jt}^{\text{L}}\). Therefore, the demand will increase to \(D_{jt}^{\text{o}} + \sum\limits_{\tau \ne t} {\gamma_{j\tau t} \cdot DRL_{j\tau }^{\hbox{max} } \cdot D_{j\tau }^{\text{o}} }\), where \(DRL_{jt}^{\hbox{max} }\) is the maximum DR level of customer j in period t and \(\gamma_{j\tau t}\) is the load-shifting rate of customer j from time t to time \(\tau\). This reflects that customers’ initiative leads to behaviors that are driven by their marginal utility.

This paper only focuses on load curtailment, the case of load-shifting is beyond the scope of this paper, namely \(\gamma_{j\tau t} = 0\). Hence the demand of customer \(j\) in period \(t\) when participating in DR is in the range as shown in (6).

$$(1 - DRL_{jt}^{\hbox{max} } )D_{jt}^{\text{o}} \le D_{jt}^{\text{a}} \le D_{jt}^{\text{o}}$$
(6)

Since the marginal utility curve passes through the two points \(((1 - DRL_{jt}^{\hbox{max} } )D_{jt}^{\text{o}} , \, MU_{jt}^{\text{H}} )\), \((D_{jt}^{\text{o}} , \, MU_{jt}^{\text{o}} )\), the parameters of marginal utility (2) can be derived as:

$$\alpha_{jt} = \frac{{MU_{jt}^{\text{H}} + (DRL_{jt}^{\hbox{max} } - 1) \cdot MU_{jt}^{\text{o}} }}{{DRL_{jt}^{\hbox{max} } }}$$
(7)
$$\beta_{jt} = \frac{{MU_{jt}^{\text{H}} - MU_{jt}^{\text{o}} }}{{DRL_{jt}^{\hbox{max} } \cdot D_{jt}^{\text{o}} }}$$
(8)

Here, for the sake of simplicity, we shall assume that the customer gross surplus for the taking load part is a constant and thus it can be neglected in the utility function. Therefore, the utility is limited to the responsive load part, namely from \((1 - DRL_{jt}^{\hbox{max} } )D_{jt}^{\text{o}}\) to \(D_{jt}^{\text{a}}\), as shown in the green shaded area in Fig. 1. Assuming that \(D_{jt} \le \bar{D}_{j}\), we have

$$\begin{aligned} U(\alpha_{jt} ,D_{jt} ) = & \int_{{(1 - DRL_{jt} ) \cdot D_{jt}^{\text{o}} }}^{{D_{jt}^{\text{a}} }} {\left( { - \beta_{jt} \tilde{D} - \alpha_{jt} } \right)} {\text{d}}\tilde{D} \\ \quad = & \,\alpha_{jt} D_{jt}^{\text{a}} - \frac{1}{2}\beta_{jt} \left( {D_{jt}^{\text{a}} } \right)^{2} - \alpha_{jt} (1 - DRL_{jt}^{\hbox{max} } )D_{jt}^{\text{o}} \\ \, & \quad - \,\frac{1}{2}\beta_{jt} \left[ {(1 - DRL_{jt}^{\hbox{max} } )D_{jt}^{\text{o}} } \right]^{2} {\kern 1pt} {\kern 1pt} \\ \end{aligned}$$
(9)

Note that (9) is a quadratic utility function which is difficult to use in analysis due to its nonlinearity. To circumvent this problem, we can use a piece-wise linear approximation to obtain the formulation shown in (11), which is suitable for mixed-integer linear programming (MILP).

$$U(\alpha_{jt} ,D_{jt} ){\kern 1pt} = \sum\limits_{k = 1}^{M} {s_{jkt} D_{jkt}^{\text{a}} }$$
(10)
$$\left\{ \begin{aligned} &s_{jkt} = \frac{{U(\alpha_{jt} ,D_{jkt} ) - U(\alpha_{jt} ,D_{j,k - 1,t} )}}{{D_{jkt} - D_{j,k - 1,t} }} \hfill \\& 0 \le D_{jkt}^{\text{a}} \le D_{jkt} - D_{j,k - 1,t} \hfill \\ &D_{jt}^{\text{a}} = (1 - DRL_{jt}^{\hbox{max} } )D_{jt}^{\text{o}} + \sum\limits_{k = 1}^{M} {D_{jkt}^{\text{a}} } \hfill \\ &D_{jkt} = (1 - DRL_{jt}^{\hbox{max} } )D_{jt}^{\text{o}} + \frac{k}{M} \cdot DRL_{jt}^{\hbox{max} } \cdot D_{jt}^{\text{o}} \hfill \\ \end{aligned} \right.$$
(11)

where \(s_{jkt}\) is the marginal utility at segment\(k\)of customer \(j\) in period t; \(D_{jkt}\) and \(D_{jkt}^{\text{a}}\) are respectively the load demand and the actual load demand at segment \(k\) of customer \(j\) in period \(t\) in the linearized DR formulation; \(M\) is the set of segments. A similar argument applies to the case when \(D_{jt} \ge \bar{D}_{j}\).

2.2 Compensation mechanism

From the perspective of maximizing social welfare, the balance between supply-side and demand-side responses must be the result of trading off generation cost and demand utility. Therefore, if DR is integrated into the conventional generation dispatch model, it will be necessary to investigate the impact of demand utility on the changes of customers’ energy consumption behaviors. The expected generation schedule will be realized by compensating the changes of the demand utility. For example, during peak periods, a reasonable compensatory mechanism can motivate customers to reduce load voluntarily. A compensation mechanism should determine the optimal amount of DR and optimal incentive fees to encourage the amount of DR that each customer is willing to curtail.

The cost of demand reduction is modelled with two key parameters: the preference parameter \(\theta_{j}\) of the willingness of customer \(j\) to curtail and the amount of his load curtailment \(dr_{jt}\) (MW) in period \(t\), such that the outage cost \(C(\theta_{j} ,dr_{jt} )\) of the customer \(\theta_{j}\) for curtailing \(dr_{jt}\) is:

$$C(\theta_{j} ,dr_{jt} ) = K_{1} (dr_{jt} )^{2} + K_{2} (1 - \theta_{j} ) \cdot dr_{jt}$$
(12)

where \(\theta_{j}\) is used to characterize customers’ preference for curtailment probabilistically, and we assume this “preference parameter” \(\theta_{j}\) possesses a uniform probability distribution \(f(\theta_{j} )\) in the interval [0,1]. It sorts the customers from the least willing to the most willing to curtail load. K1 and K2 are the quadratic and linear term coefficients, respectively, of the outage cost function. These parameters are private customer information that is unknown to the supply-side. They are all positive, and they can be estimated using existing data. For the sake of simplicity, this paper assumes that K1 = 0.5 and K2 = 1 [19]. Appendix A gives a simple example to illustrate the derivation of the assumed values of K1, K2 and all the \(\theta_{j}\).

If rational customers do not receive any compensation for their demand reduction, they will not choose to participate in the DR program. Such customers will self-select the amount of demand curtailment \(dr_{jt}\) based on the monetary compensation \(y\) offered by the energy supplier in order to maximize their utility. Hence, the customers’ benefit \(U^{\text{d}} ( \cdot )\) is defined as the amount of monetary compensation they received from the energy supplier minus their outage cost due to the curtailment.

$$U^{\text{d}} (\theta_{j} ,dr_{jt} ,y) = y - C(\theta_{j} ,dr_{jt} )$$
(13)

When the power network suffers from peak load or emergencies, LMPs will soar, which means it is expensive for an energy supplier to provide electricity to the customers at those locations. In order to maximize their own interests, the energy supplier will make a trade-off between the cost of providing electricity to the customer at each location and the cost of curtailing the customer. We define \(V^{\text{s}} (L_{j} ,dr_{jt} )\) as the value of not delivering power to customer \(j\) in period \(t\). Hence, the energy suppliers’ benefit \(U^{\text{s}}\) generated by the load curtailment of customer \(j\) can be expressed as:

$$U^{\text{s}} (L_{j} ,dr_{jt} ,y) = V^{\text{s}} (L_{j} ,dr_{jt} ) - y = L_{j} \cdot dr_{jt} - y$$
(14)

where \(L_{j}\) is parameterized value of not providing electricity to customer \(j\) at a specific location. Each customer has a fixed value of \(L_{j}\) since it cannot move and change its locational value. It also reflects the weak influence of customer location and the contribution of load curtailment for the grid. It can be obtained using optimal power flow routines [28].

The energy supplier will have a subjective estimate for each customer preference parameter \(\theta_{j}\). The DR function \(DR(dr_{jt} ,L_{j} )\) illustrates how much load curtailment is obtained from customer preference parameter \(\theta_{j}\) at location \(L_{j}\). The compensation function \(Y(dr_{jt} )\) indicates how much the energy supplier is willing to pay the DR participants for their curtailment.

The aim of designing the DR compensation mechanism is to calculate the optimal compensation fee for a customer to provide some amount megawatt of load curtailment, so as to improve security of the power grid. It is a mathematical optimization problem, which maximizes the total utility for DR. The objective function is:

$$\begin{aligned} \mathop {\hbox{max} }\limits_{DR( \cdot ),Y( \cdot )} & E_{\theta } (U^{\text{s}} (DR_{jt} (\theta_{j} ,L_{j} ),Y(DR_{jt} (\theta_{j} ,L_{j} )),L_{j} ) \\ &\quad + U^{\text{d}} (DR_{jt} (\theta_{j} ,L_{j} ),Y(DR_{jt} (\theta_{j} ,L_{j} )),\theta_{j} )) \\ = &\, (V^{\text{s}} (L_{j} ,DR_{jt} ) - Y(DR_{jt} )) + (Y(DR_{jt} ) - C(\theta_{j} ,DR_{jt} )) \\ = &\, V^{\text{s}} (L_{j} ,DR_{jt} ) - C(\theta_{j} ,DR_{jt} ) \\ \end{aligned}$$
(15)

Both energy supplier and customer are assumed to be rational. The energy supplier will only provide compensation for customers at more critical locations where it can reduce the threat to security of the power grid by curtailing some amount of load. As discussed above, customers will not curtail load unless there is a positive incentive for them. Thus both supplier and customer are subject to the individual rational constraints as (16), (17) and to the incentive compatibility constraint as (18) [19].

$$U^{\text{s}} (DR_{jt} (\theta_{j} ,L_{j} ),Y(DR_{jt} (\theta_{j} ,L_{j} )),L_{j} ) \ge 0$$
(16)
$$U^{\text{d}} (DR_{jt} (\theta_{j} ,L_{j} ),Y(DR_{jt} (\theta_{j} ,L_{j} )),\theta_{j} ) \ge 0$$
(17)
$$\begin{aligned} U^{\text{d}} (DR_{jt} (\theta_{j} ,L_{j} ),Y(DR_{jt} (\theta_{j} ,L_{j} )),\theta_{j} ) \hfill \\ \quad \ge \,U^{\text{d}} (DR_{jt} (\tilde{\theta }_{j} ,L_{j} ),Y(DR_{jt} (\tilde{\theta }_{j} ,L_{j} )),\theta_{j} ) \hfill \\ \end{aligned}$$
(18)

Equation (18) illustrates that if a customer \(\theta_{j}\) reports his information \(\tilde{\theta }_{j}\) wrongly, his benefit will not be at the maximum, so it is not wise for the customer to lie. Hence, (18) can prevent customers from lying about \(\theta_{j}\), and it ensures customers select their compensation scheme according to their true willingness to provide DR.

The diagram in Fig. 2 illustrates the principle of the compensation mechanism.

Fig. 2
figure 2

Principle diagram of the compensation mechanism

Full size image

As mentioned above, the decision variables \(DR( \cdot )\), \(Y( \cdot )\) of the proposed compensation mechanism are also the functions of the customer preference parameter \(\theta_{j}\) and the location \(L_{j}\), and they are given by (19) and (20). Their detailed mathematical derivations are given in Appendix B.

$$DR_{jt} (\theta_{j} ,L_{j} ) = \left\{ {\begin{array}{ll} 0 & {0 \le \theta_{j} < 1 - \frac{{L_{j} }}{2}} \\ {2\theta_{j} + L_{j} - 2} & {1 - \frac{{L_{j} }}{2} \le \theta_{j} < 1} \\ \end{array} } \right.$$
(19)
$$Y(DR_{jt} (\theta_{j} ,L_{j} )) = \left\{ {\begin{array}{ll} 0 \qquad\qquad\qquad\qquad\,\,{ \, 0 \le \theta_{j} < 1 - \frac{{L_{j} }}{2}} \\ {\theta_{j}^{2}- 2\theta_{j} + 2\theta_{j} L_{j} + \frac{3}{4}L_{j}^{2} - 2L_{j} +1} \\\qquad\qquad\qquad\qquad\quad 1 - \frac{{L_{j} }}{2} \le \theta_{j} < 1\\ \end{array}}\right.$$
(20)

3 Mathematical formulation

3.1 Integrated generation–consumption dispatch model

Each individual user’s behavior is assumed to be independent. Customers will make various responses to a given price according to their own utility maximization. But the individual optimal choice may not be equal to the social optimal choice. In order to unify individual optimal choice and social optimal choice, we propose the following optimization problem.

$$\mathop { \hbox{max} }\limits_{{P_{gt} ,u_{gt} }} F = {\kern 1pt} {\kern 1pt} { \hbox{max} }\sum\limits_{t = 1}^{T} {\left[ {\sum\limits_{j = 1}^{D} {U(\alpha_{jt} ,D_{jt} ) - \sum\limits_{g = 1}^{G} {(C_{gt} u_{gt} + S_{gt} v_{gt} )} } } \right]}$$
(21)
$$u_{gt} - u_{g,t - 1} - v_{gt} \le 0 \, \;\;\;\;\forall g \in G,\forall t \in T$$
(22)
$$\sum\limits_{g = 1}^{G} {P_{gt} } {\kern 1pt} {\kern 1pt} {\kern 1pt} = \sum\limits_{j = 1}^{D} {D_{jt}^{\text{o}} } \, \;\;\;\;\forall g \in G,\forall t \in T$$
(23)
$$u_{gt} P_{g}^{\hbox{min} } \le P_{gt} \le u_{gt} P_{g}^{\hbox{max} } \;\;\;\; \, \forall g \in G,\forall t \in T$$
(24)
$$P_{gt} - P_{g,t - 1} \le r_{g}^{\text{up}} u_{g,t - 1} + P_{g}^{\hbox{max} } \left( {1 - u_{g,t - 1} } \right) \, \;\;\;\;\; \, \forall g,\forall t$$
(25)
$$P_{g,t - 1} - P_{gt} \le r_{g}^{\text{dn}} u_{gt} + {\kern 1pt} P_{g}^{\hbox{max} } {\kern 1pt} {\kern 1pt} {\kern 1pt} \left( {1 - u_{gt} } \right)\;\;\;\;\; \, \forall g,\forall t$$
(26)
$$\left\{ {\begin{array}{*{20}l} {\sum\limits_{t = 1}^{{T_{g}^{\text{on}} }} {(1 - u_{gt} )} = 0} & {g = 1,2, \ldots ,G} \\ {\sum\limits_{d = t}^{{t + X_{\text{g}}^{\text{on}} - 1}} {u_{gd} } \ge X_{g}^{\text{on}} (u_{gt} - u_{g,t - 1} ) \, } & {g = 1,2, \ldots ,G ; { }t = T_{g}^{\text{on}} , \ldots ,T - X_{\text{g}}^{\text{on}} + 1} \\ {\sum\limits_{d = t}^{T} {[u_{gd} - (u_{gt} - u_{g,t - 1} )]} \ge 0} & {g = 1,2, \ldots ,G ; { }t = T - X_{\text{g}}^{\text{on}} + 2, \ldots ,T} \\ \end{array} } \right.$$
(27)
$$\left\{ {\begin{array}{*{20}l} {\sum\limits_{t = 1}^{{T_{g}^{\text{off}} }} {u_{gt} } = 0 \, } & {g = 1,2, \ldots ,G} \\ {\sum\limits_{d = t}^{{t + X_{\text{g}}^{\text{off}} - 1}} {(1 - u_{gd} )} \ge X_{g}^{\text{off}} (u_{g,t - 1} - u_{gt} )} & {g = 1,2, \ldots ,G;{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} t = T_{g}^{\text{off}} + 1, \ldots ,T - X_{\text{g}}^{\text{off}} + 1} \\ {\sum\limits_{d = t}^{T} {[1 - u_{gd} - (u_{g,t - 1} - u_{gt} )]} \ge 0} & {g = 1,2, \ldots ,G;{\kern 1pt} \;t = T - X_{\text{g}}^{\text{off}} + 2, \ldots ,T} \\ \end{array} } \right.$$
(28)
$$\sum\limits_{g = 1}^{{G_{b} }} {P_{gt} } - \sum\limits_{j = 1}^{{D_{b} }} {D_{jt}^{\text{o}} } {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} = \sum\limits_{j} {B_{ij} (\theta_{it} } - \theta_{jt} ){\kern 1pt} \quad \forall b,\forall t$$
(29)
$$- F_{l}^{\hbox{max} } \le B_{ij} (\theta_{it} - \theta_{jt} ) \le F_{l}^{\hbox{max} } \quad \forall l,\forall t$$
(30)
$$- \pi \le \theta_{bt} \le \pi {\kern 1pt} \quad \, \forall b,\forall t$$
(31)
$$\theta_{{{\text{refb}},t}} = 0\quad \forall t$$
(32)

Equation (21) is subject to (4), (10) and (11) as well. The optimization goal (21) is to maximize the social welfare, in which the first term on the right-hand side represents the aggregate utilities of all customers, and the last two terms represent the unit commitment cost and the startup cost of thermal units. \(C_{gt} = a_{g} P_{gt}^{2} + b_{g} P_{gt} + c_{g}\) is quadratic production cost function, where \(a_{g}\), \(b_{g}\), \(c_{g}\) are the cost coefficients and \(P_{gt}\) is power output of thermal unit \(g\) in period \(t\). \(u_{gt}\) is the binary decision variable: on/off status of unit \(g\) in period \(t\), “1” if the generator is on, “0” otherwise. \(v_{gt}\) is the binary decision variable: “1” if unit \(g\) is started up in period \(t\), “0” otherwise. \(T\) is the set of scheduling periods. \(D\) is set of loads. \(G\) is set of thermal units. \(S_{gt} = K_{g} + B_{g} \left[ {1 - \exp \left( {{{ - X_{gt}^{\text{off}} } \mathord{\left/ {\vphantom {{ - X_{gt}^{\text{off}} } {\tau_{g} }}} \right. \kern-0pt} {\tau_{g} }}} \right)} \right]\) is a typical exponential startup cost function, for convenient analysis, these can be linearized using the piecewise linearization method with three segments [29,30,31,32].

Equation (22) gives the start-up constraints of thermal units. Equation (23) depicts the supply-demand balance requirement of the system, associate the dual variable \(\lambda_{t}\) with (23), namely the Lagrangian multiplier, which is also the market clearing price (MCP) of the system. Equation (24) gives the generation capacity limits of thermal units. \(P_{g}^{\hbox{max} }\) and \(P_{g}^{\hbox{min} }\) are the maximum/minimum power output of thermal unit \(g\), respectively. Equations (25) and (26) are the ramping up and down rate limits of thermal units, where \(r_{g}^{\text{up}}\) and \(r_{g}^{\text{dn}}\) are the ramp-up/ramp-down rate for thermal unit\(g\). Equations (27) and (28) describe the linearized minimum up-time and down-time constraints [29,30,31,32], where \(T_{g}^{\text{on}}\) and \(T_{g}^{\text{off}}\) are the minimal on/off hour of thermal unit \(g\) (1 is on and 0 is off); \(X_{g}^{\text{on}}\) and \(X_{g}^{\text{off}}\) are the number of periods unit \(g\) has been online/offline prior to the first period of the time span (end of period 0). This formulation can be converted into a MILP problem. Equation (29) represents the power balance equations for each node, where \(\theta_{it}\) is the phase angle of bus \(i\) in period \(t\); \(D_{b}\) is set of loads in bus \(b\); \(G_{b}\) is set of thermal units in bus \(b\). Equation (30) gives the network power flow limits \(F_{l}^{\hbox{max} }\) on transmission lines \(l\), associate the dual variables \(\pi_{lt}^{ + }\) and \(\pi_{lt}^{ - }\) with (30). \(B_{ij}\) is element in row \(i\) and column \(j\) of DC power flow matrix. Equation (31) describes the upper and lower limits of the phase angle \(\theta_{bt}\) and (32) represents the reference phase angle \(\theta_{{{\text{refb}},t}}\).

After solving the model (21)–(32), the LMPs are calculated in terms of the dual variables:

$$LMP_{bt} = \lambda_{t} + \sum\limits_{l} {(\pi_{lt}^{ + } - \pi_{lt}^{ - } )}$$
(33)

3.2 Compensation mechanism in integrated model

This section proposes an integrated generation–consumption dispatch model to calculate unit commitment decisions incorporating DR. The focus is on maximizing social welfare by including the benefits and costs of customers participating in DR (15) into the decision-making. The details are as follows.

$$\begin{aligned} \mathop { \hbox{max} }\limits_{{P_{gt} ,DR_{jt} ( \cdot ),Y( \cdot )}} F = \sum\limits_{t = 1}^{T} {\left[ {\sum\limits_{j = 1}^{D} {(U(\alpha_{jt} ,D_{jt} )} + V^{\text{s}} (L_{j} ,DR_{jt} ) } \right.} \hfill \\ \, \left. {\quad -\,C(\theta_{j} ,DR_{jt} )) - \sum\limits_{g = 1}^{G} {(C_{gt} + S_{gt} )} } \right] \hfill \\ \end{aligned}$$
(34)
$$\sum\limits_{g = 1}^{G} {P_{gt} } {\kern 1pt} {\kern 1pt} {\kern 1pt} = \sum\limits_{j = 1}^{D} {(D_{jt}^{\text{o}} - DR_{jt} } ){\kern 1pt} \, \quad \forall g \in G,\forall t \in T$$
(35)
$$- F_{l}^{\hbox{max} } \le B_{ij} (\theta_{it} - \theta_{jt} ) \le F_{l}^{\hbox{max} } \quad \, \forall l,\forall t$$
(36)
$$D_{jt}^{\text{a}} = (1 - DRL_{jt}^{\hbox{max} } )D_{jt}^{\text{o}} + \sum\limits_{k = 1}^{M} {D_{jkt}^{\text{a}} } \quad \forall j,\forall t$$
(37)
$$(1 - DRL_{jt}^{\hbox{max} } )D_{jt}^{\text{o}} \le D_{jt}^{\text{a}} \le D_{jt}^{\text{o}} \, \quad \, \forall j,\forall t$$
(38)
$$0 \le DR_{jt} \le DRL_{jt}^{\hbox{max} } \cdot D_{jt}^{\text{o}} \cdot I_{jt} \quad \, \forall j,\forall t$$
(39)
$$D_{jt}^{\text{a}} = D_{jt}^{\text{o}} - DR_{jt} \, \quad \forall j,\forall t$$
(40)

Equation (34) is subject to (4), (10), (11), (19), (20), (24)–(28), (31), (32) as well. The optimization goal (34) is to maximize social welfare using the compensation mechanism, where the first term represents the aggregate utilities of all customers, the second and the third terms represent the objective of the compensation mechanism (15), and the last two terms represent the unit commitment cost and the startup cost of thermal units. Equation (35) depicts the supply-demand balance requirement of system considering DR, associate the dual variable \(\lambda_{t}\) with (35), which is also the MCP of the system. Equation (36) gives the network power flow limits on transmission lines, associate the dual variables \(\pi_{lt}^{ + }\) and \(\pi_{lt}^{ - }\) with (36). Equations (37) and (38) describe the actual load demand allowing for DR. Equation (39) gives the load curtailment capacity limits. Equation (40) calculates actual load demand. LMPs may be calculated as before using (33) after solving the model (4), (10), (11), (19), (20), (24)–(28), (31), (32), (34)–(40).

4 Solution method

Equation (4) is difficult to define explicitly in terms of load demand variables. To do this, it is first necessary to introduce a new set of binary variable \(\sigma_{jt}\) to model the demand utility in an explicit manner [33, 34]. They satisfy:

$$\sigma_{jt} = \left\{ {\begin{array}{*{20}l} 1 & {\bar{D}_{j} - D_{jt} \ge 0} \\ 0 & {\text{otherwise}} \\ \end{array} } \right.$$
(41)
$$\frac{{\bar{D}_{j} - D_{jt} }}{{\sum\limits_{g = 1}^{G} {P_{g}^{\hbox{max} } } }} \le \sigma_{jt} \le 1 + \frac{{\bar{D}_{j} - D_{jt} }}{{\sum\limits_{g = 1}^{G} {P_{g}^{\hbox{max} } } }}$$
(42)

From (41), if \(\bar{D}_{j} - D_{jt} \ge 0\), and considering \(\bar{D}_{j} \le \sum\limits_{g = 1}^{G} {P_{g}^{\hbox{max} } }\), the lower bound of (42) must be strictly greater than zero and less than 1, while the upper bound is greater than 1. Since \(\sigma_{jt}\) is a binary variable, then if \(\bar{D}_{j} - D_{jt} \ge 0\), it must be equal to 1. A similar argument applies when \(\bar{D}_{j} - D_{jt} \le 0\) in which case \(\sigma_{jt} = 0\).

Analogously, the same method is applied to solve (19) and (20).

By these means, the demand utility function can be formulated as the sum of the products of some binary variables and a bounded continuous variable. After it is linearized, the proposed mathematical model is formulated as a MILP problem and it can be solved with the commercial MILP solver CPLEX.

5 Case studies

Numerical experiments are carried on a six-bus system and the modified IEEE 118-bus system to demonstrate the validity and effectiveness of the proposed model. In order to analyze the impacts of customer demand behavior on the MCP, LMPs, customer demand utility and social welfare, all these results are analyzed comparing the social welfare maximization model with a constant demand benefit function and with the customer utility function. All experiments are implemented on a personal computer with Inter (R) Core (TM) Duo 3.2 GHz CPU and 4 GB memory and programmed using Visual Studio 2010 C++, and the MILP solver is CPLEX 12.5. The simulated time interval is one day and is divided into 24 periods.

5.1 Six-bus system

A six-bus system with 3 thermal units, 3 load nodes, and 7 branches is shown in Fig. 3. All the parameters can be found in [35]. Three cases are used to illustrate the impacts of different factors on the dispatch results.

Fig. 3
figure 3

Line diagram of six-bus system

Full size image

  • Case 1: The NCUC model without DR, i.e. DRL = 0.

  • Case 2: The integrated generation–consumption dispatch model considering the demand utility function with DRL = 0, and compare the proposed model with the traditional one that has a fixed electric energy price.

  • Case 3: The DR compensation mechanism is integrated into the generation–consumption dispatch model, \(DRL \ne 0\).

5.1.1 Case 1 analysis

The NCUC model is listed in (21)–(32) with the first term of (21) removed. Its aim is to obtain the maximum and the minimum LMPs of each load bus, and these are shown in Table 1.

Table 1 Demand LMPs for NCUC model
Full size table

We assume that the maximum and the minimum LMPs of each load bus are equal to \(MU_{jt}^{\text{H}}\) and \(MU_{jt}^{\text{o}}\), respectively. Hence, we deduce the customer’s information through (3), (7) and (8). The results are shown in Table 2.

Table 2 Estimated parameters of demand utility function at each load bus using NCUC results
Full size table

5.1.2 Case 2 analysis

  1. 1)

    Dispatch results

The dispatch results from the integrated method with DRL = 0 are shown in Table 3. The customer’s demand utility can be deduced through (4) after we obtain the customer’s information \(\alpha_{jt}\) and \(\beta_{jt}\) from Case 1 as discussed above. The demand utility for each customer and in total are shown in Table 4.

Table 3 Results of unit commitment and generation dispatch with DLR = 0
Full size table
Table 4 Demand utility of all customers with DLR = 0
Full size table

In this case, the energy production cost is 132 k$, the startup cost of units is 808 k$, the total demand utility of customers is 2095 k$, the social welfare is 19623 k$.

  1. 2)

    Comparison with traditional fixed-price strategy

The proposed distributed real-time market clearing pricing strategy (\(\lambda_{t}\) in Section 3.1) can be compared with the traditional fixed-price strategy [36]. The NCUC program has been implemented using a MILP algorithm to obtain the initial price in a centralized manner. Given the initial price at each period of time \(t \in T\), customers of each type \(\alpha\) determine their optimal demand. The type \(\alpha^{\hbox{max} }\) has the most demand, and this paper assumes that all of the customers are of type \(\alpha^{\hbox{max} }\), representing the worst case.

If a customer’s demand is \(D_{jt}\) (MW) and the price is \(\lambda_{t}\) ($/MWh) during time interval \(t \in T\), the welfare of each customer can be simply expressed as:

$$W(\alpha_{jt} ,D_{jt} ) = U(\alpha_{jt} ,D_{jt} ) - \lambda_{t} D_{jt}$$
(43)

where \(W(\alpha_{jt} ,D_{jt} )\) is customer’s welfare function; \(\lambda_{t} D_{jt}\) is customer’s electricity cost. For each price \(\lambda_{t}\), customers try to adjust their electricity consumption \(D_{jt}\) to maximize their own welfare \(W(\alpha_{jt} ,D_{jt} )\), and the optimal consumption is achieved when the first derivative of (43) equals zero. Additionally, the customer’s marginal utility is equal to the market price. Hence, the fixed price \(\lambda_{t}^{\text{fixed}}\) in each period \(t \in T\) is calculated as:

$$\lambda_{t}^{\text{fixed}} = \alpha_{jt}^{\hbox{max} } - \frac{{\beta_{jt} \sum\limits_{g = 1}^{G} {P_{gt} } u_{gt} }}{D}$$
(44)

The calculated average welfare for the proposed distributed real-time pricing and for fixed pricing are compared in Fig. 4. It should be noted that the average welfare of each customer using the proposed distributed real-time pricing algorithm is much higher than that for the traditional fixed pricing algorithm. This demonstrates that distributed real-time pricing is an effective tool because the price interacts with customer behavior.

Fig. 4
figure 4

Average customer welfare considering demand utility and with fixed price

Full size image

The computation efficiency of the two methods is considered in Table 5, where it can be seen that the total computational time and the number of iterations for distributed real-time pricing strategy are significantly reduced compared with fixed pricing strategy.

Table 5 Computation efficiency of two methods
Full size table

5.1.3 Case 3 analysis

  1. 1)

    DR compensation mechanism

As mentioned in Section 2.2, the locational value and the preference parameter of customers are the two critical parameters of the DR compensation mechanism, and the locational value can be calculated according to literature [19]. The calculated locational values and assumed preference parameters are listed in Table 6.

Table 6 Customer attributes
Full size table

To demonstrate the advantage and validity of the proposed compensation mechanism, we take customer 3 of Bus 3 in period 12 as an example. Its locational value is \(L_{3} = 0.9\), and its true preference parameter is \(\theta_{3} = 0.7\). According to (19) and (20), the optimal DR amount and compensation fees are obtained, and they are graphed in Fig. 5.

Fig. 5
figure 5

DR amount and compensation fees of proposed compensation mechanism for customer 3 on Bus 3

Full size image

Customers may lie about their private information \(\theta_{3}\) to receive more monetary compensation \(y\) offered by energy supplier. The proposed compensation mechanism avoids this because only customers who report their true private information can obtain maximum utility (18). According to (13), we can obtain customers’ benefit, and Fig. 6 shows the benefit of customer 3 for all possible private information \(\theta_{3}\). The benefit of customer 3 is maximized when he or she reports the truth at \(\theta_{3} = 0.7\). Hence, each customer better reports his or her true private information \(\theta_{3}\) and thereby selects a compensation scheme according to their true willingness to provide DR.

Fig. 6
figure 6

Customer benefit for \(L_{3} = 0.9\) and \(\theta_{3} = 0.7\)

Full size image
  1. 2)

    Dispatch results

The consumers respond to the market energy price based on their voluntary psychological feelings and then determine whether and how much to curtail. The optimal DR amounts of customers can be solved by co-optimizing DR and generation–consumption dispatch decisions simultaneously. According to the dispatch model in Section 3.2, the optimal DR amounts are shown in Table 7.

Table 7 Optimal DR amounts for three types of customers in six-bus system
Full size table

Using the data in Table 7, the DRL = 1.88% according to (5). Different types of customers have different DR preferences in different periods.

Meanwhile, the optimal DR compensation mechanism for consumers can be obtained. The curtailing capacity (i.e. DR), the compensation cost (CC), the outage cost (OC), the consumer revenue (CR), and the energy supply side revenue (SR) are shown in Table 8. The demand utility for each customer and in total are shown in Table 9. The LMPs of each bus are shown in Table 10.

Table 8 Components of DR compensation of customers
Full size table
Table 9 Demand utility with optimal DR
Full size table
Table 10 LMP of each bus with optimal DR
Full size table

In this case, the energy production cost is 122 k$, the startup cost of units is 808 k$, the total demand utility of customers is 2070 k$, the revenue generated by the DR compensation mechanism is 47 k$, and the social welfare is 1994 k$.

In addition, in order to emphasize the importance of the two critical parameters (\(L_{j} ,\theta_{j}\)), some further tests are performed. On one hand, \(L_{j}\) is fixed at 0.7 for all the customers during study cycle, as if all the customers were at the same location. On the other hand, \(\theta_{j}\) is fixed at 0.7 for all the customers during study cycle, so all customers have the same preference. Table 11 compares these results.

Table 11 Comparison of optimal DR compensation mechanism with fixed \(L_{j} = 0.7\) and \(\theta_{j} = 0.7\) in six-bus system
Full size table

With fixed \(L_{j} = 0.7\) or fixed \(\theta_{j} = 0.7\), the total demand utility of all customers is increased but the amount of DR, CC, OC, CR and SR, as well as the social welfare are reduced. Therefore, when \(L_{j}\) and \(\theta_{j}\) are fixed to a certain value, the dispatch results are non-optimal. Only customers who share their true information, and select a compensation scheme according to their true willingness to provide DR, can achieve maximum benefit. Therefore, the proposed compensation mechanism benefits both the suppliers and their customers, and realize maximum social welfare and Pareto optimal results.

The computing time is also listed in Table 11, demonstrating the applicability of our model to the six-bus system.

  1. 3)

    Comparison of Case 3 with Case 2

The system demand, the LMP, the customers’ demand utility and the social welfare with and without DR are shown in Fig. 7a to Fig. 7d, respectively.

Fig. 7
figure 7

System demand, LMP, customer demand utility and social welfare with and without DR for six-bus system

Full size image

Figure 7a compares the system demand without DR (DRL = 0) and the expected system demand with DRL = 1.88% as determined above from optimal decision-making results. It illustrates that the compensation mechanism encourages customers to curtail a certain amount of load.

Figure 7b shows the LMP with DRL = 0 and with DRL = 1.88%. It can be seen that the LMP decreases due to the demand curtailment of customers participating in DR during peak periods. However, the LMP of Bus 4 is increased to 55.55 $/MWh as compared with that of Case 2, illustrating that, when deploying the DR compensation mechanism with the objective of maximizing the social welfare, LMPs at some buses may increase. In this case, the higher LMP is primarily induced by congestion on Line 2. Dynamic unit commitment is affected by the units’ start-up costs, which changes the choice of marginal unit. Thus increasing social welfare does not necessarily reduce LMP.

Figure 7c shows the total demand utility of customers with DRL = 0 and with DRL = 1.88%. When the units’ output can meet the original demand of customers, the demand utility of customers is at a maximum. Therefore, the demand utility of customers is reduced by the demand curtailment of customers participating in DR.

Figure 7d shows the social welfare with DRL = 0 and with DRL = 1.88%. The social welfare increases very significantly with DRL = 1.88% because the customers’ utility loss from DR is compensated by the cost savings of the generation side. The proposed compensation mechanism can increase the benefits both the supply-side and demand-side. The main reason is that the reduction in demand utility of customers is approximately 1.2%, from 2095 k$ to 2070 k$, while the total cost saving for generators is approximately 7.4%, from 132 k$ to 122 k$, due to not starting up the unit with higher marginal cost.

5.2 IEEE 118-bus system

A modified IEEE 118-bus test system is used to test the proposed dispatch model at a more practical scale [37]. The system has 53 units, 186 branches, 14 capacitors, 9 tap-changers, and 91 load buses. The detailed IEEE 118-bus system data are given in motor.ece.iit.edu/data/SCUC_118. The dispatch results including DR, CC, OC, CR, and SR, and the computation time are shown in Table 12. The comments applying to Table 11 above apply equally to Table 12.

Table 12 Comparison of optimal DR compensation mechanism with fixed \(L_{j} = 0.7\) and \(\theta_{j} = 0.7\) in IEEE 118-bus system
Full size table

The system demand, LMP, demand utility and social welfare with and without DR are shown in Fig. 8a to Fig. 8d, respectively, as they were shown for the six-bus system in Fig. 7.

Fig. 8
figure 8

System demand, LMP, customer demand utility and social welfare with and without DR for IEEE118-bus system

Full size image

Figure 8a compares the system demand with DRL = 0 and with DRL = 4.56% at bus 38, as determined from the optimal decision-making results. A large amount of load is curtailed in the peak periods 10-16 and 18-22.

Figure 8b shows the LMP with DRL = 0 and with DRL = 4.56% at bus 38. Load curtailment occurs at peak periods, however, the LMP of bus 54 is increased to 14.42 $/MWh as compared with 14.13 $/MWh for DRL = 0, even the system load levels are smaller than those of the case DRL = 0. The higher LMP is primarily induced by the congestion of line 54. Dynamic unit commitment is affected by the units’ start-up cost, and the load curtailment at some buses may trigger new transmission congestions and change the marginal unit.

Figure 8c shows the demand utility of customers with DRL = 0 and with DRL = 4.56%. Figure 8d shows the social welfare with DRL = 0 and the social welfare with DRL = 4.56%. The reduction in demand utility of customers is approximately 4.1%, from 1670 k$ to 1602 k$, a larger relative change compared with the six-bus system, while the total cost savings of generators is approximately 9% from 817 k$ to 739 k$.

6 Conclusion

Based on consumer behavior theory in microeconomics, this paper, from a novel perspective, establishes an integrated generation–consumption dispatch model considering the demand utility function. Additionally, the decision-making process of the DR compensation is incorporated into this dispatch model. The individual behavior of customers is seen to impact market equilibrium. Simulation results using a six-bus system and the modified IEEE 118-bus system verify the impact of behavior changes of customers on generation dispatch, social welfare, market clearing, LMP and the demand utility. All these results are analyzed by comparison with and without DR. Finally, compared with the traditional fixed pricing, the proposed method can increase the benefits to both the supply and demand sides, and the original demand utility of customers is not lost because it is compensated by participating in DR. Therefore, the Pareto optimality of system dispatch is achieved and social welfare maximization is realized as well.