1 Introduction

In recent years, distribution systems have experienced an unprecedented growth in the deployment of distributed energy resources (DERs) including photovoltaic (PV)-equipped and smart homes. The distribution system market is anticipated to undergo a fundamental restructuring towards a transactive energy market due to increasing availability and active role of economically rational DER owners [1] and their participation in demand response (DR) programs. As a consequence of this transformation, new market pricing mechanisms that can facilitate the efficient and reliable operation of the grid are being investigated. In this line of research, pricing based on distribution locational marginal price (DLMP) has been widely studied and proposed [2,3,4,5,6,7,8,9,10]. However, it has been shown in several studies that adopting such market-based mechanisms in a distribution system with a high deployment of price-responsive DERs can adversely affect the distribution system operation by causing congestions [2, 11]. It is therefore crucial for a distribution system operator (DSO) to design congestion mitigation strategies in three-phase unbalanced distribution systems and coordinate it with the way flexibility service is utilized from price-responsive end users.

Contrary to the transmission system, distribution systems are highly unbalanced and have higher losses and voltage drops due to higher \({\textit{r}}/{\textit{x}}\) (resistance/reactance) ratio which, in addition to real power, requires power flow models that account for voltage magnitudes and reactive power flows and their associated losses. A few studies have incorporated power flow constraints to obtain losses, voltage violations, and line congestion prices using DLMP decomposition [7, 8, 10]. In these studies, the appearance of voltage violation and congestion components in a nodal DLMP signals the occurrence of an operational issue, partly caused by the marginal contribution of the nodal demand. Therefore, it is beneficial to both the DSO and a price-responsive end user, such as a smart home operated by home energy management system (HEMS) [12,13,14], to alleviate any violation or congestion by providing some flexibility service in real-time. To the best of our knowledge, there is no framework in the literature that can enhance the real-time market operation of DSO as well as add operational reliability and value through coordinated utilization of flexibility service from HEMS.

In this paper, we aim to contribute in this space and propose a DLMP-based real-time operational framework for the DSO to operate unbalanced three-phase distribution systems in a congestion-free manner by procuring a coordinated flexibility service from HEMS. The HEMS is assumed to interact with the DSO and provide its optimal flexibility range by minimizing the total expected cost and discomfort of household considering the current state of controllable appliances and their uncertain future demand. Hence, in line with the focus of this paper, we first provide some related work on DLMP-based DSO market and congestion management, HEMS decision-making under uncertainty, and real-time DR provided by HEMS, and then outline our main contributions.

At the grid level, a number of studies focus on DLMP pricing. Reference [11] proposes a dynamic tariff method for the DSO to alleviate congestion that might occur in a distribution network with high penetration of DERs. The uncertainty management is to quantify and mitigate the risk of congestion when employing the dynamic tariff method, which is achieved by formulating the problem as a chance-constrained two-level optimization and solving the problem through an iterative procedure. Reference [2] presents a method based on DLMP to mitigate grid congestion introduced by electric vehicles. References [15, 16] propose a DR-based congestion management in the distribution system. In [15], a bi-level optimization model for the day-ahead congestion management is established and a robust optimization approach is introduced to alleviate negative impacts introduced by uncertainties in DER generation and market prices. Reference [16] proposes a bilevel congestion-price-based approach to alleviate possible congestion in the distribution system. At the upper-level, the DSO communicates with the aggregators to seek flexibility service. At the lower-level, aggregators are assigned to control and schedule customers’ home-level appliances. Reference [3] proposes a DLMP-based congestion management through quadratic programming in a distribution system with high penetration of flexible demands. Aggregators are considered as intermediate agents to receive DLMP as dynamic tariffs computed by the DSO and plan for the optimal energy schedules of flexible loads. Reference [4] proposes a DLMP-based approach to congestion-free reserve and energy provision from buildings in distribution grids. The above studies assume a symmetric and balanced distribution system.

At the home level, a number of HEMS studies have used adaptive dynamic programming (ADP) for optimal management of PVs and energy storages (ESs) [17,18,19,20]. Reference [17] uses ADP with temporal difference learning to implement a computationally efficient HEMS that optimizes PV-ES system scheduling to minimize cost and discomfort to the household. Reference [18] proposes a novel distributed iterative ADP to solve the multi-ES optimal coordination control problems. Reference [19] develops a self-learning scheme based on ADP to optimize residential ES. Reference [20] compares different methods for home energy resource scheduling and analyze their performance dependency on the ES model employed, focusing on ES capacity and charge/discharge rates. A few other studies use ADP to control the energy use of building cooling systems to minimize energy consumption while preserving the comfort of occupant [21, 22]. In our recent study, we compare dynamic programming (DP) and ADP algorithms implemented by HEMS to optimize forward-looking schedules of heating, ventilation and air-conditioning (HVAC), water heater (WH), and electric vehicle charging while considering uncertainty in outside temperature, hot water usage, and non-controllable load (NCL) [23].

Lastly, a few other related papers that deal with real-time HEMS-based DR are reported in [24,25,26,27,28]. An optimization-based control of deferrable loads by HEMS in response to real-time pricing signals with the goal to reduce consumer’s electricity bill while minimizing the total daily curtailment, has been proposed in [24]. Another HEMS-level DR optimization framework under real-time pricing which uses a convex programming to relax integer variables to continuous ones for the efficient and flexible incorporation of multiple deferrable loads has been proposed in [25]. A home-level consumption scheduling framework that achieves a desired tradeoff between minimizing electricity payment due to real-time pricing and minimizing waiting time for the operation of each appliance has been proposed in [26]. Additional literature on HEMS-enabled DR based on real-time pricing has been surveyed in [27]. Unfortunately, none of the above papers and the references therein study a real-time coordinated mechanism between DSO, which prices customers according to a state-of-the-art method (DLMP), and HEMS in an unbalanced distribution system. The closest study to our work is probably [28], which assesses the benefits of residential DR in real-time distribution markets based on DLMP. However, [28] mostly focuses on the benefits of DR to the grid and assumes a balanced distribution system without dealing with any coordination mechanism between the DSO and the residential customers.

While most of the existing literature use “DC” power flow and a flat voltage profile to compute only real power DLMPs [2, 3, 8, 11], or consider balanced distribution systems [4, 29], we propose a real-time DSO optimization model that can determine real and reactive power DLMPs in three phases of an unbalanced distribution system, and utilize a coordinated flexibility service from HEMS to alleviate operation issues such as voltage violations and line congestion in different phases. The key contributions of this paper are summarized in the following three aspects:

  1. 1)

    We provide a three-phase DLMP decomposition for both real and reactive power in an unbalanced distribution system.

  2. 2)

    At the grid level, we propose a two-stage real-time operation framework using which a DSO can implement a coordinated flexibility service among a large number of HEMS-equipped homes while alleviating congestion, voltage violations, and substation-level power imbalance.

  3. 3)

    At the HEMS level, we propose an ADP-based approach that can enable HEMS to provide flexibility range to the DSO while minimizing household’s cost and discomfort by optimally deciding the schedules of appliances under future uncertainty in demand and weather temperature.

The rest of this paper is organized as follows. In Section 2 we introduce the proposed two-stage framework. In Section 3, we provide the system model which explains the mathematical formulation for the first- and second-stage optimization problems of DSO and HEMS. We present our simulation results in Section 4 and conclude in Section 5.

2 Framework

We consider an unbalanced distribution system that serves different number of homes in each phase of its three-phase nodes. We assume that homes on every phase are either regular or smart (HEMS-equipped) type. A smart home is considered to own PV panels coupled with ES, HVAC, and WH that are controlled by HEMS.

To determine the occurence of any congestion or voltage violation in the grid, we model the DSO to implement a reliability assessment by solving a first-stage optimization problem based on real-time demand/supply bids at each three-phase node. The DSO can also solve the first-stage optimization problem without any need for demand or supply bids; it can maximize the social welfare by dispatching as much power as demanded and allowed by the grid constraints. The DLMP values that are each composed of energy, losses, congestion, and voltage violation components can then be computed using the first-stage optimization solution and a sensitivity analysis. When congestion or voltage violation prices are seen, the DSO can realize the existence of an operational issue and immediately seek flexibility service from HEMS to alleviate these issues while operating within the allowable power imbalance threshold at the substation node. By using DLMP components, DSO can send two price signals (lower and upper) to each HEMS and inquire a flexible demand range.

Fig. 1
figure 1

Flowchart of proposed two-stage framework

In the next step, each HEMS can compute and send back to the DSO the inquired demand range (flexibility) by incorporating the prices of DSO in its ADP-based stochastic optimization problem to find optimal consumption amounts of its controllable appliances. In the second stage, by aggregating the flexibility ranges of all the participating HEMS, the DSO can determine an optimal dispatch point without any distribution system congestion or voltage violation and send it back to each HEMS. In its second stage, each HEMS can set the dispatch point as the maximum consumption for its controllable appliances. The flowchart in Fig. 1 summarizes this procedure.

3 System model

3.1 DSO first-stage optimization problem

The DSO first-stage optimization problem can be formulated as in (1)–(12), which minimizes the objective function in (1) by dispatching the least cost generation (\(1^{{\mathrm {st}}}\) and \(3^{{\mathrm {rd}}}\) terms, corresponding to wholesale supply and DG supply), high value loads (\(4^{{\mathrm {th}}}\) term), and avoiding curtailment at high value of lost load (VOLL) (\(2^{{\mathrm {nd}}}\) term). For conciseness and clarity, related fundamental symbols are shown in Table 1. Other symbols are abstract combinations of these symbols or explained in the text. For example, \(L_{j}^{P,\phi }\) in (2) shows the real power loss in phase \(\phi \) of line j.

$$\begin{aligned}&\underset{p_{0}^\phi , p_{i}^{G,\phi }, p_{i}^{D,\phi }, \check{p}_i^{D,\phi }}{{\text {min}}} \sum _{i \in \mathcal {N}_{{\mathrm {N}}}, \phi \in \mathcal {N}_{{\phi }}}\left[ \left( \lambda _0^p p_{0}^\phi + \nu ^p \check{p}_{i}^{D,\phi }\right) \right. \nonumber \\&\quad \left. +\left( c_{i}^{G,\phi } p_{i}^{G,\phi } - c_{i}^{D,\phi } p_{i}^{D,\phi } \right) \right] \end{aligned}$$
(1)

s.t.

$$\begin{aligned}&p_0^{\phi }+\sum _{i \in \mathcal {N}_{{\mathrm {N}}}}{p_i^{\phi }} - \sum _{j \in \mathcal {N}_{{\mathrm {L}}}}{L_j^{P,\phi }} =0 \qquad \forall \phi \in \mathcal {N}_{{\phi }} \end{aligned}$$
(2)
$$\begin{aligned}&q_0^{\phi }+\sum _{i \in \mathcal {N}_{{\mathrm {N}}}}{q_i^{\phi }} - \sum _{j \in \mathcal {N}_{{\mathrm {L}}}}{L_j^{Q,\phi }} =0\qquad \forall \phi \in \mathcal {N}_{{\phi }}\end{aligned}$$
(3)
$$\begin{aligned}&p_i^{\phi }=p_i^{D,\phi }-p_i^{G,\phi }\qquad \forall i \in \mathcal {N}_{{\mathrm {N}}}, \forall \phi \in \mathcal {N}_{{\phi }}\end{aligned}$$
(4)
$$\begin{aligned}&q_i^{\phi }=q_i^{D,\phi }-q_i^{G,\phi }\qquad \forall i \in \mathcal {N}_{{\mathrm {N}}}, \forall \phi \in \mathcal {N}_{{\phi }}\end{aligned}$$
(5)
$$\begin{aligned}&L_j^{P,\phi }=r_j \frac{(P_j^{\phi })^2 + (Q_j^{\phi })^2 }{(V_i^{\phi })^2}\qquad \forall j \in \mathcal {N}_{{\mathrm {L}}}, \forall \phi \in \mathcal {N}_{{\phi }}\end{aligned}$$
(6)
$$\begin{aligned}&L_j^{Q,\phi }=x_j \frac{(P_j^{\phi })^2 + (Q_j^{\phi })^2 }{(V_i^{\phi })^2}\qquad \forall j \in \mathcal {N}_{{\mathrm {L}}}, \forall \phi \in \mathcal {N}_{{\phi }}\end{aligned}$$
(7)
$$\begin{aligned}&1-\epsilon \leqslant V_i^\phi \leqslant 1+\epsilon \qquad \forall i \in \mathcal {N}_{{\mathrm {N}}}, \forall \phi \in \mathcal {N}_{{\phi }} \end{aligned}$$
(8)
$$\begin{aligned}&(P_j^{\phi })^2+(Q_i^\phi )^2 \leqslant (\overline{S}_j^{\phi })^2\qquad \forall j \in \mathcal {N}_{{\mathrm {L}}}, \forall \phi \in \mathcal {N}_{{\phi }} \end{aligned}$$
(9)
$$\begin{aligned}&|p_0^\phi - p_0^{\phi '} |= \Delta \phi _p\qquad \forall \phi \in \mathcal {N}_{{\phi }}, \forall \phi ' \in \{\mathcal {N}_{{\phi }}\setminus \phi \} \end{aligned}$$
(10)
$$\begin{aligned}&|q_0^\phi - q_0^{\phi '} |= \Delta \phi _q\qquad \forall \phi \in \mathcal {N}_{{\phi }}, \forall \phi ' \in \{\mathcal {N}_{{\phi }}\setminus \phi \}\end{aligned}$$
(11)
$$\begin{aligned}&\{p_{0}^\phi , p_{i}^{G,\phi }, p_{i}^{D,\phi }\} \in \left\{ \mathcal {X}^0,\mathcal {X}^G, \mathcal {X}^D\right\} \qquad \forall i \in \mathcal {N}_{{\mathrm {N}}}, \forall \phi \in \mathcal {N}_{{\phi }} \end{aligned}$$
(12)
Table 1 Nomenclature

In the above optimization problem, (2)–(5) pertain to the system and nodal real and reactive power-balance constraints in each phase. The line losses are given by (6), (7) and nodal voltage constraint is enforced by (8). The megavolt ampere (MVA) limit constraint of each line j in phase \(\phi \) is given by the quadratic constraint in (9). The three-phase real and reactive power imbalance is restricted to a small percentage \((\Delta \phi _{\textit{p}}, \Delta \phi _{\textit{q}})\) at the substation level and imposed by (10), (11) between any two phases. Additional substation, DG, and load constraints, including integer constraints from utility-scale ES units [30, 31], if any, can also be accommodated as summarized in (12).

Note that (1)–(12) is a nonlinear optimization problem due to constraints (6), (7), (8), and (9), which respectively relate to line losses, node voltages, and line MVA limits. With further derivation provided in Appendix A and details in our previous work [5], these constraints are appropriately linearized and given in (13)–(16). In (13) and (14), a Taylor series approximation around center points, \(p_i^{\phi \star }\) and \(q_i^{\phi \star }\) showing an optimal power flow solution that is obtained exogenously, has been used. In (15) and (16), a modified linearized power flow and an outer approximation are used.

$$\begin{aligned}&L_j^{P,\phi } \approx L_j^{P,\phi }(p_i^{\phi \star },q_i^{\phi \star }) \nonumber \\&\quad + \sum _{i} \frac{\partial L_j^{P,\phi }}{\partial p_i^{\phi }}(p_i^{\phi } - p_i^{\phi \star })+ \sum _{i} \frac{\partial L_j^{P,\phi }}{\partial q_i^{\phi }}(q_i^{\phi } - q_i^{\phi \star }) \end{aligned}$$
(13)
$$\begin{aligned}&L_j^{Q,\phi }\approx L_j^{Q,\phi }(p_i^{\phi \star },q_i^{\phi \star }) \nonumber \\&\quad + \sum _{i} \frac{\partial L_j^{Q,\phi }}{p_i^{\phi }}(p_i^{\phi } - p_i^{\phi \star }) + \sum _{i} \frac{\partial L_j^{Q,\phi }}{\partial q_i^{\phi }}(q_i^{\phi } - q_i^{\phi \star }) \end{aligned}$$
(14)
$$\begin{aligned} V_i^\phi &=V_0^\phi - \sum _{k \in \mathcal {N}_{{\mathrm {N}}}}{\varvec{\mathcal {G}}_p(i,k) (p_k^\phi + L_k^{P,\phi })} \nonumber \\ &\quad +\,\sum _{k \in \mathcal {N}_{{\mathrm {N}}}}{\varvec{\mathcal {G}}_q(i,k) (q_k^\phi +L_k^{Q,\phi })} \end{aligned}$$
(15)
$$\begin{aligned}&|P_j^\phi (x)|+ |Q_j^\phi (x) |\leqslant \sqrt{2}\,\overline{S}_j^\phi \end{aligned}$$
(16)

By using the above approximations and replacing (6), (7) and (9) with (13), (14) and (16), respectively, and substituting \(V_i^\phi \) in (8) with the RHS of (15), the DSO can solve a linear programming to determine its first-stage optimal dispatches. The first-stage dispatch solution can then be used by the DSO to compute the DLMPs using a sensitivity analysis.

3.2 DLMP computation using sensitivity analysis

The real power DLMP \(\pi _{p,i}^\phi \) in phase \(\phi \) of each three-phase distribution node i is defined as the sum of the energy price, \(\pi _{p,0}^{E,\phi }\), the price of losses, \(\pi _{p,i}^{L,\phi }\), the price of voltage violations, \(\pi _{p,i}^{V,\phi }\), and the price of line congestions, \(\pi _{p,i}^{C,\phi }\). The reactive power DLMP, \(\pi _{q,i}^\phi \) is similarly defined as the sum of its corresponding components in phase \(\phi \) of node i. The real and reactive power DLMPs and their respective components are defined below.

$$\begin{aligned} \pi _{p,i}^\phi & = {} \pi _{p,0}^{E,\phi }+\pi _{p,i}^{L,\phi }+\pi _{p,i}^{V,\phi }+\pi _{p,i}^{C,\phi } \nonumber \\ & = {} \lambda _p^{0,\phi }+\left( \lambda _p^{0,\phi } \sum _{j}{\frac{\partial L_j^{P,\phi }}{\partial p_i^\phi }}+\lambda _q^{0,\phi } \sum _{j}{\frac{\partial L_j^{Q,\phi }}{\partial p_i^\phi }}\right) \nonumber \\ &\quad +\, \sum _{i'}{\left( \mu _{i'}^{{\mathrm {min}},\phi } - \mu _{i'}^{{\mathrm {max}},\phi }\right) \frac{\partial V_{i'}^\phi }{\partial p_i^\phi }} + \sum _{j}{\rho _j^\phi \frac{\partial S_{j}^\phi }{\partial p_i^\phi }} \end{aligned}$$
(17)
$$\begin{aligned} \pi _{q,i}^\phi & = {} \pi _{q,0}^{E,\phi }+\pi _{q,i}^{L,\phi }+\pi _{q,i}^{V,\phi }+\pi _{q,i}^{C,\phi } \nonumber \\ & = {} \lambda _q^{0,\phi }+\left( \lambda _q^{0,\phi } \sum _{j}{\frac{\partial L_j^{Q,\phi }}{\partial q_i^\phi }}+\lambda _p^{0,\phi } \sum _{j}{\frac{\partial L_j^{P,\phi }}{\partial q_i^\phi }}\right) \nonumber \\ &\quad +\,\sum _{i'}{\left( \mu _{i'}^{{\mathrm {min}},\phi } - \mu _{i'}^{{\mathrm {max}},\phi }\right) \frac{\partial V_{i'}^\phi }{\partial q_i^\phi }} + \sum _{j}{\rho _j^\phi \frac{\partial S_{j}^\phi }{\partial q_i^\phi }} \end{aligned}$$
(18)

The first term in (17) shows the energy price of real power in phase \(\phi \) at the substation node “0”, which is typically the wholesale LMP. The second term in the bracket in (17) shows the price of real power loss caused in the grid by injecting \(p_i^\phi \). It is defined as the sum of marginal contributions of dispatch \(p_i^\phi \) in change in real and reactive power losses, \(L_j^{P,\phi }\) and \(L_j^{Q,\phi }\), of all the lines j, multiplied by their respective energy prices, \(\lambda _p^{0,\phi }\) and \(\lambda _q^{0,\phi }\). The third term in (17) is associated with the voltage violation price. The voltage violation price is activated when a node \(i'\) hits an upper or lower voltage limit given by constraints in (8). Since the dispatch of node i can cause a change in voltage of \(i'\) and can potentially make its constraint active, the voltage violation price of node i is defined as the sum of its marginal contributions to the change in voltage of \(i'\) multiplied by the difference of its respective Lagrangian multipliers associated with the lower and upper bound voltage constraints in (8). Lastly, the congestion price (the fourth term in (17)) is defined as the marginal contribution of \(p_i^\phi \) in the change in MVA flow of line j, \(S_{j}^\phi \), multiplied by its Lagrange multiplier given by constraint in (16). Similar definitions hold for the reactive power DLMP components in (18). The sensitivities \(\partial L_j^{P,\phi }/\partial p_i^\phi \), \(\partial L_j^{Q,\phi }/\partial p_i^\phi \), \(\partial V_{i'}^\phi /\partial p_i^\phi \), \(\partial S_{j}^\phi /\partial p_i^\phi \), and those with respect to \(q_i^\phi \) are provided in Appendix A.

At the end of its first-stage optimization, when the DSO determines the occurence of a congestion or voltage violation, it seeks flexibility by sending the lower and upper DLMPs, \(\underline{\pi }_{p,i}^\phi\,{\text{and}}\, \overline{\pi }_{p,i}^\phi \), where \(\underline{\pi }_{p,i}^\phi\) is the sum of energy and loss components (\(\underline{\pi }_{p,i}^\phi = \pi _{p,0}^{E,\phi }+\pi _{p,i}^{L,\phi }\)) and \(\overline{\pi }_{p,i}^\phi\) is the sum of all DLMP components (\(\overline{\pi }_{p,i}^\phi = \pi _{p,0}^{E,\phi }+\pi _{p,i}^{L,\phi }+\pi _{p,i}^{V,\phi }+\pi _{p,i}^{C,\phi }\)), to each HEMS-equipped home h served by phase \(\phi \).

3.3 HEMS first-stage optimization problem

Using the upper and lower DLMPs, each HEMS can find an optimal tradeoff between cost and discomfort by optimally determining consumption levels of its controllable appliances. In this study, each HEMS has been assumed to own PV-panels coupled with ES and NCL. Additionally, two major controllable loads, namely HVAC and WH, have been considered. A schematic of such a HEMS-equipped home is depicted in Fig. 2. The HEMS stochastic optimization problem can be modeled in terms of a DP with states, decisions, and stochastic variables. At every time step, the HVAC, WH, and ES states are respectively considered as the room temperature, water temperature, and ES state-of-charge (SOC). The decision set captures whether or not to run the HVAC system, whether or not to heat water in the WH, and how much energy to inject into the ES. To model future uncertainties, the stochastic variables in this study has been considered as the outside temperature, hot water demand, and NCL of the household. The net NCL is modeled as the difference of the NCL and PV generation, and has been assumed to be entirely served by the ES. In addition, the following timing assumptions are made:

  1. 1)

    The controllable appliance a in each home h is in state \(s_{a,t}^h\) when HEMS makes a decision \(d_{a,t}^h\) for that appliance at the beginning of the time step t.

  2. 2)

    The decision \(d_{a,t}^h\) is immediately applied which impacts the remainder of the time step t.

  3. 3)

    The stochastic variable \(\widetilde{u}_{a,t}^h\) associated with the appliance a is realized after the decision is applied and lasts for the remainder of the time step t.

Fig. 2
figure 2

HEMS illustration for a PV-equipped home with major controllable loads (ES, HVAC, WH) and NCL

Fig. 3
figure 3

Decision tree for HVAC with highlighted route that minimizes weighted sum of expected cost and discomfort

As an example, Fig. 3 illustrates a decision tree, which can best explain a DP, for the case of HVAC cooling. Here, squares represent decision nodes, circles represent uncertainty nodes, and triangles show terminal nodes. The decisions are STAY (no action), COOL, or H.COOL (highly cool). The outside temperature effect is shown by uncertainty changes (\(\hbox {UNC}\pm i, i\in \{0,\pm 1, \pm 2\}\)) degrees, where UNC represents uncertainty. The outside temperature is drawn from historical data, but its forecast is estimated as a generalized random walk that may go up or down a few degrees or remain unchanged. The optimal DP solution, shown by the bold black line, can be found using backward induction [23, 32] as follows:

Step 1: Compute the thermal discomfort at the terminal nodes.

Step 2: Compute the expected cost and discomfort of each preceding uncertainty node.

Step 3: For each decision node, pick the decision with the lowest sum of weighted decision cost (energy and discomfort) and expected future value.

Step 4: Repeat Step 2 and Step 3, working backward in time, until the optimal first period decision is found.

With \(t \in \mathcal {N}_{{\mathrm {T}}}\) showing the timeslots in forward-looking horizon \(|\mathcal {N}_{{\mathrm {T}}}|\), the HEMS objective can be achieved as modeled in (19)–(24) using a form of Bellman’s equation [33].

$$\begin{aligned} {\mathop {\min }\limits _{\varvec{d}_{t}^{h}}}\,\mathbb {E} \left( \sum \nolimits _{t\in \mathcal {N}_T}{\left( \alpha _t^{\mathcal {D},h} \varPi _{t}^{\mathcal {D},h}\left( \varvec{s}_t^h,\varvec{d}_{t}^{h},{\varvec{\widetilde{u}}_t^h}\right) +\alpha _{t}^{\mathcal {C},h}\varPi _{t}^{\mathcal {C},h} \left( {\varvec{s}}_t^h,{\varvec{d}}_t^h,{\varvec{\widetilde{u}}}_t^h\right) \right) }\vphantom{\sum}\right) \end{aligned}$$
(19)

where

$$\begin{aligned} \varPi _t^{\mathcal {D},h}\left( {\varvec{s}}_t^h,{\varvec{d}}_t^h,{\varvec{\widetilde{u}}}_t^h\right)= & {} \sum _{t \in \mathcal {N}_A}{\beta _{a,t}^h |s_{a_d,t}^h - s_{a,t}^h|} \end{aligned}$$
(20)
$$\begin{aligned} \varPi _t^{\mathcal {C},h}\left( {\varvec{s}}_t^h,{\varvec{d}}_t^h,{\varvec{\widetilde{u}}}_t^h\right)= & {} \pi _{p,t}^{h} \left( \sum _{a \in \mathcal {N}_A}{\zeta _a^h d_{a,t}^h}\right) \, \Delta {\textit{t}} \end{aligned}$$
(21)

s.t.

$$\begin{aligned} s_{a,t}^h= & {} \gamma _{a,s}^h s_{a,t-1}^h \, - \gamma _{a,d}^h d_{a,t}^h + \gamma _{a,u}^h \widetilde{u}_{a,t}^h \nonumber \\&\forall a \in \mathcal {N}_{A}, \forall t \in \mathcal {N}_{T} \end{aligned}$$
(22)
$$\begin{aligned} \underline{s}_{a,t}^h\leqslant & {} s_{a,t}^h \leqslant \overline{s}_{a,t}^h \qquad \forall a \in \mathcal {N}_{A}, \forall t \in \mathcal {N}_{T} \end{aligned}$$
(23)
$$\begin{aligned} \underline{d}_{a,t}^h\leqslant & {} d_{a,t}^h \leqslant \overline{d}_{a,t}^h \qquad \forall a \in \mathcal {N}_{A}, \forall t \in \mathcal {N}_{T} \end{aligned}$$
(24)

The objective function in (19) shows the expected weighted sum of household’s cost and discomfort over the forward-looking horizon. The symbols \(\varvec{s}_t^h,\varvec{d}_t^h,\varvec{\widetilde{u}}_t^h\) show the state, decision, and stochastic variable vectors for each house h whose \(a^{\mathrm {th}}\) entry corresponds to its controllable appliance a at timeslot t. In (20), the discomfort function, \(\varPi _t^{\mathcal {D},h}\), of each home h due to a controllable appliance a (e.g., HVAC, WH) is modeled as the absolute value of the difference of the appliance state \(s_{a,t}^h\) from the desired state \(s_{a_d,t}^h\) at each timeslot t. In (21), the energy cost function, \(\varPi _t^{\mathcal {C},h}\), of each home h at timeslot t is modeled as the price of electricity during that time multiplied by the sum of demand (in kW) in making the decision \(d_{a,t}^h\) for each appliance a and the timeslot resolution \(\Delta {\textit{t}}\). Lastly, the weighted sum coefficients, \(\alpha _t^{\mathcal {D},h}\) and \(\alpha _t^{\mathcal {C},h}\), show the tradeoff coefficients between cost and discomfort for each household h at time t.

Constraint in (22) shows the temporal state transition equation of the controllable appliance a from timeslot t to timeslot \(t-1\). Here, the state transition equation has been generalized to include adjustment coefficients and capture appliance efficiency, unit conversion, and other properties. For instance, for HVAC, \(\gamma _{a,s}^h\) (s for state) captures the HVAC efficiency in changing the room temperature from time t to \(t-1\), \(\gamma _{a,d}^h\) (d for decision) is the unit adjustment coefficient from decision (in kW) consumed to units of temperature reduced, and \(\gamma _{a,u}^h\) (u for uncertainty) is an outside air temperature driven coefficient that captures building thermal features according to building insulation properties. When applicable, the constraints in (23) and (24) enforces lower and upper state and decision bounds of each appliance a in home h at timeslot t.

By dropping the subscript t in the current timeslot appliance decision, \(d_{a,t}^h\), of each HEMS-equipped home h and summing it over all appliances, the net consumption in kW, \(p_h^{\phi }\), of each HEMS-equipped home h located in phase \(\phi \) of node i can be computed as follows:

$$\begin{aligned} p_h^{\phi } = \sum _{a \in \mathcal {N}_A}{\zeta _a^h d_{a}^h} \end{aligned}$$
(25)

In (25), \(\zeta _a^h\) is used to convert the decision units of each appliance to kW. By using the DSO’s upper and lower price range \([\underline{\pi }_{p,i}^\phi ,\, \overline{\pi }_{p,i}^\phi ]\) and solving (19)–(24) twice, the lower and upper flexibility range, \([\underline{p}_h^{\phi },\, \overline{p}_h^{\phi }]\), of each home h can be determined, respectively.

With \(\mathcal {N}_{H^+}^{i,\phi }\) and \(\mathcal {N}_{H^-}^{i,\phi }\) showing the sets of homes with and without HEMS in phase \(\phi \) of node i, the total net demand, \(p_i^{D,\phi }\), in phase \(\phi \) of the three-phase node i can be computed by summing \(p_h^{\phi }\) obtained from (25) over h, \(\forall h \in \{\mathcal {N}_{H^+}^{i,\phi } \cup \mathcal {N}_{H^-}^{i,\phi }\}\). For regular homes, \(p_h^\phi \) represents the real-time fixed demand of home h. For HEMS-equipped homes \(p_h^\phi \) is replaced with \(\underline{p}_h^\phi \) or \(\overline{p}_h^\phi \) depending on whether \(\underline{\pi }_{p,i}^\phi \) or \(\overline{\pi }_{p,i}^\phi \) is employed in (19)–(24).

$$\begin{aligned} p_i^{D,\phi } = \sum _{h \in \mathcal {N}_{H^+}^{i,\phi }}{p_h^{\phi }} +\sum _{h \in \mathcal {N}_{H^-}^{i,\phi }}{p_h^{\phi }} \end{aligned}$$
(26)

Hence, the flexibility range \([\underline{p}_i^{D,\phi },\, \overline{p}_i^{D,\phi }]\) can be determined using \([\underline{p}_h^{\phi },\, \overline{p}_h^{\phi }]\) in the first term of (26).

3.4 DSO second-stage optimization problem

In this stage, the DSO can use the flexibility range, \([\underline{p}_i^{D,\phi },\, \overline{p}_i^{D,\phi }]\), provided by nodes with HEMS-equipped homes to alleviate congestion and voltage violation issues in a social welfare maximizing manner. To do so, let \(\mathcal {N}_{{\mathrm {F}}^{+}}\), \(\mathcal {N}_{{\mathrm {F}}^{-}}\) show the set of nodes with, and without HEMS flexibility, respectively. With the dispatches \(p_i^{D,\phi }\), \( \forall i \in \mathcal {N}_{{\mathrm {F}}^{-}}\), considered fixed and equal to \(p_{i}^{D^1,\phi }\,\), \(\forall i \in \mathcal {N}_{{\mathrm {F}}^{-}}\), where superscript “1” pertains to the first-stage optimization solution, the DSO can implement the second-stage optimization problem as follows.

$$\begin{aligned}&\underset{p_{0}^{\phi }, p_{i}^{D,\phi }, \check{p}_i^{D,\phi }}{\text {min}}\nonumber \\&\quad \left[ \sum _{i \in \mathcal {N}_{{\mathrm {N}}}, \phi \in \mathcal {N}_{{\phi }}}{\left( \lambda _0^p p_{0}^\phi + \nu ^p \check{p}_{i}^{D,\phi }\right) } \right. \nonumber \\&\qquad \left. +\sum _{i \in \mathcal {N}_{{\mathrm {N}}}, \phi \in \mathcal {N}_{{\phi }}}{\left( c_{i}^{G,\phi } p_{i}^{G,\phi } - c_{i}^{D,\phi } p_{i}^{D,\phi } \right) } + \sum _{j \in \mathcal {N}_{{\mathrm {L}}_{{\mathrm {C}}}}, \phi \in \mathcal {N}_{{\phi }}}{\psi f_j^\phi } \right] \end{aligned}$$
(27)

s.t. (2)–(5), (8), (13)–(16)

$$\begin{aligned}&\{p_i^{D,\phi },q_i^{D,\phi }\} = \{p_{i}^{D^1,\phi }, q_{i}^{D^1,\phi }\} \qquad \forall i \in \mathcal {N}_{{\mathrm {F}}^{-}}, \forall \phi \in \mathcal {N}_{{\phi }} \end{aligned}$$
(28)
$$\begin{aligned}&\{p_i^{G,\phi },q_i^{G,\phi }\} = \{p_{i}^{G^1,\phi }, q_{i}^{G^1,\phi }\} \qquad \forall i \in \mathcal {N}_{{\mathrm {F}}^{-}}, \forall \phi \in \mathcal {N}_{{\phi }} \end{aligned}$$
(29)
$$\begin{aligned}&\underline{p}_i^{D,\phi } \leqslant p_i^{D,\phi } \leqslant \overline{p}_i^{D,\phi }\qquad \forall i \in \mathcal {N}_{{\mathrm {F}}^{+}}, \forall \phi \in \mathcal {N}_{{\phi }} \end{aligned}$$
(30)
$$\begin{aligned}&\underline{q}_i^{D,\phi } \leqslant q_i^{D,\phi } \leqslant \overline{q}_i^{D,\phi } \qquad \forall i \in \mathcal {N}_{{\mathrm {F}}^{+}}, \forall \phi \in \mathcal {N}_{{\phi }} \end{aligned}$$
(31)
$$\begin{aligned}&\kappa (|P_j^\phi (x)|+ |Q_j^\phi (x) |) + f_j^\phi \leqslant \sqrt{2}\,\overline{S}_j^\phi \nonumber \\&\quad \forall j \in \mathcal {N}_{{\mathrm {L}}_{{\mathrm {C}}}}, \forall \phi \in \mathcal {N}_{{\phi }} \end{aligned}$$
(32)

In (27), the objective function (1) from the first stage has been modified to include a penalty term \(\sum _{}{\psi f_j^\phi }\), where \(\psi \) shows a large number to highly penalize congested lines. Here, \(\mathcal {N}_{{\mathrm {L}}_{{\mathrm {C}}}}\) shows the set of congested lines in the first stage. By using a slack variable, \(f_j^\phi \), constraint (16) is also modified to allow up to a certain overload coefficient (\(\kappa \)) of the line capacity, e.g. \(\kappa = 0.9\), limiting the line flow to a maximum of 90% of the line capacity. By modeling the second-stage optimization of the DSO in this way, it can be clearly seen that, the flexibility provided in (30), (31) will enforce the constraint in (32) to end up binding, removing the congestion in the system. Since lower line flows are enforced, which, compared to the first stage, causes lower voltage drops, an additional merit of the second-stage optimization problem is that it can also remove voltage violations in the system. By finding an optimal solution, \(p_i^{D^*,\phi }\) in the range \([\underline{p}_i^{D,\phi },\, \overline{p}_i^{D,\phi }]\) and \(\pi _{p,i}^{\phi *}\) in the range \([\underline{\pi }_{p,i}^\phi ,\, \overline{\pi }_{p,i}^\phi ]\), the second-stage optimization problem can completely remove the congestion and voltage violation components of the DLMP.

The DSO’s second-stage solution, \(p_i^{D^*,\phi }\), is the optimal dispatch in phase \(\phi \) of the three-phase node i which serves a large number of HEMS-equipped (\(h \in \mathcal {N}_{H^+}^{i,\phi }\)) and regular (\(h \in \mathcal {N}_{H^-}^{i,\phi }\)) homes in each phase. As the last step of the proposed method, the question now is how to redistribute the optimal dispatch point, \(p_i^{D^*,\phi }\), among HEMS-equipped homes in each phase. Here, we use a convex combination to redistribute the three-phase dispatches to each HEMS-equipped home. Without loss of generality, we first assume that the flexible portion of \(p_i^{D^*,\phi }\) coming from HEMS-equipped homes is the convex combination of their flexibility range endpoints at the three-phase node level, and compute a scalar value \(\eta \in [0,1]\) according to the following linear combination.

$$\begin{aligned}&p_i^{D*,\phi } - \sum _{h \in \mathcal {N}_{H^-}^{i,\phi }}{p_h^\phi } = \eta \left( \overline{p}_i^{D,\phi } - \sum _{h \in \mathcal {N}_{H^-}^{i,\phi }}{p_h^{\phi }}\right) \nonumber \\&\quad + (1-\eta ) \left( \underline{p}_i^{D,\phi } - \sum _{h \in \mathcal {N}_{H^-}^{i,\phi }}{p_h^{\phi }}\right) \end{aligned}$$
(33)

Note that, if all homes in phase \(\phi \) of node i are HEMS-equipped, then \(\mathop {\sum}\limits_{h \in \mathcal {N}_{H^-}^{i,\phi }}{p_h^{\phi }} = 0\).

Next, we use the computed \(\eta \) from (33) and the lower and upper flexibility range of each HEMS-equipped home to find its maximum consumption amount \(p_h^{\phi *}\), given by (34) below. Note that, this technique proportionally redistributes power allocations according to the flexibility provided by each HEMS.

$$\begin{aligned} p_h^{\phi *} = \eta \underline{p}_h^{\phi } + (1-\eta ) \overline{p}_h^{\phi } \end{aligned}$$
(34)

3.5 HEMS second-stage optimization problem

Finally, the consumption amount \(p_h^{\phi *}\) from (34) is used as the upper limit constraint in the second-stage optimization problem of HEMS to find the optimal real-time decisions for its appliances as follows.

$$\begin{aligned} \mathop {{\text {min }}}\limits _{\varvec{d}_{t}^{h}} \,\mathbb {E}\left( \sum \nolimits _{t\in \mathcal {N}_T} {\left( \alpha _t^{\mathcal {D},h}\varPi _t^{\mathcal {D},h} \left( \varvec{s}_t^h,\varvec{d}_t^h,\varvec{\widetilde{u}}_t^h\right) + \alpha _t^{\mathcal {C},h} \varPi _t^{\mathcal {C}, h}\left( \varvec{s}_t^h,\varvec{d}_t^h,\varvec{\widetilde{u}}_t^h\right) \right) }\right) \end{aligned}$$
(35)

s.t.

(22)–(24)

$$\begin{aligned} \sum _{a \in \mathcal {N}_A}{\zeta _a^h \cdot d_{a}^h} \leqslant p_h^{\phi *} \qquad \forall h \in \mathcal {N}_{H^+}^{i,\phi }, \forall a \in \mathcal {N}_{A} \end{aligned}$$
(36)

By denoting the optimal solution of the above optimization problem with \(\varvec{d}_t^{h\diamond }\), and replacing it in (25), the final HEMS dispatch point, denoted by \(p_t^{h\diamond }\) at current time t, is determined.

With the flowchart shown in Fig. 1, the following algorithm provides an organized summary of the overall two-stage mechanism.

Algorithm

1. Solve (1)–(12) according to Section 3.1, compute (17), (18) and determine \([\underline{\pi }_{p,i}^\phi ,\, \overline{\pi }_{p,i}^\phi ]\) according to Section 3.2 at the DSO-level and send it to each HEMS (the first-stage DSO optimization).

2. According to Section 3.3, use \([\underline{\pi }_{p,i}^\phi ,\, \overline{\pi }_{p,i}^\phi ]\) to solve (19)–(24) twice and compute \([\underline{p}_h^{\phi },\, \overline{p}_h^{\phi }]\) using (25) for each HEMS and send it to DSO (the first-stage HEMS optimization).

3. Use (26) to compute the flexibility range, \([\underline{p}_i^{D,\phi },\, \overline{p}_i^{D,\phi }]\), of each three-phase node. According to Section 3.4, solve the problem defined by (27), (2)–(5), (8), (13)–(16), (28)–(32), and use (17), (18) to determine each three-phase node’s dispatch point and price, \((p_i^{D^*,\phi }, \pi _{p,i}^{\phi *})\). Then use (33), (34) to determine HEMS-level dispatch point, \(p_h^{\phi *}\). Send \(p_h^{\phi *}\) and \(\pi _{p,i}^{\phi *}\) to each HEMS (the second-stage DSO optimization).

4. According to Section 3.5, optimize (19)–(24), (36) using \(p_h^{\phi *}\) and \(\pi _{p,i}^{\phi *}\) at each HEMS to determine final applicance decisions and HEMS-level dispatch points, \(\varvec{d}_t^{h\diamond }\) and \(p_t^{h\diamond }\) (the second-stage HEMS optimization).

4 Simulation results

The proposed two-stage mechanism was implemented on a modified IEEE 69-bus system, as shown in Fig. 4, using CPLEX and the ADP functionality in dynamic programming for adaptive modeling and optimization (DYNAMO) toolkit developed by the National Renewable Energy Laboratory (NREL). In Fig. 4, the unbalanced three-phase loads shown by arrows and their corresponding real-time bids were set in a way to create line congestions and voltage violations in the first stage. Each node’s three-phase load was assumed to be the aggregate load of the homes connected to the respective phase. Two types of homes, regular homes (without HEMS) and smart homes (HEMS-equipped), were considered. A regular home was assumed passive whose real-time demand has to be served under any circumstance. A HEMS-equipped home was assumed to have the willingness to respond to the pricing signals from the DSO, provide a flexibility range, and adjust its controllable load (HVAC, WH and ES) consumption.

The simulation setup was designed in a way to first specify the most vulnerable lines and nodes that are prone to congestion and voltage violations. Then HEMS-equipped homes were considered for the nodes with the highest demand along the specified congested lines. In our case study, the real-time demand and bids of the three-phase nodes in the system created line congestions in lines 2 (the line connecting node 1 to node 2), 35, 46, and 52, and voltage violation in nodes 26, and 60-64. With the highest three-phase loads located at nodes 45, 49, and 60, different number of HEMS-equipped homes were considered at these nodes.

Fig. 4
figure 4

Oneline diagram of IEEE 69-bus system with HEMS-equipped homes at nodes 45, 49, 60, and regular homes elsewhere

Table 2 Number of HEMS-equipped homes of types I, II, and III in each phase of nodes 45, 49, and 60
Table 3 Parameters of HVAC used in ADP-based HEMS stochastic optimization
Table 4 Parameters of WH used in ADP-based HEMS stochastic optimization
Table 5 Parameters of ES used in ADP-based HEMS stochastic optimization

Furthermore, with everything else kept identical, and HVAC composing the major portion of its load, each HEMS-equipped home were classified into types I, II, and III according to their initial (real-time) room temperatures. Without loss of generality in specifying the types of homes, those with initial room temperatures of \(24.5\,^{\circ }{\mathrm {C}}\), \(26.1\,^{\circ }{\mathrm {C}}\), and \(27.8\,^{\circ }{\mathrm {C}}\) were considered to denote types I, II, and III, respectively. Our goal in the simulations were twofold. First, we wanted to observe how each sample HEMS type provides flexibility by taking the price of DSO and discomfort of occupants into consideration while being at different discomfort levels due to differing initial room temperatures. This aspect of our work is quite novel in the sense that it provides a way for an active consumer (HEMS-equipped home) to respond to the price signal of DSO and provide DR under real-time circumstances while accounting for the discomfort of household. Second, we wanted to see the impact of the flexibility provided by HEMS in bringing operational benefits to the grid.

Table 2 summarizes the number of sample HEMS types placed at nodes 45, 49, and 60. Here, the number of regular homes has been considered zero to provide high flexibility. Also, for each phase, the same number of each HEMS type has been considered.

For the three types of homes, all HEMS related parameters used in simulating the ADP-based stochastic optimization problem has been respectively tabulated for HVAC, WH, and ES in Tables 3, 4, and 5. The first columns in these tables show the name of parameters with its unit in parenthesis. The second columns show the corresponding values of these parameters. The first two rows in these tables show the operation range of state of appliances and their corresponding desired/minimum states. Note that, while room and water temperatures have been modeled as states for HVAC and WH, SOC has been considered as the state for the ES where only a state close to its minimum SOC is penalized. The third and fourth rows show the decision and uncertainty ranges of the stochastic variables affecting their corresponding appliance states. For instance, for HVAC in Table 3, the decision range shows how many increments of \(^\circ {\mathrm {C}}\) can be reduced. The uncertainty range shows the range in which the outside temperature varies, and as a result changes the room temperature state of HVAC over the forward-looking horizon according to the state transition equation in (22). The amount of energy needed to increase one unit (e.g., \(^{\circ }{\mathrm {C}}\)) in the state space and the state resolution is listed in the fifth and sixth rows of Tables 34, and 5. Each appliance discomfort coefficient used in (19) to compute its total discomfort at any state is given by the seventh row. Finally, the initial (current timeslot) state of each HEMS type is shown in the last three rows. Notice in the last three rows that we have kept the initial water temperature of WH and the initial SOC of the ES constant across HEMS types while altering that of HVAC to see HEMS behavior clearly.

Next, for conciseness and better illustration, we first present the simulation results of the first-stage and second-stage optimization problems of DSO. Then, we present the first- and second-stage HEMS-level results for the three HEMS types, especially for HVAC. As a sample case for HEMS type III, we also show its scheduling results for WH and ES.

The simulation results of first-stage and second-stage optimization problems of DSO are shown in Figs. 5 and 6. Figure 5 shows the DLMP values, whereas Fig. 6 shows physical grid quantities. In Fig. 5, the DLMP values and their components are stacked and shown for both (before and after HEMS flexibility) stages. We firstly focus on the first stage.

As seen, the three-phase DLMP values of the first stage are very high. The first-stage DLMPs are composed of energy price, loss price, nonzero voltage violation price, and nonzero line congestion price. It can be observed, for instance in phase B, that as a result of flow congestion in lines 2, 35, 46, and 52, and voltage violation in node 26, their corresponding DLMP components are activated in Fig. 5. Note that a congestion/voltage violation in one line/node adds the corresponding DLMP component not only to that line/node but also to the lines and nodes that are affected by it, or share a line in its route to the substation node. For instance, again in phase B, node 26 has a voltage violation (Fig. 6e), but the price is reflected mainly on nodes 1–26; other nodes are also affected. Or, due to congestion of line 2 in this phase, all nodes see a congestion price.

In the second stage, when HEMS-equipped homes in nodes 45, 49, and 60 provide flexibility, it is seen in both Figs. 5 and 6, that the congestion and voltage violation components of the DLMPs have completely vanished, reducing it to mere energy and loss prices. The obvious reason is that none of the second-stage line graphs in Fig. 6 hits a limit. Note that this is due to the penalty term in the second-stage objective function of DSO and constraint (32) to relax the congested line loading to within \(\kappa \) (0.95 in our simulation) times of its capacity. Note further that, while keeping the dispatch of the nodes without HEMS fixed, as imposed by constraints (28) and (29), the improvement in removing the violations has only been caused by the flexibility provided at nodes 45, 49, and 60.

Fig. 5
figure 5

Real power DLMPs and their components at the end of first-stage and second-stage optimizations of DSO

Fig. 6
figure 6

Line MVA flow and node voltages as a result of the first-stage and second-stage DSO optimizations

Table 6 Flexibility in \({\mathrm {kW}}\) used by DSO in each phase of nodes 45, 49, and 60

The aggregate amounts of three-phase flexibility in kW provided by nodes 45, 49, and 60, as well as their total, and the resulting reduction in system losses are reported in Table 6. Notice here that, despite our assumption of the same type and number of HEMS in each phase of these nodes, the flexibility used across their phases are different. The reason for this difference is noted as the variation across three-phase aggregate bid values of these nodes in the second-stage objective function of DSO in (27). In other words, less flexibility has been utilized from nodes with higher demand bid values, hence maximizing the resulting social welfare. This distinction makes our study interesting in the sense that it is realistically applicable to a transactive unbalanced DSO market with random real-time demand variation and bids across phases.

The ADP-based HEMS-level results of the first-stage and second-stage optimization is shown in Figs. 7 and 8. Here, we assume that HEMS minimizes its expected cost and discomfort by looking forward into the future uncertainty space in order to make an optimal real-time decision in response to the price signals of DSO. Figure 7a shows a rolling-window scheduling of the HVAC for each HEMS type for 6 consecutive timeslots while looking 6 hours into the future at each individual timeslot. We note here that, although most of our focus will be on the first timeslot, we have also plotted the next five hours schedules to show the effectiveness of HEMS decision-making under future uncertainty. The horizontal dotted line shows the desired room temperature state and the blue stairs graph shows the price of energy; the first hour is the second-stage DLMP, \(\pi _{p,i}^{\phi ^*}\), and the following hours are forecasted prices by HEMS. The height of the red shade at the end of the first timeslot shows the appliance-level flexibility range provided by HVAC in response to the DSO’s upper and lower price range, \([\underline{\pi }_{p,i}^\phi ,\, \overline{\pi }_{p,i}^\phi ]\), in the first stage. Notice that the shaded flexibility range in each HEMS type is different—narrower when the initial temperature is far from its desired state (e.g., HEMS type III) and wider when it is closer (e.g., HEMS type I). This observation can be explained as the direct artifact of the initial room temperature and the way cost and discomfort are weighted in the objective function in (19). The HVAC of HEMS type III which is at an initial high temperature of \(27.8\,^\circ {\mathrm {C}}\) (farthest from the desired temperature of \(23.9\,^\circ {\mathrm {C}}\)), does not provide much flexibility in response to the price signals of DSO as the discomfort term \(\varPi _t^{\mathcal {D},h}\) in (19) weighs more. As an intermediate case, HEMS type II shows some amount of flexibility in its tradeoff between discomfort and cost as it is closer to the desired temperature.

Fig. 7
figure 7

HEMS-level simulation results for HEMS types I, II, III

Fig. 8
figure 8

HEMS-level simulation results for WH and ES

The graphs with markers show the room temperature state set by HEMS at the end of its second-stage optimization. Here, during each timeslot 0–1, HEMS makes a decision whether to set the room for cooling or not. The temperature state transits from time “0” to “1” (the pre-decision state of next timeslot) by readily implementing the decision and applying the effect of stochastic variable (outside temperature) over the duration of the timeslot according to (22). Notice that, because of high discomfort at \(27.8\,^\circ {\mathrm {C}}\), HEMS III decides to cool the room at a maximum rate (i.e., hits the lower bound of its tiny flexibility range with a highly negative slope) in order to get closer to the desired temperature state of \(23.9\,^\circ {\mathrm {C}}\). In contrast to HEMS type III, HEMS type I decides to cool the room slightly. The net effect of this slight cooling decision and the stochastic outside temperature is seen as a small temperature increase above \(24.5\,^\circ {\mathrm {C}}\) at time “1” (slight positve slope). HEMS type II, which is at an initial room temperature of \(26.1\,^\circ {\mathrm {C}}\), decides to cool the room at an intermediate rate compared to HEMS types I and III. In this case, the net effect of cooling decision and the outside temperature is seen as a cooling temperature amount added (negative slope). The reduction in the demand coming from HVAC can be seen as the gap between the lower range of the shaded flexibility region and the marker point at time “1”. In other words, due to high initial room temperature (high discomfort), only HEMS type III denies to provide any flexibility in its HVAC.

For each HEMS type, the total flexibility limits in the first stage as well as the total HEMS and DSO’s second-stage dispatches are shown in Fig. 7b. From the solid lines, it can be seen that the demand of HEMS type I is significantly less compared to types II and III. While HEMS types I and III almost hit the dispatch of DSO, HEMS type II decides differently and picks a point lower than the DSO dispatch. The reason for this decision depends on the initial room temperature state and the tradeoff between the total expected cost and discomfort over the forward-looking horizon. Note that, the second-stage HEMS dispatch, \(p_t^{h\diamond }\), also depends on the dispatch points of its HVAC, WH and ES.

Figure 8 shows the identical WH and ES results in HEMS types I, II, and III. In Fig. 8a, while the graph with markers shows water temperature state transition, green bars show the equivalent water temperature in increments of \(^\circ {\mathrm {C}}\) (HEMS decision). Blue bars show the equivalent water temperature removed due to hot water usage (stochastice variable). The values corresponding to the bars are shown in the right vertical axis. In Fig. 8b, the decision and effect of net NCL (stochastic variable) is shown in equivalent percentage of SOC added and removed, respecively. The state transition, as the difference of the two bars plots, are shown by the graph with markers.

In HEMS type III, while the HVAC does not provide any flexibility, its ES is set to provide full flexibility (hits the lower limit of its shaded region) by drawing zero power and serving the NCL through discharging the battery. Similarly, the WH in HEMS type III (as well as in types I and II) shows full flexibility as it happens to have no water usage demand—hence no shaded flexibility region—at the first timeslot due to zero water usage. In other words, the total flexibility provided by HEMS type III is merely coming from ES. The kW amount of this flexibility can be readily seen as the difference between HEMS dispatch and the upper flexibility limit in Fig. 7b. Notice that this gap is relatively large for HEMS I and II compared to III. The reason is clearly the additional flexibility amount provided by the HVACs of HEMS types I and II.

5 Conclusion

Validated by the simulation results presented in the previous section, this paper presents a two-stage real-time and social-welfare-maximizing operation of three-phase unbalanced distribution systems through HEMS-enabled flexibility. The proposed two-stage mechanism mitigates distribution system operation issues such as line congestions and voltage violations by enabling a coordinated mechanism between DSO and HEMS. While achieving both DSO and HEMS-level objectives, it is shown that by implementing the proposed framework, flexibility service from HEMS-level appliances can be acquired efficiently. An important attribute of this research is its detailed appliance-level modeling of cost and discomfort of household in the HEMS objective function. We believe this is an interesting feature of this work that has not been previously scrutinized.

Although our simulation results focus on real power pricing and flexibility, the mathematical formulation presented is generalized enough to include three-phase reactive power pricing and flexibility. This is an additional aspect of the proposed model that can be crucial for real-time transactive energy market in distribution systems. An interesting aspect of the proposed two-stage mechanism is to conduct a detailed cost-benefit analysis of the DSO and HEMS optimization problems.