1 Introduction

1.1 Motivation

Investments in power generation assets are multi-year projects with high associated costs and lifetimes that usually last several decades. Market circumstances can significantly change over those time periods. One important aspect has been the restructuring of electricity markets undertaken in Europe, the U.S. and other parts of the world since the 1990s. Despite the introduction of competition, market concentration in the liberalized electricity markets frequently remains high. E. g. [11] report based on data from the European Commission that in 17 out of 25 reviewed countries the electricity generation business remains highly concentrated (Herfindahl–Hirschman Index (HHI) between 1800 and 5000) or even very highly concentrated (HHI above 5000). Over the last decade, market power of the big, usually incumbent firms has probably somewhat declined in Europe due to both new market entrants in the national markets and increased cross-border competition. Nevertheless (conventional) electricity generation remains a highly concentrated business, and in many countries of the world there is one major incumbent player such as EDF in France or Vattenfall in Sweden. At the same time, investments into generation assets are risky in competitive markets and require structured decision-support systems. Investment decisions and dispatch in electricity spot markets are connected, thus requiring anticipation of expected market outcomes when deciding about investments. The methods used in modeling these processes are based on a number of uncertain parameters, whose realizations are unknown at the time the decision is taken. The use of multiple scenarios is therefore an accepted strategy to ensure decisions lead to acceptable outcomes given a range of potential developments.

The use of scenarios motivates the use of stochastic methods. With asset lifetimes that can reach half a century and more, the computational efforts required to solve optimization problems can be significant. Methods for reducing these efforts may be required, while at the same time limiting the detrimental effect on solution quality. In this paper, we propose a rolling-horizon formulation for the investments of a strategic player which has computational advantages over a fixed-horizon stochastic optimization formulation, while supporting numerical results suggest an acceptable tradeoff regarding solution quality. The rolling-horizon approach is a more realistic representation of actors making their decision under imperfect information and has the advantage of allowing the players to adjust their data in between rolls. This more closely models real-world decision-making and allows for learning and other feedback in between rolls.

1.2 Literature review and contribution

In fully regulated electricity systems, generation expansion planning is a centralized optimization problem which takes into account the total cost of construction, operation, and electricity production over the optimization horizon. Usually cost minimization or welfare maximization is the objective, while other targets, such as sustainability or security of supply, may be considered in a multi-objective setting. This planning approach considers investment decisions and market operations in a single objective function of the central planner [31].

In liberalized electricity systems, investment decisions and market operations are generally decoupled. While private investors make individual decisions about new generation assets, the electricity market is run by a market operator or an independent system operator (ISO)Footnote 1. These actors have different goals: while investors and producers have incentives to maximize their revenue, market operators aim to maximize welfare, as generally they are not profit-oriented. Modeling the investment process thus requires consideration of the different goals of these actors. This may be formulated as a competitive equilibrium, i.e. a simple optimization problem (e.g. [19, 24]) or as a Cournot oligopoly game (e.g. [19, 20]). A dynamic version of the latter is found in [21]. Another approach is to formulate a bilevel problem [2, 26]. At the upper level, an investor decides on generation capacity additions as well as price bids for the electricity market. At the lower level, a market operator maximizes welfare given bids by suppliers and consumers. These types of models are also referred to as closed-loop formulations, since they represent a two-stage structure in which investment and market bidding decisions are decoupled [30]. In contrast, models such as [20, 21] provide open-loop formulations where capacity expansion and operation are decided simultaneously.

Solving the aforementioned closed-loop, bilevel generation investment problem requires advanced optimization techniques. One option is to use the conventional formulation and employ an iterative heuristic to solve the problem [27]. Another possibility is to reformulate the bilevel problem as a mathematical program with equilibrium constraints (MPEC), which allows using more conventional mixed-integer programming (MIP) solvers. This reformulation has been proposed in [19], where the authors compare three different investment models, one of which is a bilevel version separating investment decision and spot market operations. Both [14] and [29] each propose a stochastic MPEC formulation of the strategic generation investment problem. While [14] models demand in discrete load-price blocks, [29] uses a conjectured supply function resulting in a nonlinear problem, which in turn is linearized by introducing discrete steps.

The extension to multiple investors leads to multiple bilevel models that need to be solved simultaneously to form an equilibrium. These are generally formulated as an equilibrium problem with equilibrium constraints (EPEC), a structure which is significantly more complex than the previously mentioned MPECs [9]. EPEC extensions of the MPEC formulations in [14] and [29] have been proposed by [15, 16] as well as [28], respectively. Due to the increased complexity of these equilibrium models, there are significant simplifications compared to the previously proposed MPECs. While further investigations into these appears worthwhile, we focus our analysis on a single-investor MPEC formulation for concreteness.

A number of extensions to these outlined MPEC formulations have been proposed. In [12], the authors extend the MPEC to include a futures market in addition to the spot market, while [13] uses a Benders Decomposition algorithm to decrease the computational burden of the original problem. [1] proposes a model with a two-stage setup that includes risk-averse investors using conditional value at risk (CVaR) as a risk measure. The authors also focus on investments in fluctuating renewable sources rather than conventional generation assets. Extending the model to include multiple future time periods has been proposed in [2], among others.

The model used in the current paper is an extension of that in [14] by increasing the planning horizon to multiple future time periods and introducing the concepts of rolling-horizon optimization and recourse action. Rolling-horizon optimization is common in scheduling and dispatch models, where previously made dispatch decisions can be altered before physically adjusting power plant production [18]. In addition, it is a common tool for the optimization of storage systems such as gas [5] and hydropower [10]. In a multi-stage investment framework, it is perhaps more realistic to include a rolling horizon to allow changing decisions based on updated information. In addition, a limited optimization horizon is a realistic assumption for actors making decisions under imperfect information. Additionally, in between each roll, such models can allow players to learn and adjust their data (e.g. costs, scenario probabilities) which allows for a richer model that is also more realistic. For example, after solving for time periods \(\{t,...,t+n\)}, only the solutions from the first time period t are retained. After this roll, each player can evaluate if they want to adjust their data strategically (e.g., lower costs if they lost market share). As described in [5], this allows for data to become strategic and endogenous learning by the players to be modeled in a dynamic manner.

Additionally, a rolling-horizon approach can avoid “over-optimization” which may occur with full-horizon models. That is to say, having the model greatly delay an investment to later years since it considers a very long (e.g., 50 year) horizon, which seems inconsistent with real-world decision-makers. Lastly, as a side benefit, since the sizes of the rolling-horizon problems are reduced, the computation times will decrease and in some cases this could be substantial as compared to a full-horizon approach [5].

One benefit of dynamic investment models is the possibility to split investment decisions into multiple stages of differing capacity intensity (e.g., planning and construction). This concept has been described as a real-options approach in [6] and applied to energy investments in [17] and others. We implement and evaluate this stage structure, which allows recourse action, by comparing it to one-time decisions that cannot be altered at a later time. Rolling-horizon planning is and established concept in energy system modeling. However, to the best of our knowledge it has not been applied to the strategic investment problem, or any MPEC formulation, thus far. Our main contributions are thus threefold:

  1. 1.

    Reformulate a dynamic strategic investment problem with a stochastic demand tree into a rolling-horizon MPEC.

  2. 2.

    Conduct a case study over a six-bus system to highlight the computational advantages related to this structure.

  3. 3.

    Evaluate the benefit of the investment-stage structure with the option of recourse action when compared to fixed one-time investment decisions.

For our case study, we also make a thorough comparison of the solutions in terms of investment compared to full-fledged stochastic programs. We are able to show, that in the absence of major structural breaks beyond the rolling planning horizon, the solutions of our rolling planning approach are close to those of a full stochastic model.

1.3 Paper organization

The remainder of this paper is structured as follows. Section  2 contains our problem formulation with emphasis put on the improvements we make over existing models. Section  3 contains the mathematical formulation of the strategic investment model we use. In Section 4, we define a reference case and use it to test multiple model configurations. Section 5 summarizes the results and concludes the paper.

2 Problem formulation

In this section, we briefly describe the strategic generation investment problem as it has been proposed in [19]. We discuss the model assumptions and highlight our modifications to the original model as proposed in [14]. Those are the introduction of a dynamic, multi-period setup with a stochastic demand tree, a rolling-horizon optimization, and multiple investment stages.

2.1 Bilevel model and assumptions

The decision framework of a strategic investor in generation capacity can be described as a bilevel problem: at the upper level, an investor aims to maximize net revenues from investment and operation of a power plant fleet. The investor decides how much, at what location, and which kind of generation technology to invest in. A number of constraints affect this upper-level problem: there may be a constraint to the available budget, and technologies may only be for purchase at discrete capacity levels. Given an investment decision (and additional existing plants), the investor decides on price-energy bids for the electricity spot market. The fact that strategic price bidding rather than marginal cost bidding is allowed constitutes a key difference between this model and central generation expansion planning.

The electricity spot market is modeled at the lower level. We consider the strategic investor and one or several marginal-cost bidding rival producers who also own power plants. A market operator treats generator and consumer price-quantity bids as given and maximizes social welfare in the system. Constraints to this problem are upper and lower limits to supply and demand as well as the model representation of the electricity grid, which may limit transmissions between different nodes in the system. The conflicting objectives of investor/generator (who acts first) and market operator give this problem its bilevel structure. Since the lower-level market operations problem is linear, it can be reformulated and replaced by its necessary and sufficient Karush- Kuhn-Tucker (KKT) conditions and included in the upper-level problem, thus resulting in an MPEC as described in Sect. 3. Key characteristics of our formulation are notably:

  1. 1.

    We represent the electricity grid through a DC load flow approximation and do not consider transmission losses (see [9]).

  2. 2.

    Competitors are assumed to offer electricity to the spot market at marginal cost.

  3. 3.

    In our scenarios, we assume only electricity demand to be stochastic. Including further stochastic factors is possible, but increases the computational burden and we speculate that it offers limited additional insights.

  4. 4.

    We model two technologies for investment: base load (coal) and peak load (combined-cycle gas). Including renewables is possible (see [1]) but not considered in this paper.

  5. 5.

    Modeled time periods encompass 3 years each, while an investment into a power plant requires 6 years (3 years each for planning and construction).

2.2 Stochastic demand tree

Demand for electricity is driven by various fundamental factors, such as economic development, as well as structural market changes, such as the anticipated widespread rollout of electric vehicles. Given the uncertainty of these factors, the short-term and long-term level of electricity demand is an uncertain parameter for investors in generation capacity. The simplest approach to model demand uncertainty is to allow for two possible developments per time period: an increase (up) and a decrease (down). This binary approach to uncertainty may seem simplistic, but is not uncommon in applications such as option pricing [4]. A three-branch (“trinary”) decision tree, which could also include a branch with unchanged demand, was also considered. However, initial results indicated limited additional benefit but a significant increase in computational burden. Therefore, we limit our analysis to two demand scenarios per time period.

Modeling this uncertainty over multiple time periods results in a stochastic tree, which is known as a binary tree. One problem commonly associated with the use of scenarios in stochastic optimization is the “curse of dimensionality”. When considering a number of uncertain parameters with independent scenario probabilities, the number of joint realization scenarios increases exponentially with the number of considered uncertainties [8]. The same issue applies when considering a single uncertain parameter, in our case demand, and a number of time periods. We initially considered the use of recombining trees, which can be used to limit the increase in scenarios from one time period to the next [7]. A recombining tree’s defining characteristic is that the order of previous transitions does not matter to a finally reached node, as the result is identical for each variation [8]. See the recombining stochastic tree as outlined on the left-hand side of Fig. 1: From N1, an up and a down transition always leads to N5, no matter which order they occur in.

However, this approach is not feasible for use in an investment problem for the following reasons. For an investor in generation technology in a dynamic setting, decisions need to be made at multiple points in time. Take for example node N5 in the left-hand stochastic tree in Fig. 1. It represents a level of demand that is equal to the root node N1. There are two paths that lead to N5, one via N2, the other via N3. However, the decision settings in each of these nodes are different: N2 represents a higher level of demand than N3, and with a higher expected value for future demand as well. The optimal investment is likely different depending on which path is realized. For this reason, even a recombining demand scenario tree leads to a system state tree that is non-recombining (see right-hand side of Fig. 1). The latter tree is what we use in our optimization problem.

Fig. 1
figure 1

Three-stage recombining demand tree (left) and non-recombining system state tree (right)

Full size image

2.3 Rolling-horizon structure

The static single-period version of the strategic investment problem is well-studied in the existing literature (cf. Sect.  1). Since the technical lifetime of conventional generation assets usually lasts several decades over which market conditions may change significantly, a dynamic, multi-period version appears worthwhile and has been proposed and implemented multiple times [2, 30]. In a stochastic formulation, the use of multiple time periods leads to an increase in the number of decision variables due to larger scenario trees and thus its use can be limited by computational complexity [22]. This motivates the use of a rolling-horizon version of the model, as it reduces the computational burden by only including a subset of time periods from the entire optimization horizon. This structure is presented in what follows.

Figure 2 depicts the basic structure of a rolling-horizon optimization or equilibrium problem. Roll 1 (R1) consists of the time periods \(t_1\) through \(t_3\). Going forward, all decisions for time period \(t_1\) are considered final because \(t_1\) is not part of any roll other than R1. Hence the decisions for \(t_1\) cannot be altered in a future roll. This is unlike the decisions for \(t_2\) and \(t_3\). R2 consists of \(t_2\) through \(t_4\) and allows changing the decisions made previously for \(t_2\) and \(t_3\). This structure for roll R1 is referred to as “here-and-now” decisions for \(t_1\), which are final, and “wait-and-see” decisions for \(t_2\) and \(t_3\), as these can be updated when new information becomes available [3]. Further rolls such as R3 may be included as necessary for the time horizon.

Fig. 2
figure 2

Basic rolling-horizon structure

Full size image

The detail and level of information for future time periods is generally better the closer said periods are to the present. For our model, we propose the following structure: the first two time-periods, each consisting of n years, are considered individually. The subsequent m time periods consist of \(m*n\) years and are aggregated into time period \(t_{3,agg}\), depicting the part of the expected lifetime of the investment for which there is limited information available. Figure 3 shows this aggregation of time periods. This structure also offers significant computational benefit when compared to an optimization of 3-year time periods over the entire plant lifetime. Figure 4 shows the rolling-horizon structure for the discussed, aggregated time periods.

Fig. 3
figure 3

Time period aggregation

Full size image
Fig. 4
figure 4

Dynamic-period rolling-horizon structure

Full size image

2.4 Multiple investment stages and recourse action

In static models, investments for generation assets are seen as single-stage decisions. If an investment decision is deemed optimal, the plant is built and becomes operational for the modeled electricity spot market. For dynamic models, this structure may be too simple. The construction of a large-scale power plant is a complex process encompassing multiple stages that take several years to complete. Beginning with a planning stage, there are bureaucratic hurdles to clear before eventually physical construction takes place. From an optimization perspective, the fact that these stages vary in capital intensity matters to the decision process. The relatively cheap planning stage can be used to consider an investment opportunity with limited capital exposure. The decision of whether to enter the more capital-demanding construction stage can be postponed: it is only entered if it is worthwhile under updated market conditions at a later date, or can be abandoned if no longer economically feasible. This concept is known as recourse action and provides an optionality aspect which is well-suited for the rolling-horizon model.

Since we propose a dynamic model, we split the investment process into two stages. We use a simple split into a planning stage, which requires a small fraction x of total capital expenditure, and a subsequent construction stage, which requires the fraction \(1 - x\) (e.g., \(x = 20\%\)). Each stage is assumed to require n years to complete, which aligns with the time period aggregation we proposed in the previous subsection. After going through these two stages the plant is considered operational. This stage structure is outlined in Fig. 5. Since the time periods we consider in our rolling-horizon planning are aligned with the stage duration, it is possible for the investor to plan capacity for its option value and revise the construction decision at a later time.

Fig. 5
figure 5

Investment stage progression

Full size image

Note that when considering a planning horizon which is shorter than the operational lifetime of an investment, end-of-horizon effects need to be considered. To capture this, both planning and construction costs are reduced according to the length of the planning horizon relative to the asset lifetime. This implies that the remaining asset value is considered, thereby eliminating unintended end-of-horizon effects.

3 Mathematical model

3.1 Nomenclature

The following tables define the nomenclature for the mathematical model formulation. Table 1 contains sets, Tables 2 and 3 decision variables (split into upper- and lower-level variables to improve readability). Table 4 contains the lower-level, dual variables used in the MPEC reformulation. Table 5 contains the model parameters. Sets and variables start with a lower-case letter, parameters with an upper-case letter. Where possible, we use the nomenclature from [14], as our model is an extension of the work presented there.

We consider one strategic investor with a number of investment technology options i as well as existing power plants e. In addition, there are rival producers who own the existing power plants r. Electricity consumption is represented by a number of demands d, that make price-quantity bids into the spot market. These vary depending on load blocks o, which can represent peak and off-peak intervals or a more detailed representation of the demand duration curve. The investment process is split into the planning (PL), construction (CO), and operation (OP) stages, as described in Sect. 2.

Table 1 Indices
Full size table
Table 2 Upper-level decision variables
Full size table
Table 3 Lower-level primal decision variables
Full size table
Table 4 Lower-level dual decision variables
Full size table
Table 5 Parameters
Full size table

3.2 Bilevel model

The strategic generation investment optimization is formulated as a bilevel model. The upper level represents the investment optimization problem of the individual investor, while the lower level represents the spot market operated by a welfare-maximizing operator. This formulation is based on [14] and goes beyond that by including a rolling horizon and recourse action (see Sect. 2). The optimization problem is dynamic and encompasses multiple time periods. This market operation is conducted for different load blocks, i.e., peak and off-peak, in each time period considered.

3.2.1 Upper-level problem

Equation 1 represents the objective function of the strategic investor. Since it is defined as a minimization problem, the objective is the negative of the expected profit. This negative expected profit is made up of three terms: investment costs (line 1) minus spot market profits from new (line 2) and existing (line 3) power plants (consisting of revenues minus production costs).

$$\begin{aligned} \begin{array}{l} Minimize \\ \begin{array}{rl} &{} \left( \begin{array}{l} \sum _{t,i,w,n} \rho _{t} \omega _w \left( x^{PL,new}_{t,i,w,n} K^{PL}_i + x^{CO,new}_{t,i,w,n} K^{CO}_i \right) \\ - \sum _{t,o,w,n} \rho _{t} \sigma _o \omega _w \bigg ( \sum _{i \in \Psi _{n}} \left( (\lambda ^{price}_{t,o,w,n} - C^I_i) p^I_{t,o,w,i,n} \right) \\ + \sum _{e \in \Psi _{n}} ((\lambda ^{price}_{t,o,w,n} - C^E_e) p^E_{t,o,w,e,n}) \bigg ) \\ \end{array} \right) \\ \end{array} \\ \end{array} \end{aligned}$$
(1)

As just outlined in the previous section, our model reflect projects in different stages of completion. We therefore split the investment process into a planning stage (PL) and a construction stage (CO). Once finished, plants are considered operational (OP). Projects that were finished at time t are considered as new, while the total operational capacity consists of all projects finished by this time period. Equations 2 through 5 represent this structure in the model by ensuring that any additional capacity passes through all stages. Note that the \(\ge \) symbol in Eqs. 3 and 4 allow the model to abandon projects by realizing less new capacity than previously planned or constructed (i.e., recourse action). In model configurations that do not allow recourse decisions, these constraints would be equalities.

$$\begin{aligned} x^{PL,new}_{t,i,w,n}\ge & {} 0, \quad \forall t,\forall i,\forall w,\forall n \end{aligned}$$
(2)
$$\begin{aligned} x^{PL,new}_{t-1,i,w,n}\ge & {} x^{CO,new}_{t,i,w,n}, \quad \forall t,\forall i,\forall w,\forall n \end{aligned}$$
(3)
$$\begin{aligned} x^{CO,new}_{t-1,i,w,n}\ge & {} x^{OP,new}_{t,i,w,n}, \quad \forall t,\forall i,\forall w,\forall n \end{aligned}$$
(4)
$$\begin{aligned} x^{OP,total}_{t,i,w,n}= & {} x^{OP,total}_{t-1,i,w,n} + x^{OP,new}_{t,i,w,n}, \quad \forall t,\forall i,\forall w,\forall n \end{aligned}$$
(5)

As described in Sect. 2.3, the optimization represents the point in time immediately before time period t. Hence, there are scenarios for all time periods in the optimization, but decisions for the first time period need to be finalized at this time and thus are required to be consistent across scenarios, which is stated in Eqs. 6 through 8. These are the non-anticipativity constraints, which can also be considered the “here-and-now decisions”. All variables for time periods \(t_2\) and later are considered “wait-and-see decisions” and thus can be modified depending on future developments. Thus they do not need to be consistent across scenarios.

$$\begin{aligned} x^{PL,new}_{t,i,v,n} = x^{PL,new}_{t,i,w,n}, \quad t=1, \forall i,\forall w,\forall v,\forall n \end{aligned}$$
(6)
$$\begin{aligned} x^{CO,new}_{t,i,v,n} = x^{CO,new}_{t,i,w,n}, \quad t=1, \forall i,\forall w,\forall v,\forall n \end{aligned}$$
(7)
$$\begin{aligned} x^{OP,new}_{t,i,v,n} = x^{OP,new}_{t,i,w,n}, \quad t=1, \forall i,\forall w,\forall v,\forall n \end{aligned}$$
(8)

Equation 9 puts an upper limit on the possible investment capacity per technology i. This constraint is included to be able to enforce investments in multiple technologies and not just one potentially dominating one.

$$\begin{aligned} \sum _n x^{OP,total}_{t,i,w,n} \le X^{max}_{i}, \quad \forall t,\forall i,\forall w \end{aligned}$$
(9)

3.2.2 Lower-level problem

The lower-level problem represents the spot market for electricity. The problem is solved independently for each scenario w and load block o (e.g., peak and off-peak). Throughout the constraints given in Eqs. 11 through 18, the dual variables \(\lambda \) are stated since they are required for the MPEC reformulation given in Sect. 3.3.

Equation 10 represents the objective function of the lower-level optimization problem. Since we assume a welfare-maximizing market operator, the objective is to maximize the exchange of electricity between producers and consumers given price-quantity bids on both sides of the market. Note that this representation of electricity consumption is different from a minimization of production costs given fixed consumption levels, which is a common feature in central planning models. Since our model allows strategic bidding by a small number of producers, a limit on consumers’ willingness-to-pay is necessary to prevent unlimited profits.

$$\begin{aligned} \begin{aligned}&\min \sum _n \bigg ( \sum _i a^I_{t,o,w,i} p^I_{t,o,w,i,n} + \sum _e a^E_{t,o,w,e} p^E_{t,o,w,e,n}\\&\quad + \sum _r C^R_{r,w} p^R_{t,o,w,r,n} - \sum _d C^D_{d,o} p^D_{t,o,w,d,n} \bigg ), \quad \forall t, \forall o,\forall w \end{aligned} \end{aligned}$$
(10)

Equation 11 represents the electricity balance at each node in the system. The sum of all production and supply needs to equal imports and exports into the modeled grid for each node. Since we model the electricity grid using a DC load flow approximation, the flow on a line between nodes n and m is given by multiplying the difference in voltage angles \(\theta _{t,o,w,n} - \theta _{t,o,w,m}\) by the susceptance matrix B.

$$\begin{aligned} \begin{array}{lr} 0 = \sum _{d \in \Psi _{n}} p^D_{t,o,w,d,n} + \sum _{m \in \Omega _{n}} B_{n,m} (\theta _{t,o,w,n} - \theta _{t,o,w,m}) \\ - \sum _{i \in \Psi _{n}} p^I_{t,o,w,i,n} - \sum _{e \in \Psi _{n}} p^E_{t,o,w,e,n} - \sum _{r \in \Psi _{n}} p^R_{t,o,w,r,n}: \lambda ^{price}_{t,o,w,n}, \\ \quad \forall t,\forall o,\forall w,\forall n \end{array} \end{aligned}$$
(11)

Equations 12 through 15 put limits on production and consumption. The lower level is zero for these variables. The flow on a given line is required to be within the respective transmission capacity limit, as stated in Eq. 16.

$$\begin{aligned} 0 \le p^I_{t,o,w,i,n}\le & {} x^{OP,total}_{t,i,w,n}: \lambda ^{I,min}_{t,o,w,i,n}, \lambda ^{I,max}_{t,o,w,i,n}, \quad \forall t,\forall o,\forall w,\forall i,\forall n \end{aligned}$$
(12)
$$\begin{aligned} 0 \le p^E_{t,o,w,e,n}\le & {} P_{t,e,n}^{{E,max}}: \lambda ^{E,min}_{t,o,w,e,n}, \lambda ^{E,max}_{t,o,w,e,n}, \quad \forall t,\forall o,\forall w,\forall e,\forall n \end{aligned}$$
(13)
$$\begin{aligned} 0 \le p^R_{t,o,w,r,n}\le & {} P_{t,r,w,n}^{{R,max}}: \lambda ^{R,min}_{t,o,w,r,n}, \lambda ^{R,max}_{t,o,w,r,n}, \quad \forall t,\forall o,\forall w,\forall r,\forall n \end{aligned}$$
(14)
$$\begin{aligned} 0 \le p^D_{t,o,w,d,n}\le & {} P_{t,d,o,w,n}^{{D,max}}: \lambda ^{D,min}_{t,o,w,d,n}, \lambda ^{D,max}_{t,o,w,d,n}, \quad \forall t,\forall o,\forall w,\forall d,\forall n \end{aligned}$$
(15)
$$\begin{aligned} -F^{max}_{n,m}\le & {} B_{n,m} (\theta _{t,o,w,n} - \theta _{t,o,w,m}) \le F^{max}_{n,m}: \lambda ^{F,min}_{t,o,w,n,m}, \lambda ^{F,max}_{t,o,w,n,m}, \nonumber \\&\quad \forall t,\forall o,\forall w,\forall n,\forall m \in \Omega _{n} \end{aligned}$$
(16)

Finally, the DC load flow approximation requires stating the logic of voltage angles \(\theta \) in the system. For all nodes \(n \ne 1\), \(\theta \) is restricted to be between positive and negative \(\pi \). For an arbitrarily chosen node \(n = 1\), it is fixed to zero. These constraints are given in Eqs. 17 and 18.

$$\begin{aligned}&-\pi \le \theta _{t,o,w,n} \le \pi : \lambda ^{\theta ,min}_{t,o,w,n},\lambda ^{\theta ,max}_{t,o,w,n}, \quad \forall t,\forall o,\forall w,\forall n \end{aligned}$$
(17)
$$\begin{aligned}&\theta _{t,o,w,n} = 0, \lambda ^{\theta ,fix}_{t,o,w,n}, \quad n=1, \forall t,\forall o,\forall w \end{aligned}$$
(18)

3.3 MPEC

In order to solve the bilevel problem given in Sect. 3.2, we reformulate it into an MPEC. This occurs by replacing the lower-level problem by its necessary and sufficient optimality conditions, which are known as Karush- Kuhn-Tucker (KKT) conditions [9]. This reformulation is valid because the lower-level problem is linear and thus the KKT conditions are sufficient as well as necessary. All constraints of the upper level problem (as defined in Eq. 2 through 9), the lower-level problem (Eqs. 11 through 18), as well as the upper-level objective function (Eq. 1) are part of the MPEC. The latter is linearized to the form given in Eq. 19.Footnote 2 Finally, the KKT conditions given in Eqs. 20 through 36 are also part of the MPEC.

$$\begin{aligned} \begin{aligned}&\min \sum _{t,i,n} \rho _{t} \omega _w \left( x^{PL}_{t,i,w,n} K^{PL}_i + x^{CO,total}_{t,i,w,n} K^{CO}_i \right) \\&\quad - \sum _{t,o,w,n} \rho _{t} \sigma _o \omega _w \bigg ( \sum _{i \in \Psi _{n}} C^I_i p^I_{t,o,w,i,n} + \sum _{e \in \Psi _{n}} C^E_e p^E_{t,o,w,e,n} \\&\quad - \sum _{r \in \Psi _{n}} \left( C^R_{r,w} p^R_{t,o,w,r,n} + \lambda ^{R,max}_{t,o,w,r,n} P_{t,r,w,n}^{{R,max}} \right) \\&\quad + \sum _{d \in \Psi _{n}} \left( C^D_{d,o} p^D_{t,o,w,d,n} - \lambda ^{D,max}_{t,o,w,d,n} P_{t,d,o,w,n}^{{D,max}} \right) \\&\quad - \sum _{m \in \Omega _{n}} \left( \lambda ^{F,max}_{t,o,w,n,m} F^{max}_{n,m} + \lambda ^{F,min}_{t,o,w,n,m} F^{max}_{n,m} \right) \\&\quad - \lambda ^{\theta ,max}_{t,o,w,n} \pi - \lambda ^{\theta ,min}_{t,o,w,n} \pi \bigg ) \end{aligned} \end{aligned}$$
(19)

Equations 20 through 24 are the derivatives of the Lagrangian function to the lower-level decision variables (the derivatives to \(p^I\), \(p^E\), \(p^R\), \(p^D\), and \(\theta \), respectively). These are also known as stationarity conditions and are equal to zero for all equilibrium solutions to the lower-level problem.

$$\begin{aligned}&a^I_{t,o,w,i} - \lambda ^{price}_{t,o,w,n} + \lambda ^{I,max}_{t,o,w,i,n} - \lambda ^{I,min}_{t,o,w,i,n} = 0, \quad \forall t,\forall o,\forall w,\forall i \in \Psi _{n},\forall n \end{aligned}$$
(20)
$$\begin{aligned}&a^E_{t,o,w,e} - \lambda ^{price}_{t,o,w,n} + \lambda ^{E,max}_{t,o,w,e,n} - \lambda ^{E,min}_{t,o,w,e,n} = 0, \quad \forall t,\forall o,\forall w,\forall e \in \Psi _{n},\forall n \end{aligned}$$
(21)
$$\begin{aligned}&C^R_{r,w} - \lambda ^{price}_{t,o,w,n} + \lambda ^{R,max}_{t,o,w,r,n} - \lambda ^{R,min}_{t,o,w,r,n} = 0, \quad \forall t,\forall o,\forall w,\forall r \in \Psi _{n},\forall n \end{aligned}$$
(22)
$$\begin{aligned}&- C^D_{d,o} + \lambda ^{price}_{t,o,w,n} + \lambda ^{D,max}_{t,o,w,d,n} - \lambda ^{D,min}_{t,o,w,d,n} = 0, \quad \forall t,\forall o,\forall w,\forall d \in \Psi _{n},\forall n \end{aligned}$$
(23)
$$\begin{aligned}&\sum _{m \in \Omega _{n}} B_{n,m} \left( \lambda ^{price}_{t,o,w,n} - \lambda ^{price}_{t,o,w,m} \right) + \sum _{m \in \Omega _{n}} B_{n,m} \left( \lambda ^{F,max}_{t,o,w,n,m} - \lambda ^{F,max}_{t,o,w,m,n} \right) \nonumber \\&\quad + \sum _{m \in \Omega _{n}} \left( \lambda ^{F,min}_{t,o,w,n,m} - \lambda ^{F,min}_{t,o,w,m,n} \right) + \lambda ^{\theta ,max}_{t,o,w,n} - \lambda ^{\theta ,min}_{t,o,w,n} + \left( \lambda ^{\theta ,fix}_{t,o,w,n} \right) _{n=1} = 0,\nonumber \\&\quad \forall t,\forall o,\forall w,\forall n \end{aligned}$$
(24)

Equations 25 through 36 are complementarity constraints. They correspond to the values of the dual variables of the lower-level constraints, hence each of them corresponds to a constraint of the lower-level problem. In our model implementation, we use the special ordered sets of type 1 (SOS1) formulation described in [23] to linearize the complementarity constraints. This is an alternative to the “Big M” approach used by [13] and others. Both approaches use similar formulations for the complementarity constraints involving primal and dual variables (cf. Eqs. 25 through 36). Yet they differ in the way of linearizing these non-linear inequalities. The ”big M” method introduces binary variables and a somewhat arbitrarily selected scalar parameter, namely the big M. This arbitrary scalar is avoided by the SOS1-method developed in [23], as discussed in detail in the Appendix: Linearization.

Equations 25 through 32 correspond to the capacity constraints given in Eqs. 12 through 15.

$$\begin{aligned} 0 \le p^I_{t,o,w,i,n} \perp \lambda ^{I,min}_{t,o,w,i,n} \ge 0, \quad \forall t,\forall o,\forall w,\forall i,\forall n \end{aligned}$$
(25)
$$\begin{aligned} 0 \le p^E_{t,o,w,e,n} \perp \lambda ^{E,min}_{t,o,w,e,n} \ge 0, \quad \forall t,\forall o,\forall w,\forall e,\forall n \end{aligned}$$
(26)
$$\begin{aligned} 0 \le p^R_{t,o,w,r,n} \perp \lambda ^{R,min}_{t,o,w,r,n} \ge 0, \quad \forall t,\forall o,\forall w,\forall r,\forall n \end{aligned}$$
(27)
$$\begin{aligned} 0 \le p^D_{t,o,w,d,n} \perp \lambda ^{D,min}_{t,o,w,d,n} \ge 0, \quad \forall t,\forall o,\forall w,\forall d,\forall n \end{aligned}$$
(28)
$$\begin{aligned} 0 \le \left( x^{OP,total}_{t,i,w,n} - p^I_{t,o,w,i,n} \right) \perp \lambda ^{I,max}_{t,o,w,i,n} \ge 0, \quad \forall t,\forall o,\forall w,\forall i,\forall n \end{aligned}$$
(29)
$$\begin{aligned} 0 \le \left( P_{t,e,n}^{{E,max}} - p^E_{t,o,w,e,n} \right) \perp \lambda ^{E,max}_{t,o,w,e,n} \ge 0, \quad \forall t,\forall o,\forall w,\forall e,\forall n \end{aligned}$$
(30)
$$\begin{aligned} 0 \le \left( P_{t,r,w,n}^{{R,max}} - p^R_{t,o,w,r,n} \right) \perp \lambda ^{R,max}_{t,o,w,r,n} \ge 0, \quad \forall t,\forall o,\forall w,\forall r,\forall n \end{aligned}$$
(31)
$$\begin{aligned} 0 \le \left( P_{t,d,o,w,n}^{{D,max}} - p^D_{t,o,w,d,n} \right) \perp \lambda ^{D,max}_{t,o,w,d,n} \ge 0, \quad \forall t,\forall o,\forall w,\forall d,\forall n \end{aligned}$$
(32)

Equations 33 through 34 correspond to the transmission constraints given in Eq. 16.

$$\begin{aligned} 0 \le \left( F^{max}_{n,m} - B_{n,m} (\theta _{t,o,w,n} - \theta _{t,o,w,m}) \right) \perp \lambda ^{F,max}_{t,o,w,n,m} \ge 0, \quad \forall t,\forall o,\forall w,\forall n,\forall m \end{aligned}$$
(33)
$$\begin{aligned} 0 \le \left( F^{max}_{n,m} + B_{n,m} (\theta _{t,o,w,n} - \theta _{t,o,w,m}) \right) \perp \lambda ^{F,min}_{t,o,w,n,m} \ge 0, \quad \forall t,\forall o,\forall w,\forall n,\forall m \end{aligned}$$
(34)

Finally, Eqs. 35 through 36 correspond to the voltage angle constraints given in Eqs. 17 through 18.

$$\begin{aligned} 0 \le \left( \pi -\theta _{t,o,w,n} \right) \perp \lambda ^{\theta ,max}_{t,o,w,n} \ge 0, \quad \forall t,\forall o,\forall w,\forall n \end{aligned}$$
(35)
$$\begin{aligned} 0 \le \left( \pi +\theta _{t,o,w,n} \right) \perp \lambda ^{\theta ,min}_{t,o,w,n} \ge 0, \quad \forall t,\forall o,\forall w,\forall n \end{aligned}$$
(36)

4 Application

4.1 Data

In this section, we describe the data we use for a numerical application of the proposed model. Like the model, we base our case study on [14], whose authors use a six-node system for their illustrative example. Since we propose a dynamic formulation of their static model, we modify the data accordingly to create a multi-period setting. Since the dynamic formulation is significantly more challenging computationally than the original static model, we make a number of simplifications.

Our case study is based on a six-node grid (see Fig. 6). The assumed transmission capacities split the grid in a north and a south zone, each of which contain three nodes and no internal bottlenecks. A transmission capacity of 5000 MW on all internal lines satisfies this assumption. The transmission capacity on line B2–B4 and line B3–B6 is limited to 100 MW. As demand is largely concentrated in the southern zone, while existing generation capacity is mostly located in the northern zone, this limited transmission capacity can create a bottleneck for certain load blocks. This bottleneck may lead to diverging prices, and thus attractive investment opportunities for the strategic investor. Since some of our model configurations are computationally challenging, we also define nodes B3 and B6 as a subsystem for use in a limited number of model runs. This subsystem is outlined by the shaded area.

Fig. 6
figure 6

Six-node and two-node (shaded) case study test systems

Full size image
Table 6 Maximum consumption [MW] per demand and load block
Full size table
Table 7 Willingness-to-pay (WTP) [$/MWh] per demand and load block
Full size table

We model load blocks o1 through o4, which represent aggregations of the load duration curve into manageable chunks for our model. The maximum consumption per demand and load block as well as the willingness-to-pay (WTP) for the same demand-load block combinations is given in Tables 6 and 7 . Load block o1 represents a high consumption, high-WTP situation commonly associated with peak load, while load block o4 corresponds to off-peak situations. Blocks o2 and o3 represent hours with intermediate consumption and WTP. The scenario tree described in Sect. 2 is created by increasing or decreasing the given base consumption figures by 5% to create an up and a down scenario for the following time period.

We describe the data for the existing power plants in Table 8. Both the strategic investor and rival producers own existing plants, none of which are decommissioned during the modeled time horizon (i.e., we model a static preexisting power plant fleet). However, the amount of installed capacity in the system at \(t_1\) is chosen so that a positive amount of new capacity is optimal for all demand scenarios (i.e., there is sufficient demand to incentivize new investment).

Table 8 Power plants in application
Full size table

We model two investment technologies (see Table 9). The two differ in the investment and generation cost, with the base option representing a typical base load plant (e.g., coal or lignite), while the peak option corresponds more to a peak-load generator (e.g., gas). The cost structure of the gas option is meant to represent a combined-cycle gas turbine (CCGT). Since we model the entire lifetime of the plants, note that the cost components \(K^{PL}_i\) and \(K^{PL}_i\) represent the t tal costs of investment, not the annualized costs commonly used in static models. We also limit the amount of capacity that can be installed: 500 MW per technology in the two-node example and 1500 MW per technology in the six-node case study.

Table 9 Power plants investment options
Full size table

Finally, the temporal scope of our case study consists of six time periods \(t_1\) through \(t_6\). The length of time periods \(t_1\) to \(t_5\) is three years each, while t6 is defined as consisting of 24 years, resulting in a total optimization horizon of 39 years (cf. Sect. 2.3). We use a yearly discount rate of \(5\%\) to determine the discount factors \(\rho _t\). The binomial system state tree consists of 63 nodes (cf. Sect. 2.2).

4.2 Test cases

One contribution of our work is to evaluate the usefulness of a rolling-horizon optimization (see Sect. 2) and the benefit of allowing recourse action within the rolling-horizon paradigm. This requires the definition of several distinct model configurations which can be tested against all possible paths through the scenario tree. The configurations we propose vary in the number of model features included (see Table 10).

Table 10 Definition of model configurations
Full size table

We briefly describe the three model features in what follows. Perfect foresight is a deterministic optimization configuration in which the ultimate path through the scenario tree is known beforehand. This is implemented by replacing the scenario probabilities \(\omega _w\) with the actual realization paths. As such, the feature can be used as an optimal (although generally unreachable) benchmark for all other configurations. Rolling-horizon planning differs from fixed-horizon planning in the scope of time periods that are considered in a given optimization run. We define a fixed-horizon stochastic optimization as our baseline configuration. In this configuration, the model is initially solved over all scenarios and all time periods T until the final planning horizon at \(t_6\). The results are fixed for \(t_1\), and the model is rerun for the remaining five time periods starting at \(t_2\) over the stochastic sub-tree that is reachable from the current node. This structure is outlined on the left-hand side of Fig. 7. For rolling-horizon planning, only a subset \(T_{RH} \subset T\) of time-periods is considered in each optimization. This structure is shown on the right-hand side of Fig. 7.

Aside from the perfect foresight benchmark, we consider the following four additional model configurations. Configuration RH is the standard rolling-horizon formulation with multiple investment stages, i.e., including the opportunity for recourse action. Configuration LP (limited planning) is identical, except for the fact that no recourse action is allowed: all decisions are final and there is limited planning due to the rolling horizon. This lack of recourse action is implemented by replacing Eqs. 3 and 4 with strict equalities. Configuration ST is based on a stochastic optimization over the entire scenario tree, but no recourse action is allowed. Configuration RA is complementary in the sense that recourse action is allowed through investment stages, but there is no rolling-horizon structure. The characteristics of the different configurations are also given in Table 10.

Fig. 7
figure 7

Tree and sub-tree optimization horizons for fixed horizon (left) and rolling horizon (right) model configurations

Full size image

4.3 Results

We display the results of our two-node example and our six-node case study systems in what follows. The results we show are based on optimization runs over all possible realization paths. In total, there are \(2^5 = 32\) paths resulting from six time periods and the binomial uncertainty tree we use. Table 11 shows the results for the two-node example. All variable values stated are averages over all realization paths. The sole exception is computation time, which is the sum of all optimization times.

Overall, the results show that there is limited impact of model configuration on realized profits for the examples considered. While perfect foresight optimization yields just under 3.6 billion EUR, the stochastic optimization configurations are between just under 4 and 7 million EUR behind, which corresponds to a relative gap of \(0.10\%\) and \(0.21\%\). Clearly, the loss in objective function value from deviating from the perfect foresight solution is quite limited, suggesting that the likelihood of making unprofitable investment decisions is low. On the other hand, it is also noteworthy that the rolling-horizon (RH) configuration at - 6.8 million EUR performs similarly to the fully stochastic recourse action configuration (RA) at - 3.2 million EUR. Considering purely the objective function value, this implies that the (much faster) heuristic can be used as a proxy with only a limited reduction in solution quality (at least for the considered problems).

However, the objective function value is only one (albeit an important) metric for solution quality. To take a more holistic view, the average installed capacity per technology can also be considered: here, the two configurations perform almost identical, by varying only in 1 MW of peak capacity on average. The average electricity price (not shown in the table) is also identical in RH and RA. Similarly, the two configurations LP and ST (both removing recourse action, but only LP featuring the rolling-planning horizon) show a similar picture: the rolling-planning horizon leads to marginally lower revenues, but the average installed capacity is identical. A more detailed review of the results revealed that the difference in newly installed generation capacities (per time period, scenario and bus) differs by at most 2 MW between the configurations, i.e. the rolling-horizon heuristic results in almost no solution difference. This implies that for the reduced 2-node case study, over-investment or over-optimization does not play a detrimental role on the full-horizon configurations. To us the (almost) identical results suggest that in this problem instance, the tradeoff between problem size and solution quality favors the rolling-horizon heuristic.

Table 11 Results of the two-node test case
Full size table
Table 12 Results of the six-node test case
Full size table

The benefit of recourse action over fixed decisions can be evaluated as well. When comparing the configurations that allow recourse action (RH and RA) with those that do not (LP and ST), the value of being able to abandon a project becomes evident. In all cases, recourse action improves the objective function. This improvement is between 1.7 million EUR / 0.05% (two-node RA vs ST) and 6.2 million EUR / 0.1% (six-node RH vs LP). Considering the amount of installed capacity, it is noticeable that the LP and ST configurations find higher amounts of new capacity optimal than the RH and RA configurations. In combination with the lower expected profits, this finding suggests that some of the realized projects would have been abandoned if recourse action were allowed.

The five model configurations result in significantly different computation times. The PF configuration solves very quickly, as one would expect from a deterministic optimization problem. When comparing the rolling-horizon configurations RH and LP to the fixed-horizon configurations RA and ST, it is apparent that the initial planning horizon of six time periods comes at significant computational cost. In fact, the two-node example was chosen to showcase this difference while still being able to provide solutions for the fixed-horizon configurations with small, mixed-integer programming (MIP) gaps. We also give statistics about the problem size under the different configurations. It is apparent that computation time correlates with problem size, and particularly the number of SOS type 1 variables which we use to linearize the complementarity constraints. Since these variables are what make the problem mixed-integer, this is unsurprising. However, it is also noteworthy that the smaller scenario tree which is used in model configurations with rolling-horizon optimization reduces the problem size significantly.

The results of the two-node example point to a limited tradeoff between solution quality and computation time when using a rolling-planning horizon.Footnote 3 While this can be a matter of preference in small example settings, in larger problems it can mean the difference between finding a feasible solution and the MIP algorithm not terminating at all. This is the case in our six-node case study system, the results of which are presented in Table 12. As in the two-node case, the absolute and relative differences between the different configurations are small. Most of the perfect foresight revenue of just under 5.3 billion EUR is achievable with the different stochastic optimization configurations.

The RA configuration does not terminate at the target MIP gap values in adequate time. We therefore run the model at higher target MIP gaps to attempt to show the impact on results. Apparently, even a gap of 10% yields results that are within half a percent of the perfect foresight value. A 7.5% gap does not yield any additional benefits. The 5% gap (and any levels below that) do not terminate in six hours. Indeed, the solver progress slows down to a point where improvements to the solution appear to vanish. In this context, being able to solve the model through the approximation of a rolling-planning horizon offers significant benefit. While the solution quality of the RA configuration given a 10% MIP gap is slightly better than that of the RH formulation at \(10^{-6}\), this property is specific to the given problem instance and does not generalize. A small MIP gap is preferable to ensure the given solution does correspond to the global optimum of the MIP problem. The differences in newly installed generation capacities (per time period, scenario and bus) between the configurations appear unreliable due to the large MIP gap in the RA runs, i.e. there is no guarantee that these variable values are in fact optimal. Again, it is apparent that computation time is correlated to problem size, particularly the number of SOS type 1 variables.

5 Summary and conclusion

In the paper, we have introduced a dynamic investment MPEC that can be solved using a rolling-horizon optimization or as a fixed-horizon stochastic optimization. The extension of a static investment model to multiple time periods is shown to allow consideration of discrete stages in power plant investment, a representation closer to reality than an unalterable, one-off decision.

MPECs are particularly difficult to solve in dynamic multi-period settings. Using a simplified six-node system, the computational effort of strategic investment planning for assets with multi-decade lifetimes is apparent. While much smaller than real-world applications, even this problem size is shown to lead to unacceptable solution times, requiring trade-offs regarding guaranteed integer optimality and larger MIP gaps.

To provide an alternative to arbitrarily choosing a larger MIP gap, we have introduced a rolling-planning horizon motivated first by its realism advantage in representing real-world decision making. Its impact on solution time shows a clear improvement compared to integrated fixed-horizon approaches. The negative impact on solution quality, considering both objective function value and the (average) realization of decision variables, appears limited. However, this finding is clearly dependent on the problem instance considered and may not generalize. Our assessment of a smaller problem of the same type allowed to estimate the impact for larger instances, an approach which can be useful for providing first estimates regarding the tradeoff.

An additional model instance of a 118-node case study (IEEE test system) did not terminate in adequate time even for the rolling-horizon configuration. This provides evidence of the high mathematical complexity of dynamic multi-stage investment models in real-world systems, but it also indicates the rolling-horizon approach is unable to overcome all such computational limitations. We therefore recommend further investigations on larger case studies, including also comparisons with other speed-up procedures such as Benders decomposition, ADMM approaches or linear decision rules.

As an additional outcome, the usefulness of recourse action is shown for the problem at hand. Given stochastic scenarios, there are times when starting a multi-period investment project simply for the option value is economically sensible. Since modeling this behavior is not possible within a one-off optimization problem, its consideration may facilitate the use of dynamic rolling-horizon structures in the future.