Co-movements between forward prices and resource availability in hydro-dominated electricity markets

Abstract

Hydropower producers estimate the opportunity value of their water, known as a water value, by comparing current prices to future opportunities. When hydropower dominates the energy mix, the system’s hydrological state predominantly governs supply and thus prices. Despite this intuitive relationship, industry practice is to assume that inflow to reservoirs and prices are independent when they establish operational policies 1–2 years ahead. To investigate the impact of this assumption, we formulate the hydropower scheduling problem as a Markov decision process and develop a novel price model that considers the joint dynamics of forward prices and inflows. We find that producers underestimate their water value when they ignore co-movements between price and inflow. The dependency makes producers more willing to postpone generation and tolerate slightly higher spillage risk. This is because high inflow periods tend to observe low prices and the reservoir capacity is limited. Nevertheless, a case study of a hydropower plant with industry data suggests modest economic losses in practice. Our numerical results suggest a potential gain of 0.17% in expected revenue and approximately unchanged revenue variance if producers consider the co-movements when establishing an operational policy.

1 Introduction

Reservoir management is fundamental to ensure efficient allocation of resources in electricity markets dominated by hydropower [22, 44]. Decision-makers want an operational policy that maximizes expected rewards or minimizes expected costs over a given time horizon, considering production and reservoir constraints and uncertainty. As a result, they must solve a sequential decision problem under uncertainty. From a system perspective, the goal is to minimize costs [37], possibly with a risk measure [41]. Individual producers, on the other hand, aim to maximize profits [8, 17, 20, 26]. Inflow to reservoirs is a common source of uncertainty, and in liberalized wholesale markets, prices are also uncertain. A reasonable assumption in such markets is that producers are price-takers and cannot influence the price by their decisions. However, inflows impact prices. According to both economic theory and empirical observations, low inflows to reservoirs correlate with high prices and vice versa in hydro-dominated systems. This is because hydropower has no fuel price and producers instead calculate a water value to determine the expected price to sell energy [38]. When a system has abundant water in its reservoirs, the water value decreases because there is sufficient supply and even a risk of spillage. Conversely, a system with limited water has scarce supply and risk not being able to meet demand. Despite this intuitive relationship, a common industry practice is to assume independent stochastic variables when determining an operational policy. One reason is that forecasting prices and forecasting inflow are functionally distinct activities, performed by different professions. This reason is mirrored by [23], studying a problem involving stochastic wind infeed and electricity prices. For hydropower, it is difficult to identify and estimate underlying co-movements because system prices typically correlate with aggregated system inflows rather than local ones. Still, aggregate inflows are a result of the local inflows at producer locations. Consequently, there is little information about the value and effect of including co-movements between prices and inflow in medium-term hydropower planning.

[30] argue that correlations between accumulated inflow over a whole season and prices are more important than correlations between weekly prices and inflow because of storage capacity, but the authors do not provide any numerical results to support the argument. Common industry practice is to partially ignore correlations when optimizing release decisions. A common process is to first create simultaneous scenarios for price and inflow from a power market model where inflow is the input and price the output [45]. In the second step, typically these simultaneous scenarios are used as input to a more detailed optimization of reservoirs in a smaller geographical area. However, the state-of-the art algorithm used to optimize release decisions consider price and inflow as independent stochastic processes [20]. Indeed, no works properly investigate the impact of ignoring co-movements in hydropower planning. The closest effort is [32], which present a case study of a Norwegian hydropower plant and include a correlation coefficient between the stochastic price and inflow processes. Their findings suggest notable reductions in expected revenues. Nevertheless, we find that their treatment of the topic is incomplete, and we aim to address the gaps. First, they investigate a case study and report only case-specific findings. In this paper, we present analytical insights on including co-movements. Second, [32] do not explain or outline the intuition behind the results. This is important because we find that their case study contains some unintuitive results, as we discuss later in Sect. 5. By contrast, we explain all findings according to intuition and analytical insights. Third, the modeling of the co-movements by a correlation coefficient estimated using a fixed Pearson correlation coefficient is simplistic. Our work, on the other hand, introduces a sophisticated price process to consider the dependency between price and inflow that vary over the year. Contrary to [32], we find no evidence of a substantial impact on expected revenues.

In this work, we first characterize the impact of co-movements on the policy in a two-stage setting. From there, we develop a stochastic model with co-dynamics between price and inflow to reservoirs, so we can investigate the effect in a multistage setting. We formulate a Markov Decision process (MDP) of the medium-term hydropower planning problem and solve it using stochastic dual dynamic programming (SDDP), the state-of-the-art algorithm for seasonal hydropower scheduling [37, 39]. Thus we can analyze numerically the effect of co-movements between exogenous factors on the optimal policy.

Our work contributes to the extensive literature that studies the hydropower scheduling problem [20, 25, 28, 41]. In a broader perspective, we also contribute to research on general insights regarding correlations in stochastic optimization [7, 24]. An example of such work is [1], which uses a distributionally robust approach to analyze losses associated with ignoring correlations. Also related are studies that analyze the effect of correlation between hydro and wind [4, 9, 29, 36, 43] and between electricity prices and wind [34, 36], and between electricity demand and fuel prices [15]. Our specific contribution is to characterize the impact of co-movements on the policy of the hydropower scheduling problem in a two-stage setting.

The joint model for price and inflow is also a specific contribution to models for commodity pricing. We develop an additive three-factor stochastic model where the term structure of electricity prices is partly explained by inflow to reservoirs in the system, partly by a latent factor representing long-term dynamics, and partly by another latent factor for short-term deviations that cannot be explained by inflow to reservoirs. Our model can be interpreted as an additive discrete-time extension of the two-factor model by [40] with two latent factors and one factor that represents some physical phenomena explaining prices. Furthermore, we develop a risk-neutral version that can valuate hydropower cashflows. Our model allows for a linear and seasonal dependence structure between stochastic variables, and we present a calibration procedure for how to estimate the joint dynamics under a risk-neutral measure when futures contracts only exist for the calibration of a subset of the stochastic factors.

By applying our stochastic model to a hydropower case study, we find that at a given time and state, producers are more willing to postpone production and tolerate a higher probability of spillage if they incorporate co-movements when establishing their production policy. If a high-inflow state realizes in the next stage, a low-price state is more likely, which makes the expected revenue in the high-inflow state less when incorporating correlations. Therefore, since the reservoir capacities are limited, the producer is more willing to spill the incoming water and keep the reservoir at a higher level. Despite theoretical differences, our result suggests that the impact is modest on practical applications. Using industry data from a Norwegian hydropower plant, we find that the expected revenue decreases between 0.17% and 0.3% for different set-ups, including different parameter estimates of the stochastic model, and different reservoir and production capacities. Nevertheless, this is valuable information for reservoir operators. They can continue their current practices with more evidence backing their assumption of independent price and inflows does not incur large costs.

The rest of the paper is structured as follows. In Sect. 2, we formulate the hydropower scheduling problem as a Markov decision process. We characterize the optimal policy and present a simple example that illustrates the added value of correlations in a two-stage setting. Section 3 briefly discusses the SDDP algorithm, before we present the stochastic model with joint dynamics of prices and inflows to reservoirs in Sect. 4. Section 5 presents a case study, including calibration of the stochastic model, Markov-chain discretization, and numerical results. Concluding remarks are provided in Sect. 6.

2 Hydropower scheduling as a Markov decision process

Hydropower planning is a sequential decision problem under uncertainty. Endogenous states include the amount of water in the reservoirs, and exogenous states include price and inflow. Given the current state and an action of the producer, the state transition is independent of previous actions and states. State transitions therefore satisfy the Markov property, and the problem can be formulated as a Markov decision process (MDP). We denote $\pi$ as policy, s as endogenous states, $\omega$ as exogenous random variables, $X_t^\pi$ maps the next state when following policy $\pi$ at time t, and $s_t^{\pi }$ is the state reached at time t when following policy $\pi$. The problem can be formulated as

$$\begin{aligned} V^{\rho }_1 (s_0,\omega _1) = \max _{\pi \in \Pi } {\mathbb {E}} \left[ \sum _{t \in {\mathcal {T}}} \gamma ^t r(X_t^\pi (s^\pi _t,\omega _t), \omega _t) \mid \omega _1, s_1 \right] , \end{aligned}$$

(1)

where the expected value at $t = 1$ is the expected reward, r, from following policy $\pi \in \Pi$, where $\Pi$ is the feasible set of policies. Representation (1) uses discrete time where the planner makes decisions at every $t \in {\mathcal {T}}$. The set ${\mathcal {T}} =\{1,\dots ,T\}$ contains the stages of the MDP. The endogenous state, $s_t$, includes the amount of water in the reservoir at time t. The exogenous state vector $\omega _t = (\omega _t^O,\omega _t^C)$ consists of factors that determines immediate rewards, $\omega _t^O$, and components that impact the feasible stage t action set, $\omega _t^C$. Superscripts O and C denote objective uncertainty and constraint uncertainty, respectively. Superscript $\rho$ denotes that $\omega _t^C$ and $\omega _t^O$ are correlated. We adopt the notation $V^0_1(s_0,\omega _1)$ and $\alpha ^0_1(s_1,\omega _1)$ as the value function and continuation function assuming independent $\omega _t^C$ and $\omega _t^O$, respectively. Using Bellman’s principle of optimality, we can reformulate problem (1) recursively as a stochastic dynamic program

$$\begin{aligned} V^{\rho }_t(s_{t-1},\omega _t)&= \max _{(x_t,s_t,v_t) \in {\mathcal {X}}_t(s_{t-1},\omega ^C_{t})} r(x_t,\omega _t^O) + \gamma \alpha ^{\rho }_{t+1}(s_t,\omega _t), \end{aligned}$$

(2)

$$\begin{aligned} \alpha ^\rho _{t+1}(s_{t},\omega _t)&= {\mathbb {E}} \left[ V_{t+1}(s_{t},\omega _{t+1})\mid \omega _t \right] , \ \quad \quad \quad \quad t=1,\dots ,T, \end{aligned}$$

(3)

with $\alpha ^{\rho }_{T+1}(s_T,\omega _T) = 0$. The set ${\mathcal {X}}_t(s_{t-1},\omega ^C_{t})$ defines the solution space. The continuation, or the expected profit-to-go, function $\alpha ^{\rho }_{t+1}$ provides the expected value at time t by following the optimal policy from time $t+1$ to T. We want to investigate the effect of a negative relationship between $\omega _t^O$ and $\omega _t^C$, i.e., $\text {Cov}(\omega _t^O,\omega _t^C) < 0$, on the optimal policy of Problem (1). This correlation represents the influence that inflows, $\omega _t^C$, have on prices, $\omega _t^O$.

A producer’s reward is simply the revenue from selling produced energy, $x_t$, at price $\omega _t^O$:

$$\begin{aligned} r(x_t, \omega _t^O) = \omega _t^O x_t. \end{aligned}$$

(4)

Constraints (5a) to (5f) define the decision space. State $s_t$ is the volume of water in the reservoir and (5a) defines the transition from an incoming state $s_{t-1}$ to an outgoing state $s_t$. It depends on production, $x_t$, spillage, $v_t$, and inflow, $\omega _t^C$. There is no penalty for spillage except the opportunity cost of lost sales. Restriction (5b) ensures that the state is within the bounds of the reservoir, where R denotes the maximum capacity. Production is either limited by the reservoir contents, (5c), or the maximum production capacity, (5d), where G denotes maximum production capacity. Finally, (5e) and (5f) ensure that production and spillage cannot be negative, respectively.

$$\begin{aligned} {\mathcal {X}}_t(s_{t-1},\omega ^C_{t})=\{ \hspace{1cm}{} & {} \nonumber \\&s_{t}+x_t+v_t = s_{t-1} + \omega _{t}^C \ {}&(z_1) \end{aligned}$$

(5a)

$$\begin{aligned}&-x_t-v_t \le R-s_{t-1}-\omega _t^C \ {}&(z_2) \end{aligned}$$

(5b)

$$\begin{aligned}&x_t \le s_{t-1}+\omega ^C_t \ {}&(z_3) \end{aligned}$$

(5c)

$$\begin{aligned}&x_t \le G \ {}&(z_4) \end{aligned}$$

(5d)

$$\begin{aligned}&x_t \ge 0 \ {}&(z_5) \end{aligned}$$

(5e)

$$\begin{aligned}&v_t \ge 0 \ {}&(z_6) \nonumber \\ \}. \end{aligned}$$

(5f)

Proposition 2.1 states the properties of the value function $V^\rho _t$ and continuation function $\alpha ^{\rho }_t$. These properties make it possible to characterize the optimal policy for the hydropower scheduling problem defined by (2) to (3) in Proposition 2.2.

Proposition 2.1

For a given $(t,\omega ^O_t,\omega _t^C) \in {\mathcal {T}} \times {\mathbb {R}} \times {\mathbb {R}}_+$, the value function $V^{\rho }_t(\cdot ,\omega _t)$ and continuation function $\alpha ^{\rho }_{t+1}(\cdot ,\omega _{t})$ are concave, and the marginal water value, $\frac{\partial {\alpha ^{\rho }_{t+1}(\cdot ,\omega _t)}}{\partial {s_{t}}}$, is decreasing in the endogenous state $s_t$.

Proof

See A.1. $\square$

Proposition 2.2

The optimal generation policy of Problem 2–3 is given by

$$\begin{aligned} x^*_t(s_{t-1},\omega _t)&= {\left\{ \begin{array}{ll} 0 &{} \text {if } \omega ^O_t < \frac{\partial {\alpha ^\rho _{t+1}(\cdot ,\omega _t)}}{\partial {s_{t}}}\big \vert _{s_{t-1} + \omega _t^C} \\ \min \{G,K,s_{t-1}+\omega _t^C\} &{} \text {if } \omega ^O_t > \frac{\partial {\alpha ^\rho _{t+1}(\cdot ,\omega _t)}}{\partial {s_t}}\big \vert _{s_{t-1}+\omega _t^C} \end{array}\right. } \end{aligned}$$

(6)

where

$$\begin{aligned} K = s_{t-1} + \omega _t^C - \mathop {{{\,\mathrm{\text {arg min}}\,}}}\limits _{s_t} \left| \omega _t^O - \frac{\partial {\alpha ^{\rho }_{t+1}(\cdot ,\omega _t)}}{\partial {s_t}} \Big \vert _{s_t} \right| \end{aligned}$$

Proof

See A.2. $\square$

2.1 Covariance in two-stage hydropower scheduling

To study the effect of covariance on the optimal MDP policy, we consider reservoir management in two stages where the producer must determine whether to produce now or later. In other words, the producer faces problem (2) to (3) with $T=2$. Proposition 2.3 states that a decision-maker underestimates the marginal water value if it ignores the negative correlation between objective uncertainty (price) and constraint uncertainty (inflow).

Proposition 2.3

If $T=2$, $G=R \in (0,\infty )$, $\text {Cov}(\omega _2^O,\omega _2^C) < 0$, and $s_0+\omega _1^C<R$, then $\frac{\partial \alpha _2^{\rho }(\cdot ,\omega _{1})}{\partial s_1}\big \vert _{s_0+\omega _1^C} > \frac{\partial \alpha _2^{0}(\cdot ,\omega _{1})}{\partial s_1}\big \vert _{s_0+\omega _1^C}$.

Proof

See A.3. $\square$

Proposition 2.2 combined with Proposition 2.3 implies $x_{1}^{\rho *} \le x_{1}^{0 *}$ and $v_{2}^{\rho *} \ge v_{2}^{0 *}$ in a two-stage setting. A producer will therefore produce the same or less today at optimum if it considers correlations. Hence, correlations provide an incentive to postpone production. If a high inflow realizes, prices tend to be low, and the incoming water is not as valuable compared to an independent stochastic model. This is because the reservoir capacity is limited. Example 2.1 illustrates the phenomenon.

Example 2.1

Added value of including correlations under policy computation

This example demonstrates the impact of ignoring correlations on the optimal production schedule and profit losses. We consider problem (2) to (3) when $T=2$ and maximum production drains the whole reservoir, $G=R$. Let the process for the stochastic variables be

$$\begin{aligned} \omega ^O_t&= \chi _t + \mu ^O \end{aligned}$$

(7)

$$\begin{aligned} \omega ^C_t&= \nu _t + \mu ^C \end{aligned}$$

(8)

$$\begin{aligned} \chi _t&= \phi ^O \chi _{t-1} + \sigma ^O \epsilon ^O_{t} \end{aligned}$$

(9)

$$\begin{aligned} \nu _t&= \phi ^C \nu _{t-1} + \sigma ^C\epsilon ^C_{t} \end{aligned}$$

(10)

where

$$\begin{aligned} (\epsilon _t^O,\epsilon _t^C)&\sim {\mathcal {N}} \begin{bmatrix} \begin{pmatrix} 0\\ 0\\ \end{pmatrix},&{} \begin{pmatrix} 1 &{} \rho \\ \rho &{} 1\\ \end{pmatrix} \end{bmatrix} \end{aligned}$$

with numerical values and nomenclature outlined in Table 1.

Table 1 Numerical values and nomenclature for Example 2.1

Figure 1 displays the value of the continuation function and its derivative over different states. We observe that at maximum reservoir, the continuation function must be the same, since any potential new inflow cannot be stored. At low reservoirs, we observe that the continuation value differs. The independent policy sees higher value in the future, since negative correlation is ignored. When there is zero spillage risk, the difference in value is equal to the covariance between price and inflow. This is illustrated in the lower right window in Fig. 1, which shows the value at level 60 where spillage risk is close to zero. As a result, the slope of the curve based on the independent policy must decay at a lower rate. This is illustrated in Fig. 1b, which shows the marginal water value. Hence, the first stage generation decision differs with and without correlation.

From Proposition 2.2, when $G> s_{t}+\omega _t^C > K$, the optimal first stage generation decision is

$$\begin{aligned} K = s_{t-1}+\omega _t^C-\mathop {{{\,\mathrm{\text {arg min}}\,}}}\limits _{s_t} \left| \omega _t^O-\frac{\partial {\alpha ^{\rho }_{t+1}(\cdot ,\omega _t)}}{\partial {s_t}}\big \vert _{s_t}\right| . \end{aligned}$$

(11)

Table 2 outlines the optimal solution with zero and $-$0.5 correlation. The second row shows the spill probability in the last stage at optimum. An optimal policy based on the joint distribution with correlation $-$0.5 delays some production and tolerates a higher spill probability compared to the policy based on prices and inflow being independent. Table 3 show the expected revenue, where the upper left entry is the optimal policy value of the independent model relative to the optimal value of the joint model with $\rho =-0.5$. An independent policy overestimates the expected revenue by $1.3\%$. Simulating the policy on the joint model with $\rho =-0.5$ leads to an expected loss of $0.03\%$, compared to the optimal policy for the joint model, which is to produce less in the first stage.

Fig. 1

Continuation value $\alpha ^{i}_2(\cdot ,\omega _1)$ and derivative of the continuation value $\frac{\partial {\alpha ^{i}_{2}(\cdot ,\omega _t)}}{\partial {s_t}}$ as a function of reservoir volume $s_t$, with (black) and without (red) correlation, $i\in \{0,\rho \}$. We observe the continuation value with and without correlations as parallel lines with an offset of $Cov(w^O,w^C)$ when the reservoir is low. The two values get closer as the reservoir level increases. The marginal water value is higher when co-movements are accounted for. This is valid for high reservoir levels, when there is spillage risk. It leads, in principle, to delayed production releases for high reservoir levels (color figure online)

Table 2 Optimal policies for Example 2.1

Table 3 Difference in expected revenue of policies relative to $V^{\rho }_1(s_0,\omega _1)$

The state space provides a clearer comprehension of the situations where correlations matter. Figure 2a shows the difference in policy performance as a function of the current reservoir volume and the current price. The policies are the same for a current price above 30. This is because ${\mathbb {E}}(\omega ^2_2\vert \omega ^O_1>30)<\omega ^O_1$, and it is optimal to produce at maximum, which means that correlations are irrelevant. Correlations matter when $\omega _1^O < 30$ and $P(v^{*,i}_2> 0) > 0$. Here, the current water has a higher value in the future if we consider correlations. It is therefore optimal to produce less today. If the spillage probability is sufficiently small in the second stage, i.e., the reservoir level is low, and $\omega _1^O < 30$, it is optimal to store water because ${\mathbb {E}}(\omega ^O_2\vert \omega ^O_1<30)>\omega ^O_1$ according to the price model. In this case, the first-stage decision is approximately equivalent with and without correlation, as seen in Fig. 2a. We emphasize that the figures are based on simulations and hence there is zero spillage probability in this region. Although, in theory, there will be a spillage probability since the inflow domain is all real numbers, and the correlated policy will produce slightly less also in this region of the state space.

Fig. 2

Comparing policies assuming $\rho =0$ and $\rho =-0.5$. Upper left is difference in first-stage decision. Upper right is difference in policy values when evaluated on the joint distribution. The lower panel compares spill probabilities

Figure 2 outlines the policies, where Fig. 2b displays the potential for additional expected revenue from including correlations under policy computation. For example, if the price is around 15 EUR/MWh and the reservoir volume at 75 MWh, the policy with correlation gets around 0.06% higher expected revenue. Figures 2c to 2d compare spillage in the second stage with and without correlation. A policy with correlation has a higher spill probability in the region where the correlated policy delays production.

3 Solution approach

Unlike the two-stage application in Example 2.1, hydropower producers may have reservoirs that can hold water months or years into the future. Medium-term hydropower operations, for instance, plan 1–2 years into the future with time discretized into weeks [20]. This becomes a multistage stochastic problem. The hydropower scheduling problem (2) to (3) suffers from the curses of dimensionality. A particular challenge is that the state and decision variables are continuous and consequently provide infinite combinations. Although we can discretize them, the problem becomes computationally intractable before the granularity becomes representative for an actual hydropower application. Instead, the industry standard is to solve the problem using SDDP, introduced by [37], which approximates the continuation function by a piecewise linear function defined by Benders cuts. The MDP can therefore have continuous states and actions. SDDP creates a valid outer approximation of the expected future value function under certain assumptions [13]:

• A1. The value function $V_t(\cdot ,\omega _t)$ is concave (for maximization) with respect to $s_t$ for all stages $t \in {\mathcal {T}}$ for a fixed $\omega _t$.

• A2. The uncertainty sample space $\Omega _t$ is finite for all stages $t \in {\mathcal {T}}$.

• A3. The feasible region, ${\mathcal {X}}_t(\cdot , \omega _t)$, is non-empty and an optimal solution exists for every obtainable incoming state variable $s_{t-1}$ and sample $\omega _t$ for all stages $t \in {\mathcal {T}}$.

• A4. Realizations of uncertainty are independent of other stages, i.e., ${\mathbb {P}}(\omega _t \in \Omega _t |\omega _{t-1}) = {\mathbb {P}}(\omega _t \in \Omega _t)$, for all stages $t \in {\mathcal {T}}$. This is commonly referred to as stage-wise independence in the SDDP literature.

Proposition 2.1 proves that the value function $V_t(\cdot ,\omega _t)$ is concave, and hence it satisfies A1. We ensure that our scenario generation procedure makes a finite sample space for all stages and hence satisfies A2. Moreover, we select parameters so the feasible region satisfies assumption A3. Our application is a liberalized electricity market where prices are stage-wise dependent random variables, which violate assumption A4. SDDP can circumvent this issue by either including a state variable that follows an autoregressive time series (TS-SDDP) or discretize the random data process to a Markov chain (MC-SDDP) [25]. Because the objective multiplies price with a decision variable, making price a state variable, as required by TS-SDDP, introduces a bilinear term in the objective that violates assumption A1. A hybrid approach, which acts as a wrapper around the SDDP algorithm, can ensure a concave value function [20]. There are also extensions of the algorithm that can handle the bilinear term in the objective where the value function is a saddle function [11]. The main drawback of MC-SDDP is that a fixed uncertainty partition is required, which may lead to bad performance on some scenarios. However, the MC-SDDP implementation is less complex, it converges faster, and it has been shown to provide policies that perform well when applied to hydropower planning [25]. Therefore, we argue that MC-SDDP is suitable for analyzing the effect of correlation between price and inflow on hydropower operations policies.

Because the continuation function is concave, SDDP replaces it with the variable $\theta _t$, defined by the hyperplanes or cuts

$$\begin{aligned} \theta _t \ge \alpha _t^k + \beta _t^k s_t. \end{aligned}$$

(12)

Parameters $\alpha _t^k$ and $\beta _t^k$ are the intercept and slope of the hyperplanes, where k indexes iterations. SDDP solves (2) to (3) multiple times in an iterative fashion, where each iteration generates a cut, i.e., parameters $\alpha _t^k$ and $\beta _t^k$. Let K denote the current iteration. As K increases, the outer approximation of the continuation function receives more hyperplanes which improves the approximation of the concave continuation function and has convergence guarantees [19, 39]. Problem (2) to (3) is a subproblem that exists for each stage $t \in {\mathcal {T}}$. The iterative scheme to solve the multistage stochastic problem by SDDP includes a forward simulation and a backward pass. The former starts at $t=1$ with a pre-determined initial state variable. It samples $\omega _t$, solves (2) to (3), and saves the resulting state variable, ${\hat{s}}_{t}$. The algorithm proceeds to the next stage, $t+1$, samples $\omega _{t+1}$, and solves (2) to (3) again. The process continues until the terminal stage, T, where the algorithm has a list of simulated values of ${\hat{s}}_t$ for all $t \in {\mathcal {T}}$. The backward pass starts at $t = T - 1$ and moves backwards through the list. At stage t we solve (2) to (3) for all $\omega _{t+1}$ with fixed ${\hat{s}}_t$ and obtain objective value ${\hat{V}}_{t+1, \omega _{t+1}}$ and subgradient ${\hat{\lambda }}_{t+1, \omega _{t+1}}$ of the objective function with respect to $s_t$. Cut parameters at iteration K is then calculated according to (13) and (14). These cuts are included in the next forward simulation at iteration $K+1$ and provide a more detailed approximation of the continuation function.

$$\begin{aligned} \beta _i^{K+1}&= {\mathbb {E}}[{\hat{\lambda }}_{t+1, \omega _{t+1}}^K] \end{aligned}$$

(13)

$$\begin{aligned} \alpha _i^{K+1}&= {\mathbb {E}}[{\hat{V}}_{t+1, \omega _{t+1}}^K] - \beta _i^{K+1}{\hat{s}}_t \end{aligned}$$

(14)

Note that we use average cuts, which explains the expectation operator in (13) and (14). Multi-cut options, which include all the cuts, i.e., removes the expectation operator in (13) and (14), are also possible but increase the subproblem’s size. The SDDP algorithm terminates when new cuts add negligible reductions to the estimated upper bound, reaches an iteration limit, or satisfies a statistical bounds criterion. When SDDP terminates, it has approximated the continuation function for each stage $t \in {\mathcal {T}}$. The policy is simply the arguments that maximize (2) at stage t with the continuation function approximation. Because the contribution of our work is in the price process we use, and not the SDDP method itself, we refer to for instance [37, 13], or [25] for more information on SDDP and pseudocode.

4 Stochastic modelling

This section presents a model of spot prices with connection to the hydrological system state and local inflow. Local inflow often possesses strong seasonal variations. Therefore, similar to [20], we normalize inflow to reservoirs and impose an autoregressive model of order one, AR-1, to the seasonality-adjusted inflow series. This model is often used by practitioners. However, inflow generally has positive asymmetry, which motivates the use of skewed distributions, e.g. the lognormal distribution [42]. A comparison of the model fit for our Gaussian inflow model and its lognormal counterpart is provided in Appendix B. Motivated by the empirical fit, we specify a Gaussian model for inflow, but emphasize that our joint model allows for general inflow models. We let ${\bar{\mu }}_t$ denote the historical average of local inflow and ${\bar{\sigma }}_t$ the historical standard deviation of local inflow at week t, while $\nu _t$ denotes the residuals of normalized inflows. Local inflow is still $\omega ^C_t$, and it is given by

$$\begin{aligned} \omega ^C_t&= {\bar{\mu }}_t+{\bar{\sigma }}_t \nu _t \end{aligned}$$

(15)

$$\begin{aligned} \nu _t&= \phi _9 \nu _{t-1} + \sigma _4 \epsilon _{4,t}, \end{aligned}$$

(16)

where $\phi _9$ is a measure for inflow deviation autocorrelation, and $\epsilon _{4,t}$ is normally distributed with zero mean and unit variance. The local hydrological state, $h_t^{loc}$, indicates the degree to which the plant has more or less resources available than normal. This captures whether the plant is located in an area with drought or excess of water recently. We define the local hydrological state as the exponentially smoothed inflow deviations from the mean:

$$\begin{aligned} h^{loc}_t&= \phi _8 h^{loc}_{t-1} + (1-\phi _8) {\bar{\sigma }}_t \nu _t, \end{aligned}$$

(17)

where $\phi _8$ is the smoothing coefficient. The lower $\phi _8$ the higher weight is given to recent inflow deviations. A dry period for some time locally often indicates a dry period in the system, and vice versa. We propose a linear regression model with an AR-1 error structure for the system hydrological state, $h_t^{sys}$. The relevant local hydrological state at the producer’s location, $h_t^{loc}$, is an explanatory variable for the hydrological state in a larger geographical area, $h_t^{sys}$. Our motivation for this model is the fact that the system hydrological state is an aggregation of local hydrological states. An unobservable factor with AR-1 structure, $\eta _t$, captures the rest of the system hydrological state:

$$\begin{aligned} h^{sys}_t&= \phi _6 h^{loc}_{t} + \eta _t \end{aligned}$$

(18)

$$\begin{aligned} \eta _t&= \phi _7 \eta _{t-1} + \sigma _3 \epsilon _{3,t}. \end{aligned}$$

(19)

The parameter $\phi _6$ measures the effect of local hydrological state on the system state, $\phi _7$ measures the autocorrelation of system state deviations that are not explained by the local hydro system that is being modelled, and $\epsilon _{3,t}$ is normally distributed with zero mean and unit variance. The system hydrological state enters the spot price model as an independent factor. In other respects, the price model is equivalent to an additive and discrete time version of the two-factor model in [40]:

$$\begin{aligned} \omega ^O_t&= \phi _1 \cos \left( (t+\phi _2)\frac{2\pi }{52}\right) + \phi _3 h_t^{sys} + \chi _t + \xi _t \end{aligned}$$

(20)

$$\begin{aligned} \chi _t&= \phi _4 \chi _{t-1} + \sigma _1 \epsilon _{1,t} \end{aligned}$$

(21)

$$\begin{aligned} \xi _t&= \beta + \xi _{t-1} + \sigma _2 \epsilon _{2,t}. \end{aligned}$$

(22)

The first term captures price seasonality. The parameter $\phi _1$ determines the amplitude of seasonal variations, and $\phi _2$ determines when the peak is. The next term is the hydrological system state, $h_t^{sys}$, which captures the effect of resource availability on price, determined by the parameter $\phi _3$. Note that the hydrological system state is again explained by another factor, namely, local inflow deviation from the local mean, and influences $h_t^{sys}$ through (15)–(17). The equilibrium price level, $\xi _t$, captures long-term price behavior, and evolves as a discrete-time arithmetic Brownian motion (ABM) with drift $\beta$. Short-term deviations that are not explained by the hydrological state, $\chi _t$, evolves as an AR-1 process, where $\phi _4$ measures the autocorrelation in short-term price deviations that are not explained by inflow deviations. The perameters $\sigma _1$ and $\sigma _2$ is the standard deviation of short-term price deviations and the equilibrium price, respectively, and $\epsilon _1$ and $\epsilon _2$ are normally distributed with zero mean and unit variance. Overall, our model consists of four stochastic factors $(\nu _t,\eta _t,\chi _t,\xi _t)$. Note that local hydrology $h_t^{loc}$ is derived deterministically from $\nu _t$, and system hydrology $h_t^{sys}$ is derived deterministically from $h_t^{loc}$ and $\eta _t$.

4.1 Risk-neutral process

We develop a risk-neutral version of the stochastic model in the previous section. Futures contracts can then be used to calibrate the model. Given a risk-neutral price process, cashflows can be discounted at the risk-free rate [14]. The stochastic model in (15)–(22) are additive Gaussian. The transformation to an equivalent martingale measure for this type of underlying processes can be done by making assumptions regarding the risk premium [27] or convenience yield [2]. We follow the latter and obtain the following risk-neutral version:

$$\begin{aligned} \omega _t^O&= \phi _1 \cos \left( (t+\phi _2)\frac{2\pi }{52}\right) + \phi _3 h^{sys}_t + \chi _t + \xi _t \end{aligned}$$

(23)

$$\begin{aligned} \omega _t^C&= {\bar{\mu }}_t+{\bar{\sigma }}_t \nu _t \end{aligned}$$

(24)

$$\begin{aligned} \chi _t&= \phi _4^* \chi _{t-1} + \sigma _1 \epsilon ^*_{1,t} \end{aligned}$$

(25)

$$\begin{aligned} \xi _t&= \phi _5^* \xi _{t-1} + \sigma _2 \epsilon ^*_{2,t} \end{aligned}$$

(26)

$$\begin{aligned} h^{sys}_t&= \phi _6 h^{loc}_{t} + \eta _t \end{aligned}$$

(27)

$$\begin{aligned} \eta _t&= \phi ^*_7 \eta _{t-1} + \sigma _3 \epsilon ^*_{3,t} \end{aligned}$$

(28)

$$\begin{aligned} h^{loc}_t&= \phi _8 h^{loc}_{t-1} + (1-\phi _8)\left( \omega ^C_t-{\bar{\mu }}_t \right) \end{aligned}$$

(29)

$$\begin{aligned} \nu _t&= \phi ^*_9 \nu _{t-1} + \sigma _4 \epsilon ^*_{4,t} \end{aligned}$$

(30)

$$\begin{aligned} \epsilon ^*_{i,t}&\sim {\mathcal {N}}(0,\Sigma ) \end{aligned}$$

(31)

Here, $\epsilon _{i,t}^*$ has mean zero under the equivalent martingale measure. See Appendix C for derivations. In the next section, we present how to calibrate this model using forward and hydrological data.

5 Medium-term hydropower scheduling: a case study

In this section, we present a case study of a hydropower plant located in the western part of Norway. Its operator provided inflow data and plant characteristics. [31] provides system price data, while we retrieved data for system resource availability, in terms of reservoir content, from [33]. NVE represents the resource availability data series as a deviation from an estimated normal situation at any time of the year. We first present a description of the interconnected system and the aggregation into an energy equivalent reservoir (EER). Then, we present the calibration procedure. Finally, we present the numerical results.

5.1 Energy equivalent reservoir and head variations

The watercourse we study consists of multiple interconnected reservoirs. We aggregate the reservoirs into one energy equivalent reservoir (EER). The composite representation of the interconnected multireservoir system was originally proposed by [3], and has been applied in practice [16, 28]. Note that SDDP efficiently handles the situation of multiple interconnected reservoirs, but since our focus is to study the effect of covariance on the optimal MDP policy, we instead study an EER for comparison and consistency between our analytical two-stage results and numerical multi-stage results. The aggregation simplifies the modeling of uncertainties and allows us to study general insights of the effect of covariance on the optimal MDP policy. For the same reasons, we assume a constant reservoir-specific volume-energy transformation coefficient. In theory, this transformation depends on the discharge $x_t$, where the amount of discharge influences the energy efficiency. Piece-wise linear efficiency curves can approximate this behavior. Head variations, which depend on $s_t$, also influence the volume-energy transformation but require non-convex subproblems.

When aggregating reservoirs, the problem size is small enough to be solved with a standard stochastic dynamic progamming approach (SDP). However, since SDDP is state-of-the art in hydropower planning, and the model easily can be extended to a multi-reservoir setting, we demonstrate our model using SDDP. The multi-reservoir setting could potentially lead to computational difficulties, as several inflow processes must be discretized. A way to avoid this is to reduce the number of inflow factors, which often is reasonable in a limited geographical area where inflow often is highly correlated.

5.2 Calibration of the stochastic model

We use electricity futures data with time to maturity 1–5 years from 2011 to 2018, local inflow data from 2009 to 2018, and data for the system hydrological state from the same period when calibrating our model. Pricing models are commonly calibrated using Kalman filtering and maximum likelihood estimation, where observed futures prices are assessed against model predictions. Joint calibration is unattainable because future contracts for inflow do not exist. We therefore apply a two-step process. First, we employ maximum likelihood estimation with historical data for inflow and system hydrological state. We then use (18) and (19) to predict future hydrological states and extract the effect on futures prices. Afterwards we apply standard Kalman filtering and maximum likelihood estimation. We must calibrate both the joint model and a benchmark model where $\omega _t^O$ and $h_t^{sys}$ are independent. D outlines how we calibrate the benchmark model, while the following six steps explain our estimation approach for the joint model in detail.

Step 1: Computing smooth electricity forward curves and de-seasonalized prices. We decompose the forward curve into a seasonal component and an adjustment function that accounts for deviations from seasonality. The seasonal component has the same form as in the pricing model from the previous section, while the adjustment function is defined based on arbitrage arguments and the maximum smoothness criterion. See [5, 10, 18] for procedures for computing smooth synthetic forward curves. We estimate coefficients of the seasonal component by regression using front month forward contracts. Figure 3 illustrates examples of smooth synthetic forward curves computed using this procedure. Figure 4a displays synthetic forward curves for the entire electricity forward contract data set. After having estimated the seasonality in step 1, we de-seasonalize forward prices and continue the estimation procedure with forward prices adjusted for seasonality. Figure 4b outlines the de-seasonalized forward curves.

Fig. 3

Examples of smoothed futures curves by applying the method by Benth et al. [5]. Futures prices are obtained from [31]. The upper was observed on 3 January 2011 and the lower was observed on 27 May 2015

Fig. 4

Smoothed futures curves for the entire data set

Step 2: Calibration of the local inflow model. We first normalize weekly inflow data based on weekly mean and standard deviation, similarly as in Gjelsvik et al. (2010) [20], before we fit an AR-1 process to account for serial correlations. Figure 5a displays the local inflow. We assume that inflow risk is unrelated to the world economy and therefore diversifiable, commanding no risk premium.

Step 3: Constructing a time series of local hydrology. We use the exponentially weighted inflow series as the hydrological state

$$\begin{aligned} \begin{aligned} h^{loc}_t&= \phi _8 (\omega _t^C-{\bar{\mu }}_t) + (1-\phi _8)h^{loc}_{t-1} \\&= \phi _8 (\omega _t^C-{\bar{\mu }}_t) + \phi _8(1-\phi _8) (\omega _{t-1}^C-{\bar{\mu }}_{t-1}) + (1-\phi _8)^2 h^{loc}_{t-2} \\&= \phi _8(\omega _t^C-{\bar{\mu }}_t) + \phi _8(1-\phi _8) (\omega _{t-1}^C-{\bar{\mu }}_{t-1}) \\ {}&\quad + \phi _8(1-\phi _8)^2 (\omega _{t-2}^C-{\bar{\mu }}_{t-2}) + (1-\phi _8)^3 h^{loc}_{t-3} \\&\vdots \\&= \phi _8 \sum _{k=0}^K (1-\phi _8)^{k} (\omega _{t-k}^C-{\bar{\mu }}_{t-k}) + (1-\phi _8)^K h^{loc}_{t-K}, \end{aligned} \end{aligned}$$

(32)

where K denotes the number of time periods. This time series captures the situation of a local producer having either less or more resources available than normal, depending on how inflow has deviated from its mean for a longer period. This may partly explain water supply in the system. Figure 5b displays the time series for the local hydrology in red together with system hydrology in black.

Fig. 5

Hydrological data from 2009 to 2018

Step 4: Calibration of the local and system hydrology model. The system hydrology model is a linear regression model with an AR-1 error structure. It uses local hydrology as an explanatory variable, which again is driven by local inflow. We consider parameters of the inflow model fixed, and jointly estimate parameters of the local and system hydrology model. We enforce $\phi _8=\phi _7$, which ensures that the local hydrology mean reverts at the same speed as the AR-1 process. We consider this reasonable because system hydrology is an aggregation all reservoirs in the system. This restriction is necessary to get reasonable hydrology predictions in the next step. When estimating the parameters, we iterate over $\phi _6$ and $\phi _8$, or equivalently $\phi _7$, and study the likelihood surface of the linear regression model with AR-1 structure in (18) and (19). Figure 6 displays the likelihood surface of $\phi _8=\phi _7$ and $\phi _6$ with a mark at optimum.

Fig. 6

Likelihood surface for estimation of $\phi _8=\phi _7$ and $\phi _6$

Step 5: Estimating the effect of hydrology on prices and using the inflow-hydrology model to predict and extract the impact on futures prices. Given the model parameters $\theta = (\phi _1,\phi _2,\phi _3,\phi ^*_4,\phi _5^*,\phi _6,\phi ^*_7,\phi _8,\phi ^*_9)$, futures prices at any given time t with maturity $\tau$ can be expressed by our model as follows:

$$\begin{aligned} {\hat{F}}_{t,\tau }(\chi _t,\xi _t,h_t^{loc},\eta _t,\nu _t;\theta )&= {\mathbb {E}}^*(\omega ^O_{\tau }) \end{aligned}$$

(33)

$$\begin{aligned}&= \phi _1 \cos \left( (t+\phi _2)\frac{2\pi }{52}\right) + \phi _3 {\hat{h}}_{t,\tau }^{sys} + \phi _4^{* \ \tau -t}\chi _t + \phi _5^{* \ \tau -t}\xi _t \end{aligned}$$

(34)

where

$$\begin{aligned} {\hat{h}}_{t,\tau }^{sys} = \phi _6(\phi _8^{\tau -t}+h_t^{loc}+\phi _9^{* \ \tau -t}(1-\phi _8){\bar{\sigma }}_{\tau }\nu _t)+\phi _7^{* \ \tau -t}\eta _t. \end{aligned}$$

(35)

We estimate $\phi _3$ by regressing front month futures contracts on $h_t^{sys}$. Afterwards, we adjust the effect of hydrology by using model predictions and create a new data set of smooth synthetic forward curves from step 1, adjusted for hydrology. Figure 7a displays the hydrology predictions and Fig. 7b shows hydrology adjusted futures contracts, $F_{t,\tau }-\phi _3{\hat{h}}_{t,\tau }^{sys}$. By comparing Figs. 4b and 7b, we observe less variability in futures contracts after accounting for hydrology. Forward curves adjusted for hydrology are then used in the next step.

Fig. 7

Hydrology prediction and prices adjusted for hydrology

Step 6: Calibrating the long- and short-term factors to futures residuals. In the final step, we run the Kalman filter together with maximum likelihood under the restriction of zero correlation between $h_t^{sys}$ and $\chi _t$. This is motivated by the fact that the price component that explains hydrology cannot be correlated by the price component explaining everything but hydrology. See [40] and [21] for more information on the calibration of geometric models of latent factors $\chi _t$ and $\xi _t$. We modify the likelihood function to calibrate our additive model. Appendix E provides nomenclature and parameter estimates.

5.3 Numerical results

We implement the Markov chain SDDP in Julia [6] using the SDDP package [12] and the Gurobi solver. We compute policies on the dependent and independent Markov chain based on data from a Norwegian hydropower producer. The procedure for estimating the parameters of the stochastic process is explained in Sect. 5.2, and parameter estimates are provided in Table 5 (Appendix E). When discretizing the stochastic processes we focus on preserving the mean and variance of prices and inflow, and the cross-moment. The discretization procedure is described in Appendix F. We use a 2% annual discount rate, reflecting the risk-free rate of return. Revenues are from 2 years of operations to ensure both policies have the same amount of inflow to allocate. However, policies in time periods close to the terminal stage experience end of horizon effects because they want to efficiently use all resources and thus drain the reservoir. This does not happen in practice, so the policy insights we present are only from the first year of operation where end-of-horizon effects have negligible impact.

Figure 8 displays the reservoir trajectories under optimal policies. While both policies drain their reservoirs to capitalize on high winter prices, a period with traditionally high demand and prices, their decisions start to diverge in the late spring. We observe that the reservoir mean is slightly higher during the autumn (from around week 25 to 40) for the policy computed on the dependent Markov chain. Prices usually increase in this period because we move toward higher winter prices. If prices correlate with inflows to the reservoir, the producer favors a slightly higher mean reservoir if it accounts for a negative correlation. This is because if a high-inflow state occurs in the next stage, the price tends to be low. From a current viewpoint, since reservoirs are limited, this makes current inflow less valuable than if prices and inflows were uncorrelated. As a result, the producer tolerates a higher amount of spillage in this period, as shown in Fig. 9. Figure 8 also shows that considering correlations reduces the reservoir trajectory variability. We observe from the 10th percentile that a policy based on the dependent Markov chain has fewer low-reservoir trajectories during the autumn, compared to the independent Markov chain. This is the period where the covariance is highest according to our estimated model parameters and where prices are expected to increase. The policies differ less in the other parts of the year. Figure 10, which displays the production, reinforces the observations of the reservoir trajectories. We observe that the producer generates less on average until around week 30.

Fig. 8

Mean and 10th to 90th percentiles of reservoir trajectories for independent and dependent Markov chains

Fig. 9

Mean of spillage for independent and dependent Markov chains relative to weekly inflow volatility

Fig. 10

Mean and 10th to 90th percentiles of generation decisions for independent and dependent Markov chains

Table 4 outlines the expected revenues for 2 years. The first column states which Markov chain is used when computing the policy, and the two rightmost columns state which Markov chain the policy is evaluated on. The numbers are percentage differences relative to the value of the optimal policy of the dependent Markov chain, $V_1^{\rho }(s_0,\omega _1)$. We observe that the optimal value of the independent Markov chain policy overestimates the value by 2.55%. However, when simulating the policy from the independent Markov chain, we find approximately 0.17% loss incurred by ignoring correlations under policy computation. The policy computed on the independent Markov chain has a slight disadvantage because the simulations are from the dependent Markov chain, which may have a slight discrepancy in the values of the independent Markov chain’s nodes despite similar first and second moments. Therefore, our results indicate that although there is a strong relationship between inflows to reservoirs and prices, the producer does not necessarily gain much by incorporating co-movements when establishing the seasonal operational policy.

Table 4 Difference in expected revenue of policies relative to $V^{\rho }_1(s_0,\omega _1)$

Figures 11a, b show scenarios for the 100 lower and 100 upper revenues, respectively, out of 1000 scenarios. The black line is optimized revenue for the dependent Markov chain, and the red dashed line is revenue that the policy based on the independence assumption attains on the dependent Markov chain. The black solid line in Fig. 11a, b lies slightly above the dashed red line, which illustrates that the producer can slightly increase revenue in both low and high scenarios by accounting for co-movements between prices and inflows when establishing the operational policy. Moreover, we observe from Fig. 11a, b that the blue dotted line, which is optimized revenue for the independent Markov chain, mostly lies below the black line in the lower 100 scenarios and above the black line in the upper 100 scenarios. An explanation for this is that the negative correlation between prices and inflow creates a natural hedge in the lower 100 scenarios. Revenues are therefore underestimated if correlations are ignored. In the upper 100 scenarios, however, revenues are overestimated because both high price and high inflow realizations are less likely to happen if co-movements are accounted for.

Fig. 11

Revenues in upper and lower 100 scenarios out of 1000

Figure 12 displays a scatter plot of the revenues from 1000 simulations obtained when the policy is optimized based on the independence assumption, but evaluated on the dependent Markov chain. The black line illustrates the one-to-one mapping of optimized revenue for the dependent Markov chain. We observe that scenarios with low revenue obtained by the independent policy correspond to low revenue scenarios when following the dependent policy. This further accentuates that the producer cannot gain much by accounting for the negative relationship between prices and inflows to reservoirs when establishing the operational policy.

Fig. 12

Scatter plot of revenues. Red points show revenues when assuming zero co-movements when both optimizing and evaluating the policy. Red points show revenues when assuming zero co-movements when optimizing and evaluated on the dependent Markov chain (color figure online)

To test how sensitive our numerical results are to parameter estimates of the stochastic process and plant characteristics, we conduct two additional numerical experiments. The analytical results from Sect. 2.1 find that a stronger degree of co-movements between prices and inflows to reservoirs can increase the opportunity value from accounting for these co-movements when computing the policy. Moreover, a smaller reservoir capacity relative to inflow measures and more production capacity could also enhance revenue, as the opportunity value stems from spillage risk. Results from such case studies are provided in Appendix G. A summary of the results is that reservoir trajectories and policy values are highly case-specific, as expected. In line with intuition, both experiments find that the expected revenue decreases. Nonetheless, the reductions are a modest 0.24% for stronger degree of co-movements and 0.3% for smaller reservoir capacity and increased production capacity. The revenue variances have a similar behavior as the base case. The moderate changes reinforce the evidence for just modest relative gains from considering co-movements in price and inflow when establishing an operational policy.

Our findings diverge from [32], who report 2.5% and 3.1% reduced expected revenue on a case study of another Norwegian hydropower plant. We apply our model to different case studies, and we have different assumptions regarding the evolution of uncertainties and co-movements, which may explain the differences in results. We also note that there is no clear relationship between reservoir trajectories and correlation coefficient in their results. This is different from our findings, which indicate that the producer tolerates slightly more spillage if correlations are taken into account and does therefore prefer a slightly higher average reservoir trajectory.

6 Conclusions

Hydro-dominated systems in liberalized electricity markets have a clear relationship between prices and inflows to the system’s reservoirs. Full reservoirs mean abundant supply and even risk of spillage, which depress prices. Low reservoirs, on the other hand, may indicate a drought and thus high prices. Nevertheless, common industry practice is to consider price and inflow as independent stochastic processes when optimizing release decisions. This paper investigate the impact of this assumption.

In order to study the co-movements between electricity prices and inflows to reservoirs in hydro-dominated electricity markets, we develop a novel stochastic model that captures the relationship between system prices and supply. Hydrological states at the producer’s location and in the system as a whole represent the latter. The local state indicates the resource availability of the producer. This state partly explains the supply in a larger geographical area, which again partly explains prices. We provide general insights about the effect of such co-movements on the optimal policy of a production scheduling problem in a two-stage setting. We apply the stochastic model in a multistage setting by studying the medium-term hydropower scheduling problem on industry data. Policies that consider co-movements prefer a slightly higher average reservoir trajectory during periods of high correlations. Moreover, the reservoir trajectory variance reduces, and the producer tolerates a slightly higher spillage probability when prices are expected to increase in the near future. From case studies using industry data, we find that a policy that ignores correlations only incurs 0.17% to 0.30% loss in expected revenues for different reservoir and production capacities and inflow behavior. This is valuable information for reservoir operators, who receive evidence indicating that the theoretical differences only amount to modest economic losses in practice.

Appendices

Proof of Propositions

This section presents the proofs of propositions.

1.1 Proof of Proposition 2.1

Proof

The proof requires finiteness. No production results in zero value, hence $V_t(s_{t-1},\omega _t) \ge 0$, which implies $\alpha _{t+1}(s_{t},\omega _t) \ge 0$. An upper bound is provided by maximum production at every stage:

$$\begin{aligned} \alpha _{t+1}(s_{t},\omega _{t})&= \delta {\mathbb {E}}( V_{t+1}(s_{t}, \omega _{t+1})\vert \omega _t) \end{aligned}$$

(36)

$$\begin{aligned}&\le \delta G \left( \sum _{i=t+1}^{T} {\mathbb {E}}\left( \omega ^O_{i}\vert \omega ^O_t\right) \right) \end{aligned}$$

(37)

Furthermore,

$$\begin{aligned} V_t \le \omega _t^O G + \alpha _{t+1}(s_{t},\omega _{t}). \end{aligned}$$

(38)

Hence, the value function and continuation function are finite. We proceed by showing that the terminal value is concave in $s_t$ for a fixed $\omega _t$ at stage T. For a given $\omega _{T}$, we have

$$\begin{aligned} V_{T}(s_{T-1},\omega _{T})&= \max _{x_{T} \in {\mathcal {X}}(s_{T-1},\omega _{T}^C)} \omega ^O_{T}x_{T} \end{aligned}$$

(39)

$$\begin{aligned} \alpha _{T+1}(s_{T},\omega _{T})&= 0 \end{aligned}$$

(40)

This is a linear program where $R-s_{T-1}-\omega ^C_{T}$ is the upper bound of the feasible set ${\mathcal {X}}(s_{T-1},\omega _{T}^C)$. From standard linear programming results, $V_{T}(s_{T-1},\omega _{T})$ is piecewise linear concave in $s_{T-1}$. The continuation function is zero and is therefore also piecewise linear concave.

By finiteness of the continuation function and the induction hypothesis, it is easy to verify that $\alpha _{t+1}(\cdot ,\omega _t)$ is piecewise linear concave in $s_{t}$. Hence, for a feasible stage t action set ${\mathcal {X}}(s_{t-1},\omega _t^C)$ which is bounded by $R-s_t-\omega _t^C$, a linear reward function $\omega _t^O x_t$, and a concave continuation function, it follows that the value function $V_t(s_{t-1},\omega _t)$ is piecewise linear concave. This implies that $\frac{\partial {\alpha _{t+1}(\cdot ,\omega _t)}}{\partial {s_{t}}}$ is decreasing in $s_t$. $\square$

1.2 Proof of Proposition 2.2

Proof

The Lagrangian is given by

$$\begin{aligned} L(x_t,v_t,s_t)&= \omega ^O_t x_t + \gamma \alpha _{t+1}(s_t,\omega _t) + z_1(s_{t-1} + \omega _t^C-s_{t}-x_t-v_t) \\ {}&\quad + z_2(R - s_{t-1}-\omega _t^C+x_t+v_t) \\&\quad + z_3(s_{t-1} + \omega _t^C-x_t) + z_4(G-x_t) + z_5x_t + z_6 v_t. \end{aligned}$$

This yields optimality conditions

$$\begin{aligned}&\frac{\partial L(x_t,v_t,s_t)}{\partial x_t} = \omega ^O_t - z_1 +z_2 - z_3 - z_4+z_5 = 0 \\&\frac{\partial L(x_t,v_t,s_{t})}{\partial v_t} = -z_1 + z_2 + z_6 = 0 \\&\frac{\partial L(x_t,v_t,s_{t})}{\partial s_{t}} = \frac{\partial \alpha _{t+1}(s_{t},\omega _{t})}{\partial s_{t}} - z_1 = 0, \end{aligned}$$

where we have set $\gamma =1$ for ease of notation. Rearranging,

$$\begin{aligned} \frac{\partial \alpha _{t+1}(s_{t},\omega _{t})}{\partial s_t}&= \omega _t^O + z_2 - z_3 - z_4 + z_5 \end{aligned}$$

(41)

$$\begin{aligned} \frac{\partial \alpha _{t+1}(s_{t},\omega _{t})}{\partial s_t}&= z_6 + z_2 \end{aligned}$$

(42)

In the trivial case, if $\omega _t^O \le 0$ the optimal decision is $x_t^*(s_t)=0$. Moreover, the marginal water value is always non-negative, since if the price is negative, the producer can spill the water at zero cost. Therefore, this trivial situation is covered by the upper case in Proposition 2.2. Below, we prove each of the two cases, assuming $\omega _t^O>0$.

Case 1: $\frac{\partial \alpha _{t+1}(s_{t},\omega _{t})}{\partial s_t} > \omega _t^O$.

Since $\omega _t^O>0$, we can in this case safely assume $s_{t-1}+\omega _t^C-x_t<R$, since the marginal water value cannot be positive if the reservoir plus incoming inflow is larger than the reservoir plus generation capacity. From (41) we get

$$\begin{aligned} z_2+z_5&> z_3 + z_4. \end{aligned}$$

(43)

Because $s_{t-1}+\omega _t^C<R$, this constraint is not binding and thus $z_2=0$. This means that $z_5$ must be positive for the condition to hold. Hence, in this case it is optimal to produce nothing, i.e., $x_t^* = 0$. This proves the first case.

Case 2: $\frac{\partial \alpha _{t+1}(s_{t},\omega _{t})}{\partial s_t} < \omega _t^O$.

From (41) we get

$$\begin{aligned} z_2+z_5&< z_3 + z_4 \end{aligned}$$

(44)

Either $z_3$ or $z_4$ must be positive, which means $z_5=0$ since both the maximum and minimum generation constraint cannot be binding at the same time. Since either $z_3$ or $z_4$ is positive, the optimal generation quantity is $x_t^* = \min \{G,s_t+\omega _t^C\}$ if $\frac{\partial \alpha (s_{t+1},\omega _{t})}{\partial s_t} < \omega _t^O$ for all $s_{t}$. However, since $\frac{\partial \alpha _{t+1}(s_{t},\omega _{t})}{\partial s_{t}}$ is decreasing in $s_{t}$, see Proposition 2.1, it is optimal to produce until the price and derivative of the continuation function, or the marginal water value, are equal, i.e. $\frac{\partial \alpha _{t+1}(s_{t},\omega _{t})}{\partial s_{t}} = \omega _t^O$. For an illustration, see Fig. 1b. This means that in a situation where $K=s_{t-1}+\omega _t^C-\mathop {{{\,\mathrm{\text {arg min}}\,}}}\limits _{s_t} \left| \omega _t^O-\frac{\partial {\alpha _{t+1}(\cdot ,\omega _t)}}{\partial {s_t}}\big \vert s_{t}\right| \in (0,\min {\{G,s_t+\omega _t^C}\})$, the optimal generation quantity is K. Therefore, in total, the optimal generation quantity is $x_t^* = \min \{G,K,s_t+\omega _t^C\}$. This proves the second case. $\square$

1.3 Proof of Proposition 2.3

Proof

In stage 2 it is optimal to produce at maximum, and since $G=R$, the optimal production quantity is $x_2^*=\min \{s_1+\omega _2^C,R\}$. Inserting this into the expression for $\alpha _2(s_1,\omega _1)$ gives

$$\begin{aligned} \alpha _{2}^{\rho }&= \delta {\mathbb {E}}(\omega _{2}^O \min \{s_{1}+\omega _{2}^C,R\}) \nonumber \\&= {\mathbb {E}}\left( \omega _{2}^O(s_1+\omega _{2}^C)\mathbbm {1}_{\{\omega _{2}^C \le R - s_{1}\}}+R\mathbbm {1}_{\{\omega _2^C> R - s_{1}\}})\right) \nonumber \\&= \left( {\mathbb {E}}(\omega _2^O\omega _2^C)+{\mathbb {E}}(\omega _2^O)s_1\right) P(\omega _2^C \le R - s_{1})+R{\mathbb {E}}(\omega _2^O)P(\omega _2^C> R - s_{1}) \nonumber \\&= \left( {\mathbb {E}}(\omega _2^O){\mathbb {E}}(\omega _2^C) + \text {Cov}(\omega _2^O,\omega _2^C)+{\mathbb {E}}(\omega _2^O)s_1\right) P(\omega _2^C \le R - s_{1})+R{\mathbb {E}}(\omega _2^O)P(\omega _2^C > R - s_{1}) \nonumber \\&= \alpha _{2}^{0} + \text {Cov}(\omega _2^O,\omega _2^C)P(\omega _2^C \le R - s_{1}) \end{aligned}$$

(45)

The term $\text {Cov}(\omega _2^O,\omega _2^C)$ is negative and $P(\omega _2^C \le R - s_{1})$ is positive and greater than zero if $s_1 < R$, hence $\alpha _{2}^{\rho } < \alpha _{2}^{0}$ when $s_1 < R$. Note that $P(\omega _2^C \le R - s_{1})$ is monotonically increasing in $s_{1}$. Hence, $\alpha _{2}^{0} - \alpha _{2}^{\rho }$ is a decreasing function from $\text {Cov}(\omega _2^O,\omega _2^C)P(\omega _2^C \le R)$ at $s_{1}=0$ to 0 at $s_{1} = R$. For an illustration, see Fig. 1a. This means that $\frac{\partial \alpha _2^{\rho }(s_{1},\omega _{1})}{\partial s_{1}} > \frac{\partial \alpha _2^{0}(s_{1},\omega _{1})}{\partial s_{1}}$. $\square$

Assessment of the inflow model

See Fig. 13.

Fig. 13

Inflow data, and one-step ahead predictions under the Gaussian model and the lognormal counterpart

Derivation of the risk-neutral process

We first derive the continuous-time risk-neutral version and then derive the discretized version. The general dynamic of prices is

$$\begin{aligned} dS_t = \alpha (S_t,t)dt + \sigma (S_t,t) dz, \end{aligned}$$

(46)

and the following deterministic convenience yield process is

$$\begin{aligned} dC_t = \delta (S_t,t)dt. \end{aligned}$$

(47)

The discounted gains process of owning a derivative or asset on $S_t$ must be a martingale and is given by

$$\begin{aligned} dG_t&= d(e^{-rt}S_t) + e^{-rt}dC_t \end{aligned}$$

(48)

$$\begin{aligned}&= -r e^{-rt}S_tdt + e^{-rt} dS_t + e^{-rt}dC_t \end{aligned}$$

(49)

$$\begin{aligned}&= e^{-rt}\left( \left( -r S_t + \alpha (S_t,t)+\delta (S_t,t)\right) dt + \sigma (S_t,t) dz \right) \end{aligned}$$

(50)

$$\begin{aligned}&= e^{-rt}\sigma (S_t,t) (\lambda (S_t,t) dt + dz) \end{aligned}$$

(51)

$$\begin{aligned}&= e^{-rt}\sigma (S_t,t) dz^*, \end{aligned}$$

(52)

where r denotes the risk-free rate, and where the risk premium is

$$\begin{aligned} \lambda (S_t,t) = \frac{\alpha (S_t,t)+\delta (S_t,t)-rS_t}{\sigma (S_t,t)}. \end{aligned}$$

(53)

By Girsanov’s theorem, the new process $dz^*$ is a standard Brownian motion [35]. The risk-neutral dynamics can now be written as

$$\begin{aligned} dS_t = (rS_t-\delta (S_t,t))dt + \sigma (S_t,t) dz^*, \end{aligned}$$

(54)

by rearranging (49) and (52).

We now move to our specific stochastic model for prices, and start with the equilibrium price level, $\xi _t$. If we let $\alpha (\xi _t,t) = \alpha$ and $\sigma (\xi _t,t) = \sigma$, the underlying is an arithmetic Brownian motion (ABM), which can be written in discrete time as in (22). Following Alexander et al. (2012) [2], we impose the long-term convenience yield $\delta (\xi _t,t)=\delta \xi _t$. Note that since $\xi _t$ can be positive and negative, convenience yield is negative if $\xi _t$ is negative. The risk premium is then given by $\sigma _2 \lambda _{\xi }(\xi _t,t) = \alpha - (r-\delta _{\xi })\xi _t$, which inserted into (54) leads to the following risk-neutral dynamics:

$$\begin{aligned} d\xi _t = (r-\delta _{\xi })\xi _t dt + \sigma _{\xi } dz^*. \end{aligned}$$

(55)

This process has geometric expected growth and additive noise. It can be discounted by the risk-free rate for getting the value today of an asset on $\xi _t$. Rearranging, a discrete-time version risk-neutral process can be written as

$$\begin{aligned} \xi _t&= \phi _5^*\xi _{t-1}+\sigma _{2} \epsilon _2^*, \end{aligned}$$

(56)

where $\phi _5^*=e^{(r-\delta _{\xi })\Delta t}$ and $\sigma ^2_{2}=\frac{1}{2(r-\delta _{\xi })}\sigma _{\xi }^2(1-e^{(r-\delta _{\xi })\Delta t})$.

For the short-term deviations, $\chi _t$, we let $\alpha (\chi _t,t)=-\kappa \chi _t$ and $\sigma (\chi _t,t) = \sigma _{\chi }$. The underlying is then an additive Ornstein-Uhlenbeck process, which can be seen by inserting into (46). This can be written as an AR-1 process in discrete time, as in (21). Imposing the short-term convenience yield $\delta (\chi _t,t)=\delta _{\chi } \chi _t$ entails short-term risk premium $\sigma _{\chi } \lambda _{\chi }(\chi _t,t) = -\kappa _{\chi } \chi _t - (r-\delta _{\chi })\chi _t$. Inserting convenience yield into (54) gives

$$\begin{aligned} d\chi _t = (r-\delta _{\chi })\chi _t dt + \sigma _{\chi } dz^*. \end{aligned}$$

(57)

Rearranging, a discrete-time version risk-neutral process for the short-term deviations can be written as

$$\begin{aligned} \chi _t&= \phi _4^*\chi _{t-1}+\sigma _1 \epsilon _1^*, \end{aligned}$$

(58)

where $\phi _4^*=e^{(r-\delta _{\chi })\Delta t}$ and $\sigma ^2_{2}=\frac{1}{2(r-\delta _{\chi })}\sigma _{\chi }^2(1-e^{(r-\delta _{\chi })\Delta t})$. For the stochastic processes $\nu _t$ and $\eta _t$ in (16) and (19) the derivations are similar to the derivations for $\chi _t$.

Calibration with $\omega _t^O$ and $h_t^{sys}$ being independent

For the case study in Sect. 5, we need to benchmark our stochastic model against a model where price is not affected by inflows to reservoirs. This can be obtained by setting the coefficient of $h_t^{sys}$ to zero and let the short-term price factor $\chi _t$ capture all price deviations from the long-term price level, including price deviations related to hydrology. We then need to re-estimate parameters of the short-term factor, such that mean-reversion speed and variance are approximately equivalent to the model where hydrology enters as an explicit factor. Therefore, we let

$$\begin{aligned} \phi ^{I}_3&=0 \end{aligned}$$

(59)

$$\begin{aligned} \phi ^{I}_6&=0 \end{aligned}$$

(60)

$$\begin{aligned} \phi _4^{*,I}&= \frac{1}{2}(\phi _6+\phi _4^{*}) \end{aligned}$$

(61)

$$\begin{aligned} \sigma _4^{*,I}&= \sqrt{\frac{1+\phi _4^{*,I \ 2}}{1+\phi _4^{* \ 2}}\sigma _1^2+\frac{1+\phi _4^{*,I \ 2}}{1+\phi _6^{* \ 2}}\sigma _{3}^2}, \end{aligned}$$

(62)

where $\sigma _3$ is re-estimated to system hydrological data when $\phi _6$ is set to zero, and superscript I indicates independent. By defining parameters this way we can closely resemble conditional means and long-term variance in our joint model and a model with no connection between prices and inflows to reservoirs.

Nomenclature and parameter estimates

Table 5 provides the parameters used in the case study in Sect. 5. The last columns provide parameter estimates from the joint model (J) and the independent model (I). Section 5.2 explains the procedure for calibrating the joint model, while Appendix D presents a possible procedure for calibrating the independent model.

Table 5 Parameters for case study

Discretization of the stochastic model

We consider a producer planning its hydropower operation for the next 2 years. It thus considers 104 weekly stages. We construct Markov chains according to Algorithm 1 from 50,000 weekly Monte Carlo simulations of prices and inflows to reservoirs from expressions (23) to (31). Since inflow is normally distributed in our model, it can simulate negative inflow values that we set to zero. Negative inflow is a challenge in SDDP and could alternatively be addressed by penalization [20]. Each week has a price and inflow pair, and we use the k-means method to cluster. Transition probabilities are estimated as the number of scenarios that travel from one cluster to the next, divided by the total number of scenarios moving through the previous cluster. A joint clustering on price and inflow pairs created a Markov chain that did not capture the variance, mean, and covariance of the underlying stochastic model. Instead, we only cluster on prices and map the corresponding inflows to each price cluster. These inflows must transform into a representative inflow to the price cluster. Importantly, they must retain the properties of the underlying stochastic model. The average value of the inflows manages to capture the covariance and mean, but not the variance. On the other hand, a random sample of the inflows captures the variance but not the covariance and mean values. A compromise is the mean of a limited number of samples, which manages to provide representative covariances, variances, and means (experiments on our application find five samples to be representative). The variance has the same shape as the continuous process’ variance, but has a lower magnitude that decreases with the number of samples. However, an appropriate factor can scale the variance to the magnitude of the underlying stochastic model. Before scaling, we need to subtract the mean of all the associated inflows in a cluster and add it back afterwards, in order to not scale the mean as well. Algorithm 1 shows pseudocode for this operation. We use a Markov chain with five price and inflow pair nodes. Simulations verify that it is representative of the continuous price and inflow processes.

Additional case studies

This section contains results for different parameters to investigate the sensitivity of the findings. We first change parameters of the stochastic process to test stronger co-movements and higher seasonal price variations. Afterwards, we test different plant characteristics by reducing reservoir capacity and increasing generation capacity.

1.1 Extreme effect from hydrology on prices and higher seasonal price variations

To study the effect when seasonal price variations are higher and co-movements are stronger, we increase $\phi _1$ to $-$1.5 and $\phi _3$ to 10. This also introduces a higher price variance. All other model parameters remain as they were. Figure 14a displays the reservoir trajectories under optimal policies. The behavior of the mean reservoir is similar to the base case. It shows that the producer favors a higher mean reservoir from around week 25 and is therefore more willing to risk spillage in this period. The 10th to 90th percentiles are narrower throughout the year in both the dependent and independent Markov chain compared to the base case. This is a result of increased seasonal price variations. Figure 14b shows a scatter plot of revenue. As in the base case, the producer only slightly increases revenue in most scenarios when accounting for co-movements when establishing the operational policy. Table 6 outlines the percentage expected opportunity loss in the extreme case.

Fig. 14

Revenues and reservoir trajectories for the extreme case

Table 6 Difference in expected revenue of policies relative to $V^{\rho }_1(s_0,\omega _1)$ for the extreme case

1.2 Changing plant characteristics: higher production capacity and lower reservoir capacity

The next case study keeps the stochastic model parameter estimates as the base case but changes the plant characteristics to $R = G = 106,382$ MWh. Both the generation capacity and reservoir capacity are equal to the average inflow of four weeks. The reservoir capacity is 32% and generation capacity is 270% of that of the base case. Figures 15a, b show reservoir trajectories and revenues. The producer drains the reservoir already in the first week, because it can now produce the entire reservoir at any point in time, and the price seasonality trends downwards in January. Figure 15b demonstrates that the sample path revenues behave as in the other cases. Table 7 shows that the revenue improves by 0.30% in expectation.

Fig. 15

Revenues and reservoir trajectories for case with different plant characteristics

Table 7 Difference in expected revenue of policies relative to $V^{\rho }_1(s_0,\omega _1)$ with different plant characteristics $R = G = 106,382$ MWh