Adaptive partition-based SDDP algorithms for multistage stochastic linear programming with fixed recourse

Abstract

In this paper, we extend the adaptive partition-based approach for solving two-stage stochastic programs with fixed recourse matrix and fixed cost vector to the multistage stochastic programming setting where the stochastic process is assumed to be stage-wise independent. The proposed algorithms integrate the adaptive partition-based strategy with a popular approach for solving multistage stochastic programs, the stochastic dual dynamic programming (SDDP) algorithm, according to two main strategies. These two strategies are distinct from each other in the manner by which they refine the partitions during the solution process. In particular, we propose a refinement outside SDDP strategy whereby we iteratively solve a coarse scenario tree induced by the partitions, and refine the partitions in a separate step outside of SDDP, only when necessary. We also propose a refinement within SDDP strategy where the partitions are refined in conjunction with the machinery of the SDDP algorithm. We then use, within the two different refinement schemes, different tree-traversal strategies which allow us to have some control over the size of the partitions. We performed numerical experiments on a hydro-thermal power generation planning problem. Numerical results show the effectiveness of the proposed algorithms that use the refinement outside SDDP strategy in comparison to the standard SDDP algorithm and algorithms that use the refinement within SDDP strategy.

Appendices

A summary of notation

See Table 8.

Table 8 Summary of notations

B Proof of Proposition 1

Proposition 2

Convergence of the partition-based SDDP algorithms Suppose that Assumptions 1–4hold, sampling is done with replacement in the SDDP algorithm, and the employed partition refinement rule ensures that the number of clusters $L_t$ of the refined partition strictly increases whenever $\check{{\bar{\mathfrak {Q}}}}_t({\check{x}}_{t-1}) \ne \mathbb {E}[{\underline{Q}}_t({\check{x}}_{t-1},\xi _t)]$ (see definitions in Eqs. (16), (17), (22) and (25)) for any stage t unless $L_t = N_t$. Then w.p.1 after a finite number of forward and backward steps, the algorithm yields an optimal policy for (2).

Proof

Let us first consider the trivial but sufficient partition with all singletons ${\mathscr {N}}^{\max }_t = \{\{k\}_{k \in N_t}\}$ and $\check{{\bar{\mathfrak {Q}}}}^{{\max } }_t(\cdot )$ be its respective approximation of $\mathfrak {Q}_t(\cdot )$. By definition, we have that $\check{{\bar{\mathfrak {Q}}}}^{{\max } }_t(\cdot ) = {\check{\mathfrak {Q}}}_t(\cdot )$. Hence, if ${\mathscr {N}}_t = {\mathscr {N}}^{\max }_t$ for every stage $t=2, \dots , T$ then the partition-based variant of the SDDP algorithm is equivalent to the standard SDDP algorithm and the classical convergence proofs of SDDP apply (see, e.g., [36, Proposition 3.1]).

Let us now consider the situation where ${\mathscr {N}}_t \ne {\mathscr {N}}^{\max }_t$. Our goal is to show that for any arbitrary sample path $\xi ^s = (\xi _2^{s_2}, \ldots , \xi _T^{s_T})$, the forward/backward pass of the adaptive partition-based SDDP algorithms will either yield a partition ${\mathscr {N}}_t={\mathscr {N}}^{\max }_t$, or a sufficient partition such that $\check{{\bar{\mathfrak {Q}}}}_t(\cdot ) = {\check{\mathfrak {Q}}}_t(\cdot ), \ \forall t=2,\dots ,T$ along sample path $\xi ^s$. In both cases, employing the adaptive partition-based SDDP algorithms over the sample path $\xi ^s$ is equivalent to the standard SDDP algorithm. We show this by using a backward induction argument as follows.

• Let ${\check{x}}_t = {\check{x}}_t (\xi ^{s_t}_t), \forall t=1, \dots , T-1$ be the candidate solution induced by the approximate cost-to-go function $\check{{\bar{\mathfrak {Q}}}}_t(\cdot )$ along $\xi ^s$. At stage $t=T$ the candidate solution ${\check{x}}_{T-1}$ is evaluated at every partition-based subproblem (21) and by Lemma 1 we know that, if there exists an optimal $\check{{\bar{\pi }}}^\ell _{T} \in \cap _{k \in \mathscr {P}^\ell _T} \Pi ^*({\check{x}}_{T-1}, {\bar{\xi }}^k_T)$ then ${\mathscr {N}}_T$ is sufficient and $\check{{\bar{\mathfrak {Q}}}}_T({\check{x}}_{T-1}) = {\check{\mathfrak {Q}}}_T({\check{x}}_{T-1})$. Otherwise, $\check{{\bar{\mathfrak {Q}}}}_T({\check{x}}_{T-1}) < {\check{\mathfrak {Q}}}_T({\check{x}}_{T-1})$ and ${\mathscr {N}}_T$ needs to be refined. The case where $\check{{\bar{\mathfrak {Q}}}}_T({\check{x}}_{T-1}) = {\check{\mathfrak {Q}}}_T({\check{x}}_{T-1})$ is degenerate and hence, we focus on the case where $\check{{\bar{\mathfrak {Q}}}}_T({\check{x}}_{T-1}) < {\check{\mathfrak {Q}}}_T({\check{x}}_{T-1})$. After refining ${\mathscr {N}}_T$ using the absolute rule, a cut can be generated to the subproblem at stage $t=T-1$ using the refined partition ${\mathscr {N}}'_T$ whose size $L'_t > L_t$. Then an updated candidate solution ${\check{x}}'_{T-1}$ can be obtained by solving ${\underline{Q}}_{T-1}({\check{x}}_{T-2}, \xi _{T-1}^s)$. If there still exists a positive gap between $\check{{\bar{\mathfrak {Q}}}}'_T(\cdot )$ and ${\check{\mathfrak {Q}}}_T(\cdot )$ when evaluated at the updated candidate solution ${\check{x}}'_{T-1}$, the same process can be repeated again until $\check{{\bar{\mathfrak {Q}}}}'_T(\cdot ) = {\check{\mathfrak {Q}}}_T(\cdot )$ for all the candidate solutions encountered or ${\mathscr {N}}'_T = {\mathscr {N}}^{\max }_T$.

• Suppose now that $\check{{\bar{\mathfrak {Q}}}}_t({\check{x}}_{t-1}) = {\check{\mathfrak {Q}}}_t({\check{x}}_{t-1})$ or ${\mathscr {N}}_t = {\mathscr {N}}^{\max }_t$ for all $t= T-1, \dots , \tau +1$. At time $t=\tau$, similarly to the case when $t=T$, the candidate solution ${\check{x}}_{\tau -1}$ is evaluated at every partition-based subproblem (24) and if ${\mathscr {N}}_{\tau }$ is not sufficient, the absolute refinement procedure is followed until $\check{{\bar{\mathfrak {Q}}}}'_{\tau }({\check{x}}_{\tau -1}) = {\check{\mathfrak {Q}}}_{\tau }({\check{x}}_{\tau -1})$ or ${\mathscr {N}}'_{\tau } = {\mathscr {N}}^{\max }_{\tau }$.

Since (by assumption) the sampling is done with replacement, we are guaranteed that each sample path $\xi ^s$ is sampled for infinitely many times with probability 1. Therefore, with probability 1, after a finite number of iterations, for any arbitrary sample path $\xi ^s$, $\check{{\bar{\mathfrak {Q}}}}_t(\cdot ) = {\check{\mathfrak {Q}}}_t(\cdot )$ or ${\mathscr {N}}_t = {\mathscr {N}}^{\max }_t, \forall t=2, \dots , T$. At that point, employing the adaptive partition-based SDDP algorithms over sample path $\xi ^s$ is equivalent to employing the standard SDDP algorithm, and the convergence of the adaptive partition-based SDDP algorithms can be subsequently proved by applying the convergence proof of the standard SDDP algorithm. $\square$