Two-stage linear decision rules for multi-stage stochastic programming

Abstract

Multi-stage stochastic linear programs (MSLPs) are notoriously hard to solve in general. Linear decision rules (LDRs) yield an approximation of an MSLP by restricting the decisions at each stage to be an affine function of the observed uncertain parameters. Finding an optimal LDR is a static optimization problem that provides an upper bound on the optimal value of the MSLP, and, under certain assumptions, can be formulated as an explicit linear program. Similarly, as proposed by Kuhn et al. (Math Program 130(1):177–209, 2011) a lower bound for an MSLP can be obtained by restricting decisions in the dual of the MSLP to follow an LDR. We propose a new approximation approach for MSLPs, two-stage LDRs. The idea is to require only the state variables in an MSLP to follow an LDR, which is sufficient to obtain an approximation of an MSLP that is a two-stage stochastic linear program (2SLP). We similarly propose to apply LDR only to a subset of the variables in the dual of the MSLP, which yields a 2SLP approximation of the dual that provides a lower bound on the optimal value of the MSLP. Although solving the corresponding 2SLP approximations exactly is intractable in general, we investigate how approximate solution approaches that have been developed for solving 2SLP can be applied to solve these approximation problems, and derive statistical upper and lower bounds on the optimal value of the MSLP. In addition to potentially yielding better policies and bounds, this approach requires many fewer assumptions than are required to obtain an explicit reformulation when using the standard static LDR approach. A computational study on two example problems demonstrates that using a two-stage LDR can yield significantly better primal policies and modestly better dual policies than using policies based on a static LDR.

Appendices

Data for the capacity expansion problem

See Tables 6, 7 and 8.

Table 6 Fixed annual cost and operation cost (Table 1 in [15])

Table 7 Initial demand (Table 2 in [15])

Table 8 Wind regimes (Table 3 in [15])

Additional models for the capacity expansion example

1.1 Primal model and Benders decomposition

P-LDR-2S of the capacity expansion model is obtained by substituting

$$\begin{aligned} s_{ti}(\xi ^{t}) = \sum _{k \in [K_t]} {\Phi }_{tk}(\xi ^{t}) {\beta }_{tki} \end{aligned}$$

in (17). Dropping $\xi ^t$ dependences for variables to simplify the notation, we obtain

$$\begin{aligned} \min \,\,&\sum _{i \in [I]} \Big ( c_{1i}^{u^+} u^+_{1i} + c_{1i}^{u^-} u^-_{1i} + \sum _{j \in [J]} c_{1ij}^{x} x_{1ij}(\xi ^{t}) \Big ) + \sum _{j \in [J]} c_{1j}^zz_{1j} \nonumber \\&\qquad \qquad + \sum _{t \in [T]} \sum _{i \in [I]} c_{ti}^s\sum _{k \in [K_t]} \mathbb {E}\big [ {\Phi }_{tk}(\xi ^{t}) \big ] {\beta }_{tki} + \sum _{t \in [2,T]} \mathbb {E}[ Q_t({\beta },\xi ^{t}) ] \end{aligned}$$

(19a)

$$\begin{aligned} \text {s.t.}\,\,&{\beta }_{11i} - u^+_{1i} + u^-_{1i} = 0,&\quad i \in [I], \end{aligned}$$

(19b)

$$\begin{aligned}&{\beta }_{11i} - x_{1ij} \ge 0,&i \in [I], \ j \in [J], \end{aligned}$$

(19c)

$$\begin{aligned}&\sum _{i \in [I]} x_{1ij} + z_{1j} \ge d_{1j},&\quad j \in [J], \end{aligned}$$

(19d)

$$\begin{aligned}&0 \le z_{1j} \le d_{1j},&j \in [J], \end{aligned}$$

(19e)

$$\begin{aligned}&0 \le u^+_{1i} \le M_{1i}^{u^+}, \ \ 0 \le u^-_{1i} \le M_{1i}^{u^-}, \ \ 0 \le {\beta }_{11i} \le M_{1i}^{s},&i \in [I], \end{aligned}$$

(19f)

$$\begin{aligned}&x_{1ij} \ge 0,&i \in [I], \ j \in [J], \end{aligned}$$

(19g)

where, for $t \in [2,T]$, $Q_t({\beta },\xi ^{t})$ is defined as the optimal objective value of the following problem:

$$\begin{aligned} \min \,\,&\sum _{i \in [I]} \Big ( c_{ti}^{u^+} u^+_{i} + c_{ti}^{u^-} u^{-}_{i} + \sum _{j \in [J]} c_{tij}^{x} x_{ij} \Big ) + \sum _{j \in [J]} c_{tj}^zz_{j} \end{aligned}$$

(20a)

$$\begin{aligned} \text {s.t.} \,\,&u^+_{i} - u^-_{i} = \sum _{k \in [K_t]} {\Phi }_{tk}(\xi ^{t}) {\beta }_{tki} - \sum _{k \in [K_{t-1}]} {\Phi }_{tk}(\xi ^{t-1}) {\beta }_{t-1,k,i},&i \in [I], \end{aligned}$$

(20b)

$$\begin{aligned}&x_{ij} \le \sum _{k \in [K_t]} {\Phi }_{tk}(\xi ^{t}) {\beta }_{tki},&i \in [I], \ j \in [J], \end{aligned}$$

(20c)

$$\begin{aligned}&\sum _{i \in [I]} x_{ij} + z_{j} \ge d_{tj}(\xi ^{t}),&j \in [J], \end{aligned}$$

(20d)

$$\begin{aligned}&0 \le z_{j} \le d_{tj}(\xi ^{t}),&j \in [J], \end{aligned}$$

(20e)

$$\begin{aligned}&0 \le u^+_{i} \le M_{ti}^{u^+},&i \in [I], \end{aligned}$$

(20f)

$$\begin{aligned}&0 \le u^-_{i} \le M_{ti}^{u^-},&i \in [I], \end{aligned}$$

(20g)

$$\begin{aligned}&x_{ij} \ge 0,&i \in [I], \ j \in [J], \end{aligned}$$

(20h)

$$\begin{aligned}&{0\le M_{ti}^s - \sum _{k \in [K_t]} {\Phi }_{tk}(\xi ^{t}) {\beta }_{tki},}&i \in [I], \end{aligned}$$

(20i)

$$\begin{aligned}&{0 \le \sum _{k \in [K_t]} {\Phi }_{tk}(\xi ^{t}) {\beta }_{tki},}&i \in [I], \end{aligned}$$

(20j)

We note that P-LDR-2S of the capacity expansion model does not have relatively complete recourse since the recourse constraints (20i) and (20j) might be violated under some scenarios.

Let $\xi ^{T}_n, n \in [N]$, be an independent and identically distributed (i.i.d.) random sample of the random vector $\xi ^{T}$. We solve the obtained primal SAA problem with Benders decomposition, using a single aggregate cut per time-stage. That is, we have a master problem of the following form:

$$\begin{aligned} \min \,\,&\sum _{i \in [I]} \Big ( c_{1i}^{u^+} u^+_{1i} + c_{1i}^{u^-} u^-_{1i} + \sum _{j \in [J]} c_{1ij}^{x} x_{1ij}(\xi ^{t}) \Big ) + \sum _{j \in [J]} c_{1j}^zz_{1j} \nonumber \\&\qquad + \sum _{t \in [T]} \sum _{i \in [I]} c_{ti}^s\sum _{k \in [K_t]} \mathbb {E}\big [ {\Phi }_{tk}(\xi ^{t}) \big ] {\beta }_{tki} + \sum _{t \in [2,T]} \eta _t\end{aligned}$$

(21a)

$$\begin{aligned} \text {s.t.}\,\,&22b-22g, \end{aligned}$$

(21b)

$$\begin{aligned}&(\eta _t, {\beta }_{t11}, \cdots , {\beta }_{t K_tI} ) \in \mathcal {O}^t ,&\quad t \in [2,T] , \end{aligned}$$

(21c)

$$\begin{aligned}&({\beta }_{t11}, \cdots , {\beta }_{t K_tI} ) \in \mathcal {F}^t,&\quad t \in [2,T], \end{aligned}$$

(21d)

$$\begin{aligned}&{0 \le \sum _{k \in [K_t]} {\Phi }_{tk}(\xi ^{t}_n) {\beta }_{tki} \le M_{ti}^s,}&\quad t \in [2,T], i \in [I], \ n \in [N], \end{aligned}$$

(21e)

$$\begin{aligned}&\eta _t \ge 0,&\quad t \in [2,T]. \end{aligned}$$

(21f)

The variable $\eta _t$ represents the expected second-stage cost at period $t \in [2,T]$, i.e., $\frac{1}{N} \sum _{n \in [N]} [ Q_t({\beta },\xi ^{t}_n) ]$. Note that since all the original decision variables are defined to be nonnegative, and all the cost parameters are assumed to be nonnegative, $Q_t({\beta },\xi ^{t}) \ge 0$ for any given $\beta$, thus (21f) are valid. Constraints (21c) and (21d) correspond to the set of Benders optimality and feasibility cuts, respectively. As ${\beta }$ variables belong to the master problem, we add constraints (20i) and (20j) for each scenario in the sample to the master problem as (21e) which can be seen as an additional set of feasibility cuts.

The subproblem decomposes not only by scenario but also by stage. For $t \in [2,T]$ and $n \in [N]$, we have the corresponding subproblem (20a)–(20h), denoted by SP(t, n).

At every iteration of the Benders decomposition algorithm, we solve the master problem, get a candidate $\beta$ solution which is fixed in the subproblems, and solve all the subproblems. For $t \in [2,T]$, if there is at least one index $n \in [N]$ for which SP(t, n) is infeasible, then we generate a Benders feasibility cut and add it to the master problem. Otherwise, we generate a Benders optimality cut, but add it to the master problem only if it is violated at the current master problem solution. We repeat this procedure until all the subproblems are feasible and no violated optimality cuts are found.

1.2 Dual model and level method

Let $\lambda , \gamma , \theta ^+, \theta ^-, {\Gamma }^{u^+},{\Gamma }^{u^-}, {\Gamma }^{s}$ be the dual variables associated with the constraints (17b)–(17h) in (17), respectively. Then, the dual of (17) is:

$$\begin{aligned} \max \,\,&\mathbb {E}\sum _{t \in [T]} \Big [ \sum _{j \in [J]} d_{tj}(\xi ^{t}) \big ( \theta ^+_{tj}(\xi ^{t}) \, {- } \, \theta ^-_{tj}(\xi ^{t})\big ) \nonumber \\&\quad \ \ \, \, {- } \, \sum _{i \in [I]} \bigl (M_{ti}^{u^+} {\Gamma }^{u^+}_{ti}(\xi ^{t}) + M_{ti}^{u^-}{\Gamma }^{u^-}_{ti}(\xi ^{t}) + M_{ti}^{s} {\Gamma }^{s}_{ti}(\xi ^{t})\bigr ) \Big ] \end{aligned}$$

(22a)

$$\begin{aligned} \text {s.t.}\,\,&\lambda _{ti}(\xi ^{t}) - \mathbb {E}[ \lambda _{t+1,i}(\xi ^{t+1}) \mid \xi ^{t}] \nonumber \\&\ + \sum _{j \in [J]} \gamma _{tij}(\xi ^{t}) \, {- } \, {\Gamma }^{s}_{ti}(\xi ^{t}) \le c_{ti}^{s},&t \in [T], \ \mathbb {P}\text {-a.s.}, \ i \in [I], \end{aligned}$$

(22b)

$$\begin{aligned}&{- } \, {\Gamma }^{u^+}_{ti}(\xi ^{t})- \lambda _{ti}(\xi ^{t}) \le c_{ti}^{u^+},&t \in [T], \ \mathbb {P}\text {-a.s.}, \ i \in [I], \end{aligned}$$

(22c)

$$\begin{aligned}&{- } \, {\Gamma }^{u^-}_{ti}(\xi ^{t}) + \lambda _{ti}(\xi ^{t}) \le c_{ti}^{u^-},&t \in [T], \ \mathbb {P}\text {-a.s.}, \ i \in [I], \end{aligned}$$

(22d)

$$\begin{aligned}&\theta ^+_{tj}(\xi ^{t}){- } \theta ^-_{tj}(\xi ^{t}) \ \le c_{tj}^{z},&t \in [T], \ \mathbb {P}\text {-a.s.}, \ j \in [J], \end{aligned}$$

(22e)

$$\begin{aligned}&\theta ^+_{tj}(\xi ^{t}) - \gamma _{tij}(\xi ^{t}) \le c_{tij}^{x},&t \in [T], \ \mathbb {P}\text {-a.s.}, \ i \in [I], \ j \in [J], \end{aligned}$$

(22f)

$$\begin{aligned}&\gamma _{tij}(\xi ^{t}) \ge 0,&t \in [T], \ \mathbb {P}\text {-a.s.}, \ i \in [I], \ j \in [J], \end{aligned}$$

(22g)

$$\begin{aligned}&\theta ^+_{tj}(\xi ^{t}), {\theta ^-_{tj}(\xi ^{t})} \ge 0,&t \in [T], \ \mathbb {P}\text {-a.s.}, \ j \in [J], \end{aligned}$$

(22h)

$$\begin{aligned}&{\Gamma }^{u^+}_{ti}(\xi ^{t}), {\Gamma }^{u^-}_{ti}(\xi ^{t}), {\Gamma }^{s}_{ti}(\xi ^{t}) \, {\ge } \, 0&t \in [T], \ \mathbb {P}\text {-a.s.}, \ i \in [I]. \end{aligned}$$

(22i)

Observing that $\theta ^-$ variables are redundant, we remove them to simplify the model. D-LDR-2S of the capacity expansion model is obtained by substituting

$$\begin{aligned} \lambda _{ti}(\xi ^{t}) = \sum _{k \in [K_t]} {\Phi }_{tk}(\xi ^{t}) {\Lambda }_{tki} \end{aligned}$$

in (22). Dropping $\xi ^t$ dependences for variables to simplify the notation, we obtain

$$\begin{aligned} \max \,\,&\sum _{j \in [J]} d_{1j} \theta ^+_{1j} \, {-} \, \sum _{i \in [I]} \big ( M_{1i}^{u^+} {\Gamma }^{u^+}_{1i} + M_{1i}^{u^-} {\Gamma }^{u^-}_{1i} + M_{1i}^{s} {\Gamma }^{s}_{1i} \big ) + \sum _{t \in [2,T]} \mathbb {E}[G_t({\Lambda },\xi ^{t}) ] \end{aligned}$$

(23a)

$$\begin{aligned} \text {s.t.}\,\,&{\Lambda }_{11i} - \sum _{k \in [K_2]} \mathbb {E}\big [ {\Phi }_{2k}(\xi ^2) \big ] {\Lambda }_{2ki} + \sum _{j \in [J]} \gamma _{1ij} \, {-} \, {\Gamma }^{s}_{1i} \le c_{1i}^{s},&i \in [I], \end{aligned}$$

(23b)

$$\begin{aligned}&{-} \, {\Gamma }^{u^+}_{1i} -{\Lambda }_{11i} \le c_{1i}^{u^+},&i \in [I], \end{aligned}$$

(23c)

$$\begin{aligned}&{-} \, {\Gamma }^{u^-}_{1i} + {\Lambda }_{11i} \le c_{1i}^{u^-},&i \in [I], \end{aligned}$$

(23d)

$$\begin{aligned}&\theta ^+_{1j} \le c_{1j}^{z},&j \in [J], \end{aligned}$$

(23e)

$$\begin{aligned}&\theta ^+_{1j} - \gamma _{1ij} \le c_{1ij}^{x},&i \in [I], \ j \in [J] \end{aligned}$$

(23f)

$$\begin{aligned}&\gamma _{1ij} \ge 0,&i \in [I], \ j \in [J] \end{aligned}$$

(23g)

$$\begin{aligned}&\theta ^+_{1j} \ge 0,&j \in [J], \end{aligned}$$

(23h)

$$\begin{aligned}&{\Gamma }^{u^+}_{1i}, {\Gamma }^{u^-}_{1i}, {\Gamma }^{s}_{1i} \ {\ge } \ 0&i \in [I], \end{aligned}$$

(23i)

where for $t \in [2,T]$, $G_t({\Lambda },\xi ^{t})$ is defined as the optimal objective value of the following problem:

$$\begin{aligned} \max \,\,&\sum _{j \in [J]} d_{tj} \theta ^+_{j} \, {-} \, \sum _{i \in [I]} \big (M_{ti}^{u^+}{\Gamma }^{u^+}_{i}+ M_{ti}^{u^-} {\Gamma }^{u^-}_{i}+M_{ti}^{s} {\Gamma }^{s}_{i} \big ) \end{aligned}$$

(24a)

$$\begin{aligned} \text {s.t.}\,\,&\sum _{j \in [J]} \gamma _{ij} \, {-} \, {\Gamma }^{s}_{i} \le c_{ti}^{s} - \sum _{k \in [K_t]} {\Phi }_{tk}(\xi ^{t}) {\Lambda }_{tki} \nonumber \\&\qquad \quad + \sum _{k \in [K_{t+1}]} \mathbb {E}\Big [ {\Phi }_{t+1,k}(\xi ^{t+1}) \Big | \xi ^{t}\Big ] {\Lambda }_{t+1,k,i},&i \in [I], \end{aligned}$$

(24b)

$$\begin{aligned}&{-} \, {\Gamma }^{u^+}_{i} \le c_{ti}^{u^+} + \sum _{k \in [K_t]} {\Phi }_{tk}(\xi ^{t}) {\Lambda }_{tki},&i \in [I], \end{aligned}$$

(24c)

$$\begin{aligned}&{-} \, {\Gamma }^{u^-}_{i} \le c_{ti}^{u^-} - \sum _{k \in [K_t]} {\Phi }_{tk}(\xi ^{t}) {\Lambda }_{tki},&i \in [I], \end{aligned}$$

(24d)

$$\begin{aligned}&\theta ^+_{j} \le c_{tj}^{z},&j \in [J], \end{aligned}$$

(24e)

$$\begin{aligned}&\theta ^+_{j} - \gamma _{ij} \le c_{tij}^{x},&i \in [I], \ j \in [J], \end{aligned}$$

(24f)

$$\begin{aligned}&\gamma _{ij} \ge 0,&i \in [I], \ j \in [J], \end{aligned}$$

(24g)

$$\begin{aligned}&\theta ^+_{j} \ge 0,&j \in [J], \end{aligned}$$

(24h)

$$\begin{aligned}&{\Gamma }^{u^+}_{i}, {\Gamma }^{u^-}_{i}, {\Gamma }^{s}_{i} \ {\ge } \ 0&i \in [I]. \end{aligned}$$

(24i)

Let $\xi ^{T}_n, n \in [N]$, be an independent and identically distributed (i.i.d.) random sample of the random vector $\xi ^{T}$. We solve the obtained dual SAA problem with the bundle-level method, because the Benders decomposition method converged slowly for this problem. We use cuts aggregated over scenarios, thus introduce $\zeta _{t}$ variable to represent the expected second-stage cost value, i.e., $\frac{1}{N} \sum _{n \in [N]} G_t({\Lambda },\xi ^{t}_n)$, for $t \in [2,T]$.

We observe that the subproblem (24) can be further decomposed into two: one problem including only the $u^+$ and $u^-$ variables, and the other problem including the remaining set of variables.

$$\begin{aligned} (\text {DSP}^{upart}): \ \max&- \sum _{i \in [I]} \big (M_{ti}^{u^+}{\Gamma }^{u^+}_{i}+ M_{ti}^{u^-} {\Gamma }^{u^-}_{i} \big ) \\ \quad \,\, \,\text {s.t.} \,&27c,27d,27i \end{aligned}$$

$$\begin{aligned} (\text {DSP}^{rest}): \ \max&\sum _{j \in [J]} d_{tj} \theta ^+_{j} - \sum _{i \in [I]} M_{ti}^{s} {\Gamma }^{s}_{i} \\ \quad \text {s.t.}\,\,&27b,27e{-}27h \end{aligned}$$

We exploit this decomposition to disaggregate the optimality cuts in the master problem. Thus, we introduce additional variables $\zeta ^{upart}_{t}$ and $\zeta ^{rest}_{t}$ for $t \in [2,T]$ and obtain the following master problem:

$$\begin{aligned} \text {(MP)}: \ \max&\sum _{j \in [J]} d_{1j} \theta ^+_{1j} \, {-} \, \sum _{i \in [I]} \big ( M_{1i}^{u^+} {\Gamma }^{u^+}_{1i} + M_{1i}^{u^-} {\Gamma }^{u^-}_{1i} + M_{1i}^{s} {\Gamma }^{s}_{1i} \big )+ \sum _{t \in [2,T]} \zeta _{t} \end{aligned}$$

(25a)

$$\begin{aligned} \quad \quad \,\,\,\,\text {s.t.}\,\,&26b-26i, \end{aligned}$$

(25b)

$$\begin{aligned}&\zeta _{t} = \zeta ^{upart}_{t} + \zeta ^{rest}_{t},&t \in [2,T], \end{aligned}$$

(25c)

$$\begin{aligned}&(\zeta ^{upart}_{t}, {\Lambda }_{t11}, \ldots , {\Lambda }_{t K_tI} ) \in \mathcal {U}^{t} ,&t \in [2,T], \end{aligned}$$

(25d)

$$\begin{aligned}&(\zeta ^{rest}_{t}, {\Lambda }_{t11}, \ldots , {\Lambda }_{t K_tI} ) \in \mathcal {R}^{t} ,&t \in [2,T], \end{aligned}$$

(25e)

$$\begin{aligned}&\zeta ^{upart}_{t} \le 0,&t \in [2,T], \end{aligned}$$

(25f)

$$\begin{aligned}&\zeta ^{rest}_{t} \le \frac{1}{N} \sum _{n \in [N]} \sum _{j \in [J]} d_{tj}(\xi ^{t}_n) c_{tj}^{z},&t \in [2,T], \end{aligned}$$

(25g)

where $\mathcal {U}^{t}$ and $\mathcal {R}^{t}$ represent the optimality cuts derived from problems ($\text {DSP}^{upart}$) and ($\text {DSP}^{rest}$), respectively. Moreover, we introduce the upper bounds on the new auxiliary variables, which are derived from the subproblems $(\text {DSP}^{upart})$ and $(\text {DSP}^{rest})$.

The level method also uses a quadratic program for regularization which projects the previous iterate on the level set of the current approximation of the objective function. We use the following problem for this projection:

$$\begin{aligned} \text {(QP)}: \ \max&|| {\Lambda }- \hat{{\Lambda }} ||^2_2 \\ \quad \,\,\quad \,\,\,\,\text {s.t.}&28b{-}28g, \\&\sum _{j \in [J]} d_{1j} \theta ^+_{1j} \, {-} \, \sum _{i \in [I]} \big ( M_{1i}^{u^+} {\Gamma }^{u^+}_{1i} + M_{1i}^{u^-} {\Gamma }^{u^-}_{1i} + M_{1i}^{s} {\Gamma }^{s}_{1i} \big ) \ge L, \end{aligned}$$

where $\hat{\Lambda }$ and L denote the current ${\Lambda }$ solution (i.e., the previous iterate) and the level target, respectively. The optimal solution values of ${\Lambda }$ variables determine the next iterate.

The details of the level method are provided in Algorithm 1 where LB and UB denote lower bound and upper bound, respectively.