Stochastic dual dynamic integer programming

Abstract

Multistage stochastic integer programming (MSIP) combines the difficulty of uncertainty, dynamics, and non-convexity, and constitutes a class of extremely challenging problems. A common formulation for these problems is a dynamic programming formulation involving nested cost-to-go functions. In the linear setting, the cost-to-go functions are convex polyhedral, and decomposition algorithms, such as nested Benders’ decomposition and its stochastic variant, stochastic dual dynamic programming (SDDP), which proceed by iteratively approximating these functions by cuts or linear inequalities, have been established as effective approaches. However, it is difficult to directly adapt these algorithms to MSIP due to the nonconvexity of integer programming value functions. In this paper we propose an extension to SDDP—called stochastic dual dynamic integer programming (SDDiP)—for solving MSIP problems with binary state variables. The crucial component of the algorithm is a new reformulation of the subproblems in each stage and a new class of cuts, termed Lagrangian cuts, derived from a Lagrangian relaxation of a specific reformulation of the subproblems in each stage, where local copies of state variables are introduced. We show that the Lagrangian cuts satisfy a tightness condition and provide a rigorous proof of the finite convergence of SDDiP with probability one. We show that, under fairly reasonable assumptions, an MSIP problem with general state variables can be approximated by one with binary state variables to desired precision with only a modest increase in problem size. Thus our proposed SDDiP approach is applicable to very general classes of MSIP problems. Extensive computational experiments on three classes of real-world problems, namely electric generation expansion, financial portfolio management, and network revenue management, show that the proposed methodology is very effective in solving large-scale multistage stochastic integer optimization problems.

Appendix

Definition 3

A convex underestimator of $f: X\rightarrow \mathbb {R}$ is a convex function defined on $\text {conv}(X)$ that is majorized by f on X. The largest convex underestimator of f on $\text {conv}(X)$ is called the convex lower envelope of f.

Proof of Theorem 1

The graph of f, denoted as $F :=\{(x,y)\in \{0,1\}^n\times \mathbb {R} : y = f(x)\}$, is a finite set. Define $\Pi := \{(\alpha ,\beta ) \in \mathbb {R}^{n+1} : y \ge \alpha ^\top x + \beta , \; \forall (x,y)\in F\}$. Since F is a finite set, $\Pi$ is a nonempty polyhedron. Define a function $g(x):=\max _{(\alpha ,\beta )\in \Pi } \{\alpha ^\top x + \beta \}$ on $C_n :=[0,1]^n$. First, we show that g(x) is a well-defined convex piecewise linear function with finite value, i.e. $g(x)<\infty$ for all $x\in C_n$ and $g(x) = f(x)$ for all binary point $x\in \{0,1\}^n$. Therefore, g is a convex underestimator of f on $C_n$. Then we show that g(x) is the tightest convex underestimator, i.e. the convex lower envelope of f.

Consider the following linear program

$$\begin{aligned} (P)\quad \max _{\alpha ,\beta }\quad&{x}^\top \alpha + \beta \\ \text {s.t.}\quad&(\hat{x}^i)^\top \alpha + \beta \le \hat{y}^i,\quad \forall \hat{x}^i\in \{0,1\}^n, \; \hat{y}^i = f(\hat{x}^i), \end{aligned}$$

where ${x}\in C_n$, and its dual

$$\begin{aligned} (D)\quad \min \quad&\sum _{i=1}^N y^i \lambda ^i \\ \text {s.t.}\quad&\hat{X}\lambda = x \\&e^\top \lambda = 1 \\&\lambda \ge 0, \end{aligned}$$

where $\hat{X} = [\hat{x}^1, \ldots ,\hat{x}^N]$ contains all the binary vectors in $\{0,1\}^n$ as its columns and $N=2^n$. Since $C_n$ is the convex hull of $\{0,1\}^n$, the dual problem (D) is always feasible and bounded for any $x\in C_n$, which implies $g(x)<\infty$ for all $x\in C_n$. If $x \in \{0,1\}^n$, i.e. $x=\hat{x}^i$ for some $i=1, \ldots ,N$, then the feasible region of the dual problem has a unique solution $\lambda = e_i$, namely only the ith entry of $\lambda$ is 1 and all other entries of $\lambda$ are 0. Therefore, $g(x)=f(x)$ for all $x\in \{0,1\}^n$. Since $\Pi$ is a polyhedron, g(x) is a convex piecewise linear function with a finite number of linear pieces, corresponding to extreme points of $\Pi$.

Since any convex underestimator h of f on the open box $(0,1)^n$ can be expressed as a pointwise maximum of affine functions $l(x) = \alpha ^\top x + \beta$, where the halfspace $y\ge \alpha ^\top x + \beta$ contains F, then $h(x) = \max _{(\alpha ,\beta )\in S} \{\alpha ^\top x + \beta \}$ for some subset $S\subseteq \Pi$. Therefore, $g(x)\ge h(x)$ for all $x\in (0,1)^n$. On the boundary points $x\in \{0,1\}^n$, since we already have $g(x) = f(x)\ge h(x)$, thus, $g(x)\ge h(x)$ for all $x\in C_n$. Therefore, g(x) is the convex lower envelope of f. This completes the proof. $\square$

Remark

A key step in the proof of Theorem 1 uses the simple fact that if $x\in \{0,1\}^n$ and x is the convex combination of a set of binary vectors, then x coincides with one of these binary vectors. This simple fact underlies a similar argument used to prove the key strong duality result in Sect. 4.3 Theorem 3.

Proof of Theorem 4

Consider an MSIP with $d := d_1 + d_2$ mixed-integer state variables per node:

$$\begin{aligned} \begin{array}{rll} \displaystyle {\min _{x_n, y_n} } &{} \displaystyle {\sum _{n \in \mathcal {T}} }p_n f_n(x_n,y_n) &{} \\ \mathrm{s.t.} &{} (x_{a(n)}, x_n, y_n) \in X_n &{} \forall \ n \in \mathcal {T}\\ &{} x_n \in \mathbb {Z}_+^{d_1}\times \mathbb {R}_+^{d_2} &{} \forall \ n \in \mathcal {T}. \end{array} \end{aligned}$$

(7.1)

Since the state variables are bounded by (A1), we can assume that $x_{n} \in [0,U]^d$ for some positive integer U for all $n \in \mathcal {T}$.

We approximate (7.1) as follows. For an integer state variable $x \in \{0, \ldots ,U\}$, we substitute by its binary expansion: $x = \sum _{i=1}^{\kappa } 2^{i-1} \lambda _i$ where $\lambda _i \in \{0,1\}$ and $\kappa = \lfloor \log _2 U \rfloor + 1$. For a continuous state variable $x \in [0,U]$, we approximate it by binary approximation to a precision of $\epsilon \in (0,1)$, i.e. $x = \sum _{i=1}^{\kappa } 2^{i-1} \epsilon \lambda _i$ where $\lambda _i \in \{0,1\}$ and $\kappa = \lfloor \log _2 (U/\epsilon ) \rfloor + 1$ (see e.g., [35]). Note that $|x - \sum _{i=1}^{\kappa } 2^{i-1} \epsilon \lambda _i | \le \epsilon$. The total number k of binary variables introduced to approximate the d state variables thus satisfies $k \le d (\lfloor \log _2(U/\epsilon ) \rfloor + 1)$. We then have the following approximating MSIP with binary variables $\lambda _n\in \{0,1\}^k$

$$\begin{aligned} \begin{array}{rll} \displaystyle {\min _{\lambda _n, y_n} } &{} \displaystyle {\sum _{n \in \mathcal {T}} }p_n f_n(A\lambda _n,y_n) &{} \\ \mathrm{s.t.} &{} (A\lambda _{a(n)}, A\lambda _n, y_n) \in X_n &{} \forall \ n \in \mathcal {T}\\ &{} \lambda _n \in \{0,1\}^k &{} \forall \ n \in \mathcal {T}, \end{array} \end{aligned}$$

(7.2)

where the $d \times k$ matrix A encodes the coefficients of the binary expansion.

Recall that the local variables are mixed integer, i.e. $y_n = (u_n,v_n)$ with $u_n \in \mathbb {Z}_+^{\ell _1}$ and $v_n \in \mathbb {R}_+^{\ell _2}$. Given $x := \{x_n\in \mathbb {Z}^{d_1}\times \mathbb {R}^{d_2}\}_{n\in \mathcal {T}}$, let

$$\begin{aligned}\begin{array}{ll} \phi (x) &{} := \displaystyle {\min _{u,v}}\left\{ \sum _{n\in \mathcal {T}}f_n(x_n,(u_n,v_n)): (x_{a(n)}, x_n, (u_n,v_n))\in X_n,~\forall n\in \mathcal {T}\right\} \\ &{} = \displaystyle {\sum _{n\in \mathcal {T}} \min _{u_n,v_n}}\left\{ f_n(x_n,(u_n,v_n)): (x_{a(n)}, x_n, (u_n,v_n))\in X_n\right\} \\ &{} = \displaystyle {\sum _{n\in \mathcal {T}} \min _{u_n\in {\mathcal {U}}_n}}\left\{ \psi _n(x_{a(n)},x_n,u_n)\right\} ,\\ \end{array} \end{aligned}$$

where

$$\begin{aligned} \psi _n(x_{a(n)},x_n,u_n) = \min _{v_n \in \mathbb {R}_+^{\ell _2}}\left\{ f_n(x_n, (u_n, v_n)): (x_{a(n)}, x_n, (u_n,v_n))\in X_n\right\} , \end{aligned}$$

and ${\mathcal {U}}_n$ is the finite set of integer values the local variable $u_n$ can take. By the compactness assumption (A1) and the complete continuous recourse assumption (A2), the function $\psi _n$ is the value function of a linear program that is feasible and bounded for all values of $(x_{a(n)},x_n,u_n)$. By Hoffman’s lemma [43], there exists a constant $C_n(u_n)$ which is dependent on the data defining $(f_n,X_n)$ and $u_n$, such that $\psi _n(x_{a(n)},x_n,u_n)$ is Lipschitz continuous with respect to $(x_{a(n)},x_n)$ with this constant. It follows that $\phi (x)$ is Lipschitz continuous with respect to x with constant $C = \sum _{n\in T}\max _{u_n\in U_n}C_n(u_n)$, i.e.,

$$\begin{aligned} |\phi (x) - \phi (x')| \le C \Vert x - x'\Vert \;\; \forall \ x, x'. \end{aligned}$$

Let $(\tilde{\lambda }, \tilde{y})$ be an optimal solution to problem (7.2) and $v_2$ be its optimal value. Define $\tilde{x}_n = A\tilde{\lambda }_n$ for all $n\in \mathcal {T}$, then $(\tilde{x}, \tilde{y})$ is a feasible solution to (7.1) and has the objective value of $v_2$. From the definition of $\phi$ we have that $v_2 = \phi (\tilde{x})$. Now let $(\hat{x}, \hat{y})$ be an optimal solution of (7.1) and $v_1$ be its optimal value. Note that $v_1 = \phi (\hat{x})$. Let us construct a solution $(\hat{\lambda },\hat{y}')$ such that

$$\begin{aligned} \Vert \hat{x} - A\hat{\lambda }\Vert \le \sqrt{|\mathcal {T}|d} \epsilon , ~\text { and }~\hat{y}'_n = \displaystyle {{\text {argmin}}_{y_n}}\left\{ f(A\hat{\lambda }_{a(n)}, A\hat{\lambda }_n, y_n): (A\hat{\lambda }_{a(n)}, A\hat{\lambda }_n, y_n) {\in } X_n \right\} . \end{aligned}$$

Then $(\hat{\lambda },\hat{y}')$ is clearly a feasible solution to (7.2) and has the objective value $\phi (A\hat{\lambda })$. Thus we have the following inequalities

$$\begin{aligned} \phi (\hat{x}) \le \phi (\tilde{x}) \le \phi (A\hat{\lambda }). \end{aligned}$$

Thus

$$\begin{aligned} 0 \le \phi (\tilde{x}) - \phi (\hat{x}) \le | \phi (A\hat{\lambda }) - \phi (\hat{x}) | \le C \Vert A\hat{\lambda } - \hat{x}\Vert \le C\sqrt{|\mathcal {T}|d} \epsilon = C' \sqrt{d}\epsilon , \end{aligned}$$

where $C' = C\sqrt{|\mathcal {T}|}$. By choosing $\epsilon = \varepsilon /C'\sqrt{d}$ and $M = UC'$ we have that $(\tilde{x}, \tilde{y})$ is a $\varepsilon$-optimal solution of (7.1) and $k \le d (\lfloor \log _2(M\sqrt{d}/\varepsilon ) \rfloor + 1)$ as desired. $\square$