Multistage distributionally robust mixed-integer programming with decision-dependent moment-based ambiguity sets

Abstract

We study multistage distributionally robust mixed-integer programs under endogenous uncertainty, where the probability distribution of stage-wise uncertainty depends on the decisions made in previous stages. We first consider two ambiguity sets defined by decision-dependent bounds on the first and second moments of uncertain parameters and by mean and covariance matrix that exactly match decision-dependent empirical ones, respectively. For both sets, we show that the subproblem in each stage can be recast as a mixed-integer linear program (MILP). Moreover, we extend the general moment-based ambiguity set in Delage and Ye (Oper Res 58(3):595–612, 2010) to the multistage decision-dependent setting, and derive mixed-integer semidefinite programming (MISDP) reformulations of stage-wise subproblems. We develop methods for attaining lower and upper bounds of the optimal objective value of the multistage MISDPs, and approximate them using a series of MILPs. We deploy the Stochastic Dual Dynamic integer Programming (SDDiP) method for solving the problem under the three ambiguity sets with risk-neutral or risk-averse objective functions, and conduct numerical studies on multistage facility-location instances having diverse sizes under different parameter and uncertainty settings. Our results show that the SDDiP quickly finds optimal solutions for moderate-sized instances under the first two ambiguity sets, and also finds good approximate bounds for the multistage MISDPs derived under the third ambiguity set. We also demonstrate the efficacy of incorporating decision-dependent distributional ambiguity in multistage decision-making processes.

Appendices

Appendix

Reformulations of N-DDDR having continuous supports

A continuous version of the Type 1 ambiguity set ${\mathcal {P}_{t+1}^{D_1}}(\varvec{x}_{t})$ in Sect. 3.1 is given by

$$\begin{aligned} { \mathcal {P}_{t+1}^{C_1}} (\varvec{x}_{t})&:=\left\{ P\in \mathcal {M}(\varXi _{t+1},\mathcal {F}_{t+1})\ |\ \underline{\nu }(\varvec{x}_t) \le P\le \bar{\nu }(\varvec{x}_t),\right. \nonumber \\&\quad \left. \int _{{\varXi }_{t+1}}f_s(\varvec{\xi }_{t+1})P(d\varvec{\xi }_{t+1})\in [l_s(\varvec{x}_{t}),u_s(\varvec{x}_{t})],\ \forall s\in [m]\right\} ,\ \end{aligned}$$

(A.1)

where $\mathcal {M}(\varXi _{t+1},\mathcal {F}_{t+1})$ represents the set of all positive measures defined on $(\varXi _{t+1},\mathcal {F}_{t+1})$, and $\underline{\nu }(\varvec{x}_t),\ \bar{\nu }(\varvec{x}_t)\in \mathcal {M}(\varXi _{t+1},\mathcal {F}_{t+1})$ are two given measures that are lower and upper bounds for the true probability measure, respectively. To ensure that P is a probability distribution, let $l_1(\varvec{x}_t)=u_1(\varvec{x}_t)=f_1(\varvec{\xi }_{t+1})=1$ (see (5a) for more details).

Let $\varXi _{t+1}$ be a closed and bounded set in the Euclidean space, and the probability measures $P,\ \underline{\nu },\ \bar{\nu }$ be defined on the measurable space $(\varXi _{t+1},\mathcal {F}_{t+1})$, where the $\sigma$-algebra $\mathcal {F}_{t+1}$ contains all singleton subsets, i.e., $\{\xi \}\in \mathcal {F}_{t+1}$ for all $\xi \in \varXi _{t+1}$. Then, based on the ambiguity set ${ \mathcal {P}_{t+1}^{C_1}} (\varvec{x}_{t})$ in (A.1), we describe a reformulation of the Bellman equation (2) as an analogy to Theorem 1 for the continuous support case.

Theorem A.1

If for any feasible $\varvec{x}_t\in \hat{X}_t$, the ambiguity set (A.1) has a non-empty relative interior, then the Bellman equation (2) can be reformulated as $Q_t(\varvec{x}_{t-1},\varvec{\xi }_t)=$

$$\begin{aligned} \min _{\varvec{\alpha },\varvec{\beta },{\underline{\gamma },\bar{\gamma },}\varvec{x}_t,\varvec{y}_t}\quad&g_t(\varvec{x}_t,\varvec{y}_t)-{\varvec{\alpha }}^{\mathsf T} \varvec{l}(\varvec{x}_t)+{\varvec{\beta }}^{\mathsf T} \varvec{u}(\varvec{x}_t)\\&\quad -\int _{\varvec{\xi }_{t+1}\in \varXi _{t+1}}\underline{\gamma }(\varvec{\xi }_{t+1})\underline{\nu }(\varvec{x}_t,\varvec{\xi }_{t+1})d\varvec{\xi }_{t+1}\\&\quad + \int _{\varvec{\xi }_{t+1}\in \varXi _{t+1}}\bar{\gamma }(\varvec{\xi }_{t+1})\bar{\nu }(\varvec{x}_t,\varvec{\xi }_{t+1})d\varvec{\xi }_{t+1}\\ \text {s.t.}\quad&(-\varvec{\alpha }+\varvec{\beta })^{\mathsf T} \varvec{f}(\varvec{\xi }_{t+1}) {-\underline{\gamma }(\varvec{\xi }_{t+1})+\bar{\gamma }(\varvec{\xi }_{t+1})}\ge Q_{t+1}(\varvec{x}_t,\varvec{\xi }_{t+1}),\ \forall \varvec{\xi }_{t+1}\in \varXi _{t+1},\\&(\varvec{x}_t,\varvec{y}_t)\in X_t(\varvec{x}_{t-1},\varvec{\xi }_t),\\&\varvec{\alpha },\ \varvec{\beta },\ {\varvec{\underline{\gamma }}(\varvec{\xi }_{t+1}),\ \varvec{\bar{\gamma }}(\varvec{\xi }_{t+1})}\ge 0, \ \forall \varvec{\xi }_{t+1}\in \varXi _{t+1}. \end{aligned}$$

Next, a continuous version of the Type 2 ambiguity set ${ \mathcal {P}_{t+1}^{D_2}} (\varvec{x}_{t})$ in (10) is given by

$$\begin{aligned} { \mathcal {P}^{C_2}_{t+1}}(\varvec{x}_t)&:=\Biggl \{P\in \mathcal {M}(\varXi _{t+1},\mathcal {F}_{t+1})\ | \mathbb {E}_P[\varvec{\xi }_{t+1}]=\varvec{\mu }(\varvec{x}_t), \end{aligned}$$

(A.2a)

$$\begin{aligned}&\qquad \mathbb {E}_P[(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}]=\varvec{\varSigma }(\varvec{x}_t)\Biggr \}, \end{aligned}$$

(A.2b)

Then, the following result is an analogy to Theorem 2 based on the continuous ambiguity set (A.2).

Theorem A.2

If for any feasible $\varvec{x}_t\in \hat{X}_t$, the ambiguity set (A.2) has a non-empty relative interior, then the Bellman equation (2) can be reformulated as

$$\begin{aligned} Q_t(\varvec{x}_{t-1},\varvec{\xi }_t)=\min _{\varvec{x}_t,\varvec{y}_t,s,\varvec{u},\varvec{Y}}\quad&g_t(\varvec{x}_t,\varvec{y}_t)+s+\varvec{u}^{\mathsf T}\varvec{\mu }(\varvec{x}_t)+\varvec{\varSigma }(\varvec{x}_t)\cdot \varvec{Y} \end{aligned}$$

(A.3a)

$$\begin{aligned} \text {s.t.}\quad&s+\varvec{u}^{\mathsf T}\varvec{\xi }_{t+1} +(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\cdot \varvec{Y}\nonumber \\&\quad \ge Q_{t+1}(\varvec{x}_t,\varvec{\xi _{t+1}}),\nonumber \\&\qquad \forall \varvec{\xi }_{t+1}\in \varXi _{t+1},\nonumber \\&(\varvec{x}_t,\varvec{y}_t)\in X_t(\varvec{x}_{t-1},\varvec{\xi }_t). \end{aligned}$$

(A.3b)

Finally, a continuous version of the Type 3 ambiguity set ${ \mathcal {P}_{t+1}^{D_3}}(\varvec{x}_{t})$ in (14) is given by

$$\begin{aligned} { \mathcal {P}_{t+1}^{C_3}}(\varvec{x}_t)&:=\Biggl \{P\in \mathcal {M}(\varXi _{t+1},\mathcal {F}_{t+1})\ | (\mathbb {E}_P[\varvec{\xi }_{t+1}]-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\varvec{\varSigma }(\varvec{x}_t)^{-1}(\mathbb {E}_P[\varvec{\xi }_{t+1}]-\varvec{\mu }(\varvec{x}_t))\le \gamma , \end{aligned}$$

(A.4a)

$$\begin{aligned}&\qquad \mathbb {E}_P[(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}]\preceq \eta \varvec{\varSigma }(\varvec{x}_t)\Biggr \}. \end{aligned}$$

(A.4b)

We present a reformulation of the Bellman equation (2) in the following theorem that is an analogy to Theorem 3, but given the continuous ambiguity set (A.4).

Theorem A.3

Suppose that the Slater’s constraint qualification conditions are satisfied, i.e., for any feasible $\varvec{x}_t\in \hat{X}_t$, there exists a probability measure $P\in \mathcal {M}(\varXi _{t+1},\mathcal {F}_{t+1})$ such that $(\mathbb {E}_P[\varvec{\xi }_{t+1}]-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\varvec{\varSigma }(\varvec{x}_t)^{-1}(\mathbb {E}_P[\varvec{\xi }_{t+1}]-\varvec{\mu }(\varvec{x}_t))< \gamma$, and $\mathbb {E}_P[(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}]\prec \eta \varvec{\varSigma }(\varvec{x}_t)$. Using the ambiguity set defined in (A.4), the Bellman equation (2) can be recast as

$$\begin{aligned} Q_t(\varvec{x}_{t-1},\varvec{\xi }_t)=\min _{\varvec{x}_t,\varvec{y}_t,s,\varvec{Z},\varvec{Y}}\quad&g_t(\varvec{x}_t,\varvec{y}_t)+s+\varvec{\varSigma }(\varvec{x}_t)\cdot \varvec{z}_1 -2\varvec{\mu }(\varvec{x}_t)^{\mathsf T}\varvec{z}_2+\gamma z_3 +\eta \varvec{\varSigma }(\varvec{x}_t)\cdot \varvec{Y} \end{aligned}$$

(A.5a)

$$\begin{aligned} \text {s.t.}\quad&s-2\varvec{z}_2^{\mathsf T}\varvec{\xi }_{t+1}+(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\cdot Y\ge Q_{t+1}(\varvec{x}_t,\varvec{\xi }_{t+1}),\nonumber \\&\qquad \forall \varvec{\xi }_{t+1}\in \varXi _{t+1}, \end{aligned}$$

(A.5b)

$$\begin{aligned}&\varvec{Z}=\begin{pmatrix} \varvec{z}_1&{}\varvec{z}_2\\ \varvec{z}_2^{\mathsf T}&{}z_3\end{pmatrix}\succeq 0,\ \varvec{Y}\succeq 0, \end{aligned}$$

(A.5c)

$$\begin{aligned}&(\varvec{x}_t,\varvec{y}_t)\in X_t(\varvec{x}_{t-1},\varvec{\xi }_t). \end{aligned}$$

(A.5d)

The proof of Theorem A.1 is similar to the proof in [24] for the two-stage continuous-support case, and we omit its details. We provide detailed proofs for Theorems A.2 and A.3 in “Appendix C”. Note that the above three reformulations for ambiguity sets with continuous support are semi-infinite programs and thus cannot be optimized directly.

Risk-averse multistage DRO with endogenous uncertainty

We can extend N-DDDR in (1) to a more general setting. Previously, the robust counterpart chooses the worst-case distribution P from a risk-neutral aspect using expectation to measure the uncertain cost over multiple stages. However, a decision maker may measure the worst-case distribution in a risk-averse fashion, and we accordingly replace the expectations by coherent risk measures $\rho _t, \ \forall t=2,\ldots ,T$. The corresponding risk-averse multistage decision-dependent DRO model is:

$$\begin{aligned} {\mathbf{A}}\mathrm{-}{\mathbf{DDDR:}}&~\nonumber \\ \quad \min _{(\varvec{\varvec{x}_1},\varvec{y}_1)\in X_1}&\Big \{g_1(\varvec{x}_1,\varvec{y}_1)+\max _{P_2\in \mathcal {P}_2(\varvec{x}_1)}\rho _2\Big [\min _{(\varvec{x}_2,\varvec{y}_2)\in X_2(\varvec{x}_1,\varvec{\xi }_2)}g_2(\varvec{x}_2,\varvec{y}_2)+\cdots \nonumber \\&+\max _{P_{t}\in \mathcal {P}_{t}(\varvec{x}_{t-1})}\rho _{t}\Big [\min _{(\varvec{x}_{t},\varvec{y}_{t})\in X_{t}(\varvec{x}_{t-1},\varvec{\xi }_{t})}g_{t}(\varvec{x}_{t},\varvec{y}_{t})+\cdots \nonumber \\&+\max _{P_T\in \mathcal {P}_T(\varvec{x}_{T-1})}\rho _{T}\Big [\min _{(\varvec{x}_T,\varvec{y}_T)\in X_T(\varvec{x}_{T-1},\varvec{\xi }_T)}g_T(\varvec{x}_T,\varvec{y}_T)\Big ]\Big \}. \end{aligned}$$

(B.6)

We consider a special class of coherent risk measures, which is a convex combination of expectation and Conditional Value-at-Risk (CVaR) [34]:

$$\begin{aligned} \rho _t(Z)=(1-\lambda _t)\mathbb {E}[Z]+\lambda _t \text {CVaR}_{\alpha _t}[Z], \end{aligned}$$

where $\lambda _t\in [0,1]$ is a parameter that balances the expectation and CVaR measure at $\alpha _t\in (0,1)$ risk level. This risk measure is more general than expectation and it becomes the risk-neutral case when $\lambda _t=0$.

The Bellman equations for A-DDDR (B.6) then become:

$$\begin{aligned} Q_1=\min _{(\varvec{x}_1,\varvec{y}_1)\in X_1} g_1(\varvec{x}_1,\varvec{y}_1)+\max _{P_2\in \mathcal {P}_2(\varvec{x}_1)}\rho _2[Q_2(\varvec{x}_1,\varvec{\xi }_2)], \end{aligned}$$

where for $t=2,\ldots ,T-1$,

$$\begin{aligned} Q_t(\varvec{x}_{t-1},\varvec{\xi }_t)=\min _{(\varvec{x}_t,\varvec{y}_t)\in X_t(\varvec{x}_{t-1},\varvec{\xi }_t)} g_t(\varvec{x}_t,\varvec{y}_t)+\max _{P_{t+1}\in \mathcal {P}_{t+1}(\varvec{x}_t)}\rho _{t+1}[Q_{t+1}(\varvec{x}_t,\varvec{\xi }_{t+1})], \end{aligned}$$

(B.7)

and

$$\begin{aligned} Q_T(\varvec{x}_{T-1},\varvec{\xi }_T)=\min _{(\varvec{x}_T,\varvec{y}_T)\in X_T(\varvec{x}_{T-1},\varvec{\xi }_T)} g_T(\varvec{x}_T,\varvec{y}_T). \end{aligned}$$

Following the results by [33], CVaR can be attained by solving the following optimization problem:

$$\begin{aligned} \text {CVaR}_{\alpha _t}[Z]:=\inf _{\eta \in \mathbb {R}}\left\{ \eta +\frac{1}{1-\alpha _t}\mathbb {E}[Z-\eta ]_{+}\right\} , \end{aligned}$$

where $[a]_{+}:=\max \{a,0\},$ and $\eta$ is an auxiliary variable. To linearize $[Z-\eta ]_{+}$, we replace it by a variable m with two additional constraints: $m\ge 0,\ m\ge Z-\eta$.

Recall that by assumption, every $P_{t+1}\in \mathcal {P}_{t+1}(\varvec{x}_{t})$ has a decision-independent finite support $\varXi _{t+1}:=\{\varvec{\xi }_{t+1}^k\}_{k=1}^K,\ \forall \varvec{x}_{t}\in X_{t}$ for a fixed K and all $t\in [T-1]$. Each realization $k\in [K]$ is associated with probability $p_k$, and therefore the inner maximization problem $\max _{P_{t+1}\in \mathcal {P}_{t+1}(\varvec{x}_t)}\rho _{t+1}[Q_{t+1}(\varvec{x}_t,\varvec{\xi }_{t+1})]$ in (B.7) can be reformulated as

$$\begin{aligned} \max _{P_{t+1}\in \mathcal {P}_{t+1}(\varvec{x}_t)}\min _{m,\eta }\quad&\lambda _{t+1}\eta +\sum _{k=1}^Kp_k(\frac{\lambda _{t+1}}{1-\alpha _{t+1}}m+(1-\lambda _{t+1})Q_{t+1}^k) \end{aligned}$$

(B.8a)

$$\begin{aligned} \text {s.t.}\quad&m+\eta \ge Q_{t+1}^k,\ \forall k\in [K], \end{aligned}$$

(B.8b)

$$\begin{aligned}&m\ge 0,\ \forall k\in [K]. \end{aligned}$$

(B.8c)

We further simplify the notation of the recursive function $Q_{t+1}(\varvec{x}_t,\varvec{\xi }_{t+1}^k)$ as $Q_{t+1}^k$. Associating dual variables $q_k$ with constraints (B.8b) and applying strong duality result, we have

$$\begin{aligned}&\max _{P_{t+1}\in \mathcal {P}_{t+1}(\varvec{x}_t)}\Biggl \{(1-\lambda _{t+1})\sum _{k=1}^Kp_kQ_{t+1}^k+\max _{\varvec{q}}\sum _{k=1}^Kq_kQ_{t+1}^k \end{aligned}$$

(B.9a)

$$\begin{aligned}&\quad \text {s.t.}\quad q_k\le p_k\frac{\lambda _{t+1}}{1-\alpha _{t+1}},\ \forall k\in [K], \end{aligned}$$

(B.9b)

$$\begin{aligned}&\quad \sum _{k=1}^Kq_k=\lambda _{t+1}, \end{aligned}$$

(B.9c)

$$\begin{aligned}&\quad q_k\ge 0,\ \forall k\in [K].\Biggr \} \end{aligned}$$

(B.9d)

Merging the two layers of maximization problems, for each $t\in [T-1]$, we solve

$$\begin{aligned} \max _{\varvec{p,q}}\quad&(1-\lambda _{t+1})\sum _{k=1}^K p_kQ_{t+1}^k+q_kQ_{t+1}^k \end{aligned}$$

(B.10a)

$$\begin{aligned} \text {s.t.}\quad&\text{(B-10b), } \text{(B-10c) },\nonumber \\&\varvec{p}\in \mathcal {P}_{t+1}(\varvec{x}_t), \end{aligned}$$

(B.10b)

$$\begin{aligned}&q_k\ge 0,\ \forall k\in [K] . \end{aligned}$$

(B.10c)

In the following subsections, we present reformulations of A-DDDR in (B.6) under the three types of ambiguity sets mentioned in Sect. 3.

1.1 Solving A-DDDR under Type 1 ambiguity set

Using the ambiguity set defined in (3), the inner maximization problem (B.10) can be recast as

$$\begin{aligned} \max _{\varvec{p,q}}\quad&(1-\lambda _{t+1})\sum _{k=1}^Kp_kQ_{t+1}^k+q_kQ_{t+1}^k \end{aligned}$$

(B.11a)

$$\begin{aligned} \text {s.t.}\quad&\text{(B-10b), } \text{(B-10c), } \text{(C-17b)-(C-17e) }\nonumber \\&p_k,\ q_k\ge 0,\ \forall k\in [K] . \end{aligned}$$

(B.11b)

Theorem B.4

If for any feasible $\varvec{x}_t\in \hat{X}_t$, problem (B.11) is feasible, then the Bellman equation (B.7) can be reformulated as $Q_t(\varvec{x}_{t-1},\varvec{\xi }_t)=$

$$\begin{aligned} \min _{\varvec{\alpha },\varvec{\beta },\varvec{x}_t,\varvec{y}_t} \quad&g_t(\varvec{x}_t,\varvec{y}_t)+\lambda _{t+1}\theta -{\varvec{\alpha }}^{\mathsf T} \varvec{l}(\varvec{x}_t)+{\varvec{\beta }}^{\mathsf T} \varvec{u}(\varvec{x}_t){-\underline{\varvec{\gamma }}^{\mathsf T}\underline{\varvec{p}}(\varvec{x}_t) + \bar{\varvec{\gamma }}^{\mathsf T}\bar{\varvec{p}}(\varvec{x}_t)} \end{aligned}$$

(B.12a)

$$\begin{aligned} \text {s.t.}\quad&\pi _k+\theta \ge Q_{t+1}^k,\ \forall k\in [K], \end{aligned}$$

(B.12b)

$$\begin{aligned}&-\frac{\lambda _{t+1}}{1-\alpha _{t+1}}\pi _k+(-\varvec{\alpha }+\varvec{\beta })^{\mathsf T} \varvec{f}(\varvec{\xi }_{t+1}^k){-\underline{\gamma }_k+\bar{\gamma }_k}\ge (1-\lambda _{t+1}) Q_{t+1}^k, \nonumber \\&\quad \forall k\in [K], \end{aligned}$$

(B.12c)

$$\begin{aligned}&(\varvec{x}_t,\varvec{y}_t)\in X_t(\varvec{x}_{t-1},\varvec{\xi }_t), \end{aligned}$$

(B.12d)

$$\begin{aligned}&\pi _k,\ \varvec{\alpha },\ \varvec{\beta },\ {\varvec{\underline{\gamma }},\ \varvec{\bar{\gamma }}}\ge 0. \end{aligned}$$

(B.12e)

The proof of Theorem B.4 is similar to the one of Theorem 1 in “Appendix C”, where the only difference is that we introduce the two more dual variables $\pi _k$ and $\theta$, associated with constraints (B.9b) and (B.9c), respectively.

Notice here when $\lambda _{t+1}=0$, model (B.12) reduces to the risk-neutral case (4). This reformulation also has similar computational complexity as model (4) in Theorem 1. Therefore, with the same specific ambiguity set considered in (6) in Sect. 3, we can apply McCormick envelopes to obtain a multistage stochastic MILP and deploy SDDiP to solve it.

1.2 Solving A-DDDR under Type 2 ambiguity set

Under Type 2 ambiguity set in (10), the inner maximization problem (B.10) can be recast as

$$\begin{aligned} \max _{\varvec{p,q}}\quad&(1-\lambda _{t+1})\sum _{k=1}^K p_kQ_{t+1}^k+q_kQ_{t+1}^k \end{aligned}$$

(B.13a)

$$\begin{aligned} \text {s.t.}\quad&\text{(B-10b), } \text{(B-10c), } \text{(C-18b)-(C-18d) }\nonumber \\&p_k,\ q_k\ge 0, \quad \forall k\in [K]. \end{aligned}$$

(B.13b)

Theorem B.5

If for any feasible $\varvec{x}_t\in \hat{X}_t$, problem (B.13) is feasible, then the Bellman equation (B.7) can be reformulated as $Q_t(\varvec{x}_{t-1},\varvec{\xi }_t)=$

$$\begin{aligned} \min _{\varvec{x}_t,\varvec{y}_t,s,\varvec{u},\varvec{Y}} \quad&g_t(\varvec{x}_t,\varvec{y}_t)-\lambda _{t+1}\theta +s+\varvec{u}^{\mathsf T}\varvec{\mu }(\varvec{x}_t)+\varvec{\varSigma }(\varvec{x}_t)\cdot \varvec{Y} \end{aligned}$$

(B.14a)

$$\begin{aligned} \text {s.t.}\quad&s+\varvec{u}^{\mathsf T}\varvec{\xi }_{t+1}^k +(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\cdot \varvec{Y}\nonumber \\&\quad -\pi _k\frac{\lambda _{t+1}}{1-\alpha _{t+1}}\ge (1-\lambda _{t+1})Q_{t+1}^k,\nonumber \\&\qquad \forall k\in [K], \end{aligned}$$

(B.14b)

$$\begin{aligned}&\pi _k-\theta \ge Q_{t+1}^k,\quad \forall k\in [K], \end{aligned}$$

(B.14c)

$$\begin{aligned}&\pi _k\ge 0 ,\quad \forall k\in [K], \end{aligned}$$

(B.14d)

$$\begin{aligned}&(\varvec{x}_t,\varvec{y}_t)\in X_t(\varvec{x}_{t-1},\varvec{\xi }_t). \end{aligned}$$

(B.14e)

The proof of Theorem B.5 is similar to the one of Theorem 2 in “Appendix C”, where the only difference is that we introduce the two more dual variables $\pi _k$ and $\theta$, associated with constraints (B.9b) and (B.9c), respectively.

Notice here when $\lambda _{t+1}=0$, the risk-averse model (B.14) reduces to the risk-neutral case (11). We can apply McCormick envelopes to get a multistage stochastic MILP and use SDDiP algorithm to attain optimal solutions as in Sect. 3.2.

1.3 Solving A-DDDR under Type 3 ambiguity set

Given Type 3 ambiguity set defined in (14), the inner maximization problem (B.10) can be recast as

$$\begin{aligned} \max _{\varvec{p,q}}\quad&(1-\lambda _{t+1})\sum _{k=1}^Kp_kQ_{t+1}^k+q_kQ_{t+1}^k\\ \text {s.t.}\quad&\text{(B-10b), } \text{(B-10c), } \text{(C-20b)(C-20e) }\nonumber \\&p_k\ge 0,\ q_k\ge 0, \quad \forall k\in [K]. \end{aligned}$$

Theorem B.6

Suppose that Slater’s constraint qualification conditions are satisfied, i.e., for any feasible $\varvec{x}_t\in \hat{X}_t$, there exists a vector $p = (p_1,p_2,\ldots ,p_K)^{\mathsf T}$ such that $\sum _{k=1}^Kp_{k}=1$, $(\sum _{k=1}^K p_k\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\varvec{\varSigma }(\varvec{x}_t)^{-1}(\sum _{k=1}^K p_k\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))< \gamma$, and $\sum _{k=1}^Kp_k(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\prec \eta \varvec{\varSigma }(\varvec{x}_t)$. Using the ambiguity set defined in (14), the Bellman equation (B.7) can be recast as $Q_t(\varvec{x}_{t-1},\varvec{\xi }_t)=$

$$\begin{aligned} \min _{\varvec{x}_t,\varvec{y}_t,s,\varvec{Z},\varvec{Y}}\quad&g_t(\varvec{x}_t,\varvec{y}_t)-\lambda \theta +s+\varvec{\varSigma }(\varvec{x}_t)\cdot \varvec{z}_1 -2\varvec{\mu }(\varvec{x}_t)^{\mathsf T}\varvec{z}_2+\gamma z_3 +\eta \varvec{\varSigma }(\varvec{x}_t)\cdot \varvec{Y} \end{aligned}$$

(B.15a)

$$\begin{aligned} \text {s.t.}\quad&s-2\varvec{z}_2^{\mathsf T}\varvec{\xi }_{t+1}^k+(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\cdot Y-\pi _k\frac{\lambda _{t+1}}{1-\alpha _{t+1}}\ge (1-\lambda _{t+1})Q_{t+1}^k,\nonumber \\&\qquad \forall k\in [K], \end{aligned}$$

(B.15b)

$$\begin{aligned}&\varvec{Z}=\begin{pmatrix} \varvec{z}_1&{}\varvec{z}_2\\ \varvec{z}_2^{\mathsf T}&{}z_3\end{pmatrix}\succeq 0,\ \varvec{Y}\succeq 0, \end{aligned}$$

(B.15c)

$$\begin{aligned}&\pi _k - \theta \ge Q_{t+1}^k,\ \pi _k\ge 0 ,\quad \forall k\in [K], \end{aligned}$$

(B.15d)

$$\begin{aligned}&(\varvec{x}_t,\varvec{y}_t)\in X_t(\varvec{x}_{t-1},\varvec{\xi }_t). \end{aligned}$$

(B.15e)

The proof is similar to the one of Theorem 3. All the proofs in this section are omitted here due to similarity.

Notice at we obtain an MISDP in each stage and when $\lambda _{t+1}=0$, the risk-averse model (B.15) reduces to the risk-neutral case (15). We can apply McCormick envelopes and approximation schemes to obtain valid upper and lower bounds similar to the procedures in Sects. 4.2.1–4.2.2.

Details of all needed proofs

Proof

(Theorem 1) The proof follows Theorem 3.1 in [24]. Using the ambiguity set defined in (3), the inner maximization problem of (2) can be expressed as

$$\begin{aligned} \max _{\varvec{p}\in \mathbb {R}^K}\quad&\sum _{k=1}^Kp_kQ_{t+1}(\varvec{x}_t,\varvec{\xi }_{t+1}^k) \end{aligned}$$

(C.16a)

$$\begin{aligned} \text {s.t.}\quad&\sum _{k=1}^Kp_k\varvec{f}(\varvec{\xi }_{t+1}^k)\ge \varvec{l}(\varvec{x}_{t}), \end{aligned}$$

(C.16b)

$$\begin{aligned}&\sum _{k=1}^Kp_k\varvec{f}(\varvec{\xi }_{t+1}^k)\le \varvec{u}(\varvec{x}_t), \end{aligned}$$

(C.16c)

$$\begin{aligned}&{p_k\ge \underline{p}_k(\varvec{x}_t),\ \forall k\in [K],} \end{aligned}$$

(C.16d)

$$\begin{aligned}&{p_k\le \bar{p}_k(\varvec{x}_t),\ \forall k\in [K],} \end{aligned}$$

(C.16e)

$$\begin{aligned}&p_k\ge 0, \ \forall k\in [K]. \end{aligned}$$

(C.16f)

We associate dual variables $\varvec{\alpha },\ \varvec{\beta }\in \mathbb {R}^m$ with Constraints (C.16b) and (C.16c), dual variables $\varvec{\underline{\gamma }}$ and $\varvec{\bar{\gamma }}\in \mathbb {R}^K$ with Constraints (C.16d) and (C.16e), respectively. When (C.16) is feasible, strong duality holds and the Bellman equation (2) can be reformulated as (4), which completes the proof. $\square$

The proof of Theorem A.1 is omitted due to its similarity to the proof of Theorem 3.3 in [24].

Proof

(Theorem 2) Following Type 2 ambiguity set in (10), the inner maximization problem in (2) can be recast as

$$\begin{aligned} \max _{\varvec{p}\in \mathbb {R}^K}\quad&\sum _{k=1}^Kp_kQ_{t+1}(\varvec{x}_t,\varvec{\xi }_{t+1}^k) \end{aligned}$$

(C.17a)

$$\begin{aligned} \text {s.t.}\quad&\sum _{k=1}^K p_k=1, \end{aligned}$$

(C.17b)

$$\begin{aligned}&\sum _{k=1}^K p_k\varvec{\xi }_{t+1}^k=\varvec{\mu }(\varvec{x}_t), \end{aligned}$$

(C.17c)

$$\begin{aligned}&\sum _{k=1}^K p_k(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}=\varvec{\varSigma }(\varvec{x}_t), \end{aligned}$$

(C.17d)

$$\begin{aligned}&p_k\ge 0,\ \forall k\in [K]. \end{aligned}$$

(C.17e)

If the above linear program is feasible, then strong duality holds. Associate dual variables $s\in \mathbb {R},\ \varvec{u}\in \mathbb {R}^J, \ \varvec{Y}\in \mathbb {R}^{J\times J}$ with the three sets of constraints, respectively, and recast the inner maximization problem as a minimization problem. After including constraints $(\varvec{x}_t,\varvec{y}_t)\in X_t(\varvec{x}_{t-1},\varvec{\xi }_t)$, the Bellman equation (2) is equivalent to (11), and we complete the proof. $\square$

Proof

(Theorem A.2) The proof follows the conic duality in functional spaces [35]. Using the ambiguity set defined in (A.2), the inner maximization problem of (2) can be formulated as a conic linear program in a functional space as follows:

$$\begin{aligned} \max _{P\in \mathcal {M}(\varXi _{t+1},\mathcal {F}_{t+1})}\quad&\mathbb {E}_P[Q_{t+1}(\varvec{x}_t,\varvec{\xi }_{t+1})] \end{aligned}$$

(C.18a)

$$\begin{aligned} \text {s.t.}\quad&\mathbb {E}_P[1]= 1, \end{aligned}$$

(C.18b)

$$\begin{aligned}&\mathbb {E}_P[\varvec{\xi }_{t+1}]=\varvec{\mu }(\varvec{x}_t), \end{aligned}$$

(C.18c)

$$\begin{aligned}&\mathbb {E}_P[(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}]=\varvec{\varSigma }(\varvec{x}_t) \end{aligned}$$

(C.18d)

We associate dual variables $s\in \mathbb {R},\ \varvec{u}\in \mathbb {R}^J, \ \varvec{Y}\in \mathbb {R}^{J\times J}$ with the three sets of constraints, respectively. Because the primal problem has a non-empty relative interior, strong duality holds and the dual problem can be formulated as (A.3), which completes the proof. $\square$

Proof

(Theorem 3) Given Type 3 ambiguity set (14), the inner maximization problem in (2) can be recast as

$$\begin{aligned} \max _{\varvec{p}\in \mathbb {R}^K, \varvec{\tau }\in \mathbb {R}^J}\quad&\sum _{k=1}^Kp_kQ_{t+1}(\varvec{x}_t,\varvec{\xi }_{t+1}^k) \end{aligned}$$

(C.19a)

$$\begin{aligned} \text {s.t.}\quad&\sum _{k=1}^K p_k=1, \end{aligned}$$

(C.19b)

$$\begin{aligned}&\sum _{k=1}^K p_k\varvec{\xi }_{t+1}^k=\varvec{\tau }, \end{aligned}$$

(C.19c)

$$\begin{aligned}&(\varvec{\tau }-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\varvec{\varSigma }(\varvec{x}_t)^{-1}(\varvec{\tau }-\varvec{\mu }(\varvec{x}_t))\le \gamma , \end{aligned}$$

(C.19d)

$$\begin{aligned}&\sum _{k=1}^K p_k(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\preceq \eta \varvec{\varSigma }(\varvec{x}_t), \end{aligned}$$

(C.19e)

$$\begin{aligned}&p_k\ge 0,\ \forall k\in [K]. \end{aligned}$$

(C.19f)

We rewrite Constraint (C.19d) as

$$\begin{aligned} \begin{pmatrix} \varvec{\varSigma }(\varvec{x}_t) &{} \varvec{\tau }-\varvec{\mu }(\varvec{x}_t)\\ (\varvec{\tau }-\varvec{\mu }(\varvec{x}_t))^{\mathsf T} &{} \gamma \end{pmatrix} \succeq 0, \end{aligned}$$

and associate dual variables $s\in \mathbb {R},\ \varvec{u}\in \mathbb {R}^d, \ \varvec{Z}=\begin{pmatrix} \varvec{z}_1&{}\varvec{z}_2\\ \varvec{z}_2^{\mathsf T}&{}z_3\end{pmatrix}\succeq 0,\ \varvec{Y}\succeq 0$ with Constraints (C.19b)–(C.19e), respectively. The Lagrangian function of (C.19) has the following form:

$$\begin{aligned}&L(\varvec{p},\varvec{\tau },s, \varvec{u}, \varvec{Z},\varvec{Y})=\sum _{k=1}^Kp_kQ_{t+1}\left( \varvec{x}_t,\varvec{\xi }_{t+1}^k\right) -s\left( \sum _{k=1}^Kp_k-1\right) +\varvec{u}^{\mathsf T}\left( \varvec{\tau }-\sum _{k=1}^K p_k\varvec{\xi }_{t+1}^k\right) \nonumber \\&\qquad +\begin{pmatrix} \varvec{\varSigma }(\varvec{x}_t) &{} \varvec{\tau }-\varvec{\mu }(\varvec{x}_t)\\ (\varvec{\tau }-\varvec{\mu }(\varvec{x}_t))^{\mathsf T} &{} \gamma \end{pmatrix}\cdot \varvec{Z}\nonumber \\&\qquad + \left( \eta \varvec{\varSigma }(\varvec{x}_t)-\sum _{k=1}^K p_k(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\right) \cdot \varvec{Y}\nonumber \\&\quad = \sum _{k=1}^K p_k\left( Q_{t+1}(\varvec{x}_t,\varvec{\xi }_{t+1}^k)-s-\varvec{u}^{\mathsf T}\varvec{\xi }_{t+1}^k-(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\cdot \varvec{Y}\right) \nonumber \\&\qquad +\varvec{\tau }^{\mathsf T}(\varvec{u}+2\varvec{z}_2)+s+\varvec{\varSigma }(\varvec{x}_t)\cdot \varvec{z}_1 -\varvec{\mu }(\varvec{x}_t)^{\mathsf T}(2\varvec{z}_2)+\gamma z_3 +\eta \varvec{\varSigma }(\varvec{x}_t)\cdot Y. \end{aligned}$$

(C.20)

Because problem (C.19) is convex and under the Slater’s conditions, strong duality holds. The maximization problem (C.19) can be recast as

$$\begin{aligned} \min _{s,\varvec{u},\varvec{Z},\varvec{Y}}\left\{ \max _{\varvec{p},\varvec{\tau }}\left\{ L(\varvec{p},\varvec{\tau },s, \varvec{u}, \varvec{Z},\varvec{Y}): \varvec{p\ge 0},\ \varvec{\tau }\in \mathbb {R}^J\right\} \right\} . \end{aligned}$$

(C.21)

Following the Lagrangian function (C.20), after solving the inner maximization problem in (C.21) over $\varvec{p},\varvec{\tau }$, we have

$$\begin{aligned} \min _{s,\varvec{u},\varvec{Z},\varvec{Y}}\quad&s+\varvec{\varSigma }(\varvec{x}_t)\cdot \varvec{z}_1 -\varvec{\mu }(\varvec{x}_t)^{\mathsf T}(2\varvec{z}_2)+\gamma z_3 +\eta \varvec{\varSigma }(\varvec{x}_t)\cdot \varvec{Y}\\ \text {s.t.}\quad&Q_{t+1}(\varvec{x}_t,\varvec{\xi }_{t+1}^k)-s-\varvec{u}^{\mathsf T}\varvec{\xi }_{t+1}^k-(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}^k-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\cdot \varvec{Y}\le 0,\\&\quad \forall k\in [K],\\&\varvec{u}+2\varvec{z}_2=0,\\&\varvec{Z}=\begin{pmatrix} \varvec{z}_1&{}\varvec{z}_2\\ \varvec{z}_2^{\mathsf T}&{}z_3\end{pmatrix}\succeq 0,\\&\varvec{Y}\succeq 0. \end{aligned}$$

Substituting $\varvec{u}=-2\varvec{z}_2$ and combining with the outer minimization problem in (2), we complete the proof. $\square$

Proof

(Theorem A.3) The proof follows the conic duality in functional spaces [35]. Using the ambiguity set defined in (A.4), the inner maximization problem of (2) can be formulated as a conic linear program in a functional space as follows:

$$\begin{aligned} \max _{P\in \mathcal {M}(\varXi _{t+1},\mathcal {F}_{t+1})}\quad&\mathbb {E}_P[Q_{t+1}(\varvec{x}_t,\varvec{\xi }_{t+1})] \end{aligned}$$

(C.22a)

$$\begin{aligned} \text {s.t.}\quad&\mathbb {E}_P[1]= 1, \end{aligned}$$

(C.22b)

$$\begin{aligned}&(\mathbb {E}[\varvec{\xi }_{t+1}]-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}\varvec{\varSigma }(\varvec{x}_t)^{-1}(\mathbb {E}[\varvec{\xi }_{t+1}]-\varvec{\mu }(\varvec{x}_t))\le \gamma , \end{aligned}$$

(C.22c)

$$\begin{aligned}&\mathbb {E}[(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))(\varvec{\xi }_{t+1}-\varvec{\mu }(\varvec{x}_t))^{\mathsf T}]\preceq \eta \varvec{\varSigma }(\varvec{x}_t) \end{aligned}$$

(C.22d)

We associate dual variables $s\in \mathbb {R}, \ \varvec{Z}=\begin{pmatrix} \varvec{z}_1&{}\varvec{z}_2\\ \varvec{z}_2^{\mathsf T}&{}z_3\end{pmatrix}\succeq 0,\ \varvec{Y}\succeq 0$ with Constraints (C.22b)–(C.22d), respectively. The Slater’s constraint qualification conditions ensure that the primal problem has a non-empty relative interior. Therefore, strong duality holds and the dual problem can be formulated as (A.5), which completes the proof.    $\square$