Inventory control with modulated demand and a partially observed modulation process

Abstract

We consider a periodic review inventory control problem having an underlying modulation process that affects demand and that is partially observed by the uncensored demand process and a novel additional observation data (AOD) process. We present an attainability condition, AC, that guarantees the existence of an optimal myopic base stock policy if the reorder cost $K=0$ and the existence of an optimal (s, S) policy if $K>0$, where both policies depend on the belief function of the modulation process. Assuming AC holds, we show that (i) when $K=0$, the value of the optimal base stock level is constant within regions of the belief space and that each region can be described by two linear inequalities and (ii) when $K>0$, the values of s and S and upper and lower bounds on these values are constant within regions of the belief space and that these regions can be described by a finite set of linear inequalities. A heuristic and bounds for the $K=0$ case are presented when AC does not hold. Special cases of this inventory control problem include problems considered in the Markov-modulated demand and Bayesian updating literatures.

Appendix

1.1 Proof of result in section 2

Proof of Lemma 2

If $s^*(\varvec{x}) = d_m$, then

$$\begin{aligned} A_{m-1}(\varvec{x})d_{m-1} + B_{m-1}(\varvec{x})> & {} A_m(\varvec{x})d_m+ B_m(\varvec{x}), \\ A_{m+1}(\varvec{x})d_{m+1} + B_{m+1}(\varvec{x})\ge & {} A_m(\varvec{x})d_m+B_m(\varvec{x}), \end{aligned}$$

which leads to the result. $\square$

1.2 Proofs of results in section 3

Proof of Proposition 1

By induction. Letting $v_0 = 0$, we note that

$$\begin{aligned} v_1(s,\varvec{x})=\min _{y\ge s} L(\varvec{x},y) = L\big (\varvec{x},\max \{s^*(\varvec{x}),s\}\big ) \end{aligned}$$

for all $\varvec{x}$ and $L\big (\varvec{x},\max \{s^*(\varvec{x}),s\}\big )$ is non-decreasing and convex in s. Thus, the result holds true for $n=1$ (and, trivially for $n=0$). Assume the result holds for n. Then, for $s\le s^*(\varvec{x})$,

$$\begin{aligned} v_{n+1}(\varvec{x},s)\le & {} L\big (\varvec{x},s^*(\varvec{x})\big ) + \beta \sum _{d,z} \sigma (d,z,\varvec{x})v_n\big (\varvec{\lambda }(d,z,\varvec{x}) , f\big ( s^*(\varvec{x}), d\big )\big ) \\ = L\big (\varvec{x},s^*(\varvec{x})\big )+ & {} \beta \sum _{d,z} \sigma (d,z,\varvec{x})v_n\big (\varvec{\lambda }(d,z,\varvec{x}), s^*\big (\varvec{\lambda }(d,z,\varvec{x})\big ) \big ) \ \text {(using Section 2.4).} \\ \text {Also, } v_{n+1}(\varvec{x},s)\ge & {} \min _{y\ge s} L(\varvec{x},y)+ \beta \sum _{d,z} \sigma (d,z,\varvec{x}) \min _y v_n \big (\varvec{\lambda }(d,z,\varvec{x}),f\big (y, d\big ) \big ) \\ {}= & {} L\big (\varvec{x},s^*(\varvec{x})\big ) +\beta \sum _{d,z} \sigma (d,z,\varvec{x}) v_n \big (\varvec{\lambda }(d,z,\varvec{x}), s^*\big (\varvec{\lambda }(d,z,\varvec{x})\big ) \big ) \\ {}= & {} L\big (\varvec{x},s^*(\varvec{x})\big ) +\beta \sum _{d,z} \sigma (d,z,\varvec{x}) v_n \big (\varvec{\lambda }(d,z,\varvec{x}), f\big ( s^*(\varvec{x}), d\big ) \big ). \end{aligned}$$

$\text { Thus, for } s\le s^*(\varvec{x}), \text { }$

$$\begin{aligned} v_{n+1}(\varvec{x},s) = L\big (\varvec{x},s^*(\varvec{x})\big ) + \beta \sum _{d,z} \sigma (d,z,\varvec{x}) v_{n}\big (\varvec{\lambda }(d,z,\varvec{x}), f \big (s^*(\varvec{x}),d\big )\big ), \end{aligned}$$

and $v_{n+1}(\varvec{x},s) = v_{n+1}(\varvec{x}, s^*(\varvec{x})) .$

Assume $s \ge s^*(\varvec{x})$. Note

$$\begin{aligned} v_{n+1}(\varvec{x},s)\le & {} L(\varvec{x},s) + \beta \sum _{d,z} \sigma (d,z,\varvec{x}) v_n\big (\varvec{\lambda }(d,z,\varvec{x}),f\big (s, d\big )\big ). \\ \text { Also, } v_{n+1}(\varvec{x},s)\ge & {} \min _{y\ge s} L(\varvec{x},y) + \beta \sum _{d,z} \sigma (d,z,\varvec{x})\min _{y\ge s} v_n \big (\varvec{\lambda }(d,z,\varvec{x}),f\big (y, d\big )\big ) \\= & {} L(\varvec{x},s) + \beta \sum _{d,z} \sigma (d,z,\varvec{x}) v_n\big (\varvec{\lambda }(d,z,\varvec{x}), f \big (s, d \big )\big ), \\ \text { and hence for } s\ge & {} s^*(\varvec{x}), \ \text { } v_{n+1}(\varvec{x},s) = L(\varvec{x},s) + \beta \sum _{d,z} \sigma (d,z,\varvec{x}) v_n\big (\varvec{\lambda }(d,z,\varvec{x}),f \big (s, d \big )\big ) \end{aligned}$$

and is non-decreasing and convex in s. $\square$

Proof of Lemma 4

It is sufficient to show that if $y\le y'$ and $\varvec{x} \preceq \varvec{x'}$, then,

$$\begin{aligned} L(\varvec{x},y) - L(\varvec{x},y') \le L(\varvec{x'},y)-L(\varvec{x'},y'), \end{aligned}$$

which follows from the assumptions and [Puterman (1994), Lemma 4.7.2]. $\square$

Ideally, we would want to select $\widehat{\varvec{x}}^{d,z}$ so that $s^*(\varvec{x'})\le s^*(\widehat{\varvec{x}}^{d,z})$ for all $\varvec{x'}$ such that $\varvec{x'} \preceq \varvec{\lambda }(d,z,\varvec{x}) \ \ \forall \ \varvec{x}\in X$, for all (d, z), which would strengthen Lemma 4 as much as possible. We construct such an $\widehat{\varvec{x}}^{d,z}$ after the following preliminary result.

Lemma 6

The set $\{ \varvec{\lambda }(d,z,\varvec{x}): \varvec{x}\in X\}$ = $\bigg \{ \sum _i \xi _i \varvec{\lambda }(d,z,\varvec{e_i}) : \xi _i \ge 0 \ \forall i, \sum _i \xi _i = 1 \bigg \}.$

We remark that if $\varvec{x} \preceq \varvec{x'}$ and $\varvec{x} \preceq \varvec{x''}$, then $\varvec{x}\preceq \alpha \varvec{x'}+(1-\alpha )\varvec{x''}$ for all $\alpha \in [0,1]$. Thus, if $\widehat{\varvec{x}}^{d,z}$ is such that $\widehat{\varvec{x}}^{d,z} \preceq \lambda (d,z,\varvec{e_i})$ for all i, then $\widehat{\varvec{x}}^{d,z}$ is such that $\widehat{\varvec{x}}^{d,z} \preceq \varvec{x'}$ for all $\varvec{x'} \in \big \{ \varvec{\lambda }(d,z,\varvec{x}) : \varvec{x}\in X \big \}$.

1.3 Construction of $\widehat{x}^{d,z}$

We now construct $\widehat{\varvec{x}}^{d,z}$. Let

$$\begin{aligned} \widehat{x}_N^{d,z}= & {} \min \big \{ \varvec{\lambda }_N(d,z,\varvec{e_i}), i=1,\dots ,N \big \} \\ \widehat{x}_n^{d,z}= & {} \min \bigg \{ \sum _{k=n}^N \varvec{\lambda }_k(d,z,\varvec{e_i}), i =1,\dots , N \bigg \} -\sum _{k=n+1}^N \widehat{x}_k^{d,z}, \ \ n= N-1, \dots , 2 \\ \widehat{x}_1^{d,z}= & {} 1 - \sum _{k=2}^N \widehat{x}_k^{d,z}. \end{aligned}$$

By construction, $\widehat{\varvec{x}}^{d,z}\preceq \varvec{\lambda }(d,z, \varvec{x}) \ \forall \ \varvec{x}\in X$. We now show that $\widehat{\varvec{x}}^{d,z}\in X$ and that $s^*(\varvec{x'})\le s^*(\widehat{\varvec{x}}^{d,z})$ for all $\varvec{x'}\in X$ such that $\varvec{x'}\preceq \varvec{\lambda }(d,z,\varvec{x}) \ \forall \ \varvec{x}\in X$.

Lemma 7

(i) $\widehat{\varvec{x}}^{d,z} \in X$. (ii) For any $\varvec{x'}\preceq \varvec{\lambda }(d,z,\varvec{x}) \ \forall \ \varvec{x}\in X, s^*(\varvec{x'}) \le s^*(\widehat{\varvec{x}}^{d,z})$.

Proof of Lemma 7

We have the following:

• (i) Clearly, $0\le \widehat{\varvec{x}}^{d,z}_N \le 1$ and $\sum _{n=1}^N \widehat{\varvec{x}}_n^{d,z} = 1$. It is sufficient to show $0\le \widehat{\varvec{x}}^{d,z}_n, n=N-1, \dots , 1$. Note

  $$\begin{aligned} \sum _{k=n+1}^N\widehat{\varvec{x}}^{d,z}_k = \min _{1\le i\le N} \bigg \{\sum _{k=n+1}^N \lambda _k(d,z,\varvec{e_i})\bigg \} \le \sum _{k=n+1}^N \lambda _k(d,z,\varvec{e_i}) \le \sum _{k=n}^N \lambda _k(d,z,\varvec{e_i}), \ \forall \ i. \end{aligned}$$

  Thus, $\sum _{k=n+1}^N\widehat{x}^{d,z}_k \le \min _{1\le i\le N} \bigg \{\sum _{k=n}^N \lambda _k(d,z,\varvec{e_i})\bigg \} = \sum _{k=n}^N\widehat{x}^{d,z}_k$, and hence $\widehat{x}_n^{d,z}\ge 0$.

• (ii) Let $\varvec{x'}\preceq \varvec{\lambda }(d,z,\varvec{x}) \ \forall \ \varvec{x} \in X$ and assume $s^*(\widehat{\varvec{x}}^{d,z}) < s^*(\varvec{x'})$. Then, there is an $n\in \{1, \dots , N\}$ such that $\sum _{k=n}^N x_k' > \sum _{k=n}^N \widehat{x}_k^{d,z}$. However, $\sum _{k=n}^N \widehat{x}^{d,z}_k = \min _{1\le i\le N}$ $\bigg \{\sum _{k=n}^N \lambda _k(d,z,\varvec{e_i})\bigg \}$, which leads to a contradiction of the assumption that $\varvec{x'} \preceq \varvec{\lambda }(d,z,\varvec{x})$ $\forall \varvec{x} \in X$.$\square$

1.4 Computing the expected cost function, $v_n$

We now present a procedure for computing $v_n(s, \varvec{x})$. We only consider the case where $s=s^*(\varvec{x})$ due to Proposition 1 and Lemma 3. For notational simplicity, we assume that $\text {Pr}\big (z(t+1) \mid \mu (t+1),\mu (t)\big )$ is independent of $\mu (t+1)$ and $\mu (t)$. Extension to the more general case is straightforward.

Assume $v_0=0$, let $n=1$, and recall $v_1\big (\varvec{x},s^*(\varvec{x})\big ) = L\big (\varvec{x},s^*(\varvec{x})\big ).$ Note $L(\varvec{x},y) = \varvec{x}\overline{\varvec{\gamma }}_y, \ \text { where } \overline{\varvec{\gamma }}_y = \sum _{d,z} \varvec{P}(d,z)\ \underline{1}\ c(y,d)$. Let $\varGamma _1 = \{ \overline{\varvec{\gamma }}_y\}$, and note that if $c(y, d) = p(d-y)^+ + h(y-d)^+$, it is sufficient to consider only $y \in \{d_1,\dots , d_M\}$. Then, $v_1\big (\varvec{x},s^*(\varvec{x})\big )= \min \big \{\varvec{x}\overline{\varvec{\gamma }}:\overline{\varvec{\gamma }} \in \varGamma _1 \big \}.$ Assume there is a finite set $\varGamma _n$ such that $v_n \big (\varvec{x}, s^*(\varvec{x})\big ) = \min \big \{\varvec{x}\varvec{\gamma }: \varvec{\gamma }\in \varGamma _n \big \}.$ Then,

$$\begin{aligned} v_{n+1}\big (\varvec{x},s^*(\varvec{x}) \big )= & {} L\big (\varvec{x},s^*(\varvec{x})\big ) + \beta \sum _{m=1}^M \sigma (d_m,\varvec{x}) v_n\big (\varvec{\lambda }(d_m, \varvec{x}),f\big (s^*(\varvec{x}),d_m\big )\big ) \\ {}= & {} \min \big \{\varvec{x}\overline{\varvec{\gamma }}:\overline{\varvec{\gamma }} \in \varGamma _1 \big \} + \beta \sum _{m=1}^M \sigma (d_m,\varvec{x}) v_n\big (\varvec{\lambda }(d_m,\varvec{x}),s^*(\varvec{\lambda }(d_m,\varvec{x}))\big ) \\ {}= & {} \min \big \{\varvec{x}\overline{\varvec{\gamma }}:\overline{\varvec{\gamma }} \in \varGamma _1 \big \} + \beta \sum _{m=1}^M \sigma (d_m,\varvec{x}) \min \big \{\varvec{\lambda }(d_m,\varvec{x})\varvec{\gamma }: \varvec{\gamma }\in \varGamma _n \big \} \\ {}= & {} \min _{\overline{\varvec{\gamma }}} \min _{\varvec{\gamma _1}}\dots \min _{\varvec{\gamma _M}} \bigg \{ \varvec{x}\overline{\varvec{\gamma }} + \beta \sum _{m=1}^M \sigma (d_m,\varvec{x})\varvec{\lambda }(d_m,\varvec{x}) \varvec{\gamma _m} \bigg \} \\ {}= & {} \min _{\overline{\varvec{\gamma }}} \min _{\varvec{\gamma _1}}\dots \min _{\varvec{\gamma _M}} \bigg \{ \varvec{x} \bigg [\overline{\varvec{\gamma }} + \beta \sum _{m=1}^M \varvec{P}(d_m) \varvec{\gamma _m} \bigg ] \bigg \} \end{aligned}$$

Thus, $\varGamma _{n+1}$ is the set of all $\varvec{\gamma }$ such that $\varvec{\gamma } = \overline{\varvec{\gamma }} + \beta \sum _{m=1}^M \varvec{P}(d_m)\varvec{\gamma _m},$ where $\overline{\varvec{\gamma }} \in \varGamma _1$ and $\varvec{\gamma _m} \in \varGamma _n$ for all $m =1,\dots , M$, and for all n, $v_n\big (\varvec{x}, s^*(\varvec{x})\big )$ is piecewise linear and concave in $\varvec{x}$.

Let $|\varGamma |$ be the cardinality of the set $\varGamma$. Then, $|\varGamma _{n+1} | = |\varGamma _1 | |\varGamma _n |^M$, where $|\varGamma _1 | \le M$, and hence the cardinality of $\varGamma _n$ can grow rapidly. Many of the vectors in the sets $\varGamma _n$ are redundant and can be eliminated, reducing both computational and storage burdens. An exhaustive literature study of elimination procedures and other solution methods for solving POMDPs can be found in Chang et al. (2015a).

1.5 Proofs of results in section 4

Proof of Lemma 5

Assume $f(y,d) = y-d$ and $c(y,d) = p(d-y)^++h(y-d)^+$, recall that elements of $\mathcal {P}_1$ are sets of the form $\{\varvec{x}\in X: s^*(\varvec{x}) = d_m\}$ for all $d_m$ such that $\min _{\varvec{x}} s^*(\varvec{x}) \le d_m \le \max _{\varvec{x}} s^*(\varvec{x})$. Further recall that $\{\varvec{x}\in X: s^*(\varvec{x}) = d_m\}$ is the set of all $\varvec{x}$ such that

$$\begin{aligned} \sum _{k=1}^{m-1} \sigma (d_k,\varvec{x}) < p/(p+h) \le \sum _{k=1}^m \sigma (d_k,\varvec{x}), \end{aligned}$$

or equivalently,

$$\begin{aligned} \quad \quad \quad \quad \varvec{x}\sum _{k=1}^{m-1}\varvec{P}(d_k)\underline{1} < p/(p+h) \le \varvec{x}\sum _{k=1}^m \varvec{P}(d_k)\underline{1}, \end{aligned}$$

which represents two linear inequalities. Further, for $\varvec{x}\in \{ \varvec{x} \in X: s^*(\varvec{x}) =d_m\}$, $v_1^U(\varvec{x},s) = A_l(\varvec{x})d_l+B_l(\varvec{x})$ for $l=\max \{s^*(\varvec{x}),s\}$, where we note

$$\begin{aligned} A_l(\varvec{x})d_l+B_l(\varvec{x}) = \varvec{x} \big [h\sum _{k=1}^l (d_l-d_k)\varvec{P}(d_k)\underline{1} + p \sum _{k=l+1}^M(d_k-d_l)\varvec{P}(d_k)\underline{1} \big ], \end{aligned}$$

where $A_j(\varvec{x})$ and $B_j(\varvec{x})$ are defined in Section 2.3. Thus, on each element of $\mathcal {P}_1$, $v_1^U$ is linear in $\varvec{x}$ for each s and each element of $\mathcal {P}_1$ is described by a finite number of linear inequalities.

Let $(\varvec{x},s)$ be such that $d_l \le \max \{s^*(\varvec{x}),s\} \le d_{l+1}$ for all $\varvec{x}$ in an element $\{\varvec{x}\in X: s^*(\varvec{x})=d_m\}$. Further, let $d_{l(d,z)} \le \max \{s^*(\varvec{\lambda }(d,z,\varvec{x})),\max \{s^*(\varvec{x}),s\}-d\} \le d_{l(d,z) + 1}$ for all $\varvec{x}$ in an element $\{\varvec{x}\in X: s^*(\varvec{\lambda }(d,z,\varvec{x})) = d_{m(d)}\}$, which is the set of all $\varvec{x}$ such that:

$$\begin{aligned} \varvec{\lambda }(d,z,\varvec{x})\sum _{k=1}^{m(d)-1}\varvec{P}(d_k)\underline{1} < p/(p+h) \le \varvec{\lambda }(d,z,\varvec{x})\sum _{k=1}^{m(d)} \varvec{P}(d_k)\underline{1}, \end{aligned}$$

or equivalently, for all $\varvec{x}$ such that $\sigma (d,\varvec{x}) \ne 0$,

$$\begin{aligned} \varvec{x}\varvec{P}(d,z)\sum _{k=1}^{m(d)-1}\varvec{P}(d_k)\underline{1} < \big (p/(p+h)\big )\varvec{x}\varvec{P}(d,z)\underline{1} \le \varvec{x}\varvec{P}(d,z)\sum _{k=1}^{m(d)} \varvec{P}(d_k)\underline{1}, \end{aligned}$$

where we assume m and m(d) for all d have been chosen so that the finite set of linear inequalities describes a non-null subset of X. We note that for such a subset,

$$\begin{aligned} v^U_{n+1}(\varvec{x},s)= & {} A_l(\varvec{x})d_l +B_l(\varvec{x}) \\ {}{} & {} + \beta \sum _{d,z} \sigma (d,z,\varvec{x}) \bigg [A_{l(d,z)}(\varvec{\lambda }(d,z,\varvec{x}))d_{l(d,z)} + B_{l(d,z)}(\varvec{\lambda }(d,z,\varvec{x})) \bigg ] \\ {}= & {} \varvec{x}\bigg [h\sum _{k=1}^l(d_l-d_k)\varvec{P}(d_k)\underline{1} + p \sum _{k=l+1}^M(d_k-d_l)\varvec{P}(d_k)\underline{1} \\ {}{} & {} +\beta \sum _d \bigg [h \sum _z\sum _{k=1}^{l(d,z)}(d_{l(d,z)}-d_k)\varvec{P}(d,z)\varvec{P}(d_k)\underline{1} \\ {}{} & {} + p \sum _z \sum _{k=l(d,z)+1}^N(d_k-d_{l(d,z)})\varvec{P}(d,z)\varvec{P}(d_k)\underline{1}\bigg ]\bigg ]. \end{aligned}$$

The resulting partition $\mathcal {P}_2$ is at least as fine as $\mathcal {P}_1$ and each element in $\mathcal {P}_2$ is described by a finite set of linear inequalities. We have shown that on each element in $\mathcal {P}_2$, $v_2^U(\varvec{x},s)$ is linear in $\varvec{x}$ for each s. A straightforward induction argument shows these characteristics hold for all n. We illustrate by example (through Example 3) how $v_n^U(\varvec{x},s)$ may be discontinuous in $\varvec{x}$ for fixed s. $\square$

Proof of Proposition 4

It is sufficient to show the result holds for $\tau = t+1$. There are two cases. First, let $s(t) \le s^*(\varvec{x}(t))$. Then, $s(t+1) = s^*(\varvec{x}(t))-d(t)$. We note

$$\begin{aligned} \min \{s^*(\varvec{x}): \varvec{x} \in X\}- d_M\le & {} s^*(\varvec{x}(t))- d(t) \\ {}\le & {} \max \{s^*(\varvec{x}): \varvec{x} \in X\}- d_1 \text { and hence,} \\ \min \{s^*(\varvec{x}): \varvec{x} \in X\} -d_M\le & {} s(t+1) \le \max \{s^*(\varvec{x}): \varvec{x} \in X\}- d_1 \end{aligned}$$

Second, let $s^*(\varvec{x}(t)) \le s(t)$. Then, $s(t+1) = s(t)- d(t) \ge s^*(\varvec{x}(t))-d(t)$. We note

$$\begin{aligned} \min \{s^*(\varvec{x}): \varvec{x} \in X\} - d_M \le s^*(\varvec{x}(t)) - d(t) \le s(t) - d(t) \\ \le \max \{s^*(\varvec{x}): \varvec{x} \in X\} - d_1, \text { and hence, } \\ \min \{s^*(\varvec{x}): \varvec{x} \in X\} -d_M \le s(t+1) \le \max \{s^*(\varvec{x}): \varvec{x} \in X\} - d_1. \end{aligned}$$

$\square$

1.6 Design of instances for computational study

We describe the generation of computational instances for Sect. 4.4. Each instance describes a backordering system with no fixed ordering cost. For each combination of number of modulation states $N \in \{2,3\}$, number of demand outcomes $M \in \{3,4,5\}$, randomly generate M unique ordered integer demand outcomes from $[0, D_L]$ for each $D_L \in \{20, 100, 250, 500, 750, 1000\}$. Set the lowest demand outcome $d(0) = 0$ ( to encourage A1 violation), randomly sample probability transition matrix $\{P(i,j)\}$ and probability mass function for each modulation state $\{Q(d,j)\}$ such that the N ordered expected demands $ED_i, i =1, \dots , N$ are quite distinct and satisfy:

• $ED_1<= 0.5 d(M)$ and $ED_2 > 0.5 d(M)$ and $ED_2-ED_1 > 0.25 d(M)$ OR $ED_2 - ED_1 > 0.5 d(M)$, when $N = 2$ and

• $ED_1 <= 0.4 d(M)$ and $ED_2 > 0.4 d(M)$ and $ED_2 <= 0.7 d(M)$ and $ED_3 > 0.7 d(M)$ and $ED_2 - ED_1 > 0.2 d(M)$ and $ED_3 - ED_2 > 0.2 d(M)$, when $N = 3$.

Set the number of decision epochs T to 100 and vary backorder cost per unit per period p as $\{1.5, 2, 3\}$, while keeping the holding cost h at 1.

1.7 Algorithms for computational study

Here, $\varvec{e}(\mu _0)$ is the unit vector with 1 in the $\mu _0$th position.

1.8 Proofs of results in section 5

Proof of Proposition 7

The proof of Proposition 7 is a direct extension of the results in Scarf (1960). $\square$

Lemma 8

For all $\varvec{x}$ and n:

• (i) if $s\le s'$, then $v_n(\varvec{x},s) \le v_n(\varvec{x}, s') + K$

• (ii) if $y\le y'$, then $G_n(\varvec{x}, y') - G_n(\varvec{x}, y) \ge L(\varvec{x}, y') - L(\varvec{x}, y) - \beta K$

• (iii) if $s\le s' \le \underline{S}(\varvec{x})$, then $v_n(\varvec{x},s) \ge v_n(\varvec{x},s')$

• (iv) if $y\le y'\le \underline{S}(\varvec{x})$, then $G_n(\varvec{x},y') - G_n(\varvec{x},y) \le L(\varvec{x}, y') - L(\varvec{x}, y) \le 0.$

Proof of Lemma 8

• (i) This result follows from the K-convexity of $v_n(\varvec{x},s)$ in s, which is a direct implication of the second item of Proposition 7.

• (ii) This result follows from the definition of $G_n(\varvec{x},y)$, the previous result (i), and the fact that f(y, d) is convex and non-decreasing.

• (iii) $G_n(\varvec{x}, s_n(\varvec{x})) \le K+ G_n(\varvec{x},S_n(\varvec{x})) \le K+ G_n(\varvec{x}, \underline{S}(\varvec{x}))$ implies that $s_n(\varvec{x}) \le S_n(\varvec{x}) \le \underline{S}(\varvec{x})$ (This is an implication of the definitions of $s_n(\varvec{x})$ and $S_n(\varvec{x})$, and the fact that $\underline{S}(\varvec{x})$ minimizes $L(\varvec{x},y)$ while $S_n(\varvec{x})$ minimizes the sum of $L(\varvec{x},y)$ and a positive term.). It follows from the four cases of $s\le s' \le \underline{S}(\varvec{x})$ with respect to the value of $s_n(\varvec{x})$ that $v_n(\varvec{x},s) \ge v_n(\varvec{x},s')$.

• (iv) This result follows from the definition of $G_n(\varvec{x},y)$, the non-decreasing nature of f(y, d) in y and (iii).$\square$

The proof of Proposition 8 requires four lemmas.

Lemma 9

For all n and $\varvec{x}$, $\underline{S}(\varvec{x}) = S_0(\varvec{x}) \le S_n(\varvec{x})$.

Lemma 10

For all n and $\varvec{x}$, $s_n(\varvec{x})$ can be selected so that $s_n(\varvec{x}) \le \overline{s}(\varvec{x})$.

Lemma 11

For all n and $\varvec{x}$, $S_n(\varvec{x})$ can be selected so that $S_n(\varvec{x}) \le \overline{S}(\varvec{x})$.

Lemma 12

For all n and $\varvec{x}$, $\underline{s}(\varvec{x}) \le s_n(\varvec{x})$.

Proof of Proposition 8

The proof of these results follow from the proofs of Lemmas 2 - 5 in Veinott and Wagner (1965). Proof of Proposition 8(a) follows from Lemmas  9–12, and Proposition 8(b) follows from (a) and Proposition 7. $\square$

1.9 Determining $\varGamma _n(s)$

As was true for the $K=0$ case, when $K>0$, there is a finite set of vectors $\varGamma _n(s)$ such that $v_n(\varvec{x}, s) = \min \{\varvec{x}\varvec{\gamma }: \varvec{\gamma } \in \varGamma _n(s) \}$ for all s. Note that $\varGamma _0(s) = \{\underline{0}\}$ for all s, where $\underline{0}$ is the column N-vector having zero in all entries. Given $\{ \varGamma _n(s): \forall \ s \}$, we now present an approach for determining $\{\varGamma _{n+1}(s): \forall \ s \}$. Recalling Sect. A2, let $\overline{\varGamma } = \{\overline{\varvec{\gamma }}_1, \dots , \overline{\varvec{\gamma }}_M\}$ be such that $\min _y L(\varvec{x},y) = \min \{\varvec{x} \varvec{\gamma }: \varvec{\gamma } \in \overline{\varGamma } \}$. Note

$$\begin{aligned} G_n(\varvec{x}, y) = L(\varvec{x},y) + \beta \sum _{d, z} \sigma (d, z,\varvec{x}) v_n \big (\varvec{\lambda }(d,z,\varvec{x}), f(y,d)\big ), \end{aligned}$$

for $y \in \{d_1, \dots , d_M\}$. Then,

$$\begin{aligned} v_n\big (\varvec{\lambda }(d,z,\varvec{x}), f(y,d)\big ) = \min \{ \varvec{\lambda }(d,z,\varvec{x}) \varvec{\gamma }: \varvec{\gamma } \in \varGamma _n(f(y,d)) \}. \end{aligned}$$

Let $\varGamma _n'(y)$ be the set of all vectors of the form

$$\begin{aligned} \overline{\varvec{\gamma }} + \beta \sum _{d,z} \varvec{P}(d,z)\varvec{\gamma }(d,z), \end{aligned}$$

where $\overline{\varvec{\gamma }} \in \overline{\varGamma }$ and $\varvec{\gamma }(d,z) \in \varGamma _n(f(y,d))$. Then, $G_n(\varvec{x},y) = \min \{\varvec{x}\varvec{\gamma }: \varvec{\gamma } \in \varGamma _n'(y)\}$ and

$$\begin{aligned} v_{n+1}(\varvec{x},s) = {\left\{ \begin{array}{ll} K+G_n\big (\varvec{x},S_n(\varvec{x})\big ) &{} \ s\le s_n(\varvec{x}) \\ G_n(\varvec{x}, s) &{} \ \text {otherwise}, \end{array}\right. } \end{aligned}$$

where $S_n(\varvec{x})$ and $s_n(\varvec{x})$ are the smallest integers such that

$$\begin{aligned} G_n \big (\varvec{x}, S_n(\varvec{x})\big )\le & {} G_n(\varvec{x}, y) \ \forall y. \\ G_n\big (\varvec{x}, s_n(\varvec{x})\big )\le & {} K+G_n\big (\varvec{x}, S_n(\varvec{x})\big ). \end{aligned}$$

Let $X_n(s', S')$ be the set of all $\varvec{x}\in {X}$ such that $s_n(\varvec{x}) = s'$ and $S_n(\varvec{x}) = S'$. Thus, if $\varvec{x}\in X_n(s', S')$, then $s'$ and $S'$ are the smallest integers such that

$$\begin{aligned} G_n \big (\varvec{x}, S'(\varvec{x})\big )\le & {} G_n(\varvec{x}, y) \ \forall y. \\ G_n\big (\varvec{x}, s'(\varvec{x})\big )\le & {} K+G_n\big (\varvec{x}, S'(\varvec{x})\big ). \end{aligned}$$

Since $G_n(\varvec{x},y)$ is piecewise linear and convex in $\varvec{x}$ for each y, $X_n(s', S')$ is described by a finite set of linear inequalities. We remark that $\{X_n(s', S'): s' \le S', \text { and } X_n(s', S') \ne \emptyset \}$ is a partition of X. Further, we remark that if $\overline{X}(\underline{s}, \overline{s}, \underline{S}, \overline{S}) \cap X_n(s', S') \ne \emptyset$, then search for $(s', S')$ can be restricted to $\underline{s} \le s' \le \overline{s}$ and $\underline{S} \le S' \le \overline{S}$. Let $\varGamma _{n+1}(s) = \{ K\underline{1} + \varvec{\gamma }: \varvec{\gamma } \in \varGamma _n'(S') \}$ for all $s\le s'$, and let $\varGamma _{n+1}(s) = \varGamma _{n}'(s)$ for all $s>s'$. Thus, $v_{n+1}(\varvec{x},s) = \min \{ \varvec{x} \varvec{\gamma }: \varvec{\gamma } \in \varGamma _{n+1}(s) \}$ for all s.