Single Resource Revenue Management with Independent Demands

Abstract

In this chapter, we consider the single resource, independent demand revenue management problem with multiple fare classes. This problem arises in the airline industry where different fares for the same cabin are designed to cater to different market segments. As an example, a low fare may have advance purchase and length of stay restrictions and exclude ancillary services such as advance seat selection, luggage handling, and priority boarding. This low fare may target price-conscious consumers who travel for leisure on restricted budgets. On the other hand, a high fare designed for business consumers may be unrestricted, include ancillary services and be designed to be frequently available for late bookings. If requests for the low fare arrive first, the airline risks selling all of its capacity before seeing requests for the high fare. A key decision in revenue management is how much capacity to reserve for higher fare classes, or equivalently how much capacity to make available for lower fare classes. Throughout the chapter, we will refer to airline applications, but the reader should keep in mind that the models apply more generally.

Appendix

Proof of Lemma 1.4

Taking expectations yields $g(x) = G(x) \mbox{{$ {\mathbb P} \{ X \geq x \}$}} + \sum _{j \leq x-1} {\,} G(j) {\,} {\mathbb P}\{ X = j\}$ and $g(x-1) = G(x-1) {\,} {\mathbb P} \{X \geq x \} + \sum _{j \leq x-1}G(j) {\,} \mbox{{${\mathbb P} \{X = j\}$}}$. Taking the difference yields $\varDelta g(x) = \varDelta G(x) \mbox{{${\mathbb P}\{X \geq x\}$}}$. Similarly, taking expectations, we have $h(x) = H(x) {\,} \mbox{{${\mathbb P} \{X < x\}$}} + \sum _{j \geq x} H(j) {\,} {\mathbb P} \{X = j\}$. Similarly, we also have $h(x-1) = \mbox{{$H(x-1)$}} {\,} \mbox{{${\mathbb P} \{X < x\}$}} + \sum _{j\geq x}H(j) {\,} \mbox{{${\mathbb P}\{X = j\}$}}$. Taking the difference, we see that $\varDelta h(x) {=} \varDelta H(x) {\,} {\mathbb P}\{ X {<} x\}$.

Proof of Proposition 1.5

We will prove the result by induction on j. The result holds for j = 1 since $\varDelta V_1(y) = p_1 {\,} {\mathbb P}\{ D_1 \geq y\}$ is decreasing in y, and clearly $\varDelta V_1(y) = p_1 {\,} {\mathbb P}\{D_1 \geq y\} \geq \varDelta V_0(y) = 0$. Assume that the result is true for V _j−1. It follows from the dynamic programming equation that

$$\displaystyle \begin{aligned} V_j(x) = \max_{y \leq x} \left\{ W_j (y,x) \right\}, \end{aligned} $$

where, for any y ≤ x,

$$\displaystyle \begin{aligned} W_j(y,x) = {\mathbb E}\{ p_j \min \{ D_j, x - y \} \} + {\mathbb E} \{ V_{j-1} ( \max \{ x - D_j, y \} ) \}. \end{aligned} $$

Directly using the definition of W _j(y, x), for y ∈{1, …, x}, we can show that

$$\displaystyle \begin{aligned} \varDelta W_j(y,x) = W_j(y,x) - W_j(y-1,x) = [\varDelta V_{j-1}(y) - p_j] {\,} {\mathbb P}\{D_j > x-y\}. \end{aligned} $$

Since ΔV _j−1(y) is decreasing in y by the inductive hypothesis, we see that W _j(y, x) ≥ W _j(y − 1, x) if ΔV _j−1(y) > p _j and W _j(y, x) ≤ W _j(y − 1, x) if ΔV _j−1(y) ≤ p _j. Consider the expression

$$\displaystyle \begin{aligned} y_{j-1} = \max\{y \in {\mathbb N}_+: \varDelta V_{j-1}(y) > p_j\}, \end{aligned} $$

where the definition of ΔV _j(y) is extended to y = 0 for all j by setting ΔV _j(0) = p ₁. If x ≥ y _j−1, then

$$\displaystyle \begin{aligned} V_j(x) = \max_{y \leq x}W_j(y,x) = W_j(y_{j-1},x). \end{aligned} $$

On the other hand, if x < y _j−1, then

$$\displaystyle \begin{aligned} V_j(x) = \max_{y \leq x}W_j(y,x) = W_j(x,x). \end{aligned} $$

In summary, we have

$$\displaystyle \begin{aligned} V_j(x) &= W_j (\min(x,y_{j-1}),x) \\ &= \left\{ \begin{array}{ll} V_{j-1}(x), \quad&~~~~ \mathrm{if} \; x \leq y_{j-1}\\ {\mathbb E} \{ p_j {\,} \min \{ D_j, x - y_{j-1} \} \} \\ ~~~~~~~~~~~~~~~~~~~~ + {\mathbb E} \{ V_{j-1} ( \max \{ x - D_j, y_{j-1} \} ) \} &~~~~ \mathrm{if} \; x > y_{j-1}. \end{array} \right . \end{aligned} $$

Using the expression above, computing ΔV _j(x) = V _j(x) − V _j(x − 1) for $x \in {\mathbb N}_+$ results in

$$\displaystyle \begin{aligned} \varDelta V_j(x) &= \left\{ \begin{array}{ll} \varDelta V_{j-1}(x), \quad&~~~~ \mathrm{if} \; x \leq y_{j-1}\\ {\mathbb E} \{\min\{ p_j, \varDelta V_{j-1}(x-D_j) \} \} &~~~~ \mathrm{if} \; x > y_{j-1}. \end{array} \right . \end{aligned} $$

We will now use this result to show that ΔV _j(x) is itself decreasing in x. Since ΔV _j(x) = ΔV _j−1(x) for x ≤ y _j−1 and ΔV _j−1(x) is decreasing in x, we only need to worry about the case x > y _j−1. However, in this case, we have

$$\displaystyle \begin{aligned} \varDelta V_j(x) = E\min(p_j,\varDelta V_{j-1}(x-D_j)) \end{aligned} $$

is decreasing in x, since ΔV _j−1(x) is itself decreasing in x. Lastly, at y _j−1, using the expression for the first difference ΔV _j(x) above,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \varDelta V_j(y_{j-1}) &\displaystyle =&\displaystyle \varDelta V_{j-1}(y_{j-1}) > p_j \\ &\displaystyle &\displaystyle \qquad \qquad \ \ \geq {\mathbb E} \{\min\{ p_j, \varDelta V_{j-1}(x-D_j) \} \} = \varDelta V_j(y_{j-1}+1), \end{array} \end{aligned} $$

showing that ΔV _j−1(x) is decreasing at x = y _j−1 as well.

Now, we show that ΔV _j(x) ≥ ΔV _j−1(x). For x > y _j−1, we have

$$\displaystyle \begin{aligned} \min\{ p_j,\varDelta V_{j-1}(x-D_j) \} \geq \min\{ p_j,\varDelta V_{j-1}(x) \} = \varDelta V_{j-1}(x), \end{aligned} $$

where the inequality follows since ΔV _j−1(x) is decreasing in x, and the equality holds since x > y _j−1. Taking expectations we see that ΔV _j(x) ≥ ΔV _j−1(x) on x > y _j−1. Lastly, note that ΔV _j(x) = ΔV _j−1(x) on x ≤ y _j−1.

Proof of Theorem 1.6

By Lemma 1.4, we have

$$\displaystyle \begin{aligned} & {\mathbb E} \{ \min\{ x-y,D_j \} \} - {\mathbb E} \{ \min\{ x-(y-1) ,D_j \} \} \\ {}& \qquad\qquad\qquad= - {\mathbb E} \{ \min\{ x-y+1 ,D_j \} \} + {\mathbb E} \{ \min\{ x-y,D_j \} \} \\ {}& \qquad\qquad\qquad= - {\mathbb P} \{ D_j \geq x - y + 1 ) = - {\mathbb P} \{ D_j > x-y \}. \end{aligned} $$

Similarly, ${\mathbb E} \{ V_{j-1}(\max \{ y , x - D_j \}) \} - {\mathbb E} \{ V_{j-1}(\max \{ y -1 , x - D_j \}) \} = \varDelta V_{j-1}(y) \times {\mathbb P} \{ x - D_j < y \} = \varDelta V_{j-1}(y) {\,} {\mathbb P} \{ D_j > x - y \}$. This implies that

$$\displaystyle \begin{aligned} \varDelta W_j(y,x) = W_j(y,x) - W_j(y-1,x) = (\varDelta V_{j-1}(y) - p_j ) {\,} {\mathbb P}\{ D_j > x-y \}, \end{aligned} $$

thus, the sign of ΔW _j(y, x) is dictated by the sign of ΔV _j−1(y) − p _j.

We now show that W _j(y, x) is a unimodal in y. Letting $y_{j-1}^*$ be as defined in (1.7), for all $y > y_{j-1}^*$, we have ΔV _j−1(y) ≤ p _j. Therefore, ΔW _j(y, x) ≤ 0 for all $y > y_{j-1}^*$. Similarly, we have $\varDelta V_{j-1}(y_{j-1}^*) > p_j$, but since ΔV _j(x) is decreasing in x by the first part of Proposition 1.5, it follows that ΔV _j−1(y) > p _j for all $y \leq y_{j-1}^*$. Therefore, ΔW _j(y, x) ≥ 0 for all $y \leq y_{j-1}^*$. Having ΔW _j(y, x) ≥ 0 for all $y \leq y_{j-1}^*$ and ΔW _j(y, x) ≤ 0 for all $y > y_{j-1}^*$ implies that W _j(y, x) is unimodal in y and its maximizer occurs at $y_{j-1}^*$. So, the maximizer of W _j(y, x) over y ∈{0, …, x} occurs at $\min \{ y_{j-1}^* ,x\}$. Third, we show that the optimal protection levels are monotone in the fare classes. By the definition of $y_{j-1}^*$, we have $\varDelta V_{j-1}(y_{j-1}^*) > p_j$, and since ΔV _j(x) ≥ ΔV _j−1(x) by the second part of Proposition 1.5, we obtain $\varDelta V_j(y_{j-1}^*) \geq \varDelta V_{j-1}(y_{j-1}^*) > p_j > p_{j+1}$, which implies that $\varDelta V_j(y_{j-1}^*) > p_{j+1}$. In this case, since $y_j^*$ is given by ${\max \{y \in {\mathbb N}_+ : \varDelta V_j(y) > p_{j+1}\}}$, it must be the case that $y_j^* \geq y_{j-1}^*$.

Proof of Corollary 1.7

Let G(x) = p ₁ x, then $V_1(x) = g(x) = {\mathbb E} \{ G(\min \{ D_1,x \}) \}$, so $\varDelta V_1(x) = \varDelta g(x) = p_1 {\,} {\mathbb P} \{D_1 \geq x \}$. Then, by Theorem 1.6,

$$\displaystyle \begin{aligned} y_1 = \max \{y \in {\mathbb N}_+ : p_1 {\,} {\mathbb P} \{D_1 \geq x \} > p_2\} \end{aligned} $$

which coincides with Littlewood’s rule.

Proof of Proposition 1.10

Since Π _n(c, k) is the difference of a concave and a linear function, Π _n(c, k) is itself concave. The marginal value of adding the c-th unit of capacity is ΔV _n(c) − k so the c-th unit increases profits as long as ΔV _n(c) > k. Therefore, the smallest optimal capacity is given by c(k). (Notice that c(k) + 1 may be also optimal if ΔV _n(c(k) + 1) = k.) Note that c(k) is decreasing in k since ΔV _n(c) is decreasing in c. Suppose that k = p _j+1. To establish c(p _j+1) = y _j, it is enough to show that ΔV _n(y _j) > p _j+1 ≥ ΔV _n(y _j + 1). By definition, $y_j = \mbox{{$\max \{y \in {\mathbb N}_+: \varDelta V_j(y) > p_{j+1}\}$}}$, so that we have ΔV _j(y _j) > p _j+1 ≥ ΔV _j(y _j + 1). Since it is optimal to protect up to y _j units of capacity for sale at fares j, j − 1, …, 1, it follows that V _n(c) = V _j(c) for all c ≤ y _j, and consequently ΔV _n(y _j) = ΔV _j(y _j) > p _j+1. Now ΔV _n(y _j + 1) can be written as a convex combination of p _j+1 and ΔV _j(y _j + 1) ≤ p _j+1 which implies that ΔV _n(y _j + 1) ≤ p _j+1, as desired.

Proof of Theorem 1.18

Since $\mathcal {R}_t(z) \geq 0$, it follows from (1.14) that V (t, x) is increasing in t, with strict inequality as long as there is a fare k ∈ M _t such that p _k > ΔV (t, x) and λ _kt > 0. To show that V (t, x + 1) ≥ V (t, x) consider a sample path argument where the system with x + 1 units of inventory uses the optimal policy for the system with x units of inventory until either the system with x units runs out of stock or time runs out. If the system with x units of inventory runs out at time s, then the system with x + 1 units of inventory can still collect V (s, 1) ≥ 0. On the other hand, if time runs out the two systems collect the same revenue. Consequently, the system with x + 1 units of inventory makes at least as much revenue resulting in V (t, x + 1) ≥ V (t, x).

Clearly ΔV (t, 1) ≤ ΔV (t, 0) = ∞. Assume as the inductive hypothesis that ΔV (t, y) is decreasing in y ≤ x for all t ≥ 0. We want to show that ΔV (t, x + 1) ≤ ΔV (t, x), or equivalently that

$$\displaystyle \begin{aligned} V(t,x+1) + V(t,x-1) \leq V(x) + V(x). \end{aligned} $$

(1.19)

We will use a sample path argument to establish inequality (1.19). Consider four systems, one with x + 1 units of inventory, one with x − 1 units of inventory, and two with x units of inventory. Assume that we follow the optimal policy for the system with x + 1 and for the system with x − 1 that are on the left-hand side of inequality (1.19). For the two systems on the right, we use the sub-optimal policies designed for x + 1 and x − 1 units of inventory, respectively. We follow these policies until one of the following events occurs: time runs out, the difference in inventories for the systems on the left drops to 1, or the inventory of the system with x − 1 units drops to zero. After that time we follow optimal policies for all four systems. To establish inequality (1.19), we will show that the revenues obtained for the systems in the right are at least as large as for the systems on the left, even though sub-optimal policies are used for the systems in the right. This is obviously true if we run out of time since the realized revenues of the two systems on the right are exactly equal to the realized revenues from the two systems on the left. Assume now that at time s ∈ (0, t), the difference in inventories on the two systems on the left-hand side drops to 1, so that the states are (s, y + 1) and (s, y) for some y < x. This means that system on the left with x + 1 units of inventory had x − y units of sale and the system with x − 1 units of inventory had x − 1 − y units of sale. This implies that the system on the right that was following the policy designed for x + 1 reaches state (s, y), while the system that was using the policy designed for x − 1 reaches state (s, y + 1). Clearly, the additional optimal expected revenues over [0, s] for each pair of systems is V (s, y + 1) + V (s, y) = V (s, y) + V (s, y + 1), showing that the system on the right gets as much revenue as the system on the left even if sub-optimal polices are used for part of the horizon. Finally, if the inventory of the system with x − 1 units of inventory drops to 0 at some time s ∈ [0, t), so that state of the systems on the left are, respectively, (s, y) and (s, 0) for some y, such that 1 < y ≤ x, while the systems on the left are (s, y − 1) and (s, 1). From the inductive hypothesis, we know that ΔV (s, y) ≤ ΔV (s, 1) for all y ≤ x and all s ≤ t. Consequently,

$$\displaystyle \begin{aligned}V(s,y) = V(s,y) + V(s,0) \leq V(s,y-1) + V(s,1),\end{aligned}$$

and once again the pair of systems on the right result in at least as much revenue even though sub-optimal policies are used for part of the sales horizon.

We now show that ΔV (t, x) is increasing in t. This is equivalent to

$$\displaystyle \begin{aligned}\frac{\partial V(t,x)}{\partial t} = \mathcal{R}_t(\varDelta V(t,x)) \geq \mathcal{R}_t(\varDelta V(t,x-1)) = \frac{\partial V(t,x-1)}{\partial t},\end{aligned}$$

but this is true on account of $\mathcal {R}_t(z)$ being decreasing in z and ΔV (t, x) being decreasing in x.

Notice that the set a(t, x) is increasing in x since ΔV (t, x) is decreasing in x. Consequently, the set A(t, x) is also increasing in x.

We now show that V (t, x) is strictly increasing in t when d _t(p) = d(p) is time invariant. This is because

$$\displaystyle \begin{aligned}\frac{\partial V(t,x)}{\partial t} = r(\varDelta V(t,x)) \geq r(\varDelta V(t,1)) = r(V(t,1)) > 0,\end{aligned}$$

where the first inequality follows because r is decreasing and V (t, 1) = ΔV (t, 1) ≥ ΔV (t, x) for all x ≥ 1. The strict inequality follows because V (t, 1) must be below p ₁ as otherwise if V (t, 1) = p ₁, then the single unit of inventory must be priced at p ₁ over the horizon [0, t] and must sell with probability one over that interval. However, there is a positive probability equal to $e^{-\lambda _1 t}$ that the unit does not sell, so V (t, 1) < p ₁, implying that r(V (t, 1)) > 0, so V (t, x) is strictly increasing in t.

To show that V (t, x) is concave, notice that since r(z) is almost everywhere differentiable, then

$$\displaystyle \begin{aligned}\frac{\partial^2 V(t,x)}{\partial t^2} = r'(\varDelta V(t,x)) \frac{\partial \varDelta V(t,x)}{\partial t} \leq 0,\end{aligned}$$

follows since r′(z) ≤ 0, on account of r(z) being decreasing in z, and from the fact that ΔV (t, x) is increasing in t. The fact that r(z) is not differentiable at points p _j ∈ M does not change the argument because we can take the right derivative of r and things work well given that ΔV (t, x) is increasing in t.

Proof of Lemma 1.20

We will first show part that ΔV _j(t, x) is decreasing in x which is equivalent to showing that 2V _j(t, x) ≥ V _j(t, x + 1) + V _j(t, x − 1)] for all x ≥ 1. Let A be an optimal admission control rule starting from state (t, x + 1) and let B be an optimal admission control rule starting from (t, x − 1). These admission control rules are mappings from the state space to subsets S _k = {1, …, k}, k = 0, 1, …, j where S ₀ = ∅ is the optimal control whenever a system runs out of inventory. Consider four systems: two starting from state (t, x), using control rules A′ and B′, respectively, and one each starting from (t, x + 1) and (t, x − 1), using control rule A and B, respectively. Our goal is to specify heuristic control rules A′ and B′ that together make the expected revenues of the two systems starting with (t, x) at least as large as the expected revenues from the systems starting at (t, x + 1) and (t, x − 1). This will imply that 2V _j(t, x) ≥ V _j(t, x + 1) + V _j(t, x − 1).

We will use the control rules A′ = A ∩ B and B′ = A ∪ B until the first time, if ever, the remaining inventory of the system (t, x) controlled by A′ is equal to the remaining inventory of the system (t, x + 1) controlled by A. This will happen the first time, if ever, there is a sale under A and not under A′, i.e. a sale under A but not under B. Let t′ be the first time this happens, if it happens before the end of the horizon, and set t′ = 0 otherwise. If t′ > 0 then we apply policy A′ = A and B′ = B over s ∈ [0, t′). We claim that the expected revenue from the two systems starting with (t, x) is the same as the expected revenue from the other two systems. This is because the sales and revenues up to, but before t′, are the same in the two systems. At t′ sales occur only for the system (t, x) controlled by B′ and the system (t, x + 1) controlled by A, and the revenues from the two sales are identical. After the sales at t′, the inventory of the system (t, x) controlled by A′ becomes identical to the inventory of the system (t, x + 1) controlled by A while the inventory of the system (t, x) controlled by B′ becomes identical to the inventory of the system (t, x − 1) controlled by B. Since the policy switches to A′ = A and B′ = B, then sales and revenues are the same over [0, t′). If t′ = 0, then the sales of the two systems are the same during the entire horizon.

It remains to verify that inventories don’t become negative. Prior to time t′, the systems remain balance in the sense that system (t, x) governed by A′ always has one unit of inventory less than system (t, x + 1) governed by A and system (t, x) governed by B′ has one more unit of inventory than system (t, x − 1) governed by B. Thus the only two systems that could potential run out of inventory before t′ are A′ and B.

Since sales under A′ = A ∩ B are more restricted than sales under B, the inventory of system (t, x) governed by A′ will always be at least one unit since at most x − 1 units of sale are allowed under B. Therefore the only way the system can run out of inventory is if system (t, x − 1) runs out of inventory under B before t′. However, in this case, sales would stop under systems A′ and B, while sales will continue under B′ = A and A so revenues will continue to be the same until the first sale under A at which point we reached t′. This shows that even if the system (t, x − 1) runs out of inventory under B the two systems continue to have the same revenues over the entire horizon. Consequently 2ΔV _j(t, x) ≥ V _j(t, x + 1) + V _j(t, x − 1) for all x ≥ 1.

To show that ΔV _j(t, x) is increasing in j, it is enough to show that

$$\displaystyle \begin{aligned}V_j(t,x) + V_{j-1}(t,x-1) \geq V_j(t,x-1) + V_{j-1}(t,x).\end{aligned}$$

To do this, we again use a sample path argument. Let A be an optimal admission control rule for the system (j, t, x − 1) and B be an admission control rule for the system (j − 1, t, x). Let A′ and B′ be heuristic admission rules applied, respectively, to the systems (j, t, x) and (j − 1, t, x − 1). Our goal is to exhibit heuristics A′ and B′ such that when applied to the systems (j, t, x) and (j − 1, t, x − 1) they generate as much revenue as the applying A to (j, t, x − 1) and B to (j − 1, t, x). This will imply that V _j(t, x) + V _j−1(t, x − 1) ≥ V _j(t, x − 1) + V _j−1(t, x).

Let A′ = A ∪ B and B′ = A ∩ B and let t′ be the first time there is a sale under A ∪ B without a corresponding sale in A, so there is a sale under B but not under A. If t′ = 0, then the revenues of the sets of two systems are equal. If t′ > 0 switch at that point to the policy A′ = A and B′ = B. Then sales and revenues under both sets of two systems are equal up to t′. At t′ there are sales for the system (j, t, x) and (j − 1, t, x − 1) that generate the same revenues. Moreover, the inventories of the two sets of two systems have the same inventories immediately after the sale at t′. Since the policy then switches to A′ = A and B′ = B then sales and revenues are the same for the two set of systems over s ∈ [0, t′). The only system in danger to run out of inventory is system (j, t, x) under A′ = A ∪ B, but that system has the same number of sales as the system (j, t, x − 1) under A up to t′. Therefore the system (j, t, x) has at least one unit of inventory up to t′.

To show that ΔV _j(t, x) is increasing in t it is enough to show that

$$\displaystyle \begin{aligned}V_j(t,x) + V_j(t-1,x-1) \geq V_j(t,x-1) + V_j(t-1,x).\end{aligned}$$

To do this we again use a sample path argument. Let A be an optimal admission control rule for the system (t, x − 1) and B be an optimal admission control rule for the system (t − 1, x) Let A′ and B′ be heuristic admission rules applied, respectively, to the systems (t, x) and (t − 1, x − 1). Our goal is to exhibit heuristics A′ and B′ such that when applied to the systems (t, x) and (t − 1, x − 1) they generate as much revenue as the applying A to (t, x − 1) and B to (t − 1, x). This will imply that V _j(t, x) + V _j(t − 1, x − 1) ≥ V _j(t, x − 1) + V _j(t − 1, x). Let A′ = A ∪ B and B′ = A ∩ B and let t′ be the first time there is a sale under A′ without a corresponding sale in A, so there is a sale under B but not under A. If t′ = 0 then the revenues of the sets of two systems are equal. If t′ > 0 switch at that point to the policy A′ = A and B′ = B. Then sales and revenues under both sets of two systems are equal up to t′. At t′, there are sales for the system (t, x) and (t − 1, x) that generate the same revenues. Moreover, the inventories of the two sets of two systems have the same inventories immediately after the sale at t′. Since the policy then switches to A′ = A and B′ = B, then sales and revenues are the same for the two set of systems over s ∈ [0, t′). The only system in danger to run out of inventory is system (t − 1, x − 1) under B′ = A ∪ B, but that system has the same number of sales as the system (t − 1, x) under B up to t′. Therefore, the system (t − 1, x − 1) has at least one unit of inventory up to t′.

Proof of Lemma 1.21

We will first show that a _j(t, x) can also be characterized as $a_j(t,x) = \max \{k \leq j: p_k \geq \varDelta V_k(t-1,x)\}$. The result will then follow from Lemma 1.20. First notice that if a _j(t, x) = k < j then V _i(t, x) = V _k(t, x) for all i ∈{k, …, j}. Moreover, a _j(t, x) = k < j implies that W _k(t, x) > W _k+1(t, x). Consequently, 0 > W _k+1(t, x) − W _k(t, x) = (p _k+1 − ΔV _k+1(t − 1, x))λ _k+1, so p _k+1 < ΔV _k+1(t − 1, x). Conversely, if p _k ≥ ΔV _k(t − 1, x) then W _k(t, x) − W _k−1(t, x) ≥ (p _k − ΔV _k(t − 1, x))λ _k ≥ 0 so W _k(t, x) ≥ W _k−1(t, x). With the new characterization, we now turn to the monotonicity of $a_j(t,x) = \max \{k \leq j: p_k \geq \varDelta V_k(t-1,x)\}$. The monotonicity with respect to j is obvious because it expands the set over which we are maximizing. To see the monotonicity with respect to t, notice that ΔV _k(t, x) ≥ ΔV _k(t − 1, x) so k is excluded from the set whenever ΔV _k(t − 1, x) ≤ p _k < ΔV _k(t, x). To see the monotonicity with respect to x, notice that ΔV _k(t − 1, x + 1) ≤ ΔV _k(t, x) ≤ p _k implies that k contributes positively at state (t − 1, x + 1) whenever it contributes at (t − 1, x).

Proof of Theorem 1.22

The properties of A _j(t, x) follow from the properties of a _j(t, x) established in Lemma 1.21. Note that T _j = {(t, x) : a _j(t, x) < j}. From Lemma 1.21, a _j(t, x) < j implies that a _i(t, x) < i for all i > j, so T _j ⊆ T _i for all i > j. This implies that y _j(t) is increasing in j for any t ≥ 0. If t′ > t, then a _j+1(t′, y _j(t)) ≤ a _j+1(t, y _j(t)) < j + 1, so y _j(t′) ≥ y _j(t). Note that y _j(t) ≤ y _i(t) for all i > j, then x ≤ y _j(t) implies V _i+1(t, x) = V _i(t, x) for all i ≥ j and therefore V _i(t, x) = V _j(t, x) for all i > j.