Whittle indexability in egalitarian processor sharing systems

Abstract

The egalitarian processor sharing model is viewed as a restless bandit and its Whittle indexability is established. A numerical scheme for computing the Whittle indices is provided, along with supporting numerical experiments.

Appendix

Proof of Lemma 7

Recall that $\xi$ is Bernoulli(p), D is the departure random variable which is distributed as $D \sim \ Bin(x,\frac{q}{x})$ where x is the number of jobs currently in the server and q is the processing rate of the server. Consider the following:

$$\begin{aligned} f(x)&= \mathbb {E}_x[V(x - D + \xi )] - \mathbb {E}_x[V(x - D)] \\&= \sum _{d=0}^{x}{x \atopwithdelims ()d}\bigg (\frac{q}{x} \bigg )^d \bigg (1-\frac{q}{x} \bigg )^{x-d}(pV(x+1-d) + (1-p)V(x-d) - V(x-d))\\&= \sum _{d=0}^{x}{x \atopwithdelims ()d}\bigg (\frac{q}{x} \bigg )^d \bigg (1-\frac{q}{x} \bigg )^{x-d}(p(V(x+1-d) - V(x-d)))\\ \end{aligned}$$

For an optimal policy to be a threshold policy, we need $f(x+d) \ge f(x)$ for $d>0$. Therefore it is sufficient to show that $f(x+1) - f(x) > 0$. We have:

$$\begin{aligned} f(x+1)&= \sum _{d=0}^{x+1}{x+1 \atopwithdelims ()d}\bigg (\frac{q}{x+1} \bigg )^d \bigg (1-\frac{q}{x+1} \bigg )^\text {x+1-d}\\&\quad \times (p(V(x+2-d) - V(x+1-d))). \end{aligned}$$

Consider the following bound for f(x):

$$\begin{aligned} f(x)&\le \sum _{d=0}^{x}{x \atopwithdelims ()d}\bigg (\frac{q}{x+1} \bigg )^d \bigg (1-\frac{q}{x+1} \bigg )^{x-d}(p(V(x+1-d) - V(x-d))). \end{aligned}$$

This is true because the departure probability stochastically decreases as x increases. Since $(V(x+1-d) - V(x-d))$ decreases as d increases by Lemma 6, the bound follows by stochastic dominance.

Now we look at the difference

$$\begin{aligned} f(x+1) - f(x)&= \sum _{d=0}^{x}{x+1 \atopwithdelims ()d+1}\bigg (\frac{q}{x+1} \bigg )^\text {d+1} \bigg (1-\frac{q}{x+1} \bigg )^{x-d}\\&\quad \times p(V(x+1-d) - V(x-d))\\&\quad - {x \atopwithdelims ()d}\bigg (\frac{q}{x} \bigg )^d \bigg (1-\frac{q}{x} \bigg )^{x-d}p(V(x+1-d) - V(x-d)) \\&\quad + \bigg (1 - \frac{q}{x+1} \bigg )^\text {x+1} p(V(x+2) -V(x+1))\\&\ge \bigg (1 - \frac{q}{x+1} \bigg )^\text {x+1} p(V(x+2) -V(x+1))\\&\quad + p\sum _{d=0}^{x}{x \atopwithdelims ()d} \bigg ( \frac{q}{x+1} \bigg )^d \bigg (1-\frac{q}{x+1} \bigg )^{x-d}\\&\quad \times \bigg (\frac{x+1}{d+1} \frac{q}{x+1} -1\bigg )(V(x+1-d) - V(x-d)). \end{aligned}$$

Let $q_x = \frac{q}{x+1}$. Then we have

$$\begin{aligned} f(x+1) - f(x)&\ge \big (1 - q_x \big )^\text {x+1} p (V(x+2) -V(x+1))\\&\quad + p\sum _{d=0}^{x}{x \atopwithdelims ()d} \big ( q_x \big )^d \big (1-q_x \big )^{x-d}\\&\quad \times \bigg (\frac{x+1}{d+1} q_x -1\bigg )(V(x+1-d) - V(x-d)). \\ \end{aligned}$$

Let $B(x) = p(V(x+1) - V(x))$. From the Lemma 6, we know that B(.) is a non-decreasing function. The above inequality can be rewritten as

$$\begin{aligned} f(x+1) - f(x)&\ge \big (1 - q_x \big )^\text {x+1}B(x+1)\\&\quad + \sum _{d=0}^{x}{x \atopwithdelims ()d} \big ( q_x \big )^d \big (1-q_x \big )^{x-d} \bigg (\frac{x+1}{d+1} q_x -1\bigg )B(x-d). \\ \end{aligned}$$

We simplify the expression as follows:

$$\begin{aligned}&f(x+1) - f(x) \nonumber \\&\ge \big (1 - q_x \big )^\text {x+1}B(x+1) + \sum _{d=0}^{x}{x \atopwithdelims ()d} \big ( q_x \big )^d \big (1-q_x \big )^{x-d} \nonumber \\&\quad \times \bigg (\frac{x+1}{d+1} q_x -1\bigg )B(x-d) \nonumber \\&= (1-q_x)^{x+1} B(x+1) - q_x^x(1-q_x)B(0) \nonumber \\&\quad + \sum _{d=0}^{x-1} {x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x-d} \left( \frac{x+1}{d+1}q_x - 1 \right) B(x-d) \end{aligned}$$

(15)

$$\begin{aligned}&= (1-q_x)^{x+1} B(x+1) - q_x^x(1-q)B(0) \ \nonumber \\&\quad + \sum _{d=0}^{x-1} \left( {x \atopwithdelims ()d+1} q_x^{d+1} (1-q_x)^{x-d} - {x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x+1-d} \right) B(x-d) \\&= \left\{ (1-q_x)^{x+1} B(x+1) + \sum _{d=0}^{x-1}{x \atopwithdelims ()d+1} q_x^{d+1} (1-q_x)^{x-d}B(x-d)\right\} \nonumber \\&\quad - \left\{ q_x^x(1-q_x)B(0) + \sum _{d=0}^{x-1}{x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x+1-d}B(x-d) \right\} \nonumber \\&= \left\{ (1-q_x)^{x+1} B(x+1) + \sum _{d=1}^{x}{x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x+1-d}B(x+1-d)\right\} \nonumber \\&\quad - \sum _{d=0}^{x}{x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x+1-d}B(x-d)\nonumber \\&= \sum _{d=0}^{x}{x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x+1-d}B(x+1-d) - \sum _{d=0}^{x}{x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x+1-d}B(x-d) \nonumber \\&= (1-q_x) \sum _{d=0}^{x}{x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x-d} \left( B(x+1-d) - B(x-d) \right) \nonumber \\&\ge 0.\nonumber \end{aligned}$$

(16)

The passage from (15) to (16) is given in Borkar et al. (2017). The proof has been reproduced below for sake of completeness. We have proved that

$$\begin{aligned} f(x+1) - f(x) \ge 0, \end{aligned}$$

which shows that the optimal policy is a threshold policy. $\square$

Proof of term in (15) to (16)

From (15), we have:

$$\begin{aligned}&{x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x-d} \left( \frac{(x+1)q_x}{d+1} - 1 \right) \\&\quad = {x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x-d} \left( \frac{(x+1)q_x \pm q_xd}{d+1} - 1 \right) \\&\quad = {x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x-d} \left( \frac{(x-d)q_x + (d+1)q_x}{d+1} - 1 \right) \\&\quad = {x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x-d} \left( \frac{(x-d)q_x}{d+1} - 1 + q_x \right) \\&\quad = {x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x-d} \left( \frac{(x-d)q_x}{d+1} - (1 - q_x) \right) \\&\quad = {x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x-d} \frac{(x-d)q_x}{d+1} \\&\qquad - {x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x-d}(1 - q_x) \\&\quad = \frac{x!}{(x-d)! d!} \frac{(x-d)}{d+1} q_x^{d+1} (1-q_x)^{x-d} \\&\qquad - {x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x+1-d} \\&\quad = \frac{x!}{(x-d-1)! (d+1)!} q_x^{d+1} (1-q_x)^{x-d} \\&\qquad - {x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x+1-d} \\&\quad = {x \atopwithdelims ()d+1} q_x^{d+1} (1-q_x)^{x-d} \\&\qquad - {x \atopwithdelims ()d} q_x^{d} (1-q_x)^{x+1-d} \end{aligned}$$

The term on the right hand side of the final step is the desired term in (16). $\square$