Exact analysis of the M/M/k/setup class of Markov chains via recursive renewal reward

Abstract

The M/M/k/setup model, where there is a penalty for turning servers on, is common in data centers, call centers, and manufacturing systems. Setup costs take the form of a time delay, and sometimes there is additionally a power penalty, as in the case of data centers. While the M/M/1/setup was exactly analyzed in 1964, no exact analysis exists to date for the M/M/k/setup with $k>1$. In this paper, we provide the first exact, closed-form analysis for the M/M/k/setup and some of its important variants including systems in which idle servers delay for a period of time before turning off or can be put to sleep. Our analysis is made possible by a new way of combining renewal reward theory and recursive techniques to solve Markov chains with a repeating structure. Our renewal-based approach uses ideas from renewal reward theory and busy period analysis to obtain closed-form expressions for metrics of interest such as the transform of time in system and the transform of power consumed by the system. The simplicity, intuitiveness, and versatility of our renewal-based approach makes it useful for analyzing Markov chains far beyond the M/M/k/setup. In general, our renewal-based approach should be used to reduce the analysis of any 2-dimensional Markov chain which is infinite in at most one dimension and repeating to the problem of solving a system of polynomial equations. In the case where all transitions in the repeating portion of the Markov chain are skip-free and all up/down arrows are unidirectional, the resulting system of equations will yield a closed-form solution.

Appendices

Appendix 1: Recursion lemmas

Lemma 1

(Recursion lemma for mean time) For the M/M/k/setup, the mean time to move one step left from state $(i,j)$, $T^{L}_{i,j}$, is the same for all $j \ge k$.

Proof

For any $j\ge k$, observe that when moving one step left from any state $(i,j)$, we only visit states with level $j$ or greater, until the final transition to level $j-1$. Hence, $T^{L}_{i,j}$ depends only on the structure of the “subchain” of the M/M/k/setup consisting of levels $\{j,j+1,\ldots \}$, including transition rates to level $j-1$. Now consider the subchain for each $j\ge k$; these subchains are isomorphic, by the fact that the chain is repeating from level $k$ onward. Hence, the time to move one step left is the same regardless of the initial level $j\ge k$. $\square$

Lemma 2

(Recursion lemma for mean reward) For the M/M/k/setup, the mean reward earned in moving one step left from state $(i,j+1)$, $R^L_{i,j+1}$, satisfies $R^{L}_{i,j+1} = R^{L}_{i,j} + T^{L}_{i,j}$ for all $j \ge k$, where the reward tracks the number of jobs in the system.

Proof

Consider the process of moving one step left from a given state $(i,j)$ where $j\ge k$. At the same time, consider the same process where everything is shifted over one level to the right, so that the initial state is $(i,j+1)$ At any point in time, the number of jobs seen by the second process is exactly one greater than that seen by the first process. Therefore, the total number of jobs accumulated (total reward) during the second process is $T_{i,j}^L$ greater than that of the first process, since the duration of both processes is $T_{i,j}^L$ by Lemma 1.$\square$

Lemma 3

(Recursion lemma for transform of reward) For the M/M/k/setup, $\dot{R}^{L}_{i,j+1} = z\cdot \dot{R}^{L}_{i,j}$, for all $j \ge k$, where $\dot{R}$ tracks the z-transform of the number of jobs in the system.

Proof

The proof is identical to that of Lemma 2, except that in any moment in time the second process (starting in level $(i,j+1)$) earns $z$ times as much reward as the first process (starting at $(i,j)$).$\square$

Lemma 4

(Recursion lemma for transform of power) For the M/M/k/setup, $\dot{E}^{L}_{i,j+1} = \dot{E}^{L}_{i,j} = T^{L}_{i,j} \cdot z^{k \cdot P_\mathrm{peak}}$, for all $j \ge k$.

Proof

When $j\ge k$, all $k$ servers are either on or in setup, putting power consumption at $k\cdot P_\mathrm{peak}$. So the transform of power usage is $z^{k\cdot P_\mathrm{peak}}$, yielding $\dot{E}^L_{i,j} = T^{L}_{i,j}\cdot z^{k\cdot P_\mathrm{peak}}$. It then follows immediately from Lemma 1 that $\dot{E}^{L}_{i,j+1} = \dot{E}^{L}_{i,j}$.$\square$

Lemma 5

(Recursion lemma for probability) For the M/M/k/setup, for each $0 \le d \le k$ and for each $0 \le i \le k$, $p^{L}_{i \rightarrow d}$ is the same for all $j \ge k$.

Proof

Recall that $p^{L}_{i\rightarrow d}$ is the probability that, given that we start at depth $i$, we end at depth $d$ when moving one step to the left, except when $j=k$ and $d\in \{k-1,k\}$; in these cases we interpret $p_{i\rightarrow k}^L$ (or $p_{i\rightarrow k-1}^L$) as the probabilities that we first moved one step left by transitioning out of a state in depth $k$ (or $k-1$).

As with $T^{L}_{i,j}$, $p^{L}_{i\rightarrow d}$ depends only on the structure of the “subchain” consisting of levels $\{j,j+1,\ldots \}$, including transition rates to level $j-1$. Since for all $j\ge k$ the resulting subchains are isomorphic, $p^{L}_{i\rightarrow d}$ must be the same for all $j\ge k$.$\square$

Appendix 2: Solution of the system of equations for M/M/k/setup

The steps below illustrate how to solve the system of equations for M/M/k/setup. All of the operations in the steps below can be performed symbolically to obtain closed-form results.

1.1 Solving for ${p^{L}_{i \rightarrow d}}$

The system of equations for $p^{L}_{i \rightarrow d}$ consists of equation sets (28), (29) and (30). Equation (28) are $k$ quadratic equations, each in one variable: $p_{0\rightarrow 0}^L,p_{1\rightarrow 1}^L,\ldots ,p_{k-2\rightarrow k-2}^L$, $p_{k-1\rightarrow k-1}^L$. Thus, we can solve each equation easily using the quadratic formula. It can be easily shown that among the two roots of each equation, the greater root exceeds 1, and is thus disregarded. The lesser root can be shown to lie in the interval $[0,1)$, making it the unique solution of interest to the quadratic equation. Note that $p_{0\rightarrow 0}^L=0$, as expected (we cannot move to the left when we have no servers on).

The set of equations (29) is a collection of $O(k^2)$ equations involving linear terms and products of two unlike variables. However, the structure of this system of equations reduces solving the system to solving a set of linear equations via back substitution. Consider solving this set of equations for the unknown values of $p_{i\rightarrow d}^L$ in this order:

$$\begin{aligned} p_{k-1\rightarrow k-1}^L, p_{k-2\rightarrow k-2}^L, p_{k-2\rightarrow k-1}^L, \ldots , p_{0\rightarrow 1}^L\, p_{0\rightarrow 2}^L, \ldots , p_{0\rightarrow k-1}^L \end{aligned}$$

That is, solving from greatest ($k-1$) to least ($0$) “original depth,” but within each original depth, solving from least to greatest “target depth.” Solving in this order, each equation we solve will only have one unknown, as all other variables will already have been solved for in an earlier step (including the $p_{i\rightarrow i}^L$ from Eq. (28)), so these variables can be viewed as coefficients and constant terms. Once we have solved Eq. (29), we can easily solve Eq. (30), yielding $p_{0\rightarrow k}^L,p_{1\rightarrow k}^L,\ldots ,p_{k-1\rightarrow k}^L$, by taking complements. It follows that all $p^{L}_{i \rightarrow d}$ can be solved in closed forms that are, at worst, linear combinations of radicals (i.e., square roots).

1.2 Solving for ${\dot{R}_{i,k}^L}$

The system of equations for $\dot{R}_{i,k}^L$ consists of Eqs. (31) and (32), and  (33). This system is a collection of $(k+1)$ linear equations with $(k+1)$ unknowns. Although we could solve this system using standard linear algebraic techniques, the structure of this system suggests an even simpler approach using back substitution. Solving for each $\dot{R}_{i,k}^L$ only requires knowing the $\dot{R}_{i,\ell }^L$ such that $\ell \in \{i+1,\ldots ,k\}$. Equation (33) readily gives us $\dot{R}_{k,k}^L$. Thus, we can now solve for $\dot{R}_{k-1,k}^L$, then $\dot{R}_{k-2,k}^L$, and so on. In this way, each $\dot{R}_{i,k}^L$ is found by solving a linear equation for one unknown variable.

1.3 Solving for ${\dot{R}_{i,j}^H}$

The system of equations for $\dot{R}_{i,j}^H$ consists of Eqs. (34), (35), (36) and (37), and  (38). This system is a collection of $O(k^2)$ dependent linear equations with just as many unknowns. Unlike the earlier systems of equations, there is no apparent structure we can exploit, so the system can be solved via standard linear algebraic techniques such as (symbolic) matrix inversion.