Analysis and computation of the stationary distribution in a special class of Markov chains of level-dependent M/G/1-type and its application to BMAP/M/$\infty$ and BMAP/M/c+M queues

Abstract

This paper considers a special class of continuous-time Markov chains of level-dependent M/G/1-type, where block matrices representing downward jumps in the infinitesimal generator are nonsingular. This special class naturally arises in the analysis of BMAP/M/$\infty$ queues and BMAP/M/c queues with exponential impatience times (BMAP/M/c+M). We first formulate the boundary probability vector in terms of a solution of a system of infinitely many linear inequalities. We then reveal that in the above special class, this infinite system is regarded as a nested sequence of simplices, and we identify their vertices. Based on these results, we develop a simple yet efficient computational algorithm for the stationary distribution conditioned that the level is not greater than a predefined N. Note that for a large N, the conditional distribution will provide a good approximation to the stationary distribution. Some numerical examples for BMAP/M/$\infty$ and BMAP/M/c+M queues are shown.

Appendices

Appendix 1: Proof of Lemma 1

It follows from (10) and (11) that Lemma 1 holds if $\varvec{U}_{k,n}$ satisfying

$$\begin{aligned}&\begin{pmatrix} \varvec{U}_{k,0}&\varvec{U}_{k,1}&\cdots&\varvec{U}_{k,k-1} \end{pmatrix} \qquad \qquad \nonumber \\&\quad = \begin{pmatrix} \varvec{O}&\quad \varvec{O}&\quad \cdots&\quad \varvec{O}&\quad \varvec{Q}_{k,-1} \end{pmatrix}\cdot (-1)\nonumber \\&\qquad \cdot \begin{pmatrix} \varvec{Q}_{0,0} &{}\quad \varvec{Q}_{0,1} &{}\quad \cdots &{}\quad \varvec{Q}_{0,k-3} &{}\quad \varvec{Q}_{0,k-2} &{}\quad \varvec{Q}_{0,k-1}\\ \varvec{Q}_{1,-1} &{}\quad \varvec{Q}_{1,0} &{}\quad \cdots &{}\quad \varvec{Q}_{1,k-4} &{}\quad \varvec{Q}_{1,k-3} &{}\quad \varvec{Q}_{1,k-2}\\ \varvec{O} &{}\quad \varvec{Q}_{2,-1} &{}\quad \cdots &{}\quad \varvec{Q}_{2,k-5} &{}\quad \varvec{Q}_{2,k-4} &{}\quad \varvec{Q}_{2,k-3}\\ \vdots &{}\quad \vdots &{}\quad \ddots &{}\quad \vdots &{}\quad \vdots &{}\quad \vdots \\ \varvec{O} &{}\quad \varvec{O} &{}\quad \cdots &{}\quad \varvec{Q}_{k-2,-1} &{}\quad \varvec{Q}_{k-2,0} &{}\quad \varvec{Q}_{k-2,1}\\ \varvec{O} &{}\quad \varvec{O} &{}\quad \cdots &{}\quad \varvec{O} &{}\quad \varvec{Q}_{k-1,-1} &{}\quad \varvec{Q}_{k-1,0} \end{pmatrix}^{-1}, \end{aligned}$$

(45)

is given by (12). Note that (45) is equivalent to

$$\begin{aligned}&\sum _{m=0}^{n+1} \varvec{U}_{k,m} \varvec{Q}_{m,n-m} = \varvec{O}, \quad n=0,1,\ldots ,k-2, \end{aligned}$$

(46)

$$\begin{aligned}&\sum _{m=0}^{k-1} \varvec{U}_{k,m} \varvec{Q}_{m,k-1-m} = -\varvec{Q}_{k,-1}. \end{aligned}$$

(47)

On the other hand, (12) is equivalent to

$$\begin{aligned} \varvec{U}_{k,k-1}= & {} \varvec{Q}_{k,-1}(-\varvec{T}_{k-1})^{-1}, \quad k=1,2,\ldots , \end{aligned}$$

(48)

$$\begin{aligned} \varvec{U}_{k,n}= & {} \varvec{U}_{k,n+1} \varvec{Q}_{n+1,-1} (-\varvec{T}_n)^{-1}, \quad k=2,3,\ldots , n=0,1,\ldots ,k-2. \end{aligned}$$

(49)

Therefore, Lemma 1 is proved if we show that the $\varvec{U}_{k,n}$s satisfying (46) and (47) are given by (48) and (49).

From (47) for $k=1$, we have $\varvec{U}_{1,0} \varvec{Q}_{0,0} = -\varvec{Q}_{1,-1}$ and therefore

$$\begin{aligned} \varvec{U}_{1,0} = \varvec{Q}_{1,-1} (-\varvec{Q}_{0,0})^{-1} = \varvec{Q}_{1,-1} (-\varvec{T}_0)^{-1}, \end{aligned}$$

which shows that Lemma 1 holds for $k=1$.

Next, we consider the case of $k \ge 2$. For a fixed $k\;(k=2,3,\ldots )$, we first consider $\varvec{U}_{k,n} (n=0,1,\ldots ,k-2)$. It follows from (46) that for $n=0$,

$$\begin{aligned} \varvec{U}_{k,0} \varvec{Q}_{0,0} + \varvec{U}_{k,1} \varvec{Q}_{1,-1} = \varvec{O}. \end{aligned}$$

Therefore, noting that $\varvec{T}_0 = \varvec{Q}_{0,0}$, we have

$$\begin{aligned} \varvec{U}_{k,0} = \varvec{U}_{k,1} \varvec{Q}_{1,-1} (-\varvec{T}_0)^{-1}, \end{aligned}$$

which shows that (49) holds for $n=0$.

We now assume that (49) holds for some $k \ge 3$ and $n=l\, (l=0,1,\ldots ,k-3)$. It then follows from (46) that for $n=l+1$,

$$\begin{aligned} \varvec{O}= & {} \sum _{m=0}^{l+1} \varvec{U}_{k,m} \varvec{Q}_{m,l+1-m} + \varvec{U}_{k,l+2} \varvec{Q}_{l+2,-1}\\= & {} \sum _{m=0}^{l+1} \varvec{U}_{k,l+1} \varvec{Q}_{l+1,-1} (-\varvec{T}_l)^{-1} \varvec{Q}_{l,-1} (-\varvec{T}_{l-1})^{-1} \cdot \cdots \cdot \varvec{Q}_{m+1,-1} (-\varvec{T}_m)^{-1}\\&\varvec{Q}_{m,l+1-m} + \varvec{U}_{k,l+2} \varvec{Q}_{l+2,-1}\\= & {} \varvec{U}_{k,l+1} \varvec{T}_{l+1} + \varvec{U}_{k,l+2} \varvec{Q}_{l+2,-1}. \end{aligned}$$

Therefore, (49) holds for $n=l+1$. We thus conclude that (49) holds for $n=0,1,\ldots ,k-2$.

Finally, we consider (47) for $k \ge 2$. Substituting (49) for $n=0,1,\ldots ,k-2$ into (47) yields

$$\begin{aligned} -\varvec{Q}_{k,-1}= & {} \sum _{m=0}^{k-1} \varvec{U}_{k,m} \varvec{Q}_{m,k-1-m}\\= & {} \sum _{m=0}^{k-1} \varvec{U}_{k,k-1} \varvec{Q}_{k-1,-1} (-\varvec{T}_{k-2})^{-1} \varvec{Q}_{k-2,-1} (-\varvec{T}_{k-3})^{-1} \cdot \cdots \cdot \\&\varvec{Q}_{m+1,-1} (-\varvec{T}_m)^{-1} \varvec{Q}_{m,k-1-m}\\= & {} \varvec{U}_{k,k-1} \varvec{T}_{k-1}, \end{aligned}$$

so that (48) holds. $\square$

Appendix 2: Proof of Lemma 2

We prove the lemma by showing that $\overline{\varvec{p}}_0$ is a feasible solution of (15). We first consider $k=0$. By definition, $\overline{\varvec{p}}_0 \varvec{e} = 1$ and $\overline{\varvec{p}}_0 > \mathbf{0}$ owing to the positive recurrence of the Markov chain. For $k=1$, we have

$$\begin{aligned} \overline{\varvec{p}}_0 \varvec{S}_1 = p_0^{-1} \varvec{p}_0\varvec{S}_1 = p_0^{-1} \varvec{p}_0 \varvec{Q}_{0,0} (-\varvec{Q}_{k,-1})^{-1} = p_0^{-1} \varvec{p}_1 > \varvec{0}, \end{aligned}$$

where the third equality follows from the global balance equation (2) for $k=0$.

Suppose $\overline{\varvec{p}}_0 \varvec{S}_k = p_0^{-1} \varvec{p}_k$ for all $k=1,2,\ldots ,l-1\, (l \ge 2)$. We then have for $k=l$,

$$\begin{aligned} \overline{\varvec{p}}_0 \varvec{S}_l = p_0^{-1} \varvec{p}_0\varvec{S}_l = p_0^{-1} \varvec{p}_0 \left( \varvec{Q}_{0,l-1} + \sum _{n=1}^{l-1} \varvec{S}_n \varvec{Q}_{n,l-1-n} \right) (-\varvec{Q}_{l,-1})^{-1} = p_0^{-1} \varvec{p}_l, \end{aligned}$$

where the third equality follows from (2) for $k=l-1$. We thus obtain $\overline{\varvec{p}}_0 \varvec{S}_l > \varvec{0}$, which completes the proof. $\square$

Appendix 3: Tips for program implementation

The main source of numerical errors is the inverse of $\varvec{T}\, (=\varvec{T}_k)$. In our implementation, we first obtain $(-\varvec{T})^{-1}$ by means of the LU decomposition and then make a fine adjustment as follows. Because the row sum of the infinitesimal generator is equal to zero, we have (cf. (33))

$$\begin{aligned} \varvec{T}_k \varvec{e} + \sum _{n=1}^{\infty } \varvec{Q}_{k,n} \varvec{e} + \sum _{n=0}^{k-1} \varvec{Z}_{k-1} \varvec{Z}_{k-2} \cdot \cdots \cdot \varvec{Z}_n \sum _{l=k+1-n}^{\infty } \varvec{Q}_{n,l} \varvec{e} = \varvec{0}, \end{aligned}$$

from which it follows that

$$\begin{aligned} (-\varvec{T}_k)^{-1} \left[ \sum _{n=1}^{\infty } \varvec{Q}_{k,n} \varvec{e} + \sum _{n=0}^{k-1} \varvec{Z}_{k-1} \varvec{Z}_{k-2} \cdot \cdots \cdot \varvec{Z}_n \sum _{l=k+1-n}^{\infty } \varvec{Q}_{n,l} \varvec{e} \right] = \varvec{e}, \quad k=0,1,\ldots . \end{aligned}$$

(50)

Therefore, we normalize every row of $(-\varvec{T}_k)^{-1}$ in such a way that (50) holds. In our limited experience, this fine adjustment is crucial to making our procedure numerically stable for large-scale models.

Besides, we intentionally describe $\varvec{T}\, (=\varvec{T}_k)$ in terms of $\varvec{Z}_m\, (m=0,2,\ldots ,k-1)$ in Figure 1 because we expect readers to use the Horner algorithm [9, p.81]. More specifically, $\varvec{T}:=\varvec{T}_k$ is expected to be computed as follows:

$$\begin{aligned}&\varvec{Y}_{-1} = \varvec{O}, \qquad \varvec{Y}_n = \varvec{Z}_n (\varvec{Q}_{n,k-n} + \varvec{Y}_{n-1}) \quad (n=0,1,\ldots ,k-1), \\&\quad \varvec{T}=\varvec{Q}_{k,0} + \varvec{Y}_{k-1}. \end{aligned}$$

The merit of the Horner algorithm in our procedure is not only the computational efficiency but also the numerical accuracy in $\varvec{T}$. Note that the magnitude of elements in $\varvec{Z}_m$ is moderate, whereas for a large k, elements in $\varvec{Q}_{n,k}$ can be very small. By using the Horner algorithm, we can sum up terms from small ones to large ones, which maintains the numerical accuracy in $\varvec{T}$. Note that the Horner algorithm will also apply to (50).