A sequential update algorithm for computing the stationary distribution vector in upper block-Hessenberg Markov chains
This paper proposes a new algorithm for computing the stationary distribution vector in continuous-time upper block-Hessenberg Markov chains. To this end, we consider the last-block-column-linearly-augmented (LBCL-augmented) truncation of the (infinitesimal) generator of the upper block-Hessenberg Markov chain. The LBCL-augmented truncation is a linearly augmented truncation such that the augmentation distribution has its probability mass only on the last block column. We first derive an upper bound for the total variation distance between the respective stationary distribution vectors of the original generator and its LBCL-augmented truncation. Based on the upper bound, we then establish a series of linear fractional programming (LFP) problems to obtain augmentation distribution vectors such that the bound converges to zero. Using the optimal solutions of the LFP problems, we construct a matrix-infinite-product (MIP) form of the original (i.e., not approximate) stationary distribution vector and develop a sequential update algorithm for computing the MIP form. Finally, we demonstrate the applicability of our algorithm to BMAP/M/\(\infty \) queues and M/M/s retrial queues.
KeywordsUpper block-Hessenberg Markov chain Level-dependent M/G/1-type Markov chain Matrix-infinite-product (MIP) form Last-block-column-linearly-augmented truncation (LBCL-augmented truncation) BMAP/M/\(\infty \) queue M/M/s retrial queue
Mathematics Subject Classification60J22 60K25
The author thanks Mr. Masatoshi Kimura and Dr. Tetsuya Takine for providing the counterexample presented in Sect. 2.3. The author also thanks an anonymous referee for his/her valuable comments that helped to improve the paper.
- 7.Horn, R.A., Johnson, C.R.: Matrix Analysis, 2nd edn. Cambridge University Press, Cambridge (2013)Google Scholar
- 9.Kimura, M., Takine, T.: Private communication (2016)Google Scholar
- 12.Kontoyiannis, I., Meyn, S.P.: On the \(f\)-norm ergodicity of Markov processes in continuous time. Electron. Commun. Probab. 21, Paper no. 77, 1–10 (2016)Google Scholar
- 19.Masuyama, H.: Limit formulas for the normalized fundamental matrix of the northwest-corner truncation of Markov chains: Matrix-infinite-product-form solutions of block-Hessenberg Markov chains (2016). Preprint arXiv:1603.07787
- 21.Phung-Duc, T., Masuyama, H., Kasahara, S., Takahashi, Y.: A simple algorithm for the rate matrices of level-dependent QBD processes. In: Proceedings of the 5th International Conference on Queueing Theory and Network Applications (QTNA2010), pp. 46–52. ACM, New York (2010)Google Scholar
- 23.Shin, Y.W., Pearce, C.E.M.: An algorithmic approach to the Markov chain with transition probability matrix of upper block-Hessenberg form. Korean J. Comput. Appl. Math. 5(2), 361–384 (1998)Google Scholar
- 25.Yajima, M., Phung-Duc, T., Masuyama, H.: The stability condition of BMAP/M/\(\infty \) queues. In: Proceedings of the 11th International Conference on Queueing Theory and Network Applications (QTNA2016), Article no. 5, pp. 1–6. ACM, New York (2016)Google Scholar