Skip to main content
SpringerLink
Log in
Book cover

Constructive Computation in Stochastic Models with Applications pp 72–130Cite as

Block-Structured Markov Chains

  • Chapter

Abstract

In this chapter, the censoring technique is applied to be able to deal with any irreducible block-structured Markov chain, which is either discrete-time or continuous-time. The R-, U- and G-measures are iteratively defined from two different censored directions: UL-type and LU-type. An important censoring invariance for the R- and G-measures is obtained. Using the censoring invariance, the Wiener-Hopf equations are derived, and then the UL- and UL-types of RG-factorizations are given. The stationary probability vector is given an R-measure expression; while the transient probability can be computed by means of the R-, U- and G-measures. Finally, the A- and B-measures are proposed in order to discuss the state classification of the block-structured Markov chain.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   169.00
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bini D.A., G. Latouche and B. Meini (2005). Numerical Methods for Structured Markov Chains, Oxford University Press

    Google Scholar 

  2. Dudin A. and V.I. Klimenok (1999). Multi-dimensional quasi-Toeplitz Markov chains. Journal of Applied Mathematics and Stochastic Analysis 12: 393–415

    Article  MATH  MathSciNet  Google Scholar 

  3. Dudin A., C. Kim and V.I. Klimenok (2008). Markov chains with hybrid repeating rows-upper-Hessenberg, quasi-Toeplitz structure of the block transition probability matrix. Journal of Applied Probability 45: 211–225

    Article  MATH  MathSciNet  Google Scholar 

  4. Freedman D. (1983) Approximating Countable Markov Chains (Second Edition), Springer: New York

    MATH  Google Scholar 

  5. Grassmann W.K. and D.P. Heyman (1990). Equilibrium distribution of block-structured Markov chains with repeating rows. Journal of Applied Probability 27: 557–576

    Article  MATH  MathSciNet  Google Scholar 

  6. Grassmann W.K. (1993). Means and variances in Markov reward systems. In Linear Algebra, Markov Chains, and Queueing Models, C.D. Meyer and R.J. Plemmons (eds.), Springer-Verlag: New York, 193–204

    Google Scholar 

  7. Grassmann W.K. and D.P. Heyman (1993). Computation of steady-state probabilities for infinite-state Markov chains with repeating rows. ORSA Journal on Computing 5: 282–303

    Google Scholar 

  8. Heyman D.P. (1995). A decomposition theorem for infinite stochastic matrices. Journal of Applied Probability 32: 893–901

    Article  MATH  MathSciNet  Google Scholar 

  9. Karlin S. and H.W. Taylor (1975). A First Course in Stochastic Processes (Second Edition), Academic Press, INC.

    Google Scholar 

  10. Karlin S. and H.W. Taylor (1981). A Second Course in Stochastic Processes, Academic Press, INC.

    Google Scholar 

  11. Kemeny J.G., J.L. Snell and A.W. Knapp (1976). Denumerable Markov Chains (Second Edition), Springer-Verlag: NewYork

    MATH  Google Scholar 

  12. Klimenok V.I. and A. Dudin (2006). Multi-dimensional asymptotically quasi-Toeplitz Markov chains and their application in queueing theory. Queueing Systems 54: 245–259

    Article  MATH  MathSciNet  Google Scholar 

  13. Latouche G. (1992). Algorithms for infinite Markov chains with repeating columns. In Linear algebra, Markov chains, and queueing models, Minneapolis, MN, 231–265; IMA Vol. Math. Appl., 48, Springer: New York, 1993

    Google Scholar 

  14. Latouche G. and V. Ramaswami (1999). Introduction to Matrix Analytic Methods in Stochastic Models, ASA-SIAM

    Google Scholar 

  15. Lévy P. (1951). Systèmes markoviens et stationaires. Cas dénombrable. Ann. Sci. École Norm. Sup. 68: 327–381

    MATH  Google Scholar 

  16. Lévy P. (1952). Complément à l’étude des processus de Markoff. Ann. Sci. École Norm. Sup. 69: 203–212

    MATH  Google Scholar 

  17. Lévy P. (1958). Processus markoviens et stationaires. Cas dénombrable. Ann. Inst. H. Poincaré 18: 7–25

    Google Scholar 

  18. Li Q.L. and J. Cao (2004). Two types of RG-factorizations of quasi-birth-and-death processes and their applications to stochastic integral functionals. Stochastic Models 20:299–340

    Article  MATH  MathSciNet  Google Scholar 

  19. Li Q.L. and L.M. Liu (2004). An algorithmic approach on sensitivity analysis of perturbed QBD processes. Queueing Systems 48: 365–397

    Article  MATH  MathSciNet  Google Scholar 

  20. Li Q.L. and Y.Q. Zhao (2002). A constructive method for finding β-invariant measures for transition matrices of M/G/1 type. In Matrix Analytic Methods: Theory and Applications, G. Latouche and P.G. Taylor (eds), World Scientific: New Jersey, 237–263

    Google Scholar 

  21. Li Q.L. and Y.Q. Zhao (2002). The RG-factorizations in block-structured Markov renewal processes with applications. Technical Report 381, Laboratory for Research in Statistics and Probability, Carleton University and University of Ottawa, Canada, 1–40

    Google Scholar 

  22. Li Q.L. and Y.Q. Zhao (2003). β-invariant measures for transition matrices of GI/M/1 type. Stochastic Models 19: 201–233

    Article  MathSciNet  Google Scholar 

  23. Li Q.L. and Y.Q. Zhao (2004). The RG-factorization in block-structured Markov renewal processes with applications. In Observation, Theory and Modeling of Atmospheric Variability, X. Zhu (ed), World Scientific, 545–568

    Google Scholar 

  24. Li Q.L. and Y.Q. Zhao (2005). Light-tailed asymptotics of stationary probability vectors of Markov chains of GI/G/1 type. Advances in Applied Probability 37: 1075–1093

    Article  MATH  MathSciNet  Google Scholar 

  25. Li Q.L. and Y.Q. Zhao (2005). Heavy-tailed asymptotics of stationary probability vectors of Markov chains of GI/G/1 type. Advances in Applied Probability 37: 482–509

    Article  MATH  MathSciNet  Google Scholar 

  26. Neuts M.F. (1981). Matrix-Geometric Solutions in Stochastic Models-An Algorithmic Approach, The Johns Hopkins University Press: Baltimore

    MATH  Google Scholar 

  27. Neuts M.F. (1989). Structured Stochastic Matrices of M/G/1 Type and Their Applications, Marcel Dekker: New York

    MATH  Google Scholar 

  28. Zhao Y.Q. (2000). Censoring technique in studying block-structured Markov chains. In Advances in Algorithmic Methods for Stochastic Models, G. Latouche and P.G. Taylor (eds), Notable Publications Inc., NJ., 417–433

    Google Scholar 

  29. Zhao Y.Q. and D. Liu (1996). The censored Markov chain and the best augmentaion. Journal of Applied Probability 33: 623–629

    Article  MATH  MathSciNet  Google Scholar 

  30. Zhao Y.Q., W. Li and A.S. Alfa (1999). Duality results for block-structured transition matrices. Journal of Applied Probability 36: 1045–1057

    Article  MATH  MathSciNet  Google Scholar 

  31. Zhao Y.Q., W. Li and W.J. Braun (1998). Infinite block-structured transition matrices and their properties. Advances in Applied Probability 30: 365–384

    Article  MATH  MathSciNet  Google Scholar 

  32. Zhao Y.Q., W. Li and W.J. Braun (2003). Censoring, factorizations and spectral analysis for transition matrices with block-repeating entries. Methodology and Computing in Applied Probability 5: 35–58

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Industrial Engineering, Tsinghua University, Beijing, 100084, P.R. China

    Quan-Lin Li

Authors
  1. Quan-Lin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Tsinghua University Press, Beijing and Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Li, QL. (2010). Block-Structured Markov Chains. In: Constructive Computation in Stochastic Models with Applications. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-11492-2_2

Download citation

Publish with us

Policies and ethics

Search

Navigation