Skip to main content
SpringerLink
Log in

Perturbation analysis for continuous-time Markov chains

  • Articles
  • Published:
Science China Mathematics Aims and scope Submit manuscript

Abstract

We investigate perturbation for continuous-time Markov chains (CTMCs) on a countable state space. Explicit bounds on ΔD and D are derived in terms of a drift condition, where Δ and D represent the perturbation of the intensity matrices and the deviation matrix, respectively. Moreover, we obtain perturbation bounds on the stationary distributions, which extends the results by Liu (2012) for uniformly bounded CTMCs to general (possibly unbounded) CTMCs. Our arguments are mainly based on the technique of augmented truncations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

ReferencesS

  1. Altman E, Avrachenkov K E, Núñez-Queija R. Perturbation analysis for denumerable Markov chains with application to queueing models. Adv Appl Probab, 2004, 36: 839–853

    Article  MATH  Google Scholar 

  2. Breyer L, Roberts G O, Rosenthal J S. A note on geometric ergodicity and floating-point roundoff error. Statist Probab Lett, 2001, 53: 123–127

    Article  MATH  MathSciNet  Google Scholar 

  3. Chen M F. From Markov Chains to Non-Equilibrium Particle Systems, 2nd ed. Singapore: World Scientific, 2003

    Google Scholar 

  4. Coolen-Schrijner P, Van Doorn E A. The deviation matrix of a continuous-time Markov chain. Probab Engrg Inform Sci, 2002, 16: 351–366

    Article  MATH  MathSciNet  Google Scholar 

  5. Dendievel S, Latouche G, Liu Y. Poisson’s equation for discrete-time quasi-birth-death processes. Performance Evaluation, 2013, 70: 564–577

    Article  Google Scholar 

  6. Glynn P W, Meyn S P. A Liapounov bound for solutions of the Poisson equation. Ann Probab, 1996, 24: 916–931

    Article  MATH  MathSciNet  Google Scholar 

  7. Hart A G, Tweedie R L. Convergence of invariant measures of truncation approximations to Markov processes. Appl Math, 2012, 3: 2205–2215

    Article  Google Scholar 

  8. Heidergott B, Hordijk A, Leder N. Series expansions for continuous-time Markov processes. Oper Res, 2010, 58: 756–767

    Article  MATH  MathSciNet  Google Scholar 

  9. Hunter J J. Stationary distributions and mean first passage times of perturbed Markov chains. Linear Algebra Appl, 2005, 410: 217–243

    Article  MATH  MathSciNet  Google Scholar 

  10. Jiang S, Liu Y, Yao S. Poisson’s equation for discrete-time single-birth processes. Statist Probab Lett, 2014, 85: 78–83

    Article  MATH  MathSciNet  Google Scholar 

  11. Kartashov N V. Strongly stable Markov chains. J Soviet Math, 1986, 34: 1493–1498

    Article  MATH  Google Scholar 

  12. Kirkland S J, Neumann M, Xu J. Transition matrices for well-conditioned Markov chains. Linear Algebra Appl, 2007, 424: 118–131

    Article  MATH  MathSciNet  Google Scholar 

  13. Liu Y. Augmented truncation approximations of discrete-time Markov chains. Oper Res Lett, 2010, 38: 218–222

    Article  MATH  MathSciNet  Google Scholar 

  14. Liu Y. Perturbation bounds for the stationary distributions of Markov chains. SIAM J Matrix Anal Appl, 2012, 33: 1057–1074

    Article  MATH  MathSciNet  Google Scholar 

  15. Liu Y, Wang P, Xie Y. Deviation matrix and asymptotic variance for GI/M/1-type Markov chains. Front Math China, 2014, 9: 863–880

    Article  MATH  MathSciNet  Google Scholar 

  16. Liu Y, Zhang H, Zhao Y. Computable strongly ergodic rates of convergence for continuous-time Markov chains. ANZIAM J, 2008, 49: 463–478

    Article  MATH  MathSciNet  Google Scholar 

  17. Liu Y, Zhang H, Zhao Y. Subexponential ergodicity for continuous-time Markov chains. J Math Anal Appl, 2010, 368: 178–189

    Article  MATH  MathSciNet  Google Scholar 

  18. Lund R B, Meyn S P, Tweedie R L. Computable exponential convergence rates for stochastically ordered Markov processes. Ann Appl Probab, 1996, 6: 218–237

    Article  MATH  MathSciNet  Google Scholar 

  19. Mao Y H. Ergodic degrees for continuous-time Markov chains. Sci China Ser A, 2004, 47: 161–174

    Article  MATH  MathSciNet  Google Scholar 

  20. Mao Y H, Zhang Y H. Exponential ergodicity for single birth processes. J Appl Probab, 2004, 41: 1022–1032

    Article  MATH  MathSciNet  Google Scholar 

  21. Meyn S P, Tweedie R L. Stability of Markovian processes, III: Foster-Lyapunov criteria for continuous-time processes. Adv in Appl Probab, 1993, 25: 518–548

    Article  MATH  MathSciNet  Google Scholar 

  22. Meyn S P, Tweedie R L. Markov Chains and Stochastic Stability, 2nd ed. Cambridge: Cambridge University Press, 2009

    Book  MATH  Google Scholar 

  23. Mitrophanov A Y. Stability estimates for finite homogeneous continuous-time Markov chains. Theory Probab Appl, 2006, 50: 319–326

    Article  MATH  MathSciNet  Google Scholar 

  24. Mouhoubi Z, Aissani D. New perturbation bounds for denumerable Markov chains. Linear Algeba Appl, 2010, 432: 1627–1649

    Article  MATH  MathSciNet  Google Scholar 

  25. Roberts G O, Rosenthal J S. Geometric ergodicity and hybrid Markov chains. Electron Comm Probab, 1997, 2: 13–25

    Article  MATH  MathSciNet  Google Scholar 

  26. Roberts G O, Rosenthal J S, Schwartz P O. Convergence properties of perturbed Markov chains. J Appl Probab, 1998, 35: 1–11

    Article  MATH  MathSciNet  Google Scholar 

  27. Tweedie R L. Perturbations of countable Markov chains and processes. Ann Inst Statist Math, 1980, 32: 283–290

    Article  MATH  MathSciNet  Google Scholar 

  28. Tweedie R L. Truncation approximations of invariant measures for Markov chains. J Appl Probab, 1998, 35: 517–536

    Article  MATH  MathSciNet  Google Scholar 

  29. Zeifman A, Korotysheva A. Perturbation bounds for M t/M t/N queue with catastrophes. Stoch Models, 2012, 28: 49–62

    Article  MATH  MathSciNet  Google Scholar 

  30. Zhang H, Chen A, Li X, et al. Strong ergodicity of monotone transition functions. Statist Probab Lett, 2001, 55: 63–69

    Article  MATH  MathSciNet  Google Scholar 

  31. Zhao Y, Liu D. The censored Markov chain and the best augmentation. J Appl Probab, 1996, 33: 623–629

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Central South University, Changsha, 410083, China

    YuanYuan Liu

Authors
  1. YuanYuan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to YuanYuan Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y. Perturbation analysis for continuous-time Markov chains. Sci. China Math. 58, 2633–2642 (2015). https://doi.org/10.1007/s11425-015-5019-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11425-015-5019-z

Keywords

MSC(2010)

Search

Navigation