On the Regularity Property of Semi-Markov Processes with Borel State Spaces

  • Óscar Vega-AmayaEmail author
Part of the Systems & Control: Foundations & Applications book series (SCFA)


This note shows that a semi-Markov process with Borel state space is regular under a fairly weak condition on the mean sojourn or holding times and assuming that the embedded Markov chain satisfies one of the following conditions: (a) it is Harris recurrent; (b) it is recurrent and the “recurrent part” of the state space is reached with probability one for every initial state; (c) it has a unique invariant probability measure. Under the latter condition, the regularity property is only ensured for almost all initial states with respect to the invariant probability measure.


Markov Chain Regularity Property Invariant Probability Measure Borel Space Embed Markov Chain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The author takes this opportunity to thank to Professor Onésimo Hernández-Lerma for his constant encouragement and support during the author’s academic career and, in particular, for his valuable comments on an early version of present work. Thanks are also due to the referees for their useful observations which improve the writing and organization of this note.


  1. 1.
    Bhattacharya, R.N., Majumdar, M. (1989), Controlled semi-Markov models–The discounted case, J. Statist. Plann. Inf.21, 365–381.Google Scholar
  2. 2.
    Çinlar, E. (1975), Introduction to Stochastic Processes, Prentice-Hall, Englewood Cliffs, New Jersey.zbMATHGoogle Scholar
  3. 3.
    Hernández-Lerma, O., Lasserre, J.B. (2003), Markov Chains and Invariant Probabilities, Birkhäuser, Basel.zbMATHCrossRefGoogle Scholar
  4. 4.
    Jaśkiewicz, A. (2004), On the equivalence of two expected average cost criteria for semi-Markov control processes, Math. Oper. Research 29, 326–338.zbMATHCrossRefGoogle Scholar
  5. 5.
    Limnios, N., Oprişan, G. (2001), Semi-Markov Processes and Reliability, Birkhaüser, Boston.zbMATHCrossRefGoogle Scholar
  6. 6.
    Meyn, S.P., Tweedie, R.L. (1993), Markov Chains and Stochastic Stability, Springer-Verlag, London.zbMATHCrossRefGoogle Scholar
  7. 7.
    Ross, S.M. (1970), Applied Probability Models with Optimization Applications, Holden-Day, San Francisco.zbMATHGoogle Scholar
  8. 8.
    Schäl, M. (1992), On the second optimality equation for semi-Markov decision models, Math. Oper. Research 17, 470–486.zbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Departamento de MatemáticasUniversidad de SonoraHermosilloMéxico

Personalised recommendations