Abstract
This note shows that a semi-Markov process with Borel state space is regular under a fairly weak condition on the mean sojourn or holding times and assuming that the embedded Markov chain satisfies one of the following conditions: (a) it is Harris recurrent; (b) it is recurrent and the “recurrent part” of the state space is reached with probability one for every initial state; (c) it has a unique invariant probability measure. Under the latter condition, the regularity property is only ensured for almost all initial states with respect to the invariant probability measure.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bhattacharya, R.N., Majumdar, M. (1989), Controlled semi-Markov models–The discounted case, J. Statist. Plann. Inf.21, 365–381.
Çinlar, E. (1975), Introduction to Stochastic Processes, Prentice-Hall, Englewood Cliffs, New Jersey.
Hernández-Lerma, O., Lasserre, J.B. (2003), Markov Chains and Invariant Probabilities, Birkhäuser, Basel.
Jaśkiewicz, A. (2004), On the equivalence of two expected average cost criteria for semi-Markov control processes, Math. Oper. Research 29, 326–338.
Limnios, N., Oprişan, G. (2001), Semi-Markov Processes and Reliability, Birkhaüser, Boston.
Meyn, S.P., Tweedie, R.L. (1993), Markov Chains and Stochastic Stability, Springer-Verlag, London.
Ross, S.M. (1970), Applied Probability Models with Optimization Applications, Holden-Day, San Francisco.
Schäl, M. (1992), On the second optimality equation for semi-Markov decision models, Math. Oper. Research 17, 470–486.
Acknowledgements
The author takes this opportunity to thank to Professor Onésimo Hernández-Lerma for his constant encouragement and support during the author’s academic career and, in particular, for his valuable comments on an early version of present work. Thanks are also due to the referees for their useful observations which improve the writing and organization of this note.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Vega-Amaya, Ó. (2012). On the Regularity Property of Semi-Markov Processes with Borel State Spaces. In: Hernández-Hernández, D., Minjárez-Sosa, J. (eds) Optimization, Control, and Applications of Stochastic Systems. Systems & Control: Foundations & Applications. Birkhäuser, Boston. https://doi.org/10.1007/978-0-8176-8337-5_18
Download citation
DOI: https://doi.org/10.1007/978-0-8176-8337-5_18
Published:
Publisher Name: Birkhäuser, Boston
Print ISBN: 978-0-8176-8336-8
Online ISBN: 978-0-8176-8337-5
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)