Abstract
This paper concerns the construction and regularity of a transition (probability) function of a non-homogeneous continuous-time Markov process with given transition rates and a general state space. Motivating from a lot of restriction in applications of a transition function with continuous (in t ≥ 0) and conservative transition rates q(t, x, Λ), we consider the case that q(t, x, Λ) are only required to satisfy a mild measurability (in t ≥ 0) condition, which is a generalization of the continuity condition. Under the measurability condition we construct a transition function with the given transition rates, provide a necessary and sufficient condition for it to be regular, and further obtain some interesting additional results.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ash, R.B. Probability and Measure Theory, Second Edition. Academic Press, London, 2000
Breuer, L. From Markov Jump Processes to Spatial Queues. Kluwer, Dordrecht, 2003
Chen, M.F. From Markov Chains to Non-Equilibraum Particle Systems. World Scientific, 1992
Doshi, B.T. Continuous-time control of Markov provesses on an arbitrary atate space: discounted rewards. Ann. Statist. 4, 1219–1235 (1976)
Feller, W. On the integro-differential equations of purely discontinuous Markoff processes. Trans. Amer. Math. Soc., 48: 488–515 (1940)
Gihman, I.I., Skorokhod, A.V. The Theory of Stochastic Processes II. Springer-Verlag (This is a reprint of the First edition 1975), 2004
Guo, X.P., Ulirich, R. Average optimality for continuous-time Markov processes in Polish spaces. Ann. Appl. Probab., 16, 730–756 (2006)
Guo, X.P. Continuous-time Markov decision processes with discounted rewards: the case of Polish spaces. Math. Oper. Res., 32(1): 73–87 (2007)
Guo, X.P. Constrained optimizatin for average cost continuous-time Markov decision processes. IEEE Trans. Automat. Control, 52: 1139–1143 (2007)
Guo, X.P., Hernández-Lerma, O. Zero-sum games for continuous-time jump Markov processes in polish spaces: discounted payoffs. Adv. Appl. Probab., 39, 645–668 (2007)
Hu, D.H. The Analysis Theory of Markov Process with General State. Wuhan: Hubei Education Press, China, 1985 (in Chinese)
Hou, Z.T., Guo, Q.F. Time-homogeneous Markov Processes with Countable State Space. Beijing Science Press and Sprinker-Verlag, 1978
Lai, H.C., Tanaka, K. On an N-person noncooperative Markov game with a metric state space. J. Math. Anal. Appl., 101, 78–96 (1984)
Stidham, S., Weber, R. A survey of Markov decision models for the control of networks of queues. Queueing Systems, 13, 291–314 (1993)
Wang, Z.K., Yang, X.Q. Birth and Death Processes and Markov Chains. Beijing Science Press and Springer-Verlag, 1992
Author information
Authors and Affiliations
Corresponding author
Additional information
Supported by the National Natural Science Foundation of China (No.10925107), Guangdong Province Universities and Colleges Pearl River Scholar Funded Scheme, and the Fundamental Research Funds for the Central Universities (No.11612314).
Rights and permissions
About this article
Cite this article
Ye, Le., Guo, Xp. Construction and regularity of transition functions on polish spaces under measurability conditions. Acta Math. Appl. Sin. Engl. Ser. 29, 1–14 (2013). https://doi.org/10.1007/s10255-013-0208-4
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10255-013-0208-4