Abstract
As is well-known, transition probabilities of jump Markov processes satisfy Kolmogorov’s backward and forward equations. In the seminal 1940 paper, William Feller investigated solutions of Kolmogorov’s equations for jump Markov processes. Recently the authors solved the problem studied by Feller and showed that the minimal solution of Kolmogorov’s backward and forward equations is the transition probability of the corresponding jump Markov process if the transition rate at each state is bounded. This paper presents more general results. For Kolmogorov’s backward equation, the sufficient condition for the described property of the minimal solution is that the transition rate at each state is locally integrable, and for Kolmogorov’s forward equation the corresponding sufficient condition is that the transition rate at each state is locally bounded.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson, W. J. (1991). Continuous-time Markov chains: An applications-oriented approach. New York: Springer.
Bertsekas, D. P., & Shreve, S. E. (1978). Stochastic optimal control: The discrete-time case. New York: Academic Press.
Dellacherie, C., & Meyer, P. A. (1978). Probabilities and potential. Amsterdam: North-Holland.
Doob, J. L. (1990). Stochastic process., reprint of the 1953 original. New York: John Wiley.
Feinberg, E. A., Mandava, M., & Shiryaev, A. N. (2013). Sufficiency of Markov policies for continuous-time Markov decision processes and solutions to Kolmogorov’s forward equation for jump Markov processes. In Proceedings of 2013 IEEE 52nd annual conference on decision and control (pp. 5728–5732).
Feinberg, E. A., Mandava, M., & Shiryaev, A. N. (2014). On solutions of Kolmogorovs equations for nonhomogeneous jump Markov processes. Journal of Mathematical Analysis and Applications, 411(1), 261–270.
Feller, W. (1940). On the integro-differential equations of purely-discontinuous Markoff processes. Transactions of the American Mathematical Society, 48, 488–515. Errata. Transactions of the American Mathematical Society, 58, 474, 1945.
Guo, X., & Hernández-Lerma, O. (2009). Continuous-time Markov decision processes: Theory and applications. Berlin: Springer.
Halmos, P. R. (1950). Measure theory. New York: Springer.
Jacod, J. (1975). Multivariate point processes: Predictable projection, Radon-Nikodym derivatives, representation of martingales. Probability Theory and Related Fields, 31, 235–253.
Kechris, A. S. (1995). Classical descriptive set theory, Graduate texts in mathematics, Vol. 156. New York: Springer.
Kendall, D. G. (1956). Some further pathological examples in the theory of denumerable Markov processes. The Quarterly Journal of Mathematics, 7(1), 39–56.
Kolmogorov, A. N. (1992). On analytic methods in probability theory. In A.N. Shiryaev (Ed.), Selected works of A.N. Kolmogorov, Vol. II, Probability theory and mathematical statistics (pp. 62–108). Dordrecht: Kluwer (translated from Russian by G. Lindquist; originally published in 1931 in Mathematische Annalen, 104, 415–458).
Kuznetsov, S. E. (1981). Any Markov process in a Borel space has a transition function. Theory of Probability and its Applications, 25(2), 384–388.
Reuter, G. E. H. (1957). Denumerable Markov processes and the associated contraction semigroups on \(l\). Acta Mathematica, 97(1), 1–46.
Royden, H. L. (1988). Real analysis (3rd ed.). New York: Macmillan.
Ye, L., Guo, X., & Hernández-Lerma, O. (2008). Existence and regularity of a nonhomogeneous transition matrix under measurability conditions. Journal of Theoretical Probability, 21(3), 604–627.
Acknowledgements
The first two authors thank Pavlo Kasyanov for useful comments.
Author information
Authors and Affiliations
Corresponding author
Additional information
The first author was partially supported by the National Science Foundation [Grants CMMI-1335296 and CMMI-1636193].
Rights and permissions
About this article
Cite this article
Feinberg, E., Mandava, M. & Shiryaev, A.N. Kolmogorov’s equations for jump Markov processes with unbounded jump rates. Ann Oper Res 317, 587–604 (2022). https://doi.org/10.1007/s10479-017-2538-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-017-2538-8