Abstract
We study controlled jump models in continuous time using the “average reward criterion.” We show that the synthesis problem reduces to the construction of a canonical triple and formulate a characteristic property of the canonical triple and a sufficient condition for its existence.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Literature Cited
A. A. Yushkevich, “Controlled jump Markov models,” Teor. Veroyatn. Primen.,25, No. 2, 247–270 (1980).
A. A. Yushkevich and E. A. Fainberg, “On homogeneous controlled Markov models in continuous time with a finite or countable state set,” Teor,. Veroyatn. Primen.,24, No. 1, 155–160 (1979).
J. Bather, “Optimal stationary policies for denumerable Markov chains in continuous time,” Adv. Appl. Prob.,8, No. 1, 144–158 (1976).
A. B. Piunovskii and V. M. Khametov, “On optimal control of continuous-discrete jump processes,” Izv. Akad. Nauk SSSR, Tekh. Kibern., No. 3, 56–61 (1983).
V. B. Kolmanovskii, N. N. Moiseev, and F. L. Chernous'ko, “On one control technique for stochastic systems,” Izv. Akad. Nauk SSSR, Tekh. Kibern., No. 3, 5–14 (1979).
A. B. Piunovski and V. M. Khametov, “New effective solutions of optimality's equations for the controlled Markov chains with continuous parameter (the unbounded price-function),” Probl. Contr. Inform. Theory,14, No. 4, 303–318 (1985).
E. A. Fainberg, “ε-Optimal control of a finite Markov chain by the mean criterion,” Teor. Veroyatn. Primen.,25, No. 1, 71–82 (1980).
V. M. Khametov and A. B. Piunovski, “On the optimal control of information transmission,” Fund, of Teletraffic Theory, Proc. 3rd Int. Symp. on Teletraffic Theory, Moscow (1984), pp. 57–60.
D. Isaacson and B. Arnold, “Strong ergodicity for continuous-time Markov chains,” J. Appl. Prob.,15, No. 4, 699–706 (1978).
J. Van Nunen, “Markov decision processes — theory and applications,” Wissenschaft. Zeitschrift,4, No. 6, 357–371 (1980).
E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes and Their Applications [in Russian], Nauka, Moscow (1975).
A. A. Yushkevich and R. Ya. Chitashvili, “Controlled random sequences and Markov chains,” Usp. Mat. Nauk,37, No. 6, 213–242 (1982).
Additional information
Translated from Kibernetika, No. 1, pp. 45–49, January–February, 1989.
Rights and permissions
About this article
Cite this article
Piunovskii, A.B. On homogeneous controlied Markov models in continuous time. Cybern Syst Anal 25, 55–61 (1989). https://doi.org/10.1007/BF01074884
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF01074884