Summary
For a family of semi-Markov processes where the transition matrices for the embedded Markov chains and the mean sojourn times depend continuously on a parameter, we give equivalent as well as sufficient conditions for the continuity of the mean recurrence times. The results will be used in a subsequent paper on average costs in a dynamic programming model.
Article PDF
Similar content being viewed by others
References
Chung, K.L.: Markov chains with stationary transition probabilities (2nd ed.). Berlin-Heidelberg-New York: Springer 1967
Çinlar, E.: Periodicity in Markov renewal theory. Adv. Appl. Probab. 6, 61–78 (1974)
Çinlar, E.: Introduction to stochastic processes. Englewood Cliffs: Prentice-Hall 1975
Deppe, H.: Durchschnittskosten in semiregenerativen Entscheidungsmodellen. Dissertation, University of Bonn, Bonn (1981)
Deppe, H.: On the existence of average optimal policies in semiregenerative decision models. Math. Oper. Res. 9, 558–575 (1984)
Federgruen, A., Hordijk, A., Tijms, H.C.: Denumerable state semi-Markov decision processes with unbounded costs. Average cost criterion. Stochastic processes Appl. 9, 223–235 (1979)
Federgruen, A., Tijms, H.C.: The optimality equation in average cost denumerable state semiMarkov decision problems, recurrency and algorithm. J. Appl. Probab. 15, 356–373 (1978)
Hordijk, A.: A sufficient condition for the existence of an optimal policy with respect to the average cost criterion in Markovian decision processes. In: Transactions of the Sixth Prague Conference on Information Theory, Statistical Decision Functions, Random Processes, pp. 263–274. Prague: Academia 1973
Hordijk, A.: Dynamic programming and Markov potential theory. Mathematical Centre Tracts 51. Amsterdam: Mathematisch Centrum 1974
Kolonko, M.: A countable Markov chain with reward structure — continuity of the average reward. Preprint No. 415, Sonderforschungsbereich 72, University of Bonn, Bonn (1980)
Pyke, R., Schaufele, R.: The existence and uniqueness of stationary measures for Markov renewal processes. Ann. Math. Stat. 37, 1439–1462 (1966)
Royden, H.L.: Real analysis (2nd ed.). New York: Mac Millan 1968
Schäl, M.: On the M/G/1 — queue with controlled service rate. In: Proceedings of the Workshop: Optimization and Operations Research, pp. 233–239. University of Bonn, Bonn 1977
Schweitzer, P.J.: Perturbation theory and finite Markov chains. J. Appl. Probab. 5, 401–413 (1968)
Wijngaard, J.: Stationary Markovian decision problems and perturbation theory of quasicompact linear operators. Math. Oper Res. 2, 91–102 (1977)
Wijngaard, J.: Existence of average optimal strategies in Markovian decision processes on a countable state space and embedded optimality equations. OR Report No. 149, North Carolina State University, Raleigh (1979)
Author information
Authors and Affiliations
Additional information
This work was supported by the Deutsche Forschungsgemeinschaft, Sonderforschungsbereich 72 at the University of Bonn
Rights and permissions
About this article
Cite this article
Deppe, H. Continuity of mean recurrence times in denumerable semi-Markov processes. Z. Wahrscheinlichkeitstheorie verw Gebiete 69, 581–592 (1985). https://doi.org/10.1007/BF00532668
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00532668