Abstract
We study the question of geometric ergodicity in a class of Markov chains on the state space of non-negative integers for which, apart from a finite number of boundary rows and columns, the elements pjk of the one-step transition matrix are of the form c k-j where {c k} is a probability distribution on the set of integers. Such a process may be described as a general random walk on the non-negative integers with boundary conditions affecting transition probabilities into and out of a finite set of boundary states. The imbedded Markov chains of several non-Markovian queueing processes are special cases of this form. It is shown that there is an intimate connection between geometric ergodicity and geometric bounds on one of the tails of the distribution {c k}.
Article PDF
Similar content being viewed by others
References
Baum, L. E., M. Katz, and R. R. Read: Exponential convergence rates for the law of large numbers. Trans. Amer. math. Soc. 102, 187–199 (1962).
Chow, Y. S., and H. Robbins: A renewal theorem for random variables which are dependent or non-identically distributed. Ann. math. Statistics, 34, 390–395 (1963).
Chung, K. L.: Markov chains with stationary transition probabilities. Berlin-Göttingen-Heidelberg: Springer 1960.
Debreu, G. and I. N. Herstein: Non-negative square matrices. Econometrica 21, 597–607 (1953).
Heathcote, C. R.: Divergent single server queues. Proc. 1964 Chapel Hill Sympos. Congestion theory. University of North Carolina Press 1965.
Kemperman, J. H. B.: The passage problem for a stationary Markov chain. University of Chicago Press 1961.
Kendall, D. G.: Some problems in the theory of queues. J. Roy. statist. Soc. Ser. B 13, 151–173 (1951).
— Stochastic processes occurring in the theory of queues and their analysis by the method of the imbedded Markov chain. Ann. math. Statistics 24, 338–354 (1953).
—: Unitary dilations of Markov transition operators and the corresponding integral representations for transition probability matrices. In U. Grenander (ed.): Probability and Statistics. New York: Wiley 1959.
- Geometric ergodicity and the theory of queues. In K. J. Arrow, S. Karlin, and P. Suppes, (eds.). Mathematical methods in the social sciences, 1959. Stanford University Press 1960.
Lukacs, E.: Characteristic Functions. London: Griffin 1960.
Miller, H. D.: A generalization of Wald's identity with applications to random walks. Ann. math. Statistics, 24, 549–560 (1961).
Spitzer, F.: A combinatorial lemma and its applications to probability theory. Trans. Amer. math. Soc., 82, 323–339 (1956).
—: A Tauberian theorem and its probability interpretation. Trans. Amer. math. Soc., 94, 150–169 (1960).
Vere-Jones, D.: Geometric ergodicity in denumerable Markov chains. Quart. J. Math., Oxford II. Ser. 13, 7–28 (1962).
—: A rate of convergence problem in the theory of queues. Teor. Verojatn. Primen, 9, 104–112 (1964).
Widder, D. V.: The Laplace Transform. Princeton Univ. Press 1941.
Author information
Authors and Affiliations
Additional information
This research was supported by the U.S. office of Naval Research Contract No. Nonr-855(09), and carried out while the author was a visitor in the Statistics department, University of North Carolina, Chapel Hill.
Rights and permissions
About this article
Cite this article
Miller, H.D. Geometric ergodicity in a class of denumerable Markov chains. Z. Wahrscheinlichkeitstheorie verw Gebiete 4, 354–373 (1966). https://doi.org/10.1007/BF00539120
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00539120