Abstract
Let \({\text{X = }}\left\{ {X_n ;n \in \mathbb{Z}} \right\}\)be a discrete-valued stationary ergodic process distributed according to P and let x=(..., x −1, x 0, x 1,...) denote a realization from X. We investigate the asymptotic behavior of the recurrence time R n defined as the first time that the initial n-block \(x_1^n = (x_1 ,x_2 , \ldots ,x_n )\) reappears in the past of x. We identify an associated random walk, \( - \log P(X_1^n )\) on the same probability space as X, and we prove a strong approximation theorem between log R n and \( - \log P(X_1^n )\). From this we deduce an almost sure invariance principle for log R n. As a byproduct of our analysis we get unified proofs for several recent results that were previously established using methods from ergodic theory, the theory of Poisson approximation and the analysis of random trees. Similar results are proved for the waiting time W n defined as the first time until the initial n-block from one realization first appears in an independent realization generated by the same (or by a different) process.
Similar content being viewed by others
REFERENCES
Aldous, D., and Shields, P. C. (1988). A diffusion limit for a class of randomly-growing binary trees. Prob. Th. Rel. Fields 79, 509–542.
Algoet, P. H., and Cover, T. M. (1988). A sandwich proof of the Shannon-McMillan-Breiman theorem. Ann. Prob. 16, 876–898.
Barron, A. R. (1985a). Logically smooth density estimation. Ph.D. Thesis, Department of Electrical Engineering, Stanford University.
Barron, A. R. (1985b). The strong ergodic theorem for densities: Generalized Shannon-McMillan-Breiman theorem. Ann. Prob. 13, 1992–1303.
Breiman, L. (1957). The individual ergodic theorem of information theory. Ann. Math. Stat. 28, 809–811. See also, Breiman, L. (1960). A correction to “The individual ergodic theorem of information theory.” Ann. Math. Stat. 31, 809–810.
Fréchet, M. (1938). Reserches Théoriques Modernes sur le Calcul des Probabilités, Vol. II, Théorie des Événements en Chaîne dans le Cas d'un Nombre Fini d'etats Possibles. Gauthier-Villars, Paris (French).
Guibas, L., and Odlyzko, A. M. (1981). Periods in strings. J. Combin Th. Ser. A 31, 19–42.
Ibragimov, I. A. (1962). Some limit theorems for stationary processes. Th. Prob. Appl. 7, 349–382.
Jacquet, P., and Szpankowski, W. (1994). Autocorrelation of words and its applications. J. Combin. Th. Ser. A 66, 237–269.
Kac, M. (1947). On the notion of recurrence in discrete stochastic processes. Bull. Amer. Math. Soc. 53, 1002–1010.
Marton, K., and Shields, P. C. (1995). Almost-sure waiting time results for weak and very weak Bernoulli processes. Ergod. Th. Dynam. Syst. 15, 951–960.
Nobel, A., and Wyner, A. D. (1992). A recurrence theorem for dependent processes with applications to data compression. IEEE Trans Inform. Th. 38, 1561–1564.
Ornstein, D. S., and Weiss, B. (1993). Entropy and data compression schemes. IEEE Trans. Inform. Th. 39, 78–83.
Pevzner, P., Borodovsky, M., and Mironov, A. (1991). Linguistic of nucleotide sequences: the significance of deviations from mean statistical characteristics and prediction of the frequency of occurrence of words. J. Biomol. Struct. Dynam. 6, 1013–1026.
Philipp, W., and Stout, W. (1975). Almost sure invariance principles for partial sums of weakly dependent random variables. Memoirs of the AMS, Vol. 2,Issue 2, No. 161.
Pittel, B. (1985). Asymptotical growth of a class of random trees. Ann. Prob. 13, 414–427.
Shields, P. C. (1993). Waiting times: positive and negative results on the Wyner-Ziv problem. J. Theor. Prob. 6, 499–519.
Strassen, V. (1964). An almost sure invariance principle for the law of the iterated logarithm. Z. Wahrsch. verw. Gabiete 3, 23–32.
Szpankowski, W. (1993). Asymptotic properties of data compression and suffix trees. IEEE Trans. Inform. Th. 39, 1647–1659.
Willems, F. M. J. (1989). Universal data compression and repetition times. IEEE Trans. Inform. Th. 35, 54–58.
Wyner, A. J. (1993). String matching theorems and applications to data compression and statistics, Ph.D. Thesis, Department of Statistics, Stanford University.
Wyner, A. D., and Ziv, J. (1989). Some asymptotic properties of the entropy of a stationary ergodic data source with applications to data compression. IEEE Trans. Inform. Th. 35, 1250–1258.
Yushkevich, A. A. (1953). On limit theorems connected with the concept of the entropy of Markov chains. Uspehi Mat. Nauk 8, 177–180 (Russian).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kontoyiannis, I. Asymptotic Recurrence and Waiting Times for Stationary Processes. Journal of Theoretical Probability 11, 795–811 (1998). https://doi.org/10.1023/A:1022610816550
Issue Date:
DOI: https://doi.org/10.1023/A:1022610816550