Abstract
Asymptotic properties of the waiting timeW k (x,y) until an initial segment of lengthk of a sample pathx of an ergodic finite-alphabet process is seen in an independently chosen sample pathy are discussed. Wyner and Ziv have shown that for irreducible Markov chains, (1/k) logW k (x,y) converges in probability to the entropyH of the process. In this paper, almost sure convergence toH is established for the somewhat larger class of functions of irreducible Markov chains and convergence in probability toH is established for weak Bernoulli processes. A stationary coding of an i.i.d. process is constructed for which there is a subsequencek(n) such that (1/k(n) logW k(n )(x,y), converges in probability to +∞. Positive and negative results for the case when only approximate matches are required are also obtained.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Billingsley, P. (1965).Ergodic Theory and Information, John Wiley and Sons, New York, 1965.
Friedman, N., and Ornstein, D. (1970). On isomorphism of weak Bernoulli transformations,Advances in Math. 5, 365–394.
Gray, R. (1975). Sliding-block source coding,IEEE Trans. Inform. Th. IT-21, 357–368.
Marton, K., and Shields, P. (1993). Entropy and the consistent estimation of joint distributions,Ann. Prob., to appear.
Neuhoff, D., and Shields, P. (1977). Block and sliding-block source coding,IEEE. Trans. Inform. Th. IT-23, 211–215.
Nobel, A., and Wyner, A. A recurrence theorem for dependent processes with applications to data compression.IEEE Trans. Inform. Th.,IT-38, 1561–1563.
Ornstein, D. (1974).Ergodic Theory, Randomness, and Dynamical Systems, Yale University Press, New Haven.
Ornstein, D., and Shields, P. (1993). Thed-recognition of processes,Advances in Math., to appear.
Ornstein, D., and Weiss, B. (1990). How sampling reveals a process,Ann. Prob. 18, 905–930.
Schwarz, G., and Rudolph, D. (1977). The limits ind of multi-step Markov chains,Israel J of Math. 28, 103–109.
Shields, P. (1979). Stationary coding of processes,IEEE Trans. Inform. Th. IT-25, 283–291.
Shields, P. (1992). String matching-the general ergodic case,Ann. Prob. 20, 1199–1203.
Shields, P. (1992). Entropy and prefixes,Ann. Prob. 20, 403–409.
Wyner, A., and Ziv, J. (1989). Some asymptotic properties of the entropy of a stationary ergodic data source with applications to data compression,IEEE Trans. Inform. Th. IT-35, 125–1258.
Additional information
Partially supported by NSF Grant DMS-9024240.
From 9/1 to 12/1 earch year: Mathematies Institute, POB 127, 1364 Budapest, Hungary. 011-361-1-177-175. At other times: Department of Mathematics, University of Toledo, Toledo, Ohio 43606, (419) 537-2069.
Rights and permissions
About this article
Cite this article
Shields, P.C. Waiting times: Positive and negative results on the Wyner-Ziv problem. J Theor Probab 6, 499–519 (1993). https://doi.org/10.1007/BF01066715
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF01066715