Probabilistic Methods and Distributed Information pp 203-220 | Cite as

# Ergodic Theory and Encoding of Individual Sequences

## Abstract

In a famous series of papers Ziv and Lempel [1, 2, 3] studied the encoding of so-called individual sequences. In [3] Ziv gave a definition for a kind of entropy of any infinite sequence of letters drawn from a finite alphabet. The essential parameters in this definition are the numbers of different *n*-words occurring in the given infinite sequence. In particular, Ziv makes no *use of any notions concerning probabilities*. In [4] Dueck and Wolters started a second way which leads also to a notion of entropy of an individual sequence. They strongly used *definitions and theorem from ergodic theory* and connected thereby properties of *individual sequences with the results of* [5, 6]. In particular, they represented the behavior of block occurrences in a sequence \(\mathbf {u}\) by a *set* \(V_T(\mathbf {u})\) *of shift-invariant measures* . The entropy of \(\mathbf {u}\) is defined in terms of measure-theoretical entropies of the measures contained in \(V_T(\mathbf {u})\).

## References

- 1.A. Lempel, J. Ziv, On the complexity of an individual sequence. IEEE Trans. Inf. Theory
**IT–22**, 75–81 (1976)Google Scholar - 2.J. Ziv, A. Lempel, A universal algorithm for sequential data compression. IEEE Trans. Inf. Theory
**IT-23**, 337–343 (1977)MathSciNetCrossRefGoogle Scholar - 3.J. Ziv, Coding theorems for individual sequences. IEEE Trans. Inf. Theory
**IT-24**, 405–412 (1978)MathSciNetCrossRefGoogle Scholar - 4.G. Dueck, L. Wolters, Ergodic theory and encoding of individual sequences. Probl. Control. Inf. Theory
**14**(5), 329–345 (1985)MathSciNetzbMATHGoogle Scholar - 5.R.M. Gray, L.D. Davisson, Source coding theorems without the ergodic assumption. IEEE Trans. Inf. Theory
**IT-20**, 502–516 (1974)MathSciNetCrossRefGoogle Scholar - 6.R.M. Gray, L.D. Davisson, The ergodic decomposition of stationary discrete random processes. IEEE Trans. Inf. Theory
**IT-20**, 625–636 (1974)MathSciNetCrossRefGoogle Scholar - 7.M. Denker, C. Grillenberger, K. Sigmund,
*Ergodic Theory on Compact Spaces*, Lecture notes in mathematics (Springer, Berlin, 1976)CrossRefGoogle Scholar - 8.P. Billingsley,
*Ergodic Theory and Information*(Wiley, New York, 1965)zbMATHGoogle Scholar - 9.R. Ahlswede, Series: foundations in signal processing, communications and networking, in
*Storing and Transmitting Data, Rudolf Ahlswede’s Lectures on Information Theory 1*, ed. by A. Ahlswede, I. Althöfer, C. Deppe, U. Tamm, vol. 10 (Springer, Berlin, 2014)Google Scholar