Mutual information functions versus correlation functions
Received: 16 October 1989 Revised: 13 March 1990 DOI:
Cite this article as: Li, W. J Stat Phys (1990) 60: 823. doi:10.1007/BF01025996 Abstract
This paper studies one application of mutual information to symbolic sequences: the mutual information function
M(d). This function is compared with the more frequently used correlation function Γ(d). An exact relation between M(d) and Γ(d) is derived for binary sequences. For sequences with more than two symbols, no such general relation exists; in particular, Γ(d)=0 may or may not lead to M(d)=0. This linear, but not general, independence between symbols separated by a distance is studied for ternary sequences. Also included is the estimation of the finite-size effect on calculating mutual information. Finally, the concept of “symbolic noise” is discussed. Key words Mutual information function correlation functions linear and general dependence symbolic noise References
C. E. Shannon, The mathematical theory of communication,
Bell Syst. Tech. J.
A. Fraser and H. Swinney, Independent coordinates for strange attractors from mutual information,
Phys. Rev. A
A. Fraser, Reconstructing attractors from scalar time series: A comparison of singular system and redundancy criteria,
C. E. Shannon, Prediction and entropy of printed English,
Bell Syst. Tech. J.
B. Hayes, A progress report on the fine art of turning literature into drivel, Computer Recreations,
Information Theory and the Living System (Columbia University Press, 1972).
S. Wolfram, ed.,
Theory and Application of Cellular Automata (World Scientific, 1986).
G. J. Chaitin, Toward a mathematical definition of “life,” in
The Maximum Entropy Formalism, Levine and Tribus, eds. (MIT Press, 1979).
The Dripping Faucet as a Model Chaotic System (Aerial Press, 1984).
P. Grassberger, Towards a quantitative theory of self-organized complexity,
Int. J. Theor. Phys.
S. Karlin and H. Taylor,
A Second Course in Stochastic Processes (Academic Press, 1981); S. Karlin, A First Course in Stochastic Processes (Academic Press, 1968).
J. E. Hopcroft and J. D. Ullman,
Introduction to Automata Theory, Languages, and Computation (Addison-Welsey, 1979).
W. Li, Power spectra of regular languages and cellular automata,
H. Herzel, Complexity of symbolic sequences,
Syst. Anal. Model. Simul.
M. Gardner, Mathematical Games: White and brown music, fractal curves and 1/
W. Li, Mutual information functions of natural language texts, Santa Fe Institute preprint, SFI-89-008 (1989).
V. M. Alekseev and M. V. Yacobson, Symbolic dynamics and hyperbolic dynamical systems,
A. Lindenmayer, Mathematical models for cellular interactions in development I. Filaments with one-sided inputs,
J. Theor. Biol.
W. Li, Spatial 1/
spectra in open dynamical systems,
W. Li, Expansion-modification systems: Another model for 1/
f spectra, preprint (1990). Copyright information
© Plenum Publishing Corporation 1990