Abstract
This paper studies one application of mutual information to symbolic sequences: the mutual information functionM(d). This function is compared with the more frequently used correlation functionΓ(d). An exact relation betweenM(d) andΓ(d) is derived for binary sequences. For sequences with more than two symbols, no such general relation exists; in particular,Γ(d)=0 may or may not lead toM(d)=0. This linear, but not general, independence between symbols separated by a distance is studied for ternary sequences. Also included is the estimation of the finite-size effect on calculating mutual information. Finally, the concept of “symbolic noise” is discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
C. E. Shannon, The mathematical theory of communication,Bell Syst. Tech. J. 27:379–423 (1948).
A. Fraser and H. Swinney, Independent coordinates for strange attractors from mutual information,Phys. Rev. A 33:1134–1140 (1986).
A. Fraser, Reconstructing attractors from scalar time series: A comparison of singular system and redundancy criteria,Physics D 34:391–404 (1989).
C. E. Shannon, Prediction and entropy of printed English,Bell Syst. Tech. J. 1951:50–64.
B. Hayes, A progress report on the fine art of turning literature into drivel, Computer Recreations,Sci. Am. 249(5):18–28 (1983).
L. Gatlin,Information Theory and the Living System (Columbia University Press, 1972).
S. Wolfram, ed.,Theory and Application of Cellular Automata (World Scientific, 1986).
G. J. Chaitin, Toward a mathematical definition of “life,” inThe Maximum Entropy Formalism, Levine and Tribus, eds. (MIT Press, 1979).
R. Shaw,The Dripping Faucet as a Model Chaotic System (Aerial Press, 1984).
P. Grassberger, Towards a quantitative theory of self-organized complexity,Int. J. Theor. Phys. 25:907–938 (1986).
S. Karlin and H. Taylor,A Second Course in Stochastic Processes (Academic Press, 1981); S. Karlin,A First Course in Stochastic Processes (Academic Press, 1968).
J. E. Hopcroft and J. D. Ullman,Introduction to Automata Theory, Languages, and Computation (Addison-Welsey, 1979).
W. Li, Power spectra of regular languages and cellular automata,Complex Syst. 1(1):107–130 (1987).
H. Herzel, Complexity of symbolic sequences,Syst. Anal. Model. Simul. 5(5):435–444 (1988).
M. Gardner, Mathematical Games: White and brown music, fractal curves and 1/f fluctuations,Sci. Am. 238(4):16–32 (1978).
W. Li, Mutual information functions of natural language texts, Santa Fe Institute preprint, SFI-89-008 (1989).
V. M. Alekseev and M. V. Yacobson, Symbolic dynamics and hyperbolic dynamical systems,Phys. Rep. 75:287–325 (1981).
A. Lindenmayer, Mathematical models for cellular interactions in development I. Filaments with one-sided inputs,J. Theor. Biol. 18:280–299 (1968).
W. Li, Spatial 1/f spectra in open dynamical systems,Europhys. Lett. 10(5):395–400 (1989).
W. Li, Expansion-modification systems: Another model for 1/f spectra, preprint (1990).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Li, W. Mutual information functions versus correlation functions. J Stat Phys 60, 823–837 (1990). https://doi.org/10.1007/BF01025996
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01025996