Mutual information functions versus correlation functions
- Wentian Li
- … show all 1 hide
Purchase on Springer.com
$39.95 / €34.95 / £29.95*
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.
This paper studies one application of mutual information to symbolic sequences: the mutual information functionM(d). This function is compared with the more frequently used correlation functionΓ(d). An exact relation betweenM(d) andΓ(d) is derived for binary sequences. For sequences with more than two symbols, no such general relation exists; in particular,Γ(d)=0 may or may not lead toM(d)=0. This linear, but not general, independence between symbols separated by a distance is studied for ternary sequences. Also included is the estimation of the finite-size effect on calculating mutual information. Finally, the concept of “symbolic noise” is discussed.
- C. E. Shannon, The mathematical theory of communication,Bell Syst. Tech. J. 27:379–423 (1948).
- A. Fraser and H. Swinney, Independent coordinates for strange attractors from mutual information,Phys. Rev. A 33:1134–1140 (1986).
- A. Fraser, Reconstructing attractors from scalar time series: A comparison of singular system and redundancy criteria,Physics D 34:391–404 (1989).
- C. E. Shannon, Prediction and entropy of printed English,Bell Syst. Tech. J. 1951:50–64.
- B. Hayes, A progress report on the fine art of turning literature into drivel, Computer Recreations,Sci. Am. 249(5):18–28 (1983).
- L. Gatlin,Information Theory and the Living System (Columbia University Press, 1972).
- S. Wolfram, ed.,Theory and Application of Cellular Automata (World Scientific, 1986).
- G. J. Chaitin, Toward a mathematical definition of “life,” inThe Maximum Entropy Formalism, Levine and Tribus, eds. (MIT Press, 1979).
- R. Shaw,The Dripping Faucet as a Model Chaotic System (Aerial Press, 1984).
- P. Grassberger, Towards a quantitative theory of self-organized complexity,Int. J. Theor. Phys. 25:907–938 (1986).
- S. Karlin and H. Taylor,A Second Course in Stochastic Processes (Academic Press, 1981); S. Karlin,A First Course in Stochastic Processes (Academic Press, 1968).
- J. E. Hopcroft and J. D. Ullman,Introduction to Automata Theory, Languages, and Computation (Addison-Welsey, 1979).
- W. Li, Power spectra of regular languages and cellular automata,Complex Syst. 1(1):107–130 (1987).
- H. Herzel, Complexity of symbolic sequences,Syst. Anal. Model. Simul. 5(5):435–444 (1988).
- M. Gardner, Mathematical Games: White and brown music, fractal curves and 1/f fluctuations,Sci. Am. 238(4):16–32 (1978).
- W. Li, Mutual information functions of natural language texts, Santa Fe Institute preprint, SFI-89-008 (1989).
- V. M. Alekseev and M. V. Yacobson, Symbolic dynamics and hyperbolic dynamical systems,Phys. Rep. 75:287–325 (1981).
- A. Lindenmayer, Mathematical models for cellular interactions in development I. Filaments with one-sided inputs,J. Theor. Biol. 18:280–299 (1968).
- W. Li, Spatial 1/f spectra in open dynamical systems,Europhys. Lett. 10(5):395–400 (1989).
- W. Li, Expansion-modification systems: Another model for 1/f spectra, preprint (1990).
- Mutual information functions versus correlation functions
Journal of Statistical Physics
Volume 60, Issue 5-6 , pp 823-837
- Cover Date
- Print ISSN
- Online ISSN
- Kluwer Academic Publishers-Plenum Publishers
- Additional Links
- Mutual information function
- correlation functions
- linear and general dependence
- symbolic noise
- Industry Sectors
- Wentian Li (1) (2)
- Author Affiliations
- 1. Center for Complex Systems Research, Physics Department, Beckman Institute, University of Illinois, 61801, Urbana, Illinois
- 2. Department of Physics, Columbia University, 10027, New York, New York