Knowledge and Meaning: Chaos and Complexity

  • J. P. Crutchfield
Part of the Woodward Conference book series (WOODWARD)


What are models good for? Taking a largely pedagogical view, the following essay discusses the semantics of measurement and the uses to which an observer can put knowledge of a process’s structure. To do this in as concrete a way as possible, it first reviews the reconstruction of probabilistic finite automata from time series of stochastic and chaotic processes. It investigates the convergence of an observer’s knowledge of the process’s state; assuming that the process is in, and the observer also uses for internal representation, that model class. The conventional notions of phase and phase-locking are extended beyond periodic behavior to include deterministic chaotic processes. The meaning of individual measurements of an unpredictable process is then defined in terms of the computational structure in a model that an observer built.


Data Stream Binary Sequence Parse Tree Symbolic Dynamic Entropy Rate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    N. Chomsky, “Three models for the description of language,” IRE Trans. Info. Th. 2, 113 (1956).CrossRefGoogle Scholar
  2. 2.
    A. O’Hear, An Introduction to the Philosophy of Science (Oxford University Press, Oxford, 1989).Google Scholar
  3. 3.
    D. M. MacKay, Information, Meaning and Mechanism (MIT Press, Cambridge, 1969).Google Scholar
  4. 4.
    S. Omohundro, “Modelling cellular automata with partial differential equations,” Physica 10D, 128 (1984).MathSciNetADSGoogle Scholar
  5. 5.
    J. P. Crutchfield, “Turing dynamical systems,” preprint (1987).Google Scholar
  6. 6.
    L. Blum, M. Shub, and S. Smale, “On a theory of computation over the real numbers,” Bull. AMS 21, 1 (1989).MathSciNetMATHCrossRefGoogle Scholar
  7. 7.
    C. Moore, “Upredictability and undecidability in dynamical systems,” Phys. Rev. Lett. 64, 2354 (1990).MathSciNetADSMATHCrossRefGoogle Scholar
  8. 8.
    D. C. Dennett, The Intentional Stance (MIT Press, Cambridge, 1987).Google Scholar
  9. 9.
    J. P. Crutchfield and B. S. McNamara, “Equations of motion from a data series,” Complex Systems 1, 417 (1987).MathSciNetMATHGoogle Scholar
  10. 10.
    J. Rissanen, Stochastic Complexity in Statistical Inquiry (World Scientific, Singapore, 1989).MATHGoogle Scholar
  11. 11.
    J. P. Crutchfield, “Reconstructing language hierarchies,” in Information Dynamics, ed., H. A. Atmanspracher (Plenum, New York, 1990).Google Scholar
  12. 12.
    C. E. Shannon and W. Weaver, The Mathematical Theory of Communication (University of Illinois Press, Champaign-Urbana, 1962).Google Scholar
  13. 13.
    A. N. Kolmogorov, “A new metric invariant of transient dynamical systems and automorphisms in lebesgue spaces,” Dokl. Akad. Nauk. SSSR 119, 861 (1958). (Russian) Math. Rev. vol. 21, no. 2035a.MathSciNetMATHGoogle Scholar
  14. 14.
    J. P. Crutchfield and N. H. Packard, “Symbolic dynamics of noisy chaos,” Physica 7D, 201 (1983).MathSciNetADSGoogle Scholar
  15. 15.
    N. H. Packard, J. P. Crutchfield, J. D. Farmer, and R. S. Shaw, “Geometry from a time series,” Phys. Rev. Let. 45, 712 (1980).ADSCrossRefGoogle Scholar
  16. 16.
    H. Poincare, Science and Hypothesis (Dover Publications, New York, 1952).MATHGoogle Scholar
  17. 17.
    J. P. Crutchfield and K. Young, “Inferring statistical complexity,” Phys. Rev. Let. 63, 105 (1989).MathSciNetADSCrossRefGoogle Scholar
  18. 18.
    J. Pearl, Probabilistic Reasoning in Intelligent Systems (Morgan Kaufman, New York, 1988).Google Scholar
  19. 19.
    J. P. Crutchfield and K. Young, “Computation at the onset of chaos,” in Entropy, Complexity, and the Physics of Information, ed., W. Zurek, VIII of SFI Studies in the Sciences of Complexity (Addison-Wesley, Reading, Massachusetts, 1990) 223.Google Scholar
  20. 20.
    A. Fraser, “Using hidden markov models to predict chaos,” preprint (1990).Google Scholar
  21. 21.
    L. R. Rabiner, “A tutorial on hidden markov models and selected applications,” IEEE Proc. 77, 257 (1989).CrossRefGoogle Scholar
  22. 22.
    J. P. Crutchfield, Noisy Chaos. PhD thesis, University of California, Santa Cruz (1983). Published by University Microfilms Intl, Minnesota.Google Scholar
  23. 23.
    S. Wolfram, “Computation theory of cellular automata,” Comm. Math. Phys. 96, 15 (1984).MathSciNetADSMATHCrossRefGoogle Scholar
  24. 24.
    P. Grassberger, “Toward a quantitative theory of self-generated complexity,” Intl. J. Theo. Phys. 25, 907 (1986).MathSciNetMATHCrossRefGoogle Scholar
  25. 25.
    K. Lindgren and M. G. Nordahl, “Complexity measures and cellular automata,” Complex Systems 2, 409 (1988).MathSciNetMATHGoogle Scholar
  26. 26.
    J. P. Crutchfield and K. Young, “e-machine spectroscopy,” preprint (1991).Google Scholar
  27. 27.
    C. H. Bennett, “Dissipation, information, computational complexity, and the definition of organization,” in Emerging Syntheses in the Sciences, ed., D. Pines (Addison-Wesley, Redwood City, 1988).Google Scholar
  28. 28.
    A. N. Kolmogorov, “Three approaches to the concept of the amount of information,” Prob. Info. Trans. 1, 1 (1965).Google Scholar
  29. 29.
    J. P. Crutchfield and N. H. Packard, “Noise scaling of symbolic dynamics entropies,” in Evolution of Order and Chaos, ed., H. Haken (Springer-Verlag, Berlin, 1982) 215.Google Scholar
  30. 30.
    N. H. Packard, Measurements of Chaos in the Presence of Noise. PhD thesis, University of California, Santa Cruz (1982).Google Scholar
  31. 31.
    R. Shaw, The Dripping Faucet as a Model Chaotic System (Aerial Press, Santa Cruz, California, 1984).Google Scholar
  32. 32.
    C. P. Bachas and B. Huherman, “Complexity and relaxation of hierarchical structures,” Phys. Rev. Let. 57, 1965 (1986).ADSCrossRefGoogle Scholar
  33. 33.
    G. Chaitin, “On the length of programs for computing finite binary sequences,” J. ACM 13, 145 (1966).MathSciNetCrossRefGoogle Scholar
  34. 34.
    A. Lempel and J. Ziv, “On the complexity of individual sequences,” IEEE Trans. Info. Th. IT-22, 75 (1976).MathSciNetCrossRefGoogle Scholar
  35. 35.
    J. Rissanen, “Stochastic complexity and modeling,” Ann. Statistics 14, 1080 (1986).MathSciNetMATHCrossRefGoogle Scholar
  36. 36.
    W. H. Zurek, “Thermodynamic cost of computation, algorithmic complexity, and the information metric,” preprint (1989).Google Scholar
  37. 37.
    J. Ziv, “Complexity and coherence of sequences,” in The Impact of Processing Techniques on Communications, ed., J. K. Skwirzynski (Nijhoff, Dordrecht, 1985) 35.Google Scholar
  38. 38.
    A. A. Brudno, “Entropy and the complexity of the trajectories of a dynamical system,” Trans. Moscow Math. Soc. 44, 127 (1983).Google Scholar
  39. 39.
    J. P. Crutchfield, “Inferring the dynamic, quantifying physical complexity,” in Measures of Complexity and Chaos, eds., N. B. Abraham, A. M. Albano, A. Passamante, and P. E. Rapp (Plenum Press, New York, 1990) 327.Google Scholar
  40. 40.
    R. Shaw, “Strange attractors, chaotic behavior, and information flow,” Z. Naturforsh. 36a, 80 (1981).ADSMATHGoogle Scholar
  41. 41.
    J. P. Crutchfield, “Semantics and thermodynamics,” in Nonlinear Modeling, eds., M. Casdagli and S. Eubank, SFI Studies in the Sciences of Complexity (Addison-Wesley, Reading, Massachusetts, 1991)Google Scholar
  42. 42.
    R. Rammal, G. Toulouse, and M. A. Virasoro, “Ultrametricity for physicists,” Rev. Mod. Phys. 58, 765 (1986).MathSciNetADSCrossRefGoogle Scholar
  43. 43.
    B. Marcus, “Sofic systems and encoding data,” IEEE Transactions on Information Theory 31, 366 (1985).MATHCrossRefGoogle Scholar
  44. 44.
    B. Kitchens and S. Tuncel, “Finitary measures for subshifts of finite type and sofic systems,” Memoirs of the AMS 58, no. 338 (1985).MathSciNetGoogle Scholar
  45. 45.
    S. Hamad, “The symbol grounding problem,” Physica 42D, 335 (1990).Google Scholar
  46. 46.
    G. Birkhoff, Lattice Theory third ed. (American Mathematical Society, Providence, 1967).MATHGoogle Scholar
  47. 47.
    J. Hartmanis and R. E. Steams, Algebraic Structure Theory of Sequential Machines (Prentice-Hall, Englewood Cliffs, 1966).MATHGoogle Scholar
  48. 48.
    J. E. Hoperoft and J. D. Ullman, Introduction to Automata Theory, Languages, and Computation (Addison-Wesley, Reading, 1979).Google Scholar
  49. 49.
    J. P. Crutchfield, “Information and its metric,” in Nonlinear Structures in Physical Systems — Pattern Formation, Chaos and Waves, eds., L. Lam and H. C. Morris (Springer-Verlag, New York, 1990)Google Scholar
  50. 50.
    M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness (W. H. Freeman, New York, 1979).MATHGoogle Scholar
  51. 51.
    C. G. Langton, “Computation at the edge of chaos: Phase transitions and emergent computation,” in Emergent Computation, ed., S. Forrest (North-Holland, Amsterdam, 1990) 12.Google Scholar
  52. 52.
    W. Li, N. H. Packard, and C. G. Langton, “Transition phenomena in cellular automata rule space,” preprint (1990).Google Scholar

Copyright information

© Springer-Verlag New York, Inc. 1992

Authors and Affiliations

  • J. P. Crutchfield
    • 1
  1. 1.Physics DepartmentUniversity of CaliforniaBerkeleyUSA

Personalised recommendations