Journal of Statistical Physics

, Volume 155, Issue 4, pp 687–702 | Cite as

Expressing the Entropy of Lattice Systems as Sums of Conditional Entropies

  • Torbjørn Helvik
  • Kristian Lindgren


Whether a system is to be considered complex or not depends on how one searches for correlations. We propose a general scheme for calculation of entropies in lattice systems that has high flexibility in how correlations are successively taken into account. Compared to the traditional approach for estimating the entropy density, in which successive approximations build on step-wise extensions of blocks of symbols, we show that one can take larger steps when collecting the statistics necessary to calculate the entropy density of the system. In one dimension this means that, instead of a single sweep over the system in which states are read sequentially, one take several sweeps with larger steps so that eventually the whole lattice is covered. This means that the information in correlations is captured in a different way, and in some situations this will lead to a considerably much faster convergence of the entropy density estimate as a function of the size of the configurations used in the estimate. The formalism is exemplified with both an example of a free energy minimisation scheme for the two-dimensional Ising model, and an example of increasingly complex spatial correlations generated by the time evolution of elementary cellular automaton rule 60.


Entropy density Lattice systems Correlations  Complexity Spin systems Cellular automata 



Financial support from EU-FP7 project MatheMACS is gratefully acknowledged.


  1. 1.
    Alexandrowicz, Z.: Stochastic models for the statistical description of lattice systems. J. Chem. Phys. 55(6), 2765–2779 (1971)ADSCrossRefGoogle Scholar
  2. 2.
    Binder, K.: The Monte Carlo method for the study of phase-transitions - a review of some recent progress. J. Comput. Phys. 59(1), 1–55 (1985)ADSCrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Cover, T.M., Thomas, J.A.: Elements of information theory. Wiley Series in Telecommunications. Wiley, New York (1991)CrossRefzbMATHGoogle Scholar
  4. 4.
    Feldman, D.P., Crutchfield, J.P.: Structural information in two-dimensional patterns: entropy convergence and excess entropy. Phys. Rev. E 67, 051104 (2003)ADSCrossRefMathSciNetGoogle Scholar
  5. 5.
    Goldstein, S., Kuik, R., Schlijper, A.G.: Entropy and global Markov properties. Commun. Math. Phys. 126(3), 469–482 (1990)ADSCrossRefzbMATHMathSciNetGoogle Scholar
  6. 6.
    Grassberger, P.: Toward a quantitative theory of self-generated complexity. Int. J. Theor. Phys. 25, 907–938 (1986)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Helvik, T., Lindgren, K., Nordahl, M.G.: Continuity of information transport in surjective cellular automata. Commun. Math. Phys. 272, 53–74 (2007)ADSCrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Kikuchi, R.: A theory of cooperative phenomena. Phys. Rev. 81, 988–1003 (1951)ADSCrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Kramers, H.A., Wannier, G.H.: Statistics of the two-dimensional ferromagnet. Phys. Rev. 60, 252–262 (1941)ADSCrossRefMathSciNetGoogle Scholar
  10. 10.
    Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Lindgren, K.: Correlations and random information in cellular automata. Complex Syst. 1, 529–543 (1987)zbMATHMathSciNetGoogle Scholar
  12. 12.
    Lindgren, K., Nordahl, M.G.: Complexity measures and cellular automata. Complex Syst. 2, 409–440 (1988)zbMATHMathSciNetGoogle Scholar
  13. 13.
    Marcelja, S.: Entropy of phase-separated structures. Physica A 231, 168–177 (1996)ADSCrossRefGoogle Scholar
  14. 14.
    Meirovitch, H.: Calculation of entropy with computer simulation methods. Chem. Phys. Lett. 45(2), 389–392 (1977)ADSCrossRefMathSciNetGoogle Scholar
  15. 15.
    Meirovitch, H.: Methods for estimating entropy with computer-simulation—the simple cubic Ising lattice. J. Phys. A 16, 839–848 (1983)ADSCrossRefMathSciNetGoogle Scholar
  16. 16.
    Meirovitch, H.: A monte carlo study of the entropy, the pressure, and the critical behavior of the hard-square lattice gas. J. Stat. Phys. 30, 681–698 (1983)ADSCrossRefGoogle Scholar
  17. 17.
    Meirovitch, H.: Simulation of a free energy upper bound, based on the anticorrelation between an approximate free energy functional and its fluctuation. J. Chem. Phys. 111(16), 7215–7224 (1999)ADSCrossRefGoogle Scholar
  18. 18.
    Olbrich, E., Hegger, R., Kantz, H.: Local estimates for entropy densities in coupled map lattices. Phys. Rev. Lett. 84, 2132–2135 (2000)ADSCrossRefGoogle Scholar
  19. 19.
    Onsager, L.: Crystal statistics. I. A two-dimensional model with a order-disorder transition. Phys. Rev. 65, 117–149 (1944)ADSCrossRefzbMATHMathSciNetGoogle Scholar
  20. 20.
    Schlijper, A.G.: Convergence of the cluster-variation method in the thermodynamic limit. Phys. Rev. B 27, 6841 (1983)ADSCrossRefGoogle Scholar
  21. 21.
    Schlijper, A.G., Smit, B.: Two-sided bounds on the free energy from local states in Monte Carlo simulations. J. Stat. Phys. 56(3/4), 247 (1989)ADSCrossRefMathSciNetGoogle Scholar
  22. 22.
    Schlijper, A.G., van Bergen, A.R.D., Smit, B.: Local-states method for the calculation of free energies in Monte Carlo simulations of lattice models. Phys. Rev. A 41(2), 1175 (1990)ADSCrossRefGoogle Scholar
  23. 23.
    Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27(379–423), 623–656 (1948)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Complex systems group, Department of Energy and EnvironmentChalmers University of Technology GöteborgSweden

Personalised recommendations