# Expressing the Entropy of Lattice Systems as Sums of Conditional Entropies

- 219 Downloads
- 3 Citations

## Abstract

Whether a system is to be considered complex or not depends on how one searches for correlations. We propose a general scheme for calculation of entropies in lattice systems that has high flexibility in how correlations are successively taken into account. Compared to the traditional approach for estimating the entropy density, in which successive approximations build on step-wise extensions of blocks of symbols, we show that one can take larger steps when collecting the statistics necessary to calculate the entropy density of the system. In one dimension this means that, instead of a single sweep over the system in which states are read sequentially, one take several sweeps with larger steps so that eventually the whole lattice is covered. This means that the information in correlations is captured in a different way, and in some situations this will lead to a considerably much faster convergence of the entropy density estimate as a function of the size of the configurations used in the estimate. The formalism is exemplified with both an example of a free energy minimisation scheme for the two-dimensional Ising model, and an example of increasingly complex spatial correlations generated by the time evolution of elementary cellular automaton rule 60.

## Keywords

Entropy density Lattice systems Correlations Complexity Spin systems Cellular automata## Notes

### Acknowledgments

Financial support from EU-FP7 project MatheMACS is gratefully acknowledged.

## References

- 1.Alexandrowicz, Z.: Stochastic models for the statistical description of lattice systems. J. Chem. Phys.
**55**(6), 2765–2779 (1971)ADSCrossRefGoogle Scholar - 2.Binder, K.: The Monte Carlo method for the study of phase-transitions - a review of some recent progress. J. Comput. Phys.
**59**(1), 1–55 (1985)ADSCrossRefzbMATHMathSciNetGoogle Scholar - 3.Cover, T.M., Thomas, J.A.: Elements of information theory. Wiley Series in Telecommunications. Wiley, New York (1991)CrossRefzbMATHGoogle Scholar
- 4.Feldman, D.P., Crutchfield, J.P.: Structural information in two-dimensional patterns: entropy convergence and excess entropy. Phys. Rev. E
**67**, 051104 (2003)ADSCrossRefMathSciNetGoogle Scholar - 5.Goldstein, S., Kuik, R., Schlijper, A.G.: Entropy and global Markov properties. Commun. Math. Phys.
**126**(3), 469–482 (1990)ADSCrossRefzbMATHMathSciNetGoogle Scholar - 6.Grassberger, P.: Toward a quantitative theory of self-generated complexity. Int. J. Theor. Phys.
**25**, 907–938 (1986)CrossRefzbMATHMathSciNetGoogle Scholar - 7.Helvik, T., Lindgren, K., Nordahl, M.G.: Continuity of information transport in surjective cellular automata. Commun. Math. Phys.
**272**, 53–74 (2007)ADSCrossRefzbMATHMathSciNetGoogle Scholar - 8.Kikuchi, R.: A theory of cooperative phenomena. Phys. Rev.
**81**, 988–1003 (1951)ADSCrossRefzbMATHMathSciNetGoogle Scholar - 9.Kramers, H.A., Wannier, G.H.: Statistics of the two-dimensional ferromagnet. Phys. Rev.
**60**, 252–262 (1941)ADSCrossRefMathSciNetGoogle Scholar - 10.Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat.
**22**(1), 79–86 (1951)CrossRefzbMATHMathSciNetGoogle Scholar - 11.Lindgren, K.: Correlations and random information in cellular automata. Complex Syst.
**1**, 529–543 (1987)zbMATHMathSciNetGoogle Scholar - 12.Lindgren, K., Nordahl, M.G.: Complexity measures and cellular automata. Complex Syst.
**2**, 409–440 (1988)zbMATHMathSciNetGoogle Scholar - 13.Marcelja, S.: Entropy of phase-separated structures. Physica A
**231**, 168–177 (1996)ADSCrossRefGoogle Scholar - 14.Meirovitch, H.: Calculation of entropy with computer simulation methods. Chem. Phys. Lett.
**45**(2), 389–392 (1977)ADSCrossRefMathSciNetGoogle Scholar - 15.Meirovitch, H.: Methods for estimating entropy with computer-simulation—the simple cubic Ising lattice. J. Phys. A
**16**, 839–848 (1983)ADSCrossRefMathSciNetGoogle Scholar - 16.Meirovitch, H.: A monte carlo study of the entropy, the pressure, and the critical behavior of the hard-square lattice gas. J. Stat. Phys.
**30**, 681–698 (1983)ADSCrossRefGoogle Scholar - 17.Meirovitch, H.: Simulation of a free energy upper bound, based on the anticorrelation between an approximate free energy functional and its fluctuation. J. Chem. Phys.
**111**(16), 7215–7224 (1999)ADSCrossRefGoogle Scholar - 18.Olbrich, E., Hegger, R., Kantz, H.: Local estimates for entropy densities in coupled map lattices. Phys. Rev. Lett.
**84**, 2132–2135 (2000)ADSCrossRefGoogle Scholar - 19.Onsager, L.: Crystal statistics. I. A two-dimensional model with a order-disorder transition. Phys. Rev.
**65**, 117–149 (1944)ADSCrossRefzbMATHMathSciNetGoogle Scholar - 20.Schlijper, A.G.: Convergence of the cluster-variation method in the thermodynamic limit. Phys. Rev. B
**27**, 6841 (1983)ADSCrossRefGoogle Scholar - 21.Schlijper, A.G., Smit, B.: Two-sided bounds on the free energy from local states in Monte Carlo simulations. J. Stat. Phys.
**56**(3/4), 247 (1989)ADSCrossRefMathSciNetGoogle Scholar - 22.Schlijper, A.G., van Bergen, A.R.D., Smit, B.: Local-states method for the calculation of free energies in Monte Carlo simulations of lattice models. Phys. Rev. A
**41**(2), 1175 (1990)ADSCrossRefGoogle Scholar - 23.Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J.
**27**(379–423), 623–656 (1948)CrossRefMathSciNetGoogle Scholar