Foundations of Physics

, Volume 42, Issue 2, pp 319–339 | Cite as

Macroscopic Time Evolution and MaxEnt Inference for Closed Systems with Hamiltonian Dynamics

  • Domagoj KuićEmail author
  • Paško Županović
  • Davor Juretić


MaxEnt inference algorithm and information theory are relevant for the time evolution of macroscopic systems considered as problem of incomplete information. Two different MaxEnt approaches are introduced in this work, both applied to prediction of time evolution for closed Hamiltonian systems. The first one is based on Liouville equation for the conditional probability distribution, introduced as a strict microscopic constraint on time evolution in phase space. The conditional probability distribution is defined for the set of microstates associated with the set of phase space paths determined by solutions of Hamilton’s equations. The MaxEnt inference algorithm with Shannon’s concept of the conditional information entropy is then applied to prediction, consistently with this strict microscopic constraint on time evolution in phase space. The second approach is based on the same concepts, with a difference that Liouville equation for the conditional probability distribution is introduced as a macroscopic constraint given by a phase space average. We consider the incomplete nature of our information about microscopic dynamics in a rational way that is consistent with Jaynes’ formulation of predictive statistical mechanics, and the concept of macroscopic reproducibility for time dependent processes. Maximization of the conditional information entropy subject to this macroscopic constraint leads to a loss of correlation between the initial phase space paths and final microstates. Information entropy is the theoretic upper bound on the conditional information entropy, with the upper bound attained only in case of the complete loss of correlation. In this alternative approach to prediction of macroscopic time evolution, maximization of the conditional information entropy is equivalent to the loss of statistical correlation, and leads to corresponding loss of information. In accordance with the original idea of Jaynes, irreversibility appears as a consequence of gradual loss of information about possible microstates of the system.


MaxEnt algorithm Information theory Statistical mechanics Hamiltonian dynamics 



Authors wish to thank the anonymous reviewer for insightful suggestions that significantly improved the submitted manuscript. The present work was supported by Croatian MZOS project no. 177-1770495-0476.


  1. 1.
    Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev. 106, 620–630 (1957) MathSciNetADSCrossRefGoogle Scholar
  2. 2.
    Jaynes, E.T.: Information theory and statistical mechanics. II. Phys. Rev. 108, 171–190 (1957) MathSciNetADSCrossRefGoogle Scholar
  3. 3.
    Gibbs, J.W.: Elementary Principles in Statistical Mechanics. Yale University Press, New Haven (1902) zbMATHGoogle Scholar
  4. 4.
    Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423, 623–656 (1948). Reprinted in: Shannon, C.E., Weaver, W.: The Mathematical Theory of Communication. University of Illinois Press, Urbana (1949) Google Scholar
  5. 5.
    Jaynes, E.T.: Information theory and statistical mechanics. In: Ford, K.W. (ed.) 1962 Brandeis Lectures in Theoretical Physics, vol. 3, pp. 181–218. W.A. Benjamin, Inc., New York (1963) Google Scholar
  6. 6.
    Jaynes, E.T.: Gibbs vs Boltzmann entropies. Am. J. Phys. 33, 391–398 (1965) ADSzbMATHCrossRefGoogle Scholar
  7. 7.
    Jaynes, E.T.: Where do we stand on maximum entropy? In: Levine, R.D., Tribus, M. (eds.) The Maximum Entropy Formalism, pp. 15–118. MIT Press, Cambridge (1979) Google Scholar
  8. 8.
    Jaynes, E.T.: The minimum entropy production principle. Annu. Rev. Phys. Chem. 31, 579–601 (1980) ADSCrossRefGoogle Scholar
  9. 9.
    Jaynes, E.T.: Macroscopic prediction. In: Haken, H. (ed.) Complex Systems—Operational Approaches in Neurobiology, Physics, and Computers, pp. 254–269. Springer, Berlin (1985) CrossRefGoogle Scholar
  10. 10.
    Grandy, W.T.: Principle of maximum entropy and irreversible processes. Phys. Rep. 62, 175–266 (1980) MathSciNetADSCrossRefGoogle Scholar
  11. 11.
    Garrod, C.: Statistical Mechanics and Thermodynamics. Oxford University Press, New York (1995) Google Scholar
  12. 12.
    Jaynes, E.T.: The second law as physical fact and as human inference. Unpublished manuscript (1990).
  13. 13.
    Zurek, W.H.: Algorithmic randomness and physical entropy. Phys. Rev. A 40, 4731–4751 (1989) MathSciNetADSCrossRefGoogle Scholar
  14. 14.
    Duncan, T.L., Semura, J.S.: The deep physics behind the second law: information and energy as independent forms of bookkeeping. Entropy 6, 21–29 (2004) ADSzbMATHCrossRefGoogle Scholar
  15. 15.
    Duncan, T.L., Semura, J.S.: Information loss as a foundational principle for the second law of thermodynamics. Found. Phys. 37, 1767–1773 (2007) MathSciNetADSzbMATHCrossRefGoogle Scholar
  16. 16.
    Grandy, W.T.: Time evolution in macroscopic systems. I. Equations of motion. Found. Phys. 34, 1–20 (2004) MathSciNetADSzbMATHCrossRefGoogle Scholar
  17. 17.
    Grandy, W.T.: Time evolution in macroscopic systems. II. The entropy. Found. Phys. 34, 21–57 (2004) MathSciNetADSzbMATHCrossRefGoogle Scholar
  18. 18.
    Grandy, W.T.: Time evolution in macroscopic systems. III. Selected applications. Found. Phys. 34, 771–813 (2004) MathSciNetADSzbMATHCrossRefGoogle Scholar
  19. 19.
    Tishby, N.Z., Levine, R.D.: Time evolution via a self-consistent maximal-entropy propagation: the reversible case. Phys. Rev. A 30, 1477–1490 (1984) MathSciNetADSCrossRefGoogle Scholar
  20. 20.
    Plastino, A.R., Plastino, A.: Statistical treatment of autonomous systems with divergenceless flows. Physica A 232, 458–476 (1996) MathSciNetADSCrossRefGoogle Scholar
  21. 21.
    Plastino, A., Plastino, A.R., Miller, H.G.: Continuity equations, H-theorems, and maximum entropy. Phys. Lett. A 232, 349–355 (1997) MathSciNetADSzbMATHCrossRefGoogle Scholar
  22. 22.
    Plastino, A.R., Plastino, A.: Universality of Jaynes’ approach to the evolution of time-dependent probability distributions. Physica A 258, 429–445 (1998) MathSciNetADSCrossRefGoogle Scholar
  23. 23.
    Schönfeldt, J.-H., Jimenez, N., Plastino, A.R., Plastino, A., Casas, M.: Maximum entropy principle and classical evolution equations with source terms. Physica A 374, 573–584 (2007) ADSCrossRefGoogle Scholar
  24. 24.
    Khinchin, A.I.: Mathematical Foundations of Statistical Mechanics. Dover, New York (1949) zbMATHGoogle Scholar
  25. 25.
    Wan, F.Y.M.: Introduction to the Calculus of Variations and Its Applications. Chapman & Hall, New York (1995) zbMATHGoogle Scholar
  26. 26.
    Kittel, C.: Elementary Statistical Physics. Wiley, New York (1958) Google Scholar
  27. 27.
    Zurek, W.H., Paz, J.-P.: Decoherence, chaos, and the second law. Phys. Rev. Lett. 72, 2508–2512 (1994) ADSCrossRefGoogle Scholar
  28. 28.
    Zurek, W.H.: Decoherence, einselection, and the quantum origins of the classical. Rev. Mod. Phys. 75, 715–775 (2003) MathSciNetADSzbMATHCrossRefGoogle Scholar
  29. 29.
    Jaynes, E.T.: Probability Theory: The Logic of Science. Cambridge University Press, Cambridge (2003). Bretthorst, G.L. (ed.) zbMATHCrossRefGoogle Scholar
  30. 30.
    Zwick, M.: Quantum measurement and Gödel’s proof. Specul. Sci. Technol. 1, 135–145 (1978) Google Scholar
  31. 31.
    Gödel, K.: On Formally Undecidable Propositions of Principia Mathematica and Related Systems. Basic Books, Inc., New York (1962) zbMATHGoogle Scholar
  32. 32.
    Nagel, E., Newman, J.R.: Gödel’s Proof. New York University Press, New York (1960) Google Scholar
  33. 33.
    Pattee, H.H.: Postscript. In: Pattee, H.H. (ed.) Hierarchy Theory, pp. 131–156. George Braziller, New York (1973) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Domagoj Kuić
    • 1
    Email author
  • Paško Županović
    • 1
  • Davor Juretić
    • 1
  1. 1.Faculty of ScienceUniversity of SplitSplitCroatia

Personalised recommendations