Journal of Statistical Physics

, Volume 177, Issue 3, pp 485–505 | Cite as

On Entropy Minimization and Convergence

  • S. DostoglouEmail author
  • A. Hughes
  • Jianfei Xue


We examine the minimization of information entropy for measures on the phase space of bounded domains, subject to constraints that are averages of grand canonical distributions. We describe the set of all such constraints and show that it equals the set of averages of all probability measures absolutely continuous with respect to the standard measure on the phase space (with the exception of the measure concentrated on the empty configuration). We also investigate how the set of constrains relates to the domain of the microcanonical thermodynamic limit entropy. We then show that, for fixed constraints, the parameters of the corresponding grand canonical distribution converge, as volume increases, to the corresponding parameters (derivatives, when they exist) of the thermodynamic limit entropy. The results hold when the energy is the sum of any stable, tempered interaction potential that satisfies the Gibbs variational principle (e.g. Lennard-Jones) and the kinetic energy. The same tools and the strict convexity of the thermodynamic limit pressure for continuous systems (valid whenever the Gibbs variational principle holds) give solid foundation to the folklore local homeomorphism between thermodynamic and macroscopic quantities.


Entropy minimization Thermodynamic limit Grand canonical ensemble Strict convexity Convex conjugate 

Mathematics Subject Classification

82B05 82B21 



  1. 1.
    Balian, R., Alhassid, Y., Reinhardt, H.: Dissipation in many-body systems: a geometric approach based on information theory. Phys. Rep. 131, 1–46 (1986)ADSMathSciNetCrossRefGoogle Scholar
  2. 2.
    Barlow, H.B., Kaushal, T.P., Mitchison, G.J.: Finding minimum entropy codes. Neural Comput. 1, 412–423 (1989)CrossRefGoogle Scholar
  3. 3.
    Bot, R.I., Csetnek, E.R., Moldovan, A.: Revisiting some duality theorems via the quasirelative interior in convex optimization. J. Optim. Theory Appl. 139, 67–84 (2008)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Comets, F., Gidas, B.: Parameter estimation for Gibbs distributions from partially observed data. Ann. Appl. Probab. 2(1), 142–70 (1992)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Dereudre, D.: Variational principle for Gibbs point processes with finite range interaction. Electron. Commun. Probab. 21(10), 1–11 (2016)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Dereudre, D., Lavancier, F.: Consistency of likelihood estimation for Gibbs point processes. Ann. Stat. 45(2), 744–770 (2017)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Georgii, H.-O.: The equivalence of ensembles for classical systems of particles. J. Stat. Phys. 80(5/6), 1341–1378 (1995)ADSMathSciNetCrossRefGoogle Scholar
  8. 8.
    Gibbs, J.W.: Elementary Principles In Statistical Mechanics. Ox Bow Press, Connecticut (1981)Google Scholar
  9. 9.
    Gidas, B.: Parameter estimation for Gibbs distributions from fully observed data. In: Chellappa, R., Jain, A. (eds.) Markov Random Fields, Theory and Application, pp. 471–498. Academic Press, Cambridge (1993)Google Scholar
  10. 10.
    Griffiths, R.B., Ruelle, D.: Strict convexity (“continuity”) of the pressure in lattice systems. Commun. Math. Phys. 23(3), 169–75 (1971)ADSMathSciNetCrossRefGoogle Scholar
  11. 11.
    Jaynes, E.T.: Information Theory and Statistical Mechanics. Statistical Physics. Brandeis Lectures, vol. 3. Benjamin Inc, New York (1963)zbMATHGoogle Scholar
  12. 12.
    Khinchin, A.I.: Mathematical Foundations of Statistical Mechanics. Dover, New York (1949)zbMATHGoogle Scholar
  13. 13.
    Kullback, S.: Certain inequalities in information theory and the Cramer-Rao inequality. Ann. Math. Stat. 25(4), 745–751 (1954)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Kullback, S.: Information Theory and Statistics. Dover, New York (1968)zbMATHGoogle Scholar
  15. 15.
    Lanford, O.E.: Entropy and equilibrium states in classical statistical mechanics. In: Lenard, A., et al. (eds.) Statistical Mechanics and Mathematical Problems, pp. 1–113. Springer, Basel (1973)Google Scholar
  16. 16.
    Lanford, O.E., Ruedin, L.: Statistical mechanical methods and continued fractions. Helv. Phys. Acta 69(5), 908–948 (1996)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Lebowitz, J.L., Penrose, O.: Analytic and clustering properties of thermodynamic functions and distribution functions for classical lattice and continuum systems. Commun. Math. Phys. 11, 99–124 (1968)ADSMathSciNetCrossRefGoogle Scholar
  18. 18.
    Martin-Löf, A.: Statistical Mechanics and the Foundations of Thermodynamics. Springer, Berlin (1979)Google Scholar
  19. 19.
    Olla, S., Varadhan, S.R., Yau, H.T.: Hydrodynamical limit for a Hamiltonian system with weak noise. Commun. Math. Phys. 155(3), 523–60 (1993)ADSMathSciNetCrossRefGoogle Scholar
  20. 20.
    Pistone, G., Sempi, C.: An infinite-dimensional geometric structure on the space of all the probability measures equivalent to a given one. Ann. Stat. 23(5), 1543–61 (1995)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Preston, C.: Random Fields. Lecture Notes in Math, vol. 534. Springer, Berlin (1976)CrossRefGoogle Scholar
  22. 22.
    Rechtman, R., Penrose, O.: Continuity of the temperature and derivation of the Gibbs canonical distribution in classical statistical mechanics. J. Stat. Phys. 19(4), 359–366 (1978)ADSMathSciNetCrossRefGoogle Scholar
  23. 23.
    Roberts, A.W., Varberg, D.E.: Convex Functions. Academic Press, Cambridge (1973)zbMATHGoogle Scholar
  24. 24.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)CrossRefGoogle Scholar
  25. 25.
    Rockafellar, R.T., Wets, R.: Variational Analysis. Springer, Berlin (1998)CrossRefGoogle Scholar
  26. 26.
    Ruelle, D.: Statistical Mechanics; Rigorous Results. Benjamin Inc, New York (1969)zbMATHGoogle Scholar
  27. 27.
    Varadhan, S.R.S.: Scaling limits for interaction diffusions. Comm. Math. Phys. 135, 313–353 (1991)ADSMathSciNetCrossRefGoogle Scholar
  28. 28.
    Wainwright, M.J., Jordan, M.I.: Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn. 1(1–2), 1–305 (2008)CrossRefGoogle Scholar
  29. 29.
    Xue, J.: On hydrodynamic equations and their relation to kinetic theory and statistical mechanics. University of Missouri Ph.D. Thesis (2017)Google Scholar
  30. 30.
    Zeidler, E.: Nonlinear Functional Analysis and its Applications III. Springer, New York (1985)CrossRefGoogle Scholar
  31. 31.
    Zubarev, D.N., Morozov, V., Röpke, G.: Statistical Mechanics of Nonequilibrium Processes, vol. 1. Akademie Verlag, Berlin (1996)zbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of MathematicsUniversity of MissouriColumbiaUSA
  2. 2.Department of MathematicsUniversity of ArizonaTucsonUSA

Personalised recommendations