Communications in Mathematical Physics

, Volume 140, Issue 2, pp 339–371 | Cite as

Entropy production by block variable summation and central limit theorems

  • E. A. Carlen
  • A. Soffer
Article

Abstract

We prove a strict lower bound on the entropy produced when independent random variables are summed and rescaled. Using this, we develop an approach to central limit theorems from a dynamical point of view in which the entropy is a Lyapunov functional governing approach to the Gaussian limit. This dynamical approach naturally extends to cover dependent variables, and leads to new results in pure probability theory as well as in statistical mechanics. It also provides a unified framework within which many previous results are easily derived.

Keywords

Entropy Neural Network Statistical Physic Complex System Nonlinear Dynamics 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [BaEm] Bakry, D., Emery, M.: Diffusions hypercontractives. Seminaire de Probabilities XIX, Lect. Notes in Math. vol.1123, pp. 179–206. Berlin, Heidelberg, New York: Springer 1985Google Scholar
  2. [Bar] Barron, A.R.: Entropy and the central limit theorem. Ann. Prob.14, 336–342 (1986)Google Scholar
  3. [Bla] Blachman, N.M.: The convolution inequality for entropy powers. IEEE Trans. Info. Thy.2, 267–271 (1965)CrossRefGoogle Scholar
  4. [Bri] Bricmont, J.: Correlation inequalities for two component fields. Ann. Soc. Sr. Bru.,90, 245–252 (1976)Google Scholar
  5. [Bro] Brown, L.D.: A proof of the central limit theorem motivated by the Cramer-Rao inequality. In: Statistics and Probability: Essays in Honor of C.R. Rao Kallianpur et al. (eds.) pp.314–328. Amsterdam: North-Holland 1982Google Scholar
  6. [BFS] Brydges, D., Fröhlich, J., Spencer, T.: A random walk representation of classical spin systems and correlation inequalities. Commun. Math. Phys.83, 123–150 (1982)CrossRefGoogle Scholar
  7. [BrKu] Bricmont, J., Kupiainen, A.: Phase transition in the 3d random field Ising model. Commun. Math. Phys.116, 539–572 (1988)CrossRefGoogle Scholar
  8. [Car] Carlen, E.A.: Superadditivity of Fisher's Information and Logarithmic Sobolev Inequalities. J. Funct. Anal. (to appear)Google Scholar
  9. [CaLo] Carlen, E.A., Loss, M.: Extremals of functionals with competing symmetries. J. Funct. Anal.88, 437–456 (1990)CrossRefGoogle Scholar
  10. [CaSo] Carlen, E.A., Soffer, A.: Entropy production by convolution and central limit theorems with strong rate information, Princeton preprint, in preparationGoogle Scholar
  11. [Cra] Cramér, H.: Über eine Eigenschaft der normalen Verteilungsfunktion. Math. Zeit.41, 405–414 (1936)CrossRefGoogle Scholar
  12. [Csi] Csiszar, I.: Informationstheoretische Konvergenzbegriffe im Raum der Wahrschein-lichkeitsverteilungen, Publications of the Mathematical Institute, Hungarian Academy of Sciences,VII Series A, 137–157 (1962)Google Scholar
  13. [Dem] Dembo, A.: Information inequalities and uncertainty principles, Stanford University Technical Report No. 75, (1990)Google Scholar
  14. [DeC] DeConinck, J.: On limit theorems for the bivariate (magnetization, energy) variable at the critical point. Commun. Math. Phys.109, 53–59 (1984)CrossRefGoogle Scholar
  15. [Des] Desvillettes, L.: Entropy dissipation rate and convergence to equilibrium in Kinetic Equations. Commun. Math. Phys.123, 687–702 (1989)CrossRefGoogle Scholar
  16. [DeSt] Deuschel, J.D., Stroock D.W.: Large deviations. Boston: Academic Press 1989Google Scholar
  17. [DN] Dunlop, F., Newman, C.: Multicomponent field theories and classical rotators. Commun. Math. Phys.44, 223–235 (1975)CrossRefGoogle Scholar
  18. [DoVa] Donsker, M.D., Varadhan, S.R.S.: Asymptotic evaluation of certain Markov process expectations for large time, I. Comm. Pure Appl. Math.28, 1–47 (1975)Google Scholar
  19. [ElNe] Ellis, R., Newman, C.: Necessary and sufficient conditions for the application of the GHS inequality with applications to analysis and probability. Trans. Am. Math. Soc.237, 83–99 (1978)Google Scholar
  20. [Ens] Enss, V.: Asymptotic completeness for quantum mechanical scattering I: Short range potentials. Commun. Math. Phys.61, 285–291 (1978)CrossRefGoogle Scholar
  21. [Fis] Fisher, R.A.: Theory of statistical estimation. Proc. Cam. Phil. Soc.22, 700–725 (1925)Google Scholar
  22. [FKG] Fortuin, C.M., Kasteleyn, P.W., Ginibre, J.: Correlation inequalities on some partially ordered sets. Commun. Math. Phys.22, 89–103 (1971)CrossRefGoogle Scholar
  23. [GaJL] Gallavotti, G., Jona-Lasinio, G.: Limit theorems for multidimensional Markov processes. Commun. Math. Phys.41, 301–307 (1975)CrossRefGoogle Scholar
  24. [GaML] Gallavotti, G., Martin-Löf, A.: Block spin distribution for short range attractive Ising models. Nuovo Cim.25B, 425–441 (1975)Google Scholar
  25. [Gaw] Gawedzki, K.: Block Spin Renormalization. In: Mathematics and Physics: Recent Results. Vol. I. L. Striet (ed.). Singapore: World Scientific 1983Google Scholar
  26. [GlJa] Glimm, J., Jaffe, A.: Quantum physics a functional integral point of view. New York: Springer 1981Google Scholar
  27. [Gro] Gross, L.: Logarithmic Sobolev inequalities. Am. J. Math.97, 1061–1083 (1975)Google Scholar
  28. [Gri] Griffiths, R.B., Hurst, C.A., Sherman, S.: Concavity of magnetization of an Ising ferromagnet in a positive externfal field. J. Math. Phys.11, 790–795 (1970)CrossRefGoogle Scholar
  29. [GPV] Guo, M.Z., Papanicolau, G.C., Varadhan, S.R.S.: Nonlinear diffusion limit for a system with nearest neighbor interactions. Commun. Math. Phys.118, 31–67 (1988)CrossRefGoogle Scholar
  30. [HeNa] Hegerfeldt, G., Nappi, C.: Mixing properties of lattice systems. Commun. Math. Phys.53, 1–7 (1977)CrossRefGoogle Scholar
  31. [Ho] Holley, R.: Free energy in a Markovian model of a lattice spin system. Commun. Math. Phys.23, 8–28 (1971)CrossRefGoogle Scholar
  32. [JoL] Jona-Lasinio, J.: The renormalization group: A probabilistic view. Nuovo Cim.26B, 99–119 (1975)Google Scholar
  33. [Kac] Kac, M.: On a characterization of the normal distribution. Am. J. Math.61, 473–476 (1939)Google Scholar
  34. [KeSh] Kelly, D.G., Sherman, S.: General Griffiths inequalities on correlations in Ising ferromagnetis. J. Math. Phys.9, 466–484 (1968)CrossRefGoogle Scholar
  35. [kul] Kullback, S.: A lower bound for discrimination information in terms of variation. IEEE Trans. Info. Thy.4, 126–127 (1967)CrossRefGoogle Scholar
  36. [Le74] Lebowitz, J.: GHS and other Inequalities. Commun. Math. Phys.35, 87–92 (1974)CrossRefGoogle Scholar
  37. [Le72] Lebowitz, J.: Bounds on the Correlations and Analyticity Properties of Ferromagnetic Ising Spin Systems. Commun. Math. Phys.28, 313–321 (1972)CrossRefGoogle Scholar
  38. [LePe] Lebowitz, J., Penrose, O.: Analytic and clustering properties of thermodynamic functions for classical lattice and continuum systems. Commun. Math. Phys.11, 99–124 (1968)CrossRefGoogle Scholar
  39. [Lie78] Lieb, E.H.: Proof of an entropy conjecture of Wehrl. Commun. Math. Phys.62, 35–41 (1978)CrossRefGoogle Scholar
  40. [Lie89] Lieb, E.H.: Gaussian kernels have Gaussian maximisers, Princeton preprint, 1989Google Scholar
  41. [Lin59] Linnik, Ju.V.: An information theoretic proof of the central limit theorem with Lindeberg conditions. Theory Probab. Appl.4, 288–299 (1959)CrossRefGoogle Scholar
  42. [Lin60] Linnik, Ju.V.: On certain connections of the information theory of C. Shannon and R. Fisher with the theory of symmetrization of random vectors, Transactions of the Second Prague Conference on Information Theory, 313–327, Publ. House Szech. Akad. Sci., Prague; New York: Academic Press 1960Google Scholar
  43. [McK] McKean, H.: Speed of approach to equilibrium for Kac's caricature of a Maxwellian gas. Arch. Rat. Mech. Anal.21, 342–367 (1966)CrossRefGoogle Scholar
  44. [Mou] Mourre, E.: Absence of singular spectrum of certain self-adjoint operators. Commun. Math. Phys.78, 391–408 (1981)CrossRefGoogle Scholar
  45. [Ne75a] Newman, C.M.: Moment inequalities for ferromagnetic Gibbs distributions. J. Math. Phys.16, 1956–1959 (1975)CrossRefGoogle Scholar
  46. [Ne80] Newman, C.M.: Normal Fluctuations and the FKG Inequalities. Commun. Math. Phys.74, 129–140 (1980)CrossRefGoogle Scholar
  47. [Ne83] Newman, C.M.: A general central limit theorem for FKG systems. Commun. Math. Phys.91, 75–80 (1983)CrossRefGoogle Scholar
  48. [Ne75b] Newman, C.M.: Gaussian correlation inequalities for ferromagnets. Z. Wahrsch. Gebiete33, 75–93 (1975)CrossRefGoogle Scholar
  49. [Sap] Sapagov, N.A.: On independent terms of a sum of random variables which is distributed almost normally. Vestnik Leningrad Univ.14, 78–105 (1959)Google Scholar
  50. [ShWe] Shannon, C.E., Weaver, W.: The mathematical theory of communication, Urbana, IL: University of Illinois Press 1949Google Scholar
  51. [Shi74] Shimizu, R.: On Fisher's amount of information for location family. In: A modern course on statistical distributions in scientific work. Patil et al. (eds.), Boston, MA: D. Reidel 1974Google Scholar
  52. [Shi82] Shimizu, R.: On the stability of characterization of the normal distribution. In: Statistics and Probability: Essays in Honor of C.R. Rao. Kallianpur et al. (eds.) pp. 661–670. Amsterdam: North-Holland 1982Google Scholar
  53. [SiSo1] Sigal, I.M., Soffer, A.: Long range many body scattering: Asymptotic clustering for Coulomb type potentials. Inv. Math.99, 115–143 (1990)CrossRefGoogle Scholar
  54. [SiSo2] Sigal, I.M., Soffer, A.: TheN-particle scattering problem: Asymptotic completeness for short range systems. Ann. Math.126, 35–108 (1987)Google Scholar
  55. [Sim] Simon, B.: Functional integration and quantum physics. New York: Academic Press 1979Google Scholar
  56. [Ski] Skitovich, V.P.: Linear forms of independent random variables and the normal distribution. Isvestiya Akad. Nauk U.S.S.R.18, 185–200 (1954)Google Scholar
  57. [Sta] Some inequalities satisfied by the quantities of information of Fisher and Shannon. Info. Contr.2, 101–112 (1959)Google Scholar

Copyright information

© Springer-Verlag 1991

Authors and Affiliations

  • E. A. Carlen
    • 1
  • A. Soffer
    • 1
  1. 1.Department of MathematicsPrinceton UniversityPrincetonUSA

Personalised recommendations