Skip to main content
SpringerLink
Log in

Probability, Information and Statistical Physics

  • Published:
International Journal of Theoretical Physics Aims and scope Submit manuscript

Abstract

In this short survey review we discuss foundational issues of the probabilistic approach to information theory and statistical mechanics from a unified standpoint. Emphasis is on the inter-relations between theories. The basic aim is tutorial, i.e. to carry out a basic introduction to the analysis and applications of probabilistic concepts to the description of various aspects of complexity and stochasticity. We consider probability as a foundational concept in statistical mechanics and review selected advances in the theoretical understanding of interrelation of the probability, information and statistical description with regard to basic notions of statistical mechanics of complex systems. It includes also a synthesis of past and present researches and a survey of methodology. The purpose of this terse overview is to discuss and partially describe those probabilistic methods and approaches that are used in statistical mechanics with the purpose of making these ideas easier to understanding and to apply.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. http://theor.jinr.ru/~kuzemsky/ankolmogbio.html

References

  1. Tsonis, A.A.: Randomnicity: Rules and Randomness in the Realm of the Infinite. Imperial College Press, London (2008)

    Book  Google Scholar 

  2. Bogoliubov, N.N.: Problems of a dynamical theory in statistical physics. In: de Boer, J., Uhlenbeck, G.E. (eds.) Studies in Statistical Mechanics, vol. 1, p 1. North-Holland, Amsterdam (1962)

    Google Scholar 

  3. Bogoliubov, N.N.: On the stochastic processes in the dynamical systems. Sov. J. Part. Nucl 9, 205 (1978)

    ADS  Google Scholar 

  4. Kuzemsky, A.L.: Theory of transport processes and the method of the nonequilibrium statistical operator. Int. J. Mod. Phys. B 21, 2821 (2007)

    Article  ADS  MATH  Google Scholar 

  5. Kuzemsky, A.L.: Statistical mechanics and the physics of many-particle model systems. Phys. Part. Nucl. 40, 949 (2009)

    Article  Google Scholar 

  6. Aubin, D., Dalmedico, A.D.: Writing the history of dynamical systems and chaos: Longue duree and revolution, disciplines and cultures. Hist. Math. 29, 273 (2002)

    Article  MATH  Google Scholar 

  7. Kossakowski, A., Ohya, M., Togawa, Y.: How can we observe and describe chaos. Open. Syst. Inf. Dyn. 10, 221 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  8. Leoncini, X., Leonetti, M. (eds.): From Hamiltonian Chaos to Complex Systems: A Nonlinear Physics Approach. Springer, Berlin (2013)

    Google Scholar 

  9. Kuzemsky, A.L.: Bogoliubov’s vision: Quasiaverages and broken symmetry to quantum protectorate and emergence. Int. J. Mod. Phys. B 24, 835 (2010)

    Article  ADS  MATH  MathSciNet  Google Scholar 

  10. Bertin, E.: A Concise Introduction to the Statistical Physics of Complex Systems. Springer, Berlin (2011)

    Google Scholar 

  11. Baranger, M.: Chaos, Complexity, and Entropy. A Physics Talk for Non-Physicists. New England Complex Systems Institute, Cambridge (2000)

    Google Scholar 

  12. Roederer, J.G.: Information and its Role in Nature. Springer, Berlin (2005)

    MATH  Google Scholar 

  13. Applebaum, D.: Probability and Information. An Integrated Approach. Cambridge University Press, Cambridge (2008)

    Book  MATH  Google Scholar 

  14. Rissanen, J.: Information and Complexity in Statistical Modeling. Springer, Berlin (2007)

    MATH  Google Scholar 

  15. Harmuth, H.F.: Information Theory Applied to Space-Time Physics. World Scientific, Singapore (1993)

    Book  Google Scholar 

  16. Grandy, W.T.: Information theory in physics. Am. J. Phys 65, 466 (1997)

    Article  ADS  Google Scholar 

  17. Beisbart, C., Hartmann, S. (eds.): Probabilities in Physics. Oxford University Press, Oxford (2011)

    MATH  Google Scholar 

  18. Aguirre, A., Foster, B., Merali, Z. (eds.): It From bit or bit From It? On Physics And Information. Springer, Berlin (2015)

  19. Kadomtsev, B.B.: Dynamics and information. Phys. Usp 37, 425 (1994)

    Article  ADS  Google Scholar 

  20. Green, H.S.: Information Theory and Quantum Physics. Springer, Berlin (2000)

    Book  MATH  Google Scholar 

  21. Brukner, C., Zeilinger, A.: Young’s experiment and the finiteness of information. Phil. Trans. R. Soc. Lond. A 360, 1061 (2002)

    Article  ADS  MATH  MathSciNet  Google Scholar 

  22. Diosi, L.: A Short Course in Quantum Information Theory. An Approach From Theoretical Physics, 2nd ed. Springer, Berlin (2011)

    Book  MATH  Google Scholar 

  23. Schuster, P.: Less is more and the art of modeling complex phenomena. Complexity 11, 11 (2005)

    Article  Google Scholar 

  24. Volkenstein, M.V.: The amount and value of information in biology. Found. Phys 7, 97 (1977)

    Article  ADS  Google Scholar 

  25. Crofts, A.R.: Life, information, entropy, and time: vehicles for semantic inheritance. Complexity 13, 14 (2007)

    Article  Google Scholar 

  26. Asano, M., Basieva, I., Khrennikov, A., Ohya, M., Tanaka, Y., Yamato, I.: Quantum information biology: from information interpretation of quantum mechanics to applications in molecular biology and cognitive psychology (2015) arXiv:1503.02515 [quant ph]

  27. Nielsen, M.A., Chuang, I.L.: Quantum Computation and Quantum Information. Cambridge University Press, Cambridge (2000)

    MATH  Google Scholar 

  28. Ohya, M., Volovich, I.: Mathematical Foundations of Quantum Information and Computation and Its Applications to Nano- and Bio-systems. Springer, Berlin (2011)

    Book  MATH  Google Scholar 

  29. Stenholm, S., Suominen, K.A.: Elements of Information Theory. Wiley, New York (2005)

    Google Scholar 

  30. Vedral, V.: Decoding Reality. The Universe as Quantum Information. Oxford University Press, Oxford (2010)

    MATH  Google Scholar 

  31. Yeh, M.-C., Leggett, A.J.: The escape physics of single shot measurement of flux qubit with dcSQUID (2014). arXiv:1401.4186v1[cond-mat.supr-con]

  32. Kac, M.: Some Stochastic Problems in Physics and Mathematics. In: Colloquium Lectures in Pure and Applied Science, no 2. Texas, Dallas (1956)

  33. Kac, M.: Probability and Related Topics in Physical Sciences. Interscience Publ., New York (1958)

    Google Scholar 

  34. Feller, W.: An Introduction to Probability Theory and Its Applications, vol. 1. Wiley, New York (1970)

    Google Scholar 

  35. Feller, W.: An Introduction to Probability Theory and Its Applications, vol. 2. Wiley, New York (1971)

    Google Scholar 

  36. Gnedenko, B.: The Theory Probability. Mir Publ., Moscow (1975)

    Google Scholar 

  37. Ambegaokar, V.: Reasoning About Luck: Probability and its Uses in Physics. Cambridge University Press, New York (1996)

    Book  Google Scholar 

  38. Kallenberg, O.: Foundations of Modern Probability. Springer, Berlin (1997)

    MATH  Google Scholar 

  39. Jaynes, E.T. In: Rosenkrantz, R.D. (ed.) : Papers on Probability, Statistics and Statistical Physics. D.Reidel Publ., Dordrecht (1983)

  40. Grandy, W.T., Milloni, P.W., et al.: Physics and Probability: Essays in Honor of Edwin T. Jaynes. Cambridge University Press, New York (1993)

    Book  Google Scholar 

  41. Von Plato, J.: Creating Modern Probability: Its Mathematics, Physics And Philosophy In Historical Perspective. Cambridge University Press, Cambridge (1994)

    Book  Google Scholar 

  42. Jaynes, E.T.: Probability Theory: The Logic of Science. Cambridge University Press, New York (2003)

    Book  Google Scholar 

  43. McCullagh, P.: What is a statistical model?. Ann. Stat. 30, 1225 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  44. Graben, P.B., Barrett, A., Atmanspacher, H.: Stability criteria for the contextual emergence of macrostates in neural networks. Netw. Comput. Neural Syst. 20, 178 (2009)

    Article  Google Scholar 

  45. Gibbs, J. W.: Elementary Principles in Statistical Mechanics Developed with Especial Reference to the Rational Foundations of Thermodynamics. Dover Publ., New York (1960)

    Google Scholar 

  46. Khinchin, A.Ya.: Mathematical Foundations of Statistical Mechanics. Dover Publ., New York (1949)

    MATH  Google Scholar 

  47. Sklar, L.: Physics and Chance. Cambridge University Press, New York (1993)

    Book  Google Scholar 

  48. Guttmann, Y.: The Concept of Probability in Statistical Physics. Cambridge University Press, Cambridge (2002)

    Google Scholar 

  49. Ernst, G., Huttemann, A. (eds.): Time, Chance and Reduction. Philosophical Aspects of Statistical Mechanics. Cambridge University Press, Cambridge (2010)

    Google Scholar 

  50. Kozlov, V.V., Smolyanov, O.G.: Information entropy in problems of classical and quantum statistical mechanics. Dokl. Math. 74, 910 (2006)

    Article  MATH  Google Scholar 

  51. Kozlov, V.V.: Gibbs ensembles, equidistribution of the energy of sympathetic oscillators and statistical models of thermostat. Regul. Chaotic Dynam 13, 141 (2008)

    Article  ADS  MATH  Google Scholar 

  52. Home, D., Whitaker, M.A.B.: Ensemble interpretations of quantum mechanics. A modern perspective. Phys. Rep 210, 223 (1992)

    Article  ADS  MathSciNet  Google Scholar 

  53. Omnes, R.: The Interpretation of Quantum Mechanics. Princeton University Press, Princeton (1994)

    MATH  Google Scholar 

  54. Auletta, G.: Foundations and Interpretation of Quantum Mechanics. World Scientific, Singapore (2001)

    Google Scholar 

  55. Kuzemsky, A.L.: Works of D. I. Blokhintsev and development of quantum Physics. Phys. Part. Nucl. 39, 137 (2008)

    Article  Google Scholar 

  56. Khrennikov, A.: Interpretations of Probability, 2nd ed. Walter de Gruyter Press, Berlin (2009)

    Book  MATH  Google Scholar 

  57. Khrennikov, A. (ed.): Foundations of Probability and Physics. World Scientific, Singapore (2002)

    Google Scholar 

  58. Khrennikov, A.: Contextual Approach to Quantum Formalism. Springer, Berlin (2009)

    Book  MATH  Google Scholar 

  59. von Mises, R.: Probability, Statistics, and Truth. Dover Publ., New York (1981)

    MATH  Google Scholar 

  60. Mugur-Schachter, M.: On the concept of probability. Math. Struct. Comput. Sci. 24(e240309), 91 (2014)

    Google Scholar 

  61. Howie, D.: Interpreting Probability: Controversies and Developments in the Early Twentieth Century. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  62. Tijms, H.: Understanding Probability. Chance Rules in Everyday Life, 2nd ed. Cambridge University Press, Cambridge (2007)

    Book  Google Scholar 

  63. Narens, L.: Theories of Probability. An Examination of Logical and Qualitative Foundations. World Scientific, Singapore (2007)

    MATH  Google Scholar 

  64. Suarez, M. (ed.): Probabilities, Causes and Propensities in Physics. (Studies in Epistemology, Logic, Methodology, and Philosophy of Science). Springer, New York (2011)

    Google Scholar 

  65. Ruelle, D.: Statistical Mechanics. Rigorous Results, 2nd ed. World Scientific, Singapore (1999)

    Book  MATH  Google Scholar 

  66. Ruelle, D.: Chance and Chaos. Princeton University Press, Princeton (1991)

    Google Scholar 

  67. Laughlin, R.B.: A Different Universe. Basic Books, New York (2005)

    Google Scholar 

  68. Laughlin, R.B.: The Crime of Reason: And the Closing of the Scientific Mind. Basic Books, New York (2008)

    Google Scholar 

  69. Laughlin, R.B., Pines, D.: Theory of everything. Proc. Natl. Acad. Sci. (USA) 97, 28 (2000)

    Article  ADS  MathSciNet  Google Scholar 

  70. Cox, D.L., Pines, D.: Complex adaptive matter: emergent phenomena in materials. MRS Bull. 30, 425 (2005)

    Article  Google Scholar 

  71. Carroll, R.: On the Emergence Theme of Physics. World Scientific, Singapore (2010)

    Book  MATH  Google Scholar 

  72. Cramer, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)

    MATH  Google Scholar 

  73. Kolmogorov, A.N.: Foundations of the Theory of Probability. Chelsea Publishing Co., New York (1950)

    Google Scholar 

  74. Kolmogorov, A.N.: On Logical Foundations of Probability Theory. In: Lecture Notes in Mathematics. Probability Theory and Mathematical Statistics, vol. 1021, p 1. Springer, Berlin (1983)

    Google Scholar 

  75. Kolmogorov, A.N.: The theory of probability. In: Mathematics: Its Content, Methods and Meaning. Dover Publ., New York (1999)

  76. Doob, J.L.: Stochastic Processes. Wiley, New York (1953)

    MATH  Google Scholar 

  77. Papoulis, A., Pillai, S.U.: Probability, Random Variables and Stochastic Processes, 4th ed. McGraw-Hill, New York (2002)

    Google Scholar 

  78. van Kampen, N.G.: Stochastic Processes in Physics and Chemistry, 3rd ed. North-Holland, Amsterdam (2007)

    Google Scholar 

  79. Wio, H.S.: An Introduction to Stochastic Processes and Nonequilibrium Statistical Physics. World Scientific, Singapore (1994)

    Book  MATH  Google Scholar 

  80. Mahnke, R., Kaupuzs, J., Lubashevsky, I.: Physics of Stochastic Processes: How Randomness Acts in Time. Wiley, New York (2008)

    Book  Google Scholar 

  81. Jeffreys, H.: Theory of Probability. Clarendon Press, Oxford (1998)

    MATH  Google Scholar 

  82. Kennedy, B.: Celebrations of creativity. Phys. World 3, 52 (1994)

    Article  Google Scholar 

  83. Fine, T.L.: The only acceptable approach to probabilistic reasoning. SIAM News 37(2), 723 (2004)

    Google Scholar 

  84. van Kampen, N.G.: Views of a Physicist. World Scientific, Singapore (2000)

    MATH  Google Scholar 

  85. Khinchin, A.I.: Mathematical Foundations of Information Theory. Dover Publ., New York (1957)

    MATH  Google Scholar 

  86. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1991)

    Book  MATH  Google Scholar 

  87. Gray, R.M.: Entropy and Information Theory. Springer, Berlin (2000)

    Google Scholar 

  88. MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  89. Gitt, W.: In the Beginning was Information. Master Books, Green Forest, Arkansas (2005)

    Google Scholar 

  90. Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev 106, 620 (1957)

    Article  ADS  MATH  MathSciNet  Google Scholar 

  91. Jaynes, E.T.: Information theory and statistical mechanics - II. Phys. Rev 108, 171 (1957)

    Article  ADS  MathSciNet  Google Scholar 

  92. Katz, A.: Principles of Statistical Mechanics: The Information Theory Approach. W. H. Freeman and Co., San Francisco (1967)

    Google Scholar 

  93. Hobson, A.: Concepts in Statistical Mechanics. Gordon and Breach, New York (1970)

    Google Scholar 

  94. Morse, P.M.: Thermal Physics, 2nd ed. W. A. Benjamin, Inc., New York (1969)

    Google Scholar 

  95. Grandy, W.T.: Foundations of Statistical Mechanics: Equilibrium Theory, vol. 1. D. Reidel Publ., Dordrecht (1987)

    Book  Google Scholar 

  96. Grandy, W.T.: Foundations of Statistical Mechanics: Nonequilibrium Phenomena, vol. 2. D. Reidel Publ., Dordrecht (1988)

    Book  Google Scholar 

  97. Wiener, N.: The Human Use of Human Beings: Cybernetics and Society. Da Capo Press, New York (1988)

    Google Scholar 

  98. Keller, G., Warnecke, G. (eds.): Entropy. (Princeton Studies in Applied Mathematics). Princeton University Press, Princeton (2003)

    Google Scholar 

  99. Evans, L.C.: Entropy, Large Deviations, and Statistical Mechanics. Springer, Berlin (2005)

    Google Scholar 

  100. Evans, L.C.: Entropy and Partial Differential Equations. Wiley, New York (2008)

    Google Scholar 

  101. Minlos, R.A.: Introduction to Mathematical Statistical Physics. (University Lecture Series) American Mathematical Society (2000)

  102. Zubarev, D.N.: Nonequilibrium Statistical Thermodynamics. Consultant Bureau, New York (1974)

    Google Scholar 

  103. Muller, I.: A History of Thermodynamics. The Doctrine of Energy and Entropy. Springer, Berlin (2007)

    Google Scholar 

  104. Muller, I., Muller, W.H.: Fundamentals of Thermodynamics and Applications. Springer, Berlin (2009)

    Google Scholar 

  105. Starzak, M.E.: Energy and Entropy. Equilibrium to Stationary States. Springer, Berlin (2010)

    Book  Google Scholar 

  106. Kuzemsky, A.L.: Thermodynamic limit in statistical physics. Int. J. Mod. Phys. B 28, 1430004 (2014)

    Article  ADS  MathSciNet  Google Scholar 

  107. Gzyl, H.: The Method of Maximum Entropy. World Scientific, Singapore (1995)

    MATH  Google Scholar 

  108. Charpentier, E., Lesne, A., Nikolskii, N.K. (eds.): Kolmogorov’s Heritage in Mathematics. Springer, Berlin (2007)

    MATH  Google Scholar 

  109. Li, M., Vitanyi, P.: An Introduction to Kolmogorov Complexity and its Applications. Springer, Berlin (1997)

    Book  MATH  Google Scholar 

  110. Livi, R., Vulpiani, A. (eds.): The Kolmogorov Legacy in Physics. Springer, Berlin (2004)

    Google Scholar 

  111. Tihomirov, V.M.: Andrei Nikolaevich Kolmogorov (1903-1987). The great Russian scientist. Teach. Math. 1, 25 (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Bogoliubov Laboratory of Theoretical Physics, Joint Institute for Nuclear Research, 141980 Dubna, Moscow Region, Russia

    A. L. Kuzemsky

Authors
  1. A. L. Kuzemsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. L. Kuzemsky.

Appendix A: Biography of A. N. Kolmogorov

Appendix A: Biography of A. N. Kolmogorov

Andrei Nikolaevich KolmogorovFootnote 1(1903 - 1987) was one of the most prominent twentieth-century mathematicians. Throughout his mathematical work, A.N. Kolmogorov showed great creativity and versatility and his wide-ranging studies in many different areas led to the solution of conceptual and fundamental problems and the posing of new, important questions [108]. His lasting contributions embrace probability theory and statistics, the theory of dynamical systems, mathematical logic, geometry and topology, the theory of functions and functional analysis, classical mechanics, the theory of turbulence, and information theory. A.N. Kolmogorov made major contributions to almost all areas of mathematics and many fields of science [109, 110] and is considered one of the 20th century’s most eminent mathematicians. He was the founder of modern probability theory, having formulated its axiomatic foundations and developed many of its mathematical tools. Kolmogorov also helped make advances in many applied sciences, from physics to linguistics.

In 1920, at the age of 17, Kolmogorov enrolled in Moscow University. During his years as a university student, he published 18 mathematical papers including the strong law of large numbers, generalizations of calculus operations, and discourses in intuitionistic logic. In 1925, Kolmogorov received a doctoral degree from the department of physics and mathematics and became a research associate atMoscow University. At the age of 28, he was made a full professor of mathematics; two years later, in 1933, he was appointed director of the university’s Institute of Mathematics. While he was still a research associate, Kolmogorov published a paper, ”General Theory of Measure and Probability Theory,” in which he gave an axiomatic representation of some aspects of probability theoryon the basis of measure theory. His work in this area, which a younger colleague once called the ”New Testament” of mathematics, was fully described in a monograph that was published in 1933. The paper was translated into English and published in 1950 as Foundations of the Theory of Probability. Kolmogorov’s contribution to probability theory has been compared to Euclid’s role in establishing the basis of geometry. He also made major contributions to the understanding of stochastic processes (involving random variables), and he advanced the knowledge of chains of linked probabilities. Kolmogorov developed many applications of probability theory. He published a lot of papers on probability theory and mathematical statistics, and embraces topics such as limit theorems, axiomatic and logical foundations of probability theory, Markov chains and processes, stationary processes and branching processes.

A. N. Kolmogorov was a genius and a person proficient in a wide range of fields [111]. He was interested in sciences, for both, exact ones, and humanities, and he had a keen interest for philosophical problems as well as for problems of ethics and morality. He was an expert and a delicate judge of arts-of poetry, of paintings, and above all, of sculptures. He was deeply concerned for the future problems of humankind. Andrej Nikolaevich Kolmogorov undoubtedly was one of the greatest mathematicians and researchers of laws of nature of the Twentieth Century, (a Natural Philosopher, as such one would have been called in earlier times), and one among the greatest Russian scientists in the entire history of the Russian science. Additional information and analysis of Kolmogorov’s heritage can be found in the following books [108–110]

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kuzemsky, A.L. Probability, Information and Statistical Physics. Int J Theor Phys 55, 1378–1404 (2016). https://doi.org/10.1007/s10773-015-2779-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10773-015-2779-8

Keywords

Search

Navigation