Abstract
In recent years we extended Shannon static statistical information theory to dynamic processes and established a Shannon dynamic statistical information theory, whose core is the evolution law of dynamic entropy and dynamic information. We also proposed a corresponding Boltzmman dynamic statistical information theory. Based on the fact that the state variable evolution equation of respective dynamic systems, i.e. Fokker-Planck equation and Liouville diffusion equation can be regarded as their information symbol evolution equation, we derived the nonlinear evolution equations of Shannon dynamic entropy density and dynamic information density and the nonlinear evolution equations of Boltzmann dynamic entropy density and dynamic information density, that describe respectively the evolution law of dynamic entropy and dynamic information. The evolution equations of these two kinds of dynamic entropies and dynamic informations show in unison that the time rate of change of dynamic entropy densities is caused by their drift, diffusion and production in state variable space inside the systems and coordinate space in the transmission processes; and that the time rate of change of dynamic information densities originates from their drift, diffusion and dissipation in state variable space inside the systems and coordinate space in the transmission processes. Entropy and information have been combined with the state and its law of motion of the systems. Furthermore we presented the formulas of two kinds of entropy production rates and information dissipation rates, the expressions of two kinds of drift information flows and diffusion information flows. We proved that two kinds of information dissipation rates (or the decrease rates of the total information) were equal to their corresponding entropy production rates (or the increase rates of the total entropy) in the same dynamic system. We obtained the formulas of two kinds of dynamic mutual informations and dynamic channel capacities reflecting the dynamic dissipation characteristics in the transmission processes, which change into their maximum—the present static mutual information and static channel capacity under the limit case where the proportion of channel length to information transmission rate approaches to zero. All these unified and rigorous theoretical formulas and results are derived from the evolution equations of dynamic information and dynamic entropy without adding any extra assumption. In this review, we give an overview on the above main ideas, methods and results, and discuss the similarity and difference between two kinds of dynamic statistical information theories.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cover, T. M., Thomas, J. A., Elements of Information Theory, New York: John Wiley & Sons, 1991.
Zhu, X. L., Fundamentals of Applied Information Theory (in Chinese), Beijing: Tsinghua University Press, 2000.
Zhong, Y. X., Principles of Information Science (in Chinese), Beijing: Beijing University of Posts and Telecommunications Press, 1996
Verdu, S., McLaughlin, S. W., eds., Information Theory: 50 years of Discovery, New York: IEEE Press, 2000.
Weber, B. H., Depew, D. J., Smith, J. D., eds., Entropy, Information and Evolution, Cambridge: The MIT Press, 1988.
Kapur, J. N., Kesavan, H. K., Entropy Optimization Principle with Application, San Diego: Academic Press, 1992.
Zurek, W. H., ed., Complexity, Entropy and the Physics of Information, Reading Mass: Addison-Wesley, 1990.
Haken, H., Information and Self-organization, Berlin: Springer-Verlag, 1988.
Ingarden, H. S., Kossakowski, A., Ohya, M., Information Dynamics and Open System, Dordrecht: Kluwer Academic Publishers, 1997.
Atmanspacher, H., Scheingraber, H., eds., Information Dynamics, New York: Plenum Press, 1991.
Barwise, J., Information Flow, Cambridge: Cambridge University Press, 1997.
Weenig, W. H., Information Diffusion and Persuation in Communication Networks, Leiden: Leiden University, 1991.
Xing, X. S., On basic equation of statistical physics, Science in China, Ser. A, 1996, 39(11): 1193–1203.
Xing, X. S., On the fundamental equation of nonequilibrium statistical physics, Int. J. Mod. Phys. B, 1998, 12(20): 2005–2029.
Xing, X. S., New progress in the principle of nonequilibrium statistical physics, Chinese Science Bulletin, 2001,46(6): 448–454
Xing, X. S., On the formula for entropy production rate, Acta Physica Sinica (in Chinese), 2003, 52(12): 2969–2976.
Xing, X. S., On dynamic statistical information theory, Transactions of Beijing Institute of Technology (in Chinese), 2004, 24(1): 1–15
Xing, X. S., Nonequilibrium statistical information theory, Acta Physica Sinica (in Chinese), 2004, 53(9): 2852–2863.
Xing, X. S., Physical entropy, information entropy and their evolution equations, Science in China, Ser. A, 2001, 44(10): 1331–1339.
Huang, C. F., Principle of information diffusion, Fuzzy Sets and System, 1997, 91(1): 69–90.
Haken, H., Synergetics, Berlin: SpringerVerlag, 1983.
Magnasco, M. O., Forced thermal ratchets. Phys. Rev. Lett., 1993, 71(10): 1477–1480.
Bartussek, R., Hanggi, P., Kisser, J. G., Periodically rocked thermal ratchets. Europhys Lett., 1994, 28: 459–464.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Xing, X. Dynamic statistical information theory. SCI CHINA SER G 49, 1–37 (2006). https://doi.org/10.1007/s11433-005-0102-z
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s11433-005-0102-z