Classical Dynamics, Measurement, and Information

Abstract

In this chapter, we review the basic concepts in the classical information theory [1, 2], which is needed to quantitatively discuss the relationship between thermodynamics and information. First, we formulate stochastic dynamics in classical systems. Second, we introduce the basic quantities in the classical information theory: the Shannon information, the Kullback–Leibler divergence (the relative entropy), and the mutual information. Third, we discuss classical measurement theory with stochastic errors by using the information theory. We illustrate three typical examples of classical measurements.