Computation of entropy for special cases. Entropy of stochastic processes
In the present chapter, we set out the methods for computation of entropy of many random variables or of a stochastic process in discrete and continuous time.
From a fundamental and practical points of view, of particular interest are the stationary stochastic processes and their information-theoretic characteristics, specifically their entropy. Such processes are relatively simple objects, particularly a discrete process, i.e. a stationary process with discrete states and running in discrete time. Therefore, this process is a very good example for demonstrating the basic points of the theory, and so we shall start from its presentation.
- 8.Doob, J.: Stochastic Processes (Translation to Russian). Inostrannaya Literatura, Moscow (1956)Google Scholar
- 36.Ryzhik, J.M., Gradstein, I.S.: Tables of Series, Products and Integrals. Gostekhizdat, Moscow (1951, in Russian)Google Scholar
- 37.Ryzhik, J.M., Gradstein, I.S.: Tables of Series, Products and Integrals (Translation from Russian). Academic, New York (1965)Google Scholar
- 48.Stratonovich, R.L.: Conditional Markov Processes and Their Application to the Theory of Optimal Control. Moscow State University, Moscow (1966, in Russian)Google Scholar
- 50.Stratonovich, R.L.: Amount of information and entropy of segments of stationary Gaussian processes. Problemy Peredachi Informacii 3(2), 3–21 (1967, in Russian)Google Scholar
- 52.Stratonovich, R.L.: Conditional Markov Processes and Their Application to the Theory of Optimal Control (Translation from Russian). Modern Analytic and Computational Methods in Science and Mathematics. Elsevier, New York (1968)Google Scholar
- 56.Stratonovich, R.L., Grishanin, B.A.: Game problems with constraints of an informational type. Izv. USSR Acad. Sci. Tech. Cybern. 1, 3–12 (1968, in Russian)Google Scholar