Advertisement

Computation of entropy for special cases. Entropy of stochastic processes

  • Roman V. Belavkin
  • Panos M. Pardalos
  • Jose C. Principe
Chapter

Abstract

In the present chapter, we set out the methods for computation of entropy of many random variables or of a stochastic process in discrete and continuous time.

From a fundamental and practical points of view, of particular interest are the stationary stochastic processes and their information-theoretic characteristics, specifically their entropy. Such processes are relatively simple objects, particularly a discrete process, i.e. a stationary process with discrete states and running in discrete time. Therefore, this process is a very good example for demonstrating the basic points of the theory, and so we shall start from its presentation.

References

  1. 7.
    Doob, J.: Stochastic Processes. Wiley Publications in Statistics. Wiley, New York (1953)zbMATHGoogle Scholar
  2. 8.
    Doob, J.: Stochastic Processes (Translation to Russian). Inostrannaya Literatura, Moscow (1956)Google Scholar
  3. 36.
    Ryzhik, J.M., Gradstein, I.S.: Tables of Series, Products and Integrals. Gostekhizdat, Moscow (1951, in Russian)Google Scholar
  4. 37.
    Ryzhik, J.M., Gradstein, I.S.: Tables of Series, Products and Integrals (Translation from Russian). Academic, New York (1965)Google Scholar
  5. 48.
    Stratonovich, R.L.: Conditional Markov Processes and Their Application to the Theory of Optimal Control. Moscow State University, Moscow (1966, in Russian)Google Scholar
  6. 50.
    Stratonovich, R.L.: Amount of information and entropy of segments of stationary Gaussian processes. Problemy Peredachi Informacii 3(2), 3–21 (1967, in Russian)Google Scholar
  7. 52.
    Stratonovich, R.L.: Conditional Markov Processes and Their Application to the Theory of Optimal Control (Translation from Russian). Modern Analytic and Computational Methods in Science and Mathematics. Elsevier, New York (1968)Google Scholar
  8. 56.
    Stratonovich, R.L., Grishanin, B.A.: Game problems with constraints of an informational type. Izv. USSR Acad. Sci. Tech. Cybern. 1, 3–12 (1968, in Russian)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Roman V. Belavkin
    • 1
  • Panos M. Pardalos
    • 2
  • Jose C. Principe
    • 3
  1. 1.Faculty of Science and TechnologyMiddlesex UniversityLondonUK
  2. 2.Industrial and Systems EngineeringUniversity of FloridaGainesvilleUSA
  3. 3.Electrical & Computer EngineeringUniversity of FloridaGainesvilleUSA

Personalised recommendations