The Entropy Ergodic Theorem

  • Robert M. Gray


The goal of this chapter is to prove an ergodic theorem for sample entropy of finite-alphabet random processes. The result is sometimes called the ergodic theorem of information theory or the asymptotic equipartition (AEP) theorem, but it is best known as the Shannon-McMillan-Breiman theorem. It provides a common foundation to many of the results of both ergodic theory and information theory.


Ergodic Theorem Invariant Function Sample Entropy Entropy Rate Martingale Theory 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science + Business Media, LLC 2011

Authors and Affiliations

  1. 1.Department of Electrical EngineeringStanford UniversityStanfordUSA

Personalised recommendations