The Entropy Ergodic Theorem
The goal of this chapter is to prove an ergodic theorem for sample entropy of finite-alphabet random processes. The result is sometimes called the ergodic theorem of information theory or the asymptotic equipartition (AEP) theorem, but it is best known as the Shannon-McMillan-Breiman theorem. It provides a common foundation to many of the results of both ergodic theory and information theory.
KeywordsErgodic Theorem Invariant Function Sample Entropy Entropy Rate Martingale Theory
Unable to display preview. Download preview PDF.