Different Closed-Form Expressions for Generalized Entropy Rates of Markov Chains
- 22 Downloads
Closed-form expressions for generalized entropy rates of Markov chains are obtained through pertinent averaging. First, the rates are expressed in terms of Perron-Frobenius eigenvalues of perturbations of the transition matrices. This leads to a classification of generalized entropy functionals into five exclusive types. Then, a weighted expression is obtained in which the associated Perron-Frobenius eigenvectors play the same role as the stationary distribution in the well-known weighted expression of Shannon entropy rate. Finally, all terms are shown to bear a meaning in terms of dynamics of an auxiliary absorbing Markov chain through the notion of quasi-limit distribution. Illustration of important properties of the involved spectral elements is provided through application to binary Markov chains.
KeywordsEntropy rate Ergodic Markov chain Entropy distribution Quasi-limit distribution
Mathematics Subject Classification (2010)94A17 60J10 15B48 47B65
Unable to display preview. Download preview PDF.
- Amari SI, Nagaoka H (2007) Methods of information geometry (Vol. 191), American Mathematical Soc.Google Scholar
- Beck C, Schögl F (1995) Thermodynamics of chaotic systems: an introduction (No. 4). Cambridge University Press, CambridgeGoogle Scholar
- Girardin V (2005) On the different extensions of the Ergodic Theorem of information theory. In: Baeza-Yates R, Glaz J, Gzyl H, Hüsler J, Palacios JL (eds) Recent advances in applied probability. Springer, San Francisco, pp 163–179Google Scholar
- Kafsi M, Grossglauser M, Thiran P (2015) Traveling salesman in reverse: Conditional Markov entropy for trajectory segmentation. IEEE Int Conf Data Min 2015:201–210Google Scholar
- Meyer CD (2000) Matrix analysis and applied linear algebra, SIAM PhiladelphiaGoogle Scholar
- Pronzato L, Wynn HP, Zhigljavsky AA (1997) Using Renyi entropies to measure uncertainty in search problems. In: Mathematics of stochastic manufacturing systems: AMS-SIAM summer seminar in applied mathematics 33. Williamsburg, USA, pp 253–268Google Scholar
- Rényi A (1961) On measures of entropy and information. In: Proceedings of the fourth Berkeley symposium on mathematical statistics and probability volume 1: contributions to the theory of statistics, The Regents of the University of CaliforniaGoogle Scholar
- Regnault P, Girardin V, Lhote L (2017) Escort distributions and the Rényi entropy rates of Markov chains, Geometric science of information, ParisGoogle Scholar
- R Core Team (2018) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/
- Varma RS (1966) Generalizations of Rényi’s entropy of order α. J Math Sc 1:34–48Google Scholar