Abstract
The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. In this paper, we establish a number of simple inequalities for the weighted entropies (general as well as specific), mirroring similar bounds on standard (Shannon) entropies and related quantities. The required assumptions are written in terms of various expectations of weight functions. Examples are weighted Ky Fan and weighted Hadamard inequalities involving determinants of positive-definite matrices, and weighted Cramér-Rao inequalities involving the weighted Fisher information matrix.
Similar content being viewed by others
References
Belis M., Guiasu S.: A Quantitative and qualitative measure of information in cybernetic systems. IEEE Trans. Inf. Theory 14, 593–594 (1968)
Clim, A.: Weighted entropy with application. In: Analele Universităţii Bucureşti, Matematică, Anul LVII, pp. 223–231 (2008)
Cover T., Thomas J.: Elements of Information Theory. Wiley, New York (2006)
Cover, T.M., Thomas, J.A.: Determinant inequalities via information theory. SIAM J. Matrix Anal. Appl. 9, 384–392 (1988)
Dembo A., Cover T.M., Thomas J.A.: Information theoretic inequalities. IEEE Trans. Inf. Theory 37, 1501–1518 (1991)
Di Crescenzo A., Longobardi M.: Entropy based measure of uncertainty in past lifetime distributions. J. Appl. Prob. 39(3), 434–440 (2002)
Dial G., Taneja I.J.: On weighted entropy of type (α,β) and its generalizations. Appl. Math. 26, 418–425 (1981)
Frizelle G., Suhov Y.M.: An entropic measurement of queueing behaviour in a class of manufacturing operations. Proc. R. Soc. A 457, 1579–1601 (2001)
Frizelle G., Suhov Y.M.: The measurement of complexity in production and other commercial systems. Proc. R. Soc. A 464, 2649–2668 (2008)
Fuchs, A., Letta, G.: L’inégalité de Kullback. Application à la théorie de l’estimation. Séminaire de probabilités 4, pp. 108–131. Strasbourg (1970)
Guiasu S.: Weighted entropy. Rep. Math. Phys. 2, 165–179 (1971)
Ito K.: Introduction to Probability Theory. Cambridge University Press, Cambridge (1984)
Johnson D.H., Glantz R.M.: When does interval coding occur? Neurocopmuting 59:60, 13–18 (2004)
Kannappan P.L., Sahoo P.K.: On the general solution of a functional equation connected to sum form information measures on open domain. Math. Sci. 9, 545–550 (1986)
Kapur J.N.: Measures of Information and Their Applications, Chapter 17. Wiley, New Delhi (1994)
Fan K.: On a theorem of Weyl concerning eigenvalues of linear transformations. I. Proc. Natl. Acad. USA 35, 652–655 (1949)
Fan K.: On a theorem of Weyl concerning eigenvalues of linear transformations. II. Proc. Natl. Acad. USA 36, 31–35 (1950)
Fan K.: Maximum properies and inequalities for the eigenvalues of completely continuous operators. Proc. Natl. Acad. USA 37, 760–766 (1951)
Kelbert, M., Suhov, Y.: Continuity of mutual entropy in the limiting signal-to-noise ratio regimes. In: Stochastic Analysis, pp. 281–299. Springer, Berlin (2010)
Kelbert M., Suhov Y.: Information Theory and Coding by Example. Cambridge University Press, Cambridge (2013)
Moslehian, M.: Ky Fan inequalities. arXiv:1108.1467 (2011)
Muandet, K., Marukatat, S., Nattee, C.: Query selection via weighted entropy in graph-based semi-supervised classification. In: Advances in Machine Learning. Lecture Notes in Computer Science, vol. 5828, pp. 278–292 (2009)
Parkash O., Taneja H.C.: Characterization of the quantitative-qualitative measure of inaccuracy for discrete generalized probability distributions. Commun. Stat. Theory Methods 15, 3763–3771 (1986)
Sharma B.D., Mitter J., Mohan M.: On measure of ‘useful‘ information. Inf. Control 39, 323–336 (1978)
Singh R.P., Bhardwaj J.D.: On parametric weighted information improvement. Inf. Sci. 59, 149–163 (1992)
Sreevally A., Varma S.K.: Generating measure of cross entropy by using measure of weighted entropy. Soochow J. Math. 30(2), 237–243 (2004)
Srivastava A.: Some new bounds of weighted entropy measures. Cybern. Inf. Technol. 11(3), 60–65 (2011)
Suhov, Y., Yasaei Sekeh, S., Kelbert, M.: Entropy-power inequality for weighted entropy. arXiv:1502.02188
Suhov, Y., Stuhl, I., Yasaei Sekeh, S.: Weighted Gaussian entropy and determinant inequalities. arXiv:1505.01753
Suhov, Y., Stuhl, I., Kelbert, M.: Weight functions and log-optimal investment portfolios. arXiv:1505.01437
Tuteja R.K., Chaudhary Sh., Jain P.: Weighted entropy of orders α and type β information energy. Soochow J. Math. 19(2), 129–138 (1993)
Zamir R.: A proof of the Fisher information inequality via a data processing argument. IEEE Trans. Inf. Theory 44(3), 1246–1250 (1998)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Suhov, Y., Stuhl, I., Yasaei Sekeh, S. et al. Basic inequalities for weighted entropies. Aequat. Math. 90, 817–848 (2016). https://doi.org/10.1007/s00010-015-0396-5
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00010-015-0396-5