Abstract
The notion of the Shannon entropy appears frequently and is important in many works. In this Chapter we will review some of the characterizations of it and of the concept of the gain of information with functional inequalities. Similarly, we shall present a characterization of Rényi’s generalized concept of information measure and gain of information with the aid of functional inequalities. These inequalities, to be discussed, have also other interpretations.
This Chapter is partly based on a manuscript of P. Fischer of the University of Guelph. It has been edited to conform to the style of the rest of the book. In addition other minor editing has been done.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
SHANNON, C. E., The mathematical theory of communication, Bell Syst. Techn. Journal 27 (1948), 379–423; 623–656.
HAMMING, R. W., “Coding and Information Theory”, New Jersey, 1980.
KIRMSER, P. G., Problem E1274, Amer. Math. Monthly 64 (1957), 432.
RÉNYI, A., “Probability Theory”, Amsterdam-London, 1970.
FISCHER, P., On some new generalizations of Shannon’s inequality, Pac. J. Math. 70 (1977), 351–360.
FISCHER, P., On the inequality \(\sum {pif(pi)} \sum {pif(qi)} \), Metrika 18 (1972), 199 - 208.
ACZÉL, J. and J. PFANZAGL, Remarks on the measurement of subjective probability and information, Metrika 11 (1966), 91–105.
FISCHER, P., Sur l’inegalite \(\sum {pif(pi)} \sum {pif(qi)} \), Aequationes Math. 2 (1969), 363.
MUSZELY, Gy., On continuous solutions of a functional inequality, Metrika 20 (1973), 65–69.
CSASZAR, A., Sur une class des fonctions non mesurables, Fund. Math. 36 (1949), 72–76.
FISCHER P., On the inequality \(\sum\nolimits_{i = 1}^n {{p_i}\frac{{f\left( {{p_i}} \right)}}{{f\left( {{q_i}} \right)}}} \geqslant 1\), Pac. J. Math. 60 (1975), 65 - 74.
FISCHER P., On the inequality \(\sum {pi\frac{{f(pi)}}{{f(qi)}} \leqslant 1} \), Canad. Math. Bull. 17 (1974), 193–199.
ACZÉL, J. and A. M. OSTROWSKI, On the characterization of Shannon’s entropy by Shannon’s inequality, J. Austral. Math. Soc. 16 (1973), 368–374.
WALTER, W., Remark on a Paper by Aczél and Ostrowski, J. Austral. Math. Soc. 22A (1976), 165–166.
FISCHER, P., On the inequality \(\sum {g(pi)f(pi)\sum {g(pi)f(qi)} } \), Aequationes Math. 10 (1974), 23 - 33.
FISCHER, P., Sur l’inégalité \(\sum {\left[ {pif(pi) + qif(qi)} \right]} \sum {\left[ {pif(qi) + qif(pi)} \right]} \), Periodica Mathematica Hungarica 5 (1974), 87 - 92.
KARDOS, P., On the inequality \(\sum\nolimits_{i = 1}^n {{p_i}\frac{{{f_i}\left( {{p_i}} \right)}}{{{f_i}\left( {{q_i}} \right)}} \leqslant 1} \), Canad. Math. Bull. 22 (1979), 483 - 489.
FEINSTEIN, A., “Foundations of Information Theory”, New York, 1958.
STOLARSKY, K. B., A stronger logarithmic inequality suggested by the entropy inequality, SIAM J. Math. Anal. 11 (1980), 242–247.
KHINCHIN, A. I., “Mathematical Foundations of Information Theory”, New York, 1957.
CSISZAR, I., Information-type measures of difference of probability distributions and indirect observations, Studia Sci. Math. Hungarica 2 (1967), 299–318.
CSISZAR, I., Informtion measures: A critical survey, in Trana. 7th Prague Conf. Inf. Th. Stat. Decision Funct. Rand. Prob., Prague, 1978.
J., On Shannon’s inequality, optimal coding, and characterizations of Shannon’s and Rényi’s entropies, Inst. Naz. Alta Mat. Symp. Math. 15 (1975), 153–179.
NATH, P. and D. P. MITTAL, A generalization of Shannon’s inequality and its application in coding theory, Information and Control 23 (1973), 439–445.
CLAUSING, A., Type t entropy and majorization, SIAM J. Math. Anal. 14 (1983), 203–208.
ISIHARA, A., Information loss and entropy increase, J. Math. A.al. Appl. 39 (1972), 314–317.
KAPUR, J. N., On convex semimetric spaces and generalized Shannon inequalities, Indian J. Pure Appl. Math. 18 (1987), 122–135.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1993 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Mitrinović, D.S., Pečarić, J.E., Fink, A.M. (1993). Shannon’s Inequality. In: Classical and New Inequalities in Analysis. Mathematics and Its Applications (East European Series), vol 61. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-1043-5_23
Download citation
DOI: https://doi.org/10.1007/978-94-017-1043-5_23
Publisher Name: Springer, Dordrecht
Print ISBN: 978-90-481-4225-5
Online ISBN: 978-94-017-1043-5
eBook Packages: Springer Book Archive