Skip to main content
SpringerLink
Log in

A class of measures of informativity of observation channels

  • Published:
Periodica Mathematica Hungarica Aims and scope Submit manuscript

Summary

A class of numerical measures of informativity of observation channels or statistical experiments is defined by the aid off-divergences introduced by the author as measures of difference of two probability distributions. For observation channels with given prior probabilities, thef-informativity measures are generalizations of Shannon's mutual information and include Gallager's functionE 0(ϱQ) appearing in the derivation of error exponent for noisy channels, as well. For observation channels without prior probabilities, the suggested informativity measures have the geometric interpretation of a radius.

Thef-informativity defined for the Bayesian case shares several useful properties of the mutual information, such as e. g. the “data processing theorem”. Its maximum with respect to all possible prior distributions is shown by a minimax argument to be just thef-radius, thus the latter is a generalization of channel capacity. Thef-informativity measures can also be used to characterize the statistical sufficiency of indirect observations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. U. Augustin, Error estimates for low rate codes,Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 14 (1969), 61–88.

    Google Scholar 

  2. D. Blackwell, Comparison of experiments,Proc. 2nd Berkeley Sympos. Math. Statist. and Probability, Vol. 1, Berkeley, 1951, 93–102.

    Google Scholar 

  3. D. Blackwell, Equivalent comparison of experiments,Ann. Math. Statist. 24 (1953), 265–272.

    Google Scholar 

  4. D. Blackwell andM. A. Girshick,Theory of Games and Statistical Decisions, New York, 1954.

  5. L. L. Campbell, A coding theorem and Rényi's entropy,Information and Control 8 (1965), 423–429.

    Google Scholar 

  6. H. Chernoff, A measure of asymptotic efficiency for tests of a hypothesis based on a sum of observations,Ann. Math. Statist. 23 (1952), 493–507.

    Google Scholar 

  7. I. Csiszár, Eine Informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizität von Markoffschen Ketten,Magyar. Tud. Akad. Mat. Kutató Int. Közl. 8 (1963), 85–108.

    Google Scholar 

  8. I. Csiszár, Information-type measures of difference of probability distributions and indirect observations,Studia Sci. Math. Hungar. 2 (1967), 299–318.

    Google Scholar 

  9. R. G. Gallager,Information Theory and Reliable Communication, New York, 1968.

  10. H. Heyer, Erschöpftheit und Invarianz beim Vergleich von Experimenten,Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 12 (1969), 21–55.

    Google Scholar 

  11. F. Jelinek, Buffer overflow in variable length coding of fixed rate sources,IEEE Trans. Information Theory 14 (1968), 490–501.

    Google Scholar 

  12. S. Kullback,Information Theory and Statistics, New York, 1959

  13. D. V. Lindley, On a measure of information provided by an experiment,Ann. Math. Statist. 27 (1956), 986–1005.

    Google Scholar 

  14. N. Morse andR. Sackstedter, Statistical isomorphism,Ann. Math. Statist. 37 (1966), 203–214.

    Google Scholar 

  15. A. Perez, Sur la théorie de l'information dans le cas d'un alphabet abstrait,Trans. First Prague Confer. on Information Theory Statist. Decision Functions, Random Process, Prague, 1957, 209–243.

  16. A. Perez, Information-theoretic risk estimates in statistical decision,Kybernetika (Prague)3 (1967), 1–21.

    Google Scholar 

  17. A. Rényi, On measures of entropy and information,Proc. 4th Berkeley Sympos. Math. Statist. and Probability, Vol. 1, Berkeley 1960, 547–561.

    Google Scholar 

  18. A. Rényi, On the amount of information concerning an unknown parameter in a sequence of observations,Magyar Tud. Akad. Mat. Kutató Int. Közl. 9 (1964), 617–624.

    Google Scholar 

  19. A. Rényi, On some problems of statistics from the point of view of information theory,Proc. Colloq. on Information Theory, Debrecen, 1967, 343–347.

  20. R. Sackstedter, A note on statistical equivalence,Ann. Math. Statist. 38 (1967), 787–794.

    Google Scholar 

  21. R. Sibson, Information radius,Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 14 (1969), 149–161.

    Google Scholar 

  22. F. Topsoe, An information theoretical identity and a problem involving capacity,Studia Sci. Math. Hungar. 2 (1967), 291–292.

    Google Scholar 

  23. I. Vajda, On the convergence of information contained in a sequence of observations,Proc. Colloq. on Information Theory, Vol. 2, Debrecen, 1967, 489–501.

  24. I. Vajda, A contribution to the informational analysis of pattern,Proc. Fourth Hawaii Internat. Confer. on System Sci., Western Period. Co.

Download references

Author information

Authors and Affiliations

  1. Mta Matematikai Kutató Intézete, V., Reáltanoda U. 13-15, Budapest, Hungary

    I. Csiszár

Authors
  1. I. Csiszár
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Dedicated to the memory of Alfréd Rényi

This work was done while the author was visiting professor at The Catholic University of America, Washington, D.C., sponsored by National Science Foundation Grant No. GP-9396.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Csiszár, I. A class of measures of informativity of observation channels. Period Math Hung 2, 191–213 (1972). https://doi.org/10.1007/BF02018661

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02018661

Keywords

Search

Navigation