Skip to main content
SpringerLink
Log in

The Compound Notions for Logical and Shannon Entropies

  • Chapter
  • First Online:
New Foundations for Information Theory

Part of the book series: SpringerBriefs in Philosophy ((BRIEFSPHILOSOPH))

  • 542 Accesses

Abstract

In this chapter, all the compound notions of simple, joint, conditional, and mutual logical entropy are defined and then the corresponding notions of Shannon entropy are derived via the dit-bit transform. Moreover, a number of other notions of divergence, cross entropy, and Hamming distance are developed for logical entropy along with the corresponding notions for Shannon entropy. And finally, a number of intriguing parallels between the two entropies and related inequalities are developed which allow some inequalities directly relating the two entropies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 16.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    This is investigated in Rossi [6].

References

  1. Cover, Thomas, and Joy Thomas. 1991. Elements of Information Theory. New York: John Wiley.

    Book  Google Scholar 

  2. Csiszar, Imre, and Janos Körner. 1981. Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic Press.

    Google Scholar 

  3. Kullback, Solomon, and Richard A. Leibler. 1951. On information and sufficiency. Annals of Mathematical Statistics 22: 79–86. https://doi.org/10.1214/aoms/1177729694.

    Article  Google Scholar 

  4. McEliece, Robert J. 1977. The Theory of Information and Coding: A Mathematical Framework for Communication (Encyclopedia of Mathematics and its Applications, Vol. 3). Reading MA: Addison-Wesley.

    Google Scholar 

  5. Rao, C. R. 1982. Diversity and Dissimilarity Coefficients: A Unified Approach. Theoretical Population Biology. 21: 24–43.

    Article  Google Scholar 

  6. Rossi, Giovanni. 2011. Partition Distances. arXiv:1106.4579v1.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Social Sciences, University of Ljubljana, Ljubljana, Slovenia

    David Ellerman

Authors
  1. David Ellerman
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Ellerman, D. (2021). The Compound Notions for Logical and Shannon Entropies. In: New Foundations for Information Theory. SpringerBriefs in Philosophy. Springer, Cham. https://doi.org/10.1007/978-3-030-86552-8_3

Download citation

Publish with us

Policies and ethics

Search

Navigation