Shannon Entropy vs. Kolmogorov Complexity
Most assertions involving Shannon entropy have their Kolmogorov complexity counterparts. A general theorem of Romashchenko  states that every information inequality that is valid in Shannon’s theory is also valid in Kolmogorov’s theory, and vice verse. In this paper we prove that this is no longer true for ∀ ∃-assertions, exhibiting the first example where the formal analogy between Shannon entropy and Kolmogorov complexity fails.
KeywordsShannon Entropy Binary String Kolmogorov Complexity Vice Verse Kolmogorov Theory
Unable to display preview. Download preview PDF.
- 3.Chernov, A., Muchnik, A., Romashchenko, A., Shen, A., Vereshchagin, N.: Upper semi-lattice of binary strings with the relation ‘x is simple conditional to y. Theoretical Computer Science 271, 69–95 (2002); Preliminary version in: 14th Annual IEEE Conference on Computational Complexity, Atlanta, May 4-6, pp. 114–122 (1999)Google Scholar
- 9.Shannon, C.E.: A mathematical theory of communication. Bell Sys. Tech. J. 27, 379–423 and 623–656 (1948)Google Scholar
- 10.Solomonoff, R.J.: A formal theory of inductive inference, Part 1 and Part 2. Information and Control 7, 1–22 and 224–254 (1964)Google Scholar