Advertisement

Journal of Automated Reasoning

, Volume 53, Issue 1, pp 63–103 | Cite as

Formalization of Shannon’s Theorems

  • Reynald Affeldt
  • Manabu Hagiwara
  • Jonas Sénizergues
Article

Abstract

The most fundamental results of information theory are Shannon’s theorems. These theorems express the bounds for (1) reliable data compression and (2) data transmission over a noisy channel. Their proofs are non-trivial but are rarely detailed, even in the introductory literature. This lack of formal foundations is all the more unfortunate that crucial results in computer security rely solely on information theory: this is the so-called “unconditional security”. In this article, we report on the formalization of a library for information theory in the SSReflect extension of the Coq proof-assistant. In particular, we produce the first formal proofs of the source coding theorem, that introduces the entropy as the bound for lossless compression, and of the channel coding theorem, that introduces the capacity as the bound for reliable communication over a noisy channel.

Keywords

Information theory Shannon’s theorems Noisy-channel coding theorem Coq SSReflect 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Affeldt, R., Hagiwara, M.: Formalization of Shannon’s Theorems in SSReflect-Coq. In: Proceedings of the 3rd International Conference on Interactive Theorem Proving (ITP 2012), Princeton, NJ, USA, August 13–15, 2012. Lecture Notes in Computer Science, vol. 7406, pp. 233–249. Springer, Heidelberg (2012)Google Scholar
  2. 2.
    Affeldt, R., Hagiwara, M., Sénizergues, J.: Formalization of Shannon’s Theorems in SSReflect-Coq. Coq documentation and scripts available at http://staff.aist.go.jp/reynald.affeldt/shannon
  3. 3.
    Audebaud, P., Paulin-Mohring, C.: Proofs of randomized algorithms in Coq. Sci. Comput. Prog. 74(8), 568–589 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Barthe, G., Crespo, J.M., Grégoire, B., Kunz, C., Zanella Béguelin, S.: Computer-Aided Cryptographic Proofs. In: Proceedings of the 3rd International Conference on Interactive Theorem Proving (ITP 2012), Princeton, NJ, USA, August 2012. Lecture Notes in Computer Science, vol. 7406, pp. 11–27. Springer, Heidelberg (2012)Google Scholar
  5. 5.
    Bertot, Y., Gonthier, G., Ould Biha, S., Pasca, I.: Canonical Big Operators. In: Proceedings of the 21st International Conference on Theorem Proving in Higher Order Logics (TPHOLs 2008), Montreal, Canada, August 18–21, 2008. Lecture Notes in Computer Science, vol. 5170, pp. 86–101. Springer, Heidelberg (2008)Google Scholar
  6. 6.
    Coble, A.R.: Anonymity, Information, and Machine-Assisted Proof. PhD Thesis, King’s College. University of Cambridge, UK (2010)Google Scholar
  7. 7.
    The Coq Development Team. Reference Manual. Version 8.4. Available at http://coq.inria.fr. INRIA (2004–2012)
  8. 8.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. Wiley-Interscience (2006)Google Scholar
  9. 9.
    Csiszár, I.: The method of types. IEEE Trans. Inform. Theory 44(6), 2505–2523 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Csiszár, I., Körner, J.: Information Theory—Coding Theorems for Discrete Memoryless Systems, 2nd edn. Cambridge University Press (2011)Google Scholar
  11. 11.
    Gonthier, G., Mahboubi, A., Tassi, E.: A Small Scale Reflection Extension for the Coq system. Version 10. Technical Report RR-6455, INRIA (2011)Google Scholar
  12. 12.
    Hagiwara, M.: Coding Theory: Mathematics for Digital Communication. In Japanese. http://www.nippyo.co.jp/book/5977.html. Nippon Hyoron Sha (2012)
  13. 13.
    Hasan, O., Tahar, S.: Verification of expectation using theorem proving to verify expectation and variance for discrete random variables. J. Autom. Reason. 41, 295–323 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Hölzl, J., Heller, A.: Three Chapters of Measure Theory in Isabelle/HOL. In: Proceedings of the 2nd International Conference on Interactive Theorem Proving (ITP 2011), Berg en Dal, The Netherlands, August 22–25, 2011. Lecture Notes in Computer Science, vol. 6898, pp. 135–151. Springer, Heidelberg (2011)Google Scholar
  15. 15.
    Hurd, J.: Formal Verification of Probabilistic Algorithms. PhD Thesis, Trinity College, University of Cambridge, UK (2001)Google Scholar
  16. 16.
    Mhamdi, T., Hasan, O., Tahar, S.: On the Formalization of the Lebesgue Integration Theory in HOL. In: Proceedings of the 1st International Conference on Interactive Theorem Proving (ITP 2010), Edinburgh, UK, July 11–14, 2010. Lecture Notes in Computer Science, vol. 6172, pp. 387–402. Springer, Heidelberg (2010)Google Scholar
  17. 17.
    Mhamdi, T., Hasan, O., Tahar, S.: Formalization of Entropy Measures in HOL. In: Proceedings of the 2nd International Conference on Interactive Theorem Proving (ITP 2011), Berg en Dal, The Netherlands, August 22–25, 2011. Lecture Notes in Computer Science, vol. 6898, pp. 233–248. Springer, Heidelberg (2011)Google Scholar
  18. 18.
    Khudanpur, S.: Information Theoretic Methods in Statistics. Lecture Notes. Available at http://old-site.clsp.jhu.edu/sanjeev/520.674/notes/GISAlgorithm.pdf (1999). Accessed 02 May 2013
  19. 19.
    Shannon, C.E.: A Mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 and 623–656 (1948)CrossRefzbMATHMathSciNetGoogle Scholar
  20. 20.
    Shannon, C.E.: Communication theory of secrecy systems. Bell Sys. Tech. J. 28, 656–715 (1949)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Uyematsu, T.: Modern Shannon Theory, Information Theory with Types. In Japanese. Baifukan (1998)Google Scholar
  22. 22.
    Verdú, S.: Fifty years of Shannon theory. IEEE Trans. Inform. Theory 44(6), 2057–2078 (1998)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Reynald Affeldt
    • 1
  • Manabu Hagiwara
    • 2
  • Jonas Sénizergues
    • 3
  1. 1.Research Institute for Secure SystemsNational Institute of Advanced Industrial Scienceand TechnologyTsukubaJapan
  2. 2.Department of Mathematics and Informatics, Faculty of ScienceChiba UniversityChibaJapan
  3. 3.École Normale Supérieure de CachanCachanFrance

Personalised recommendations