Background on Information Theory and Coding Theory

  • David JoynerEmail author
  • Jon-Lark Kim
Part of the Applied and Numerical Harmonic Analysis book series (ANHA)


This chapter summarizing background information assumes that the reader has some familiarity with linear algebra and basic probability. The basic model of information theory and error-correcting block codes is introduced. The basic example of the Hamming [7,4,3] code is presented in detail.

What is ironic is that even in basic background issues, coding theory has interesting open questions. For example, for a given length and dimension, which code is the best 2-error-correcting code? Another example: see Manin’s theorem 19 and the closely related Conjecture 22 below.


Linear Code Cyclic Code Asymptotic Bound Cyclic Shift Quadratic Residue 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. [Af]
    Aftab, Cheung, Kim, Thakkar, Yeddanapudi: Information theory. Student term project in a course at MIT Preprint (2001). Available:
  2. [Ash]
    Ash, R.: Information Theory. Dover, New York (1965)zbMATHGoogle Scholar
  3. [Bal]
    Ball, S.: On large subsets of a nite vector space in which every subset of basis size is a basis. Preprint, Dec. (2010).
  4. [Ba]
    A. Barg’s Coding Theory webpage
  5. [BGT]
    Berrou, C., Glavieux, A., Thitimajshima, P.: Near Shannon limit error-correcting coding and decoding: Turbo-codes. Communications (1993). ICC 93. Geneva. Technical Program, Conference Record, IEEE International Conference on, vol. 2, pp. 1064–1070 (1993). Available:
  6. [Bi]
    Bierbrauer, J.: Introduction to Coding Theory. Chapman & Hall/CRC, New York (2005)zbMATHGoogle Scholar
  7. [CSi]
    Conway, F., Siegelman, J.: Dark Hero of the Information Age. MIT Press, New York (2005)zbMATHGoogle Scholar
  8. [dLG]
    de Launey, W., Gordon, D.: A remark on Plotkin’s bound. IEEE Trans. Inf. Theory 47, 352–355 (2001). Available: MathSciNetCrossRefGoogle Scholar
  9. [GZ]
    Gaborit, P., Zemor, G.: Asymptotic improvement of the Gilbert–Varshamov bound for linear codes. IEEE Trans. Inf. Theory IT-54(9), 3865–3872 (2008). Available: MathSciNetCrossRefGoogle Scholar
  10. [Hil]
    Hill, R.: A First Course in Coding Theory. Oxford Univ Press, Oxford (1986)zbMATHGoogle Scholar
  11. [Hi]
    Hirschfeld, J.W.P.: The main conjecture for MDS codes. In: Cryptography and Coding. Lecture Notes in Computer Science, vol. 1025. Springer, Berlin (1995). Avaialble from: CrossRefGoogle Scholar
  12. [HP1]
    Huffman, W.C., Pless, V.: Fundamentals of Error-Correcting Codes. Cambridge Univ. Press, Cambridge (2003)CrossRefGoogle Scholar
  13. [JV]
    Jiang, T., Vardy, A.: Asymptotic improvement of the Gilbert–Varshamov bound on the size of binary codes. IEEE Trans. Inf. Theory 50, 1655–1664 (2004)MathSciNetCrossRefGoogle Scholar
  14. [JKTu]
    Joyner, D., Kreminski, R., Turisco, J.: Applied Abstract Algebra. Johns Hopkins Univ. Press, Baltimore (2004)zbMATHGoogle Scholar
  15. [MS]
    MacWilliams, F., Sloane, N.: The Theory of Error-Correcting Codes. North-Holland, Amsterdam (1977)zbMATHGoogle Scholar
  16. [MN]
    MacKay, D.J.C., Radford, M.N.: Near Shannon limit performance of low density parity check codes. Electronics Letters, July (1996). Available: CrossRefGoogle Scholar
  17. [Ni]
    Niven, I.: Coding theory applied to a problem of Ulam. Math. Mag. 61, 275–281 (1988)MathSciNetCrossRefGoogle Scholar
  18. [S]
    The SAGE Group: SAGE: Mathematical software, Version 4.6.
  19. [SS]
    Shokranian, S., Shokrollahi, M.A.: Coding Theory and Bilinear Complexity. Scientific Series of the International Bureau, vol. 21. KFA Jülich (1994)zbMATHGoogle Scholar
  20. [Tho]
    Thompson, T.: From Error-Correcting Codes Through Sphere Packings to Simple Groups. Cambridge Univ. Press, Cambridge (2004)zbMATHGoogle Scholar
  21. [vL1]
    van Lint, J.: Introduction to Coding Theory, 3rd edn. Springer, Berlin (1999).CrossRefGoogle Scholar
  22. [VVS]
    Viterbi, A.J., Viterbi, A.M., Sindhushayana, N.T.: Interleaved concatenated codes: New perspectives on approaching the Shannon limit. Proc. Natl. Acad. Sci. USA 94, 9525–9531 (1997)MathSciNetCrossRefGoogle Scholar
  23. [V3]
    Voloch, F.: Computing the minimum distance of cyclic codes. Preprint. Available:
  24. [War]
    Ward, H.N.: Quadratic residue codes and symplectic groups. J. Algebra 29, 150–171 (1974)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Mathematics DepartmentUS Naval AcademyAnnapolisUSA
  2. 2.Department of MathematicsUniversity of LouisvilleLouisvilleUSA

Personalised recommendations