Channel Capacity and Channel Coding

Abstract

Chapter 5 continues the discussion of Shannon’s information theory as regards channel capacity and channel coding. Simple channel models are introduced and their capacity is computed. It is shown that channel coding needs redundancy and the fundamental theorem of channel coding is stated. Its proof relies on Shannon’s random coding, the principle of which is stated and illustrated. A geometrical picture of a code as a sparse set of points within the high-dimensional Hamming space which represents sequences is proposed. The practical implementation of channel coding uses error-correcting codes, which are briefly defined and illustrated by describing some code families: recursive convolutional codes , turbocodes and low-density parity-check codes . The last two families can be interpreted as approximately implementing random coding by deterministic means . Contrary to true random coding, their decoding is of moderate complexity and both achieve performance close to the theoretical limit. How their decoding is implemented is briefly described. The first and more important step of decoding enables regenerating an encoded sequence. Finally, it is stated that the constraints which endow error-correcting codes with resilience to errors can be of any kind (e.g., physical-chemical or linguistic), and not necessarily mathematical as in communication engineering.