In the late 1940’s two wartime research efforts were troubled by the problem of faithfully reproducing a signal in the presence of interfering noise. At MIT, Norbert Wiener, better known as the inventor of cybernetics, was seeking reliable prediction for automatic fire control. Claude Shannon, working at the Bell Telephone Laboratories, wanted to make optimal use of communication channels for transmission of coded messages. These two men laid the foundations of what is now known as statistical or mathematical communication theory (or, more popularly, information theory). In this chapter we explore aspects of Shannon’s contributions, as set forth in his 1948 paper entitled “The Mathematical Theory of Communication.”
KeywordsChannel Capacity Communication Theory Noisy Channel Input Symbol Output Symbol
Unable to display preview. Download preview PDF.
- Arbib, Michael A. Brains, Machines, and Mathematics. McGraw-Hill, 1964. Section 3.3 of this comprehensive and stimulating little book contains a development similar to ours in many respects but mainly building toward a proof of Shannon’s second fundamental theorem.Google Scholar
- Fano, Robert M. Transmission of Information. Wiley, 1961. A systematic textbook treatment of the work of Shannon and others. Chapters 1, 2, 4, and 5 contain an extended mathematical treatment of the material covered in this chapter.Google Scholar
- Pierce, John R. Symbols, Signals, and Noise. Harper, 1961. A delightful popular treatment of communication theory and its applications to areas like psychology and art.Google Scholar
- Reza, Fazlollah M. An Introduction to Information Theory. McGraw-Hill, 1961. Despite its title, a comprehensive text for engineers which includes considerable material on stochastic processes. Chapters 1 and 3 relate to our discussion. Chapter 2 is a useful condensed introduction to discrete probability theory.Google Scholar
- Shannon, Claude E., and Weaver, Warren. The Mathematical Theory of Communication. University of Illinois Press, 1959. Contains Shannon’s original monograph (which first appeared in two parts in the July and October issues of the 1948 Bell System Technical Journal) and Weaver’s excellent nontechnical essay.Google Scholar