On Shannon’s Duality of a Source and a Channel and Nonanticipative Communication and Communication for Control

  • Charalambos D. Charalambous
  • Christos K. Kourtellaris
  • Photios A. Stavrou
Part of the Lecture Notes in Control and Information Sciences book series (LNCIS, volume 456)


This chapter provides a brief introduction to the Fundamental Problem of Communication, as formulated by Shannon, and evolved over the years into various generalities, including the authors’ views on Duality of a Source to a Channel. Suggestions for further research are described, with emphasis on the importance of this duality in nonanticipative or real-time information transmission in both communication and communication for control, of delay-sensitive applications.


Channel Capacity Channel Code Achievable Rate Channel Output Noisy Channel 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors are grateful to Jan H. van Schuppen for stimulating discussions and technical suggestions, during the preparation of this manuscript and throughout the years.


  1. 1.
    Shannon CE (1948) A mathematical theory of communication. Bell Sys Tech J 27:379–423, 1948.Google Scholar
  2. 2.
    Berger T (1971) Rate Distortion Theory: a mathematical basis for data compression. Prentice-Hall, Englewood CliffsGoogle Scholar
  3. 3.
    Shannon CE (1959) Coding theorems for a discrete source with a fidelity criterion. IRE Nat Conv Rec 4:142–163Google Scholar
  4. 4.
    Richardson T, Urbanke R (2008) Modern coding theory. Cambridge University Press, CambridgeGoogle Scholar
  5. 5.
    Marko H (1973) The bidirectional communication theory-A generalization of information theory. IEEE Trans Commun 21(12):1345–1351Google Scholar
  6. 6.
    Csiszár I, Körner J (1981) Information theory: coding theorems for discrete memoryless systems. Academic Press, WalthamGoogle Scholar
  7. 7.
    Gallager RG (1968) Information theory and reliable communication. Wiley, New YorkGoogle Scholar
  8. 8.
    Cover TM, Thomas JA (1991) Elements of information theory. Wiley-Interscience, New YorkGoogle Scholar
  9. 9.
    Gray RM (1990) Entropy and information theory. Springer, BerlinGoogle Scholar
  10. 10.
    Han TS (2003) Information-spectrum methods in information theory. Springer, BerlinGoogle Scholar
  11. 11.
    Gamal AE, Kim YH (2011) Network information theory. Cambridge University Press, CambridgeGoogle Scholar
  12. 12.
    Ihara S (1993) Information theory for continuous systems. World-Scientific, SingaporeGoogle Scholar
  13. 13.
    Massey JL (1990) Causality, feedback and directed information. In: International symposium on information theory and its applications (ISITA). Nov 27–30:303–305Google Scholar
  14. 14.
    Kramer G (1998) Directed information for channels with feedback. Ph.D thesis, Swiss Federal Institute of Technology, Zurich, SwitzerlandGoogle Scholar
  15. 15.
    Tatikonda S (2000) Control under communication constraints. Ph.D thesis, Massachusetts Institute of Technology (MIT), MA, USAGoogle Scholar
  16. 16.
    Chen J, Berger T (2005) The capacity of finite-state channels with feedback. IEEE Trans Inf Theory 51(3):780–798Google Scholar
  17. 17.
    Tatikonda S, Mitter S (2009) The capacity of channels with feedback. IEEE Trans Inf Theory 55(1):323–349Google Scholar
  18. 18.
    Permuter HH, Weissman T, Goldsmith A (2009) Finite state channels with time-invariant deterministic feedback. IEEE Trans Inf Theory 55(2):644–662Google Scholar
  19. 19.
    Kramer G (2008) Topics in multi-user information theory. Found Trends Inf Theory 4(4–5):265–444Google Scholar
  20. 20.
    Cover TM, Pombra S (1989) Gaussian feedback capacity. IEEE Trans Inf Theory 35(1):37–43Google Scholar
  21. 21.
    Neuhoff DL, Gilbert R (1982) Causal source codes. IEEE Trans Inf Theory 28(5):701–713Google Scholar
  22. 22.
    Berger T (2003) Shannon lecture: living information theory. IEEE Inf Theory Soc Newslett 53(1)Google Scholar
  23. 23.
    Gastpar M, Rimoldi B, Vetterli M (2003) To code or not to code: lossy source-channel communication revisited. IEEE Trans Inf Theory 49(5):1147–1158Google Scholar
  24. 24.
    Wong W, Brockett R (1997) Systems with finite communication bandwidth constraints I: state estimation problems. IEEE Trans Autom Control 42(9):1294–1299Google Scholar
  25. 25.
    Nair GN, Evans RJ, Mareels IMY, Moran W (2004) Topological feedback entropy and nonlinear stabilization. IEEE Trans Autom Control 49(9):1585–1597Google Scholar
  26. 26.
    Gorbunov AK, Pinsker MS (1973) Nonanticipatory and prognostic epsilon entropies and message generation rates. Probl Inf Transm 9(3):184–191MathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Charalambos D. Charalambous
    • 1
  • Christos K. Kourtellaris
    • 1
  • Photios A. Stavrou
    • 1
  1. 1.Department of Electrical and Computer Engineering (ECE)University of CyprusNicosiaCyprus

Personalised recommendations