Turbo Coding pp 165-198 | Cite as

Belief Propagation and Parallel Decoding

  • Chris Heegard
  • Stephen B. Wicker
Part of the The Springer International Series in Engineering and Computer Science book series (SECS, volume 476)


Probabilistic reasoning can be modeled through the use of graphs — the vertices in the graphs represent random variables, while the edges represent dependencies between the random variables. Such representations play a fundamental role in the development of expert systems, in part because they allow for a rapid factorization and evaluation of the joint probability distributions of the graph variables [CGH97].


Bayesian Network Belief Propagation Turbo Decode Belief Propagation Algorithm Parallel Decode 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [AM97]
    S.M. Aji and RJ. McEliece. A general algorithm for distributing information on a graph. In Proc. 1997 IEEE Int. Symp. on Inform. Theory, Ulm, Germany, July 1997.Google Scholar
  2. [BGT93]
    C. Berrou, A. Glavieux, and P. Thitimajshima. Near Shannon limit error-correcting coding and decoding: Turbo Codes. Proceedings of the 1993 International Conference on Communications, pages 1064–1070, 1993.Google Scholar
  3. [CGH97]
    E. Castillo, J.M. Gutierrez and A.S. Hadi. Expert Systems and Probabilistic Network Models. Monographs in Computer Science, Springer-Verlag, 1997.CrossRefGoogle Scholar
  4. [Coo90]
    G.F. Cooper. The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks. Artificial Intelligence 42:393–405, 1990.MathSciNetMATHCrossRefGoogle Scholar
  5. [JLO90]
    F.V. Jensen, S.L. Lauritzen and K.G. Olesen, “Bayesian updating in recursive graphical models by local computation,” Computational Statistical Quarterly 4:269–282, 1990.MathSciNetGoogle Scholar
  6. [Kim98]
    S. Kim. Probabilistic Reasoning, Parameter Estimation, and Issues in Turbo Decoding. Ph.D. dissertation, Cornell University, 1998.Google Scholar
  7. [KW98a]
    S. Kim and S.B. Wicker. A Connection Between the Baum-Welch Algorithm and Turbo Decoding. Proceedings of the 1998 Information Theory Workshop, Killarney, Ireland, June 22–26, pp. 12–13, 1998.Google Scholar
  8. [KW98c]
    S. Kim and S.B. Wicker. Belief Propagation in Graphs and Parallel Turbo Decoding. Submitted to the IEEE Transactions on Information Theory, 1998.Google Scholar
  9. [KF98]
    F.R. Kschischang and B.J. Frey. Iterative Decoding of Compound Codes by Probability Propagation in Graphical Models. IEEE Journal on Selected Areas in Communications, SAC-16(2):219–230, 1998.CrossRefGoogle Scholar
  10. [LS88]
    S.L. Lauritzen and DJ. Spiegelhalter. Local Computation with Probabilities on Graphical Structures and Their Application to Expert Systems. Journal of the Royal Statistical Society, Series B, 50:157–224, 1988.MathSciNetMATHGoogle Scholar
  11. [MN95]
    D.J.C. MacKay and R.M. Neal. Good error-correcting codes based on very sparse matrices. In Cryptography and Coding, 5th IMA Conference no. 1025 in Lecture Notes in Computer Science, pp. 100–111, Springer-Verlag, 1995.Google Scholar
  12. [MacL98]
    D.J.C. MacKay. Good Error-Correcting Codes based on Very Sparse Matrices. Submitted to IEEE Trans. Inform. Theory. Google Scholar
  13. [MRC95]
    R.J. McEliece, E. Rodemich and J.-F. Cheng. The Turbo Decision Algorithm. In Proc. 33rd Allerton Conference on Communications, Control and Computing, 1995.Google Scholar
  14. [MMC98]
    R.J. McEliece, D.J.C. MacKay and J.-F. Cheng. Turbo Decoding as an Instance of Pearl’s;Belief Propagation1 Algorithm. IEEE Journal on Selected Areas in Communications, SAC-16(2):140–152, 1998.CrossRefGoogle Scholar
  15. [Pea88]
    J. Pearl, Probablistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers, Inc., San Mateo, CA, 1988.Google Scholar
  16. [Shi94]
    S.E. Shimony. Finding MAPs for belief networks is NP-hard. Artificial Intelligence 68:399–410, 1994.MathSciNetMATHCrossRefGoogle Scholar
  17. [SS96]
    M. Sipser and D.A. Spielman. Expander Codes. IEEE Trans. Inform. Theory, IT-42:1710–1722, 1996.MathSciNetMATHCrossRefGoogle Scholar
  18. [SHJ97]
    P. Smyth, D. Heckerman and M.I. Jordan. Probabilistic Independence Networks for Hidden Markov Probability Models. Neural Computation 9:227–269, 1997.MATHCrossRefGoogle Scholar
  19. [Tan81]
    R.M. Tanner, “A recursive approach to low complexity codes,” IEEE Trans. Inform. Theory, vol.IT-27, pp. 533–547, Sept. 1981.MathSciNetMATHCrossRefGoogle Scholar
  20. [Wei97]
    Y. Weiss. Belief Propagation and Revision in Networks with Loops. M.I.T. A.I. Memo No. 1616 and C.B.C.L. Paper No. 155, Nov. 1997.Google Scholar
  21. [WLK95]
    N. Wiberg, H.-A. Loeliger and R. Kotter. Codes and iterative decoding on general graphs. European Trans. on Telecommunication,6:513–525, 1995CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1999

Authors and Affiliations

  • Chris Heegard
    • 1
    • 2
  • Stephen B. Wicker
    • 2
  1. 1.Alantro Communications, Inc.USA
  2. 2.Cornell UniversityUSA

Personalised recommendations