Advertisement

Local Conditioning: Exact Message Passing for Cyclic Undirected Distributed Networks

  • Matthew G. ReyesEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 881)

Abstract

This paper addresses practical implementation of summing out, expanding, and reordering of messages in Local Conditioning (LC) for undirected networks. In particular, incoming messages conditioned on potentially different subsets of the receiving node’s relevant set must be expanded to be conditioned on this relevant set, then reordered so that corresponding columns of the conditioned matrices can be fused through element-wise multiplication. An outgoing message is then reduced by summing out loop cutset nodes that are upstream of the outgoing edge. The emphasis on implementation is the primary contribution over the theoretical justification of LC given in Fay et al. Nevertheless, the complexity of Local Conditioning in grid networks is still no better than that of Clustering.

Keywords

Local Conditioning Belief Propagation Distributed Systems Message passing Cyclic networks Recursive algorithms 

Notes

Acknowledgments

The author would like to thank David Neuhoff for comments on an earlier draft.

References

  1. 1.
    Aji, S.M., McCliese, R.J.: The generalized distributive law. IEEE Trans. Info. Theor. 46(2), 325–343 (2000)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Becker, A., Geiger, D.: Optimization of Pearl’s method of conditioning and greedy-like approximation algorithms for the vertex feedback set problem. Artif. Intell. 83, 167–188 (1996)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, Hoboken (1991)CrossRefGoogle Scholar
  4. 4.
    Diez, F.J.: Local conditioning in Bayesian networks. Arti. Intell. 87, 1–20 (1996)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Fay, A., Jaffray, J.-Y.: A justification of local conditioning in Bayesian networks. Int. J. Approximate Reasoning 24, 59–81 (2000)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Frey, B.J., MacKay, D.J.C.: A revolution: belief propagation in graphs with cycles. In: Advances in Neural Information Processing Systems 10. MIT Press, Denver (1997)Google Scholar
  7. 7.
    Grimmett, G.R.: A theorem on random fields. Bull. Lond. Math. Soc. 5, 81–84 (1973)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Lauritzen, S.L., Spiegelhalter, D.J.: Local computations with probabilities on graphical structure and their application to expert systems. J. Roy. Stat. Soc. B 50(2), 157–224 (1988)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Murphy, K., Weiss, Y., Jordan, M.: Loopy belief propagation for approximate inference: an empirical study. In: UAI (1999)Google Scholar
  10. 10.
    Pearl, J.: A constraint propagation approach to probabilistic reasoning. In: Uncertainty in Artificial Intelligence, pp. 357–369. Elsevier, New York (1986)CrossRefGoogle Scholar
  11. 11.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufman, Burlington (2014)zbMATHGoogle Scholar
  12. 12.
    Reyes, M.G., Neuhoff, D.L.: Local conditioning for undirected graphs. In: Information Theory and Applications workshop, San Diego (2017)Google Scholar
  13. 13.
    Suermondt, H.J., Cooper, G.F.: Probabilistic inference in multiply connected belief networks using loop cutsets. Int. J. Approximate Reasoning 4, 283–306 (1990)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Taleb, N.N.: The Black Swan: The Impact of the Highly Improbably. Random House, New York (2007)Google Scholar
  15. 15.
    Wainwright, M.J., Jaakkola, T.S., Willsky, A.S.: Tree-based reparametrization framework for analysis of sum-product and related algorithms. IEEE Trans. Inf. Theor. 49(5), 1120–1146 (2003)CrossRefGoogle Scholar
  16. 16.
    Yedidia, J.S., Freeman, W.T., Weiss, Y.: Constructing free-energy approximations and generlized belief propagation algorithms. IEEE Trans. Inf. Theor. 51(7), 2282–2312 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Independent Researcher and ConsultantAnn ArborUSA

Personalised recommendations