Smoothed Analysis of Belief Propagation for Minimum-Cost Flow and Matching

  • Tobias Brunsch
  • Kamiel Cornelissen
  • Bodo Manthey
  • Heiko Röglin
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7748)

Abstract

Belief propagation (BP) is a message-passing heuristic for statistical inference in graphical models such as Bayesian networks and Markov random fields. BP is used to compute marginal distributions or maximum likelihood assignments and has applications in many areas, including machine learning, image processing, and computer vision. However, the theoretical understanding of the performance of BP is unsatisfactory. Recently, BP has been applied to combinatorial optimization problems. It has been proved that BP can be used to compute maximum-weight matchings and minimum-cost flows for instances with a unique optimum. The number of iterations needed for this is pseudo-polynomial and hence BP is not efficient in general.

We study belief propagation in the framework of smoothed analysis and prove that with high probability the number of iterations needed to compute maximum-weight matchings and minimum-cost flows is bounded by a polynomial if the weights/costs of the edges are randomly perturbed. To prove our upper bounds, we use an isolation lemma by Beier and Vöcking (SIAM J. Comput., 2006) for matching and generalize an isolation lemma for min-cost flow by Gamarnik, Shah, and Wei (Oper. Res., 2012). We also prove almost matching lower tail bounds for the number of iterations that BP needs to converge.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ahuja, R.K., Magnanti, T.L., Orlin, J.B.: Network flows: theory, algorithms, and applications. Prentice-Hall (1993)Google Scholar
  2. 2.
    Bayati, M., Borgs, C., Chayes, J., Zecchina, R.: Belief-propagation for weighted b-matching on arbitrary graphs and its relation to linear programs with integer solutions. SIAM Journal on Discrete Mathematics 25(2), 989–1011 (2011)MATHCrossRefGoogle Scholar
  3. 3.
    Bayati, M., Braunstein, A., Zecchina, R.: A rigorous analysis of the cavity equations for the minimum spanning tree. Journal of Mathematical Physics 49(12), 125206 (2008)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Bayati, M., Shah, D., Sharma, M.: Max-product for maximum weight matching: Convergence, correctness, and LP duality. IEEE Transactions on Information Theory 54(3), 1241–1251 (2008)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Beier, R., Vöcking, B.: Typical properties of winners and losers in discrete optimization. SIAM Journal in Computing 35(4), 855–881 (2006)MATHCrossRefGoogle Scholar
  6. 6.
    Gamarnik, D., Shah, D., Wei, Y.: Belief propagation for min-cost network flow: Convergence & correctness. Operations Research 60(2), 410–428 (2012)MathSciNetMATHCrossRefGoogle Scholar
  7. 7.
    Manthey, B., Röglin, H.: Smoothed analysis: Analysis of algorithms beyond worst case. IT – Information Technology 53(6), 280–286 (2011)CrossRefGoogle Scholar
  8. 8.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann (1988)Google Scholar
  9. 9.
    Röglin, H., Vöcking, B.: Smoothed analysis of integer programming. Mathematical Programming 110(1), 21–56 (2007)MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Sanghavi, S., Malioutov, D.M., Willsky, A.S.: Belief propagation and LP relaxation for weighted matching in general graphs. IEEE Transactions on Information Theory 57(4), 2203–2212 (2011)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Sanghavi, S., Shah, D., Willsky, A.S.: Message passing for maximum weight independent set. IEEE Transactions on Information Theory 55(11), 4822–4834 (2009)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Spielman, D.A., Teng, S.-H.: Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time. Journal of the ACM 51(3), 385–463 (2004)MathSciNetMATHCrossRefGoogle Scholar
  13. 13.
    Spielman, D.A., Teng, S.-H.: Smoothed analysis: An attempt to explain the behavior of algorithms in practice. Communications of the ACM 52(10), 76–84 (2009)CrossRefGoogle Scholar
  14. 14.
    Yedidia, J.S., Freeman, W.T., Weiss, Y.: Understanding belief propagation and its generalizations. In: Lakemeyer, G., Nebel, B. (eds.) Exploring Artificial Intelligence in the New Millennium, ch. 8, pp. 239–269. Morgan Kaufmann (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Tobias Brunsch
    • 1
  • Kamiel Cornelissen
    • 2
  • Bodo Manthey
    • 2
  • Heiko Röglin
    • 1
  1. 1.University of BonnGermany
  2. 2.University of TwenteThe Netherlands

Personalised recommendations