Convergence and correctness of belief propagation for the Chinese postman problem

Abstract

Belief Propagation (BP), a distributed, message-passing algorithm, has been widely used in different disciplines including information theory, artificial intelligence, statistics and optimization problems in graphical models such as Bayesian networks and Markov random fields. Despite BP algorithm has a great success in many application fields and many progress about BP algorithm has been made, the rigorous analysis about the correctness and convergence of BP algorithm are known in only a few cases for arbitrary graph. In this paper, we will investigate the correctness and convergence of BP algorithm for determining the optimal solutions of the Chinese postman problem in both undirected and directed graphs. As a main result, we prove that BP algorithm converges to the optimal solution of the undirected Chinese postman problem within O(n) iterations where n represents the number of vertices, provided that the optimal solution is unique. For the directed case, we consider the directed Chinese postman problem with capacity and show that BP algorithm also converges to its optimal solution after O(n) iterations, as long as its corresponding linear programming relaxation has the unique optimal solution.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

References

  1. 1.

    Achlioptas, D., Ricci-Tersenghi, F.: On the solution-space geometry of random constraint satisfaction problems. In: Proceedings of the 38th Annual ACM Symposium on Theory of Computing, pp. 130–139, ACM (2006)

  2. 2.

    Aji, S.M., Horn, G.B., Mceliece, R.J.: On the convergence of iterative decoding on graphs with a single cycle. In: Proceedings of IEEE International Symposium Information Theory, p. 276. Cambridge (1998)

  3. 3.

    Aji, S.M., Mceliece, R.J.: The generalized distributive law. IEEE Trans. Inf. Theory 46, 325–343 (2000)

    MathSciNet  Article  Google Scholar 

  4. 4.

    Bayati, M., Braunstein, A., Zecchina, R.: A rigorous analysis of the cavity equations for the minimum spanning tree. J. Math. Phys. 49, 857–883 (2008)

    MathSciNet  Article  Google Scholar 

  5. 5.

    Bayati, M., Borgs, C., Chayes, J., Zecchina, R.: Belief propagation for weighted b-matchings on arbitrary graphs and its relation to linear programs with integer solutions. SIAM J. Discrete Math. 25, 989–1011 (2011)

    MathSciNet  Article  Google Scholar 

  6. 6.

    Bayati, M., Shah, D., Sharma, M.: A simpler max-product maximum weight matching algorithm and the auction algorithm. In: IEEE International Symposium Information Theory, pp. 557–561 (2006)

  7. 7.

    Bayati, M., Shah, D., Sharma, M.: Max-product for maximum weight matching: convergence, correctness, and LP duality. IEEE Trans. Inf. Theory 54, 1241–1251 (2008)

    MathSciNet  Article  Google Scholar 

  8. 8.

    Brunsch, T., Cornelissen, K., Manthey, B., Röglin, H.: Smoothed Analysis of Belief Propagation for Minimum-Cost Flow and Matching, WALCOM: algorithms and Computation, pp. 182–193. Springer, Berlin (2012)

    Google Scholar 

  9. 9.

    Chen, D.J., Lee, C.Y., Park, C.H., Mendes, P.: Parallelizing simulated annealing algorithms based on high-performance computer. J Glob. Optim. 39, 261–289 (2007)

    Article  Google Scholar 

  10. 10.

    Cheng, Y., Neely, M., Chugg, K.M.: Iterative message passing algorithm for bipartite maximum weighted matching. In: IEEE International Symposium Information Theory, Cambridge, pp. 1934–1938, (2006)

  11. 11.

    Coja-Oghlan, A., Mossel, E., Vilenchik, D.: A spectral approach to analysing belief propagation for 3-colouring. Comb. Probab. Comput. 18, 881–912 (2009)

    MathSciNet  Article  Google Scholar 

  12. 12.

    Edmonds, J.: The Chinese postman problem. Oper. Res. 13, 1–73 (1965)

    Article  Google Scholar 

  13. 13.

    Edmonds, J., Johnson, E.: Matching, Euler tour and Chinese postman. Math. Program. 5, 88–124 (1973)

    MathSciNet  Article  Google Scholar 

  14. 14.

    Even, G., Halabi, N.: Analysis of the min-sum algorithm for packing and covering problems via linear programming. IEEE Trans. Inf. Theory 61, 5295–5305 (2015)

    MathSciNet  Article  Google Scholar 

  15. 15.

    Feige, U., Mossel, E., Vilenchik, D.: Complete convergence of message passing algorithms for some satisfiability problems. Random 4110, 339–350 (2010)

    MathSciNet  MATH  Google Scholar 

  16. 16.

    Frey, B.J., Dueck, D.: Clustering by passing messages between data points. Science 315, 972–976 (2007)

    MathSciNet  Article  Google Scholar 

  17. 17.

    Frey, B.J., Koetter, R., Forney Jr., G.D., Kschischang, F.R., Spielman, D.A.: Introduction to the special issue on codes on graphs and iterative algorithms. IEEE Trans. Inf. Theory 47, 493–497 (2001)

    Article  Google Scholar 

  18. 18.

    Gallager, R.G.: Low density parity check codes. IEEE Trans. Inf. Theory 8, 21–28 (1962)

    MathSciNet  Article  Google Scholar 

  19. 19.

    Gamarnik, D., Shah, D., Wei, Y.: Belief propagation for min-cost network flow: convergence and correctness. Oper. Res. 60, 410–428 (2012)

    MathSciNet  Article  Google Scholar 

  20. 20.

    Guan, M.G.: Graphic programming using odd or even points. Chin. Math. 1, 273–277 (1962)

    MathSciNet  MATH  Google Scholar 

  21. 21.

    Kschischang, F.R., Frey, B.J., Loeliger, H.-A.: Factor graphs and sum-product algorithm. IEEE Trans. Inf. Theory 47, 498–519 (2001)

    MathSciNet  Article  Google Scholar 

  22. 22.

    Mézard, M.: Passing messages between disciplines. Science 301, 1685–1686 (2003)

    Article  Google Scholar 

  23. 23.

    Mézard, M., Parisi, G., Zecchina, R.: Analytic and algorithmic solution of random satisfiability problems. Science 297, 812–815 (2002)

    Article  Google Scholar 

  24. 24.

    Mézard, M., Zecchina, R.: Random k-satisfiability problem: from an analytic solution to an efficient algorithm. Phys. Rev. 66, 249–264 (2002)

    Google Scholar 

  25. 25.

    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Reasoning. Morgan Kaufman, San Mateo (1988)

    Google Scholar 

  26. 26.

    Richardson, T., Urbanke, R.: The capacity of low-density parity check codes under message-passing decoding. IEEE Trans. Inf. Theory 47, 599–618 (2001)

    MathSciNet  Article  Google Scholar 

  27. 27.

    Sanghavi, S., Malioutov, D., Willsky, A.: Linear programming analysis of loopy belief propagation for weighted matching. In: Advances in Neural Information Processing Systems, pp. 1273–1280. Cambridge, (2007)

  28. 28.

    Sanghavi, S., Malioutov, D., Willsky, A.: Belief propagation and LP relaxation for weighted matching in general graphs. IEEE Trans. Inf. Theory 54, 2203–2212 (2011)

    MathSciNet  Article  Google Scholar 

  29. 29.

    Sanghavi, S., Shah, D., Willsky, A.S.: Message passing for maximum weight independent set. IEEE Trans. Inf. Theory 55, 4822–4834 (2009)

    MathSciNet  Article  Google Scholar 

  30. 30.

    Schrijver, A.: Combinatorial Optimization - Polyhedra and Efficiency. Springer, Berlin (2003)

    Google Scholar 

  31. 31.

    Vontobel, P.O., Koetter, R.: Graph-cover decoding and finite-length analysis of message passing iterative decoding of LDPC codes, arXiv preprint cs/0512078 (2005)

  32. 32.

    Wainwright, M., Jaakkola, T., Willsky, A.: Tree consistency and bounds on the performance of the max-product algorithm and its generalizations. Stat. Comput. 14, 143–166 (2004)

    MathSciNet  Article  Google Scholar 

  33. 33.

    Weiss, Y.: Correctness of local probability propagation in graphical models with loops. Neural Comput. 12, 1–42 (2000)

    Article  Google Scholar 

  34. 34.

    Weiss, Y., Freeman, W.: On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs. IEEE Trans. Inf. Theory 47, 736–744 (2001)

    MathSciNet  Article  Google Scholar 

  35. 35.

    Weiss, Y., Freeman, W.: Correctness of belief propagation in Gaussian graphical models of arbitrary topology. Neural Comput. 13, 2173–2200 (2001)

    Article  Google Scholar 

  36. 36.

    Yedidia, J., Freeman, W., Weiss, Y.: Understanding belief propagation and its generalizations. Explor. Artif. Intell. New Millenn. 8, 236–239 (2003)

    Google Scholar 

Download references

Acknowledgements

We are truly grateful to the reviewers’ comments and suggestions on improving the manuscript, which helped us greatly to improve the quality of our paper. The research was partially supported by the National Natural Science Foundation of China under Grant Nos. 11871280, 11471003, 11401389,11531014, 11871081, China Scholarship Council under Grant Nos. 201607910003, 201608330111, Zhejiang Provincial Natural Science Foundation of China under Grant Nos. LY17A010017, 11401389 and Qing Lan Project.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Xiaoyan Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dai, G., Li, F., Sun, Y. et al. Convergence and correctness of belief propagation for the Chinese postman problem. J Glob Optim 75, 813–831 (2019). https://doi.org/10.1007/s10898-019-00749-2

Download citation

Keywords

  • Belief propagation (BP)
  • Message-passing algorithm
  • Min-sum algorithm
  • Convergence
  • Undirected Chinese postman problem
  • Directed Chinese postman problem