Comparison of Energy Minimization Algorithms for Highly Connected Graphs

  • Vladimir Kolmogorov
  • Carsten Rother
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3952)


Algorithms for discrete energy minimization play a fundamental role for low-level vision. Known techniques include graph cuts, belief propagation (BP) and recently introduced tree-reweighted message passing (TRW). So far, the standard benchmark for their comparison has been a 4-connected grid-graph arising in pixel-labelling stereo. This minimization problem, however, has been largely solved: recent work shows that for many scenes TRW finds the global optimum. Furthermore, it is known that a 4-connected grid-graph is a poor stereo model since it does not take occlusions into account.

We propose the problem of stereo with occlusions as a new test bed for minimization algorithms. This is a more challenging graph since it has much larger connectivity, and it also serves as a better stereo model. An attractive feature of this problem is that increased connectivity does not result in increased complexity of message passing algorithms. Indeed, one contribution of this paper is to show that sophisticated implementations of BP and TRW have the same time and memory complexity as that of 4-connected grid-graph stereo.

The main conclusion of our experimental study is that for our problem graph cut outperforms both TRW and BP considerably. TRW achieves consistently a lower energy than BP. However, as connectivity increases the speed of convergence of TRW becomes slower. Unlike 4-connected grids, the difference between the energy of the best optimization method and the lower bound of TRW appears significant. This shows the hardness of the problem and motivates future research.


Ground Truth Belief Propagation Message Passing Stereo Match Sequential Schedule 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Machine Intell. 6, 721–741 (1984)CrossRefzbMATHGoogle Scholar
  2. 2.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(11) (2001)Google Scholar
  3. 3.
    Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: IEEE International Conference on Computer Vision (2001)Google Scholar
  4. 4.
    Kolmogorov, V., Zabih, R.: Multi-camera scene reconstruction via graph cuts. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 82–96. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  5. 5.
    Sun, J., Zheng, N., Shum, H.: Stereo matching using belief propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(7), 787–800 (2003)CrossRefzbMATHGoogle Scholar
  6. 6.
    Lin, M., Tomasi, C.: Surfaces with occlusions from layered stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(8), 710–717 (2004)CrossRefGoogle Scholar
  7. 7.
    Sun, J., Li, Y., Kang, S.B., Shum, H.: Symmetric stereo matching for occlusion handling. In: IEEE Conf. on Comp. Vis. and Pat. Recog. (2005)Google Scholar
  8. 8.
    Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. In: Proc. Int. Conf. Comp. Vision (2001)Google Scholar
  9. 9.
    Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: Image and video synthesis using graph cuts. In: ACM Transactions on Graphics, SIGGRAPH (2003)Google Scholar
  10. 10.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Computer Vision 47, 7–42 (2002)CrossRefzbMATHGoogle Scholar
  11. 11.
    Tappen, M.F., Freeman, W.T.: Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters. In: Proc. Int. Conf. Comp. Vision (2003)Google Scholar
  12. 12.
    Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. In: Artificial Intelligence and Statistics (2005)Google Scholar
  13. 13.
    Meltzer, T., Yanover, C., Weiss, Y.: Globally optimal solutions for energy minimization in stereo vision using reweighted belief propagation. In: Proc. Int. Conf. Comp. Vision (2005)Google Scholar
  14. 14.
    Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 16–29. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  15. 15.
    Rother, C., Kumar, S., Kolmogorov, V., Blake, A.: Digital tapestry. In: IEEE Conf. on Comp. Vis. and Pat. Recog (2005)Google Scholar
  16. 16.
    Felzenszwalb, P., Huttenlocher, D.: Efficient belief propagation for early vision. In: IEEE Conf. on Comp. Vis. and Pat. Recog (2004)Google Scholar
  17. 17.
    Greig, D., Porteous, B., Seheult, A.: Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society, Series B 51, 271–279 (1989)Google Scholar
  18. 18.
    Ishikawa, H.: Exact optimization for Markov Random Fields with convex priors. IEEE Trans. Pattern Anal. Machine Intell. 25(10), 1333–1336 (2003)CrossRefGoogle Scholar
  19. 19.
    Veksler, O.: Efficient graph-based energy minimization methods in computer vision. PhD thesis, Cornell University, Dept. of Computer Science, Ithaca, NY (1999)Google Scholar
  20. 20.
    Freeman, W.T., Pasztor, E.C., Carmichael, O.T.: Learning low-level vision. Int. J. Computer Vision 40, 25–47 (2000)CrossRefzbMATHGoogle Scholar
  21. 21.
    Kumar, S., Herbert, M.: Discriminative fields for modeling spatial dependencies in natural images. In: Advances in Neural Information Processing Systems (2004)Google Scholar
  22. 22.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)zbMATHGoogle Scholar
  23. 23.
    Barbu, A., Yuille, A.L.: Motion estimation by Swendsen-Wang cuts. In: CVPR (2004)Google Scholar
  24. 24.
    Wainwright, M., Jaakkola, T., Willsky, A.: MAP estimation via agreement on (hyper)trees: Message-passing and linear-programming approaches. IEEE Transactions on Information Theory 51(11), 3697–3717 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Scharstein, D., Szelsiki, R.: High-accuracy stereo depth maps using structured light. In: IEEE Conf. on Comp. Vis. and Pat. Recog (2003)Google Scholar
  26. 26.
    Kolmogorov, V., Rother, C.: Comparison of energy minimization algorithms for highly connected graphs. Technical Report MSR-TR-2006-19 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Vladimir Kolmogorov
    • 1
  • Carsten Rother
    • 2
  1. 1.University College LondonUK
  2. 2.Microsoft Research Ltd.CambridgeUK

Personalised recommendations