Comparison of Energy Minimization Algorithms for Highly Connected Graphs
Algorithms for discrete energy minimization play a fundamental role for low-level vision. Known techniques include graph cuts, belief propagation (BP) and recently introduced tree-reweighted message passing (TRW). So far, the standard benchmark for their comparison has been a 4-connected grid-graph arising in pixel-labelling stereo. This minimization problem, however, has been largely solved: recent work shows that for many scenes TRW finds the global optimum. Furthermore, it is known that a 4-connected grid-graph is a poor stereo model since it does not take occlusions into account.
We propose the problem of stereo with occlusions as a new test bed for minimization algorithms. This is a more challenging graph since it has much larger connectivity, and it also serves as a better stereo model. An attractive feature of this problem is that increased connectivity does not result in increased complexity of message passing algorithms. Indeed, one contribution of this paper is to show that sophisticated implementations of BP and TRW have the same time and memory complexity as that of 4-connected grid-graph stereo.
The main conclusion of our experimental study is that for our problem graph cut outperforms both TRW and BP considerably. TRW achieves consistently a lower energy than BP. However, as connectivity increases the speed of convergence of TRW becomes slower. Unlike 4-connected grids, the difference between the energy of the best optimization method and the lower bound of TRW appears significant. This shows the hardness of the problem and motivates future research.
KeywordsGround Truth Belief Propagation Message Passing Stereo Match Sequential Schedule
- 2.Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(11) (2001)Google Scholar
- 3.Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: IEEE International Conference on Computer Vision (2001)Google Scholar
- 7.Sun, J., Li, Y., Kang, S.B., Shum, H.: Symmetric stereo matching for occlusion handling. In: IEEE Conf. on Comp. Vis. and Pat. Recog. (2005)Google Scholar
- 8.Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. In: Proc. Int. Conf. Comp. Vision (2001)Google Scholar
- 9.Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: Image and video synthesis using graph cuts. In: ACM Transactions on Graphics, SIGGRAPH (2003)Google Scholar
- 11.Tappen, M.F., Freeman, W.T.: Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters. In: Proc. Int. Conf. Comp. Vision (2003)Google Scholar
- 12.Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. In: Artificial Intelligence and Statistics (2005)Google Scholar
- 13.Meltzer, T., Yanover, C., Weiss, Y.: Globally optimal solutions for energy minimization in stereo vision using reweighted belief propagation. In: Proc. Int. Conf. Comp. Vision (2005)Google Scholar
- 14.Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 16–29. Springer, Heidelberg (2006)CrossRefGoogle Scholar
- 15.Rother, C., Kumar, S., Kolmogorov, V., Blake, A.: Digital tapestry. In: IEEE Conf. on Comp. Vis. and Pat. Recog (2005)Google Scholar
- 16.Felzenszwalb, P., Huttenlocher, D.: Efficient belief propagation for early vision. In: IEEE Conf. on Comp. Vis. and Pat. Recog (2004)Google Scholar
- 17.Greig, D., Porteous, B., Seheult, A.: Exact maximum a posteriori estimation for binary images. Journal of the Royal Statistical Society, Series B 51, 271–279 (1989)Google Scholar
- 19.Veksler, O.: Efficient graph-based energy minimization methods in computer vision. PhD thesis, Cornell University, Dept. of Computer Science, Ithaca, NY (1999)Google Scholar
- 21.Kumar, S., Herbert, M.: Discriminative fields for modeling spatial dependencies in natural images. In: Advances in Neural Information Processing Systems (2004)Google Scholar
- 23.Barbu, A., Yuille, A.L.: Motion estimation by Swendsen-Wang cuts. In: CVPR (2004)Google Scholar
- 25.Scharstein, D., Szelsiki, R.: High-accuracy stereo depth maps using structured light. In: IEEE Conf. on Comp. Vis. and Pat. Recog (2003)Google Scholar
- 26.Kolmogorov, V., Rother, C.: Comparison of energy minimization algorithms for highly connected graphs. Technical Report MSR-TR-2006-19 (2006)Google Scholar