MAP-Inference on Large Scale Higher-Order Discrete Graphical Models by Fusion Moves

  • Jörg Hendrik KappesEmail author
  • Thorsten Beier
  • Christoph Schnörr
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8926)


Many computer vision problems can be cast into optimization problems over discrete graphical models also known as Markov or conditional random fields. Standard methods are able to solve those problems quite efficiently. However, problems with huge label spaces and or higher-order structure remain challenging or intractable even for approximate methods.

We reconsider the work of Lempitsky et al. 2010 on fusion moves and apply it to general discrete graphical models. We propose two alternatives for calculating fusion moves that outperform the standard in several applications. Our generic software framework allows us to easily use different proposal generators which spans a large class of inference algorithms and thus makes exhaustive evaluation feasible.

Because these fusion algorithms can be applied to models with huge label spaces and higher-order terms, they might stimulate and support research of such models which may have not been possible so far due to the lack of adequate inference methods.


Integer Linear Programming Fusion Algorithm Optimal Move Unary Term Proposal Generator 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

336126_1_En_37_MOESM1_ESM.pdf (473 kb)
Supplementary material (PDF 473 KB)


  1. 1.
    Andres, B., Beier, T., Kappes, J.H.: OpenGM2 (2012).
  2. 2.
    Andres, B., Kappes, J.H., Beier, T., Köthe, U., Hamprecht, F.A.: The lazy flipper: Efficient depth-limited exhaustive search in discrete graphical models. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VII. LNCS, vol. 7578, pp. 154–166. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  3. 3.
    Besag, J.: On the Statistical Analysis of Dirty Pictures. Journal of the Royal Statistical Society. Series B (Methodological) 48(3), 259–302 (1986)zbMATHMathSciNetGoogle Scholar
  4. 4.
    Boros, E., Hammer, P.L.: Pseudo-boolean optimization. Discrete Appl. Math. 123(1–3), 155–225 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  6. 6.
    Delong, A., Osokin, A., Isack, H., Boykov, Y.: Fast approximate energy minimization with label costs. International Journal of Computer Vision 96, 1–27 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Elidan, G., Globerson, A.: The probabilistic inference challenge (PIC 2011).
  8. 8.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. Int. J. Comput. Vision 70(1), 41–54 (2006). CrossRefGoogle Scholar
  9. 9.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. International Journal of Computer Vision 70(1), 41–54 (2006)CrossRefGoogle Scholar
  10. 10.
    Fix, A., Gruber, A., Boros, E., Zabih, R.: A graph cut algorithm for higher-order Markov random fields. In: ICCV (2011).
  11. 11.
    Globerson, A., Jaakkola, T.: Fixing max-product: Convergent message passing algorithms for map lp-relaxations. In: NIPS (2007)Google Scholar
  12. 12.
    Goldluecke, B., Strekalovskiy, E., Cremers, D.: Tight convex relaxations for vector-valued labeling. SIAM Journal on Imaging Sciences 6(3), 1626–1664 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Gorelick, L., Boykov, Y., Veksler, O., Ayed, I.B., Delong, A.: Submodularization for binary pairwise energies. In: CVPR. IEEE (2014) (in press)Google Scholar
  14. 14.
  15. 15.
    Ishikawa, H.: Transformation of general binary mrf minimization to the first-order case. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(6), 1234–1249 (2011)CrossRefGoogle Scholar
  16. 16.
    Jaimovich, A., Elidan, G., Margalit, H., Friedman, N.: Towards an integrated protein-protein interaction network: A relational markov network approach. Journal of Computational Biology 13(2), 145–164 (2006)CrossRefMathSciNetGoogle Scholar
  17. 17.
    Kahl, F., Strandmark, P.: Generalized roof duality. Discrete Applied Mathematics 160(16–17), 2419–2434 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Kappes, J.H., Andres, B., Hamprecht, F.A., Schnörr, C., Nowozin, S., Batra, D., Kim, S., Kausler, B.X., Kröger, T., Lellmann, J., Komodakis, N., Savchynskyy, B., Rother, C.: A comparative study of modern inference techniques for structured discrete energy minimization problems. CoRR abs/1404.0533 (2014)Google Scholar
  19. 19.
    Kappes, J.H., Andres, B., Hamprecht, F.A., Schnörr, C., Nowozin, S., Batra, D., Kim, S., Kausler, B.X., Lellmann, J., Komodakis, N., Rother, C.: A comparative study of modern inference techniques for discrete energy minimization problems. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  20. 20.
    Kappes, J.H., Speth, M., Reinelt, G., Schnörr, C.: Towards efficient and exact MAP-inference for large scale discrete computer vision problems via combinatorial optimization. In: CVPR (2013)Google Scholar
  21. 21.
    Kim, S., Nowozin, S., Kohli, P., Yoo, C.D.: Higher-order correlation clustering for image segmentation. In: Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS) (2011)Google Scholar
  22. 22.
    Kohli, P., Ladicky, L., Torr, P.H.: Robust higher order potentials for enforcing label consistency. International Journal of Computer Vision 82(3), 302–324 (2009). CrossRefGoogle Scholar
  23. 23.
    Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press (2009)Google Scholar
  24. 24.
    Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(10), 1568–1583 (2006)CrossRefGoogle Scholar
  25. 25.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part III. LNCS, vol. 2352, pp. 65–81. springer, heidelberg (2002) CrossRefGoogle Scholar
  26. 26.
    Lempitsky, V., Rother, C., Roth, S., Blake, A.: Fusion moves for markov random field optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(8), 1392–1405 (2010)CrossRefGoogle Scholar
  27. 27.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV (2001)Google Scholar
  28. 28.
    Nowozin, S., Lampert, C.H.: Structured learning and prediction in computer vision. Foundations and Trends in Computer Graphics and Vision 6(3–4), 185–365 (2011)zbMATHGoogle Scholar
  29. 29.
    Nowozin, S., Rother, C., Bagon, S., Sharp, T., Yao, B., Kohli, P.: Decision tree fields. In: ICCV, pp. 1668–1675. IEEE (2011)Google Scholar
  30. 30.
    Roth, S., Black, M.J.: Fields of experts. International Journal of Computer Vision 82(2), 205–229 (2009)CrossRefGoogle Scholar
  31. 31.
    Rother, C., Kolmogorov, V., Lempitsky, V.S., Szummer, M.: Optimizing binary MRFs via extended roof duality. In: CVPR (2007)Google Scholar
  32. 32.
    Savchynskyy, B., Kappes, J.H., Swoboda, P., Schnörr, C.: Global MAP-optimality by shrinking the combinatorial search area with convex relaxation. In: NIPS (2013)Google Scholar
  33. 33.
    Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for Markov random fields with smoothness-based priors. IEEE PAMI 30(6), 1068–1080 (2008). CrossRefGoogle Scholar
  34. 34.
    Yanover, C., Schueler-Furman, O., Weiss, Y.: Minimizing and learning energy functions for side-chain prediction. Journal of Computational Biology 15(7), 899–911 (2008)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Jörg Hendrik Kappes
    • 1
    Email author
  • Thorsten Beier
    • 1
  • Christoph Schnörr
    • 1
  1. 1.Heidelberg Collaboratory for Image ProcessingHeidelberg UniversityHeidelbergGermany

Personalised recommendations