MRF Inference by k-Fan Decomposition and Tight Lagrangian Relaxation

  • Jörg Hendrik Kappes
  • Stefan Schmidt
  • Christoph Schnörr
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6313)


We present a novel dual decomposition approach to MAP inference with highly connected discrete graphical models. Decompositions into cyclic k-fan structured subproblems are shown to significantly tighten the Lagrangian relaxation relative to the standard local polytope relaxation, while enabling efficient integer programming for solving the subproblems. Additionally, we introduce modified update rules for maximizing the dual function that avoid oscillations and converge faster to an optimum of the relaxed problem, and never get stuck in non-optimal fixed points.


Markov Random Field Lagrangian Relaxation Linear Programming Relaxation Single View Problem Decomposition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bergtholdt, M., Kappes, J., Schmidt, S., Schnörr, C.: A study of parts-based object class detection using complete graphs. Int. J. Comp. Vision 87(1-2), 93–117 (2010)Google Scholar
  2. 2.
    Bertsekas, D.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)zbMATHGoogle Scholar
  3. 3.
    Bertsimas, D., Tsitsiklis, J.: Introduction to Linear Optimization. Athena Scientific, Belmont (1997)Google Scholar
  4. 4.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, New York (2004)zbMATHGoogle Scholar
  5. 5.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Patt. Anal. Mach. Intell. 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  6. 6.
    Cowell, R.G., Dawid, A.P., Lauritzen, S.L., Spiegelhalter, D.J.: Probabilistic Networks and Expert Systems: Exact Computational Methods for Bayesian Networks. Springer Publishing Company, Incorporated, Heidelberg (2007)zbMATHGoogle Scholar
  7. 7.
    Ermoliev, Y.: Methods for solving nonlinear extremal problems. Cybernetics 2(4), 1–17 (1966)CrossRefGoogle Scholar
  8. 8.
    Johnson, J.K., Malioutov, D., Willsky, A.S.: Lagrangian relaxation for MAP estimation in graphical models. In: 45th Annual Allerton Conference on Communication, Control and Computing (September 2007)Google Scholar
  9. 9.
    Kohli, P., Ladický, L., Torr, P.H.: Robust higher order potentials for enforcing label consistency. Int. J. Comput. Vision 82(3), 302–324 (2009)CrossRefGoogle Scholar
  10. 10.
    Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1568–1583 (2006)CrossRefGoogle Scholar
  11. 11.
    Komodakis, N., Paragios, N.: Beyond pairwise energies: Efficient optimization for higher-order MRFs. In: CVPR, pp. 2985–2992. IEEE, Los Alamitos (2009)Google Scholar
  12. 12.
    Komodakis, N., Paragios, N., Tziritas, G.: MRF optimization via dual decomposition: Message-passing revisited. In: ICCV, pp. 1–8. IEEE, Los Alamitos (2007)Google Scholar
  13. 13.
    Kumar, M.P., Kolmogorov, V., Torr, P.H.S.: An analysis of convex relaxations for MAP estimation of discrete MRFs. J. Mach. Learn. Res. 10, 71–106 (2009)MathSciNetGoogle Scholar
  14. 14.
    Polyak, B.: A general method for solving extremum problems. Soviet Math. 8, 593–597 (1966)Google Scholar
  15. 15.
    Potetz, B., Lee, T.S.: Efficient belief propagation for higher-order cliques using linear constraint nodes. Comput. Vis. Image Underst. 112(1), 39–54 (2008)CrossRefGoogle Scholar
  16. 16.
    Rother, C., Kohli, P., Feng, W., Jia, J.: Minimizing sparse higher order energy functions of discrete variables. In: CVPR, pp. 1382–1389. IEEE, Los Alamitos (2009)Google Scholar
  17. 17.
    Rother, C., Kolmogorov, V., Lempitsky, V.S., Szummer, M.: Optimizing binary MRFs via extended roof duality. In: CVPR. IEEE Computer Society, Los Alamitos (2007)Google Scholar
  18. 18.
    Ruszczynski, A.: A merit function approach to the subgradient method with averaging. Optimization Methods Software 23(1), 161–172 (2008)zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Sigal, L., Black, M.: HumanEva: synchronized video and motion capture dataset for evaluation of articulated human motion. Tech. Rep. Technical Report CS-06-08, Brown University (2006)Google Scholar
  20. 20.
    Sontag, D., Meltzer, T., Globerson, A., Jaakkola, T., Weiss, Y.: Tightening LP relaxations for MAP using message passing. In: McAllester, D.A., Myllymki, P. (eds.) UAI, pp. 503–510. AUAI Press (2008)Google Scholar
  21. 21.
    Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields with smoothness-based priors. IEEE Trans. Patt. Mach. Intell. 30(6), 1068–1080 (2008)CrossRefGoogle Scholar
  22. 22.
    Vicente, S., Kolmogorov, V., Rother, C.: Joint optimization of segmentation and appearance models. In: Proc. ICCV 2009 (2009)Google Scholar
  23. 23.
    Wainwright, M.J., Jordan, M.I.: Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning 1(1-2), 1–305 (2008)zbMATHGoogle Scholar
  24. 24.
    Werner, T.: Revisiting the decomposition approach to inference in exponential families and graphical models. Tech. rep., Center for Machine Perception, Czech Technical University (May 2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Jörg Hendrik Kappes
    • 1
  • Stefan Schmidt
    • 1
  • Christoph Schnörr
    • 1
  1. 1.Image and Pattern Analysis Group & HCIUniversity of HeidelbergGermany

Personalised recommendations