Closed-Form Approximate CRF Training for Scalable Image Segmentation

  • Alexander Kolesnikov
  • Matthieu Guillaumin
  • Vittorio Ferrari
  • Christoph H. Lampert
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8691)


We present LS-CRF, a new method for training cyclic Conditional Random Fields (CRFs) from large datasets that is inspired by classical closed-form expressions for the maximum likelihood parameters of a generative graphical model with tree topology. Training a CRF with LS-CRF requires only solving a set of independent regression problems, each of which can be solved efficiently in closed form or by an iterative solver. This makes LS-CRF orders of magnitude faster than classical CRF training based on probabilistic inference, and at the same time more flexible and easier to implement than other approximate techniques, such as pseudolikelihood or piecewise training. We apply LS-CRF to the task of semantic image segmentation, showing that it achieves on par accuracy to other training techniques at higher speed, thereby allowing efficient CRF training from very large training sets. For example, training a linearly parameterized pairwise CRF on 150,000 images requires less than one hour on a modern workstation.


Image Segmentation Training Image Conditional Random Field Probabilistic Inference Segmentation Mask 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-319-10578-9_36_MOESM1_ESM.pdf (3.1 mb)
Electronic Supplementary Material (PDF 3,124 KB)


  1. 1.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. PAMI 34(11) (2012)Google Scholar
  2. 2.
    Andres, B., Beier, T., Kappes, J.H.: OpenGM: A C++ library for discrete graphical models. ArXiv e-prints 1206.0111 (2012),
  3. 3.
    Arandjelović, R., Zisserman, A.: Three things everyone should know to improve object retrieval. In: CVPR (2012)Google Scholar
  4. 4.
    Besag, J.: Statistical analysis of non-lattice data. The Statistician (1975)Google Scholar
  5. 5.
    Bottou, L., Bousquet, O.: The tradeoffs of large scale learning. In: NIPS (2007)Google Scholar
  6. 6.
    Bulatov, A., Grohe, M.: The complexity of partition functions. Theoretical Computer Science 348(2) (2005)Google Scholar
  7. 7.
    Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing (SISC) 16(5), 1190–1208 (1995)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A large-scale hierarchical image database. In: CVPR (2009)Google Scholar
  9. 9.
    Domke, J.: Learning graphical model parameters with approximate marginals inference. PAMI (2013)Google Scholar
  10. 10.
    Domke, J.: Structured learning via logistic regression. In: NIPS (2013)Google Scholar
  11. 11.
    Everingham, M., van Gool, L., Williams, C., Winn, J., Zisserman, A.: The Pascal visual object classes (VOC) challenge. IJCV 88(2) (2010)Google Scholar
  12. 12.
    Finley, T., Joachims, T.: Training structural SVMs when exact inference is intractable. In: ICML (2008)Google Scholar
  13. 13.
    Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Annals of Statistics (2001)Google Scholar
  14. 14.
    Fulkerson, B., Vedaldi, A., Soatto, S.: Class segmentation and object localization with superpixel neighborhoods. In: ICCV (2009)Google Scholar
  15. 15.
    Gould, S.: Multiclass pixel labeling with non-local matching constraints. In: CVPR, pp. 2783–2790 (2012)Google Scholar
  16. 16.
    Gould, S., Fulton, R., Koller, D.: Decomposing a scene into geometric and semantically consistent regions. In: ICCV (2009)Google Scholar
  17. 17.
    Gould, S., Zhang, Y.: patchMatchGraph: Building a graph of dense patch correspondences for label transfer. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 439–452. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  18. 18.
    Guillaumin, M., Kuettel, D., Ferrari, V.: ImageNet Auto-annotation with Segmentation Propagation. IJCV (2014)Google Scholar
  19. 19.
    Kohli, P., Shekhovtsov, A., Rother, C., Kolmogorov, V., Torr, P.: On partial optimality in multi-label MRFs. In: ICML (2008)Google Scholar
  20. 20.
    Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. PAMI 28(10) (2006)Google Scholar
  21. 21.
    Komodakis, N.: Efficient training for pairwise or higher order CRFs via dual decomposition. In: CVPR, pp. 1841–1848 (2011)Google Scholar
  22. 22.
    Lempitsky, V.S., Vedaldi, A., Zisserman, A.: A pylon model for semantic segmentation. NIPS 24, 1485–1493 (2011)Google Scholar
  23. 23.
    Meshi, O., Sontag, D., Jaakkola, T., Globerson, A.: Learning efficiently with approximate inference via dual losses (2010)Google Scholar
  24. 24.
    Nowozin, S., Lampert, C.H.: Structured learning and prediction in computer vision. Foundations and Trends in Computer Graphics and Vision 6 (2011)Google Scholar
  25. 25.
    Nowozin, S., Rother, C., Bagon, S., Sharp, T., Yao, B., Kohli, P.: Decision tree fields. In: ICCV (2011)Google Scholar
  26. 26.
    Schroff, F., Criminisi, A., Zisserman, A.: Object class segmentation using random forests. In: BMVC (2008)Google Scholar
  27. 27.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context. IJCV 81(1) (2009)Google Scholar
  28. 28.
    Sontag, D., Meshi, O., Globerson, A., Jaakkola, T.S.: More data means less inference: A pseudo-max approach to structured learning. In: NIPS (2010)Google Scholar
  29. 29.
    Sutton, C., McCallum, A.: Piecewise training of undirected models. In: UAI (2005)Google Scholar
  30. 30.
    Tighe, J., Lazebnik, S.: SuperParsing: Scalable nonparametric image parsing with superpixels. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 352–365. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  31. 31.
    Trofimov, I., Kornetova, A., Topinskiy, V.: Using boosted trees for click-through rate prediction for sponsored search. In: International Workshop on Data Mining for Online Advertising and Internet Economy (2012)Google Scholar
  32. 32.
    Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. JMLR 6 (2005)Google Scholar
  33. 33.
    Wainwright, M.J., Jordan, M.I.: Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning (2008)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Alexander Kolesnikov
    • 1
  • Matthieu Guillaumin
    • 2
  • Vittorio Ferrari
    • 3
  • Christoph H. Lampert
    • 1
  1. 1.IST AustriaAustria
  2. 2.ETH ZürichSwitzerland
  3. 3.University of EdinburghUK

Personalised recommendations