Convex Multi-class Image Labeling by Simplex-Constrained Total Variation

  • Jan Lellmann
  • Jörg Kappes
  • Jing Yuan
  • Florian Becker
  • Christoph Schnörr
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5567)


Multi-class labeling is one of the core problems in image analysis. We show how this combinatorial problem can be approximately solved using tools from convex optimization. We suggest a novel functional based on a multidimensional total variation formulation, allowing for a broad range of data terms. Optimization is carried out in the operator splitting framework using Douglas-Rachford Splitting. In this connection, we compare two methods to solve the Rudin-Osher-Fatemi type subproblems and demonstrate the performance of our approach on single- and multichannel images.


Maximal Monotone Convex Optimization Problem Data Term Total Variation Minimization Total Variation Term 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. PAMI 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  2. 2.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. PAMI 26(9), 1124–1137 (2004)CrossRefzbMATHGoogle Scholar
  3. 3.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? PAMI 26(2), 147–159 (2004)CrossRefzbMATHGoogle Scholar
  4. 4.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Strang, G.: Maximal flow through a domain. Math. Prog. 26, 123–143 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Chan, T.F., Esedoḡlu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. J. Appl. Math. 66(5), 1632–1648 (2006)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Pock, T., Schönemann, T., Graber, G., Bischof, H., Cremers, D.: A convex formulation of continuous multi-label problems. In: ECCV, vol. 3, pp. 792–805 (2008)Google Scholar
  8. 8.
    Ishikawa, H.: Exact optimization for Markov random fields with convex priors. PAMI 25(10), 1333–1336 (2003)CrossRefGoogle Scholar
  9. 9.
    Zach, C., Gallup, D., Frahm, J.M., Niethammer, M.: Fast global labeling for real-time stereo using multiple plane sweeps. In: VMV (2008)Google Scholar
  10. 10.
    Kleinberg, J., Tardos, E.: Approximation algorithms for classification problems with pairwise relationships: Metric labeling and MRFs. In: FOCS, pp. 14–23 (1999)Google Scholar
  11. 11.
    Ziemer, W.: Weakly Differentiable Functions. Springer, Heidelberg (1989)CrossRefzbMATHGoogle Scholar
  12. 12.
    Meyer, Y.: Oscillating Patterns in Image Processing and Nonlinear Evolution Equations. Univ. Lect. Series, vol. 22. AMS (2001)Google Scholar
  13. 13.
    Sapiro, G., Ringach, D.L.: Anisotropic diffusion of multi-valued images with applications to color filtering. Trans. Image Process. 5, 1582–1586 (1996)CrossRefGoogle Scholar
  14. 14.
    Chan, T.F., Shen, J.: Image processing and analysis. SIAM, Philadelphia (2005)CrossRefzbMATHGoogle Scholar
  15. 15.
    Yang, J., Yin, W., Zhang, Y., Wang, Y.: A fast algorithm for edge-preserving variational multichannel image restoration. Tech. Rep. 08-09, Rice Univ. (2008)Google Scholar
  16. 16.
    Duval, V., Aujol, J.F., Vese, L.: A projected gradient algorithm for color image decomposition. CMLA Preprint (2008-21) (2008)Google Scholar
  17. 17.
    Chan, T., Esedoglu, S., Park, F., Yip, A.: Total variation image restoration: Overview and recent developments. In: The Handbook of Mathematical Models in Computer Vision. Springer, Heidelberg (2005)Google Scholar
  18. 18.
    Lellmann, J., Kappes, J., Yuan, J., Becker, F., Schnörr, C.: Convex multi-class image labeling by simplex-constrained total variation. TR, U. of Heidelberg (2008)Google Scholar
  19. 19.
    Rockafellar, R., Wets, R.J.B.: Variational Analysis, 2nd edn. Springer, Heidelberg (2004)zbMATHGoogle Scholar
  20. 20.
    Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. of the AMS 82(2), 421–439 (1956)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM Journal on Numerical Analysis 16(6), 964–979 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Eckstein, J.: Splitting Methods for Monotone Operators with Application to Parallel Optimization. PhD thesis, MIT (1989)Google Scholar
  23. 23.
    Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for max. mon. operators. M. Prog. 55, 293–318 (1992)CrossRefzbMATHGoogle Scholar
  24. 24.
    Michelot, C.: A finite algorithm for finding the projection of a point onto the canonical simplex of ℝn. J. Optim. Theory and Appl. 50(1), 195–200 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Dobson, D.C., Curtis, Vogel, R.: Iterative methods for total variation denoising. J. Sci. Comput 17, 227–238 (1996)MathSciNetGoogle Scholar
  26. 26.
    Chambolle, A.: An algorithm for total variation minimization and applications. JMIV 20, 89–97 (2004)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Chambolle, A.: Total variation minimization and a class of binary MRF models. In: Rangarajan, A., Vemuri, B.C., Yuille, A.L. (eds.) EMMCVPR 2005. LNCS, vol. 3757, pp. 136–152. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  28. 28.
    Aujol, J.F.: Some algorithms for total variation based image restoration. CMLA Preprint (2008-05) (2008)Google Scholar
  29. 29.
    Chan, T.F., Golub, G.H., Mulet, P.: A nonlinear primal-dual method for total variation-based image restoration. J. Sci. Comput. 20, 1964–1977 (1999)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. SIAM J. Multisc. Model. Sim. 4(4), 1168–1200 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Bresson, X., Chan, T.: Fast minimization of the vectorial total variation norm and applications to color image processing. Tech. Rep. 07-25, UCLA (2007)Google Scholar
  32. 32.
    Geman, D., Yang, C.: Nonlinear image recovery with halfquadratic regularization. IEEE Trans. Image Proc. 4(7), 932–946 (1995)CrossRefGoogle Scholar
  33. 33.
    Cohen, L.: Auxiliary variables and two-step iterative algorithms in computer vision problems. JMIV 6(1), 59–83 (1996)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Strang, G.: The discrete cosine transform. SIAM Review 41(1), 135–147 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for Markov random fields. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 16–29. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  36. 36.
    Hintermüller, M., Stadler, G.: An infeasible primal-dual algorithm for total bounded variation-based inf-convolution-type image restoration. J. Sci. Comput. 28(1), 1–23 (2006)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Jan Lellmann
    • 1
  • Jörg Kappes
    • 1
  • Jing Yuan
    • 1
  • Florian Becker
    • 1
  • Christoph Schnörr
    • 1
  1. 1.Image and Pattern Analysis Group (IPA) HCI, Dept. of Mathematics and Computer ScienceUniversity of HeidelbergGermany

Personalised recommendations