Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem

Abstract

We consider a variational convex relaxation of a class of optimal partitioning and multiclass labeling problems, which has recently proven quite successful and can be seen as a continuous analogue of Linear Programming (LP) relaxation methods for finite-dimensional problems. While for the latter several optimality bounds are known, to our knowledge no such bounds exist in the infinite-dimensional setting. We provide such a bound by analyzing a probabilistic rounding method, showing that it is possible to obtain an integral solution of the original partitioning problem from a solution of the relaxed problem with an a priori upper bound on the objective. The approach has a natural interpretation as an approximate, multiclass variant of the celebrated coarea formula.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Algorithm 1

References

  1. 1.

    Alberti, G.: The calibration method for the Mumford-Shah functional and free-discontinuity problems. Calc. Var. Partial Differ. Equ. 16(3), 299–333 (2003)

    MathSciNet  MATH  Article  Google Scholar 

  2. 2.

    Ambrosio, L., Fusco, N., Pallara, D.: Functions of Bounded Variation and Free Discontinuity Problems. Clarendon, Oxford (2000)

    MATH  Google Scholar 

  3. 3.

    Bae, E., Yuan, J., Tai, X.C.: Global minimization for continuous multiphase partitioning problems using a dual approach. Int. J. Comput. Vis. 92, 112–129 (2011)

    MathSciNet  MATH  Article  Google Scholar 

  4. 4.

    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  5. 5.

    Chambolle, A., Cremers, D., Pock, T.: A convex approach for computing minimal partitions. Tech. Rep. 649, Ecole Polytechnique CMAP (2008)

  6. 6.

    Chambolle, A., Darbon, J.: On total variation minimization and surface evolution using parametric maximum flows. Int. J. Comput. Vis. 84, 288–307 (2009)

    Article  Google Scholar 

  7. 7.

    Chan, T.F., Esedoḡlu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. J. Appl. Math. 66(5), 1632–1648 (2006)

    MathSciNet  MATH  Google Scholar 

  8. 8.

    Darbon, J., Sigelle, M.: Image restoration with discrete constrained total variation part I: fast and exact optimization. J. Math. Imaging Vis. 26(3), 261–276 (2006)

    MathSciNet  Article  Google Scholar 

  9. 9.

    Darbon, J., Sigelle, M.: Image restoration with discrete constrained total variation part II: levelable functions, convex priors and non-convex cases. J. Math. Imaging Vis. 26(3), 277–291 (2006)

    MathSciNet  Article  Google Scholar 

  10. 10.

    Delaunoy, A., Fundana, K., Prados, E., Heyden, A.: Convex multi-region segmentation on manifolds. In: Int. Conf. Comp. Vis (2009)

    Google Scholar 

  11. 11.

    Goldstein, T., Bresson, X., Osher, S.: Global minimization of Markov random field with applications to optical flow. CAM Report 09-77, UCLA (2009)

  12. 12.

    Kleinberg, J.M., Tardos, E.: Approximation algorithms for classification problems with pairwise relationships: metric labeling and Markov random fields. In: Found. Comp. Sci., pp. 14–23 (1999)

    Google Scholar 

  13. 13.

    Klodt, M., Schoenemann, T., Kolev, K., Schikora, M., Cremers, D.: An experimental comparison of discrete and continuous shape optimization methods. In: Europ. Conf. Comp. Vis, Marseille, France (2008)

    Google Scholar 

  14. 14.

    Kolev, K., Klodt, M., Brox, T., Cremers, D.: Continuous global optimization in multiview 3d reconstruction. Int. J. Comput. Vis. 84(1) (2009). doi:10.1007/s11263-009-0233-1

  15. 15.

    Komodakis, N., Tziritas, G.: Approximate labeling via graph cuts based on linear programming. IEEE Trans. Pattern Anal. Mach. Intell. 29(8), 1436–1453 (2007)

    Article  Google Scholar 

  16. 16.

    Lellmann, J., Becker, F., Schnörr, C.: Convex optimization for multi-class image labeling with a novel family of total variation based regularizers. In: Int. Conf. Comp. Vis (2009)

    Google Scholar 

  17. 17.

    Lellmann, J., Kappes, J., Yuan, J., Becker, F., Schnörr, C.: Convex multi-class image labeling by simplex-constrained total variation. In: Scale Space and Var. Meth. LNCS, vol. 5567, pp. 150–162 (2009)

    Google Scholar 

  18. 18.

    Lellmann, J., Lenzen, F., Schnörr, C.: Optimality bounds for a variational relaxation of the image partitioning problem. In: Energy Min. Meth. Comp. Vis. Patt. Recogn. (2011)

    Google Scholar 

  19. 19.

    Lellmann, J., Schnörr, C.: Continuous multiclass labeling approaches and algorithms. SIAM J. Imaging Sci. (2011). doi:10.1137/100805844

  20. 20.

    Lysaker, M., Tai, X.C.: Iterative image restoration combining total variation minimization and a second-order functional. Int. J. Comput. Vis. 66(1), 5–18 (2006)

    Article  Google Scholar 

  21. 21.

    Olsson, C.: Global optimization in computer vision: convexity, cuts and approximation algorithms. Ph.D. Thesis, Lund Univ. (2009)

  22. 22.

    Olsson, C., Byröd, M., Overgaard, N.C., Kahl, F.: Extending continuous cuts: anisotropic metrics and expansion moves. In: Int. Conf. Comp. Vis (2009)

    Google Scholar 

  23. 23.

    Paragios, N., Chen, Y., Faugeras, O. (eds.): The Handbook of Mathematical Models in Computer Vision. Springer, Berlin (2006)

    Google Scholar 

  24. 24.

    Pock, T., Cremers, D., Bischof, H., Chambolle, A.: Global solutions of variational models with convex regularization. J. Imaging Sci. 3(4), 1122–1145 (2010)

    MathSciNet  MATH  Article  Google Scholar 

  25. 25.

    Rockafellar, R.T., Wets, R.J.B.: Variational Analysis, 2nd edn. Springer, Berlin (2004)

    Google Scholar 

  26. 26.

    Strandmark, P., Kahl, F., Overgaard, N.C.: Optimizing parametric total variation models. In: Int. Conf. Comp. Vis. (2009)

    Google Scholar 

  27. 27.

    Trobin, W., Pock, T., Cremers, D., Bischof, H.: Continuous energy minimization by repeated binary fusion. In: Europ. Conf. Comp. Vis., vol. 4, pp. 667–690 (2008)

    Google Scholar 

  28. 28.

    Vazirani, V.V.: Approximation Algorithms. Springer, Berlin (2010)

    Google Scholar 

  29. 29.

    Yuan, J., Bae, E., Tai, X.C., Boykov, Y.: A continuous max-flow approach to Potts model. In: Europ. Conf. Comp. Vis, pp. 379–392 (2010)

    Google Scholar 

  30. 30.

    Zach, C., Gallup, D., Frahm, J.M., Niethammer, M.: Fast global labeling for real-time stereo using multiple plane sweeps. In: Vis. Mod. Vis. (2008)

    Google Scholar 

Download references

Acknowledgements

This publication is partly based on work supported by Award No. KUK-I1-007-43, made by King Abdullah University of Science and Technology (KAUST).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jan Lellmann.

Appendix

Appendix

Proof of Proposition 3

In order to prove the first assertion (88), note that the mapping wΨ(νw ) is convex, therefore it must assume its maximum on the polytope Δ l −Δ l :={z 1z 2|z 1,z 2∈Δ l } in a vertex of the polytope. Since the polytope Δ l −Δ l is the difference of two polytopes, its vertex set is at most the difference of their vertex sets, V:={e ie j|i,j∈{1,…,l}}. On this set, the bound Ψ(νw )⩽λ u holds for wV due to the upper-boundedness condition (25), which proves (88).

The second equality (90) follows from the fact that G:={b ik:=e k(e ie i+1)∣1⩽kd,1⩽il−1} is a basis of the linear subspace W, satisfying Ψ(b ik)⩽λ u , and Ψ is positively homogeneous and convex, and thus subadditive. Specifically, there is a linear transform T:W→ℝd×(l−1) such that w=∑ i,k b ik α ik for α=T(w). Then

(172)
(173)
(174)

Since (25) ensures Ψb ik)⩽λ u , we obtain

$$ \varPsi(w) \leqslant\lambda_u \sum_{i k} | \alpha_{i k} | \leqslant \lambda_u \|T\| \|w \|_2 $$
(175)

for any suitable operator norm ∥⋅∥ and any wW. □

Proof of Proposition 4

Denote \(\mathcal{B}_{\delta} :=\mathcal{B}_{\delta}(x)\). We prove mutual inclusion:

“⊆”: From the definition of the measure-theoretic interior,

(176)

Since \(|\mathcal{B}_{\delta} | \geqslant|\mathcal{B}_{\delta} \cap E| \geqslant|\mathcal{B}_{\delta} \cap E \cap F|\) (and vice versa for \(|\mathcal{B}_{\delta} \cap F|\)), it follows by the “sandwich” criterion that both \(\lim_{\delta\searrow0} |\mathcal{B}_{\delta} \cap E| / |\mathcal{B}_{\delta} |\) and \(\lim_{\delta\searrow0} |\mathcal{B}_{\delta} \cap F| / |\mathcal{B}_{\delta} |\) exist and are equal to 1, which shows xE 1F 1.

“⊇”: Assume that xE 1F 1. Then

(177)
(178)
(179)

We obtain equality,

(180)
(181)
(182)

from which we conclude that

$$ \lim_{\delta\searrow0} \sup\frac{|\mathcal{B}_{\delta} \cap E \cap F|}{|\mathcal{B}_{\delta} |} = \lim_{\delta\searrow0} \inf \frac{|\mathcal{B}_{\delta} \cap E \cap F|}{|\mathcal{B}_{\delta} |} = 1, $$

i.e., x∈(EF)1. □

Proof of Proposition 5

First note that

(183)
(184)
(185)
(186)
(187)

The inequality (∗) is a consequence of the definition of \(w^{\pm}_{\mathcal{F}E}\) and [2, Theorem 3.77], and (∗∗) follows directly from w(x),w(y)∈Δ l a.e. on Ω. The upper bound (187) permits applying [2, Theorem 3.84] on w, which provides \(w \in\operatorname{BV} (\varOmega)^{l}\) and (94). Due to [2, Proposition 3.61], the sets (E)0,(E)1 and \(\mathcal{F}E\) form a (pairwise disjoint) partition of Ω, up to an \(\mathcal{H}^{d - 1}\)-zero set. Therefore, since \(\varPsi(D u) \ll|D u| \ll\mathcal{H}^{d - 1}\) by construction, from [2, Theorem 2.37, 3.84] we obtain, for any Borel set A,

(188)
(189)

Since w(x)∈Δ l a.e. by assumption, we conclude that \(w^{+}_{\mathcal{F}E}\) and \(w^{-}_{\mathcal{F}E}\) must have values in Δ l as well, see [2, Theorem 3.77]. Therefore we can apply Proposition 3 to obtain

(190)
(191)
(192)

We rewrite Ψ(Dw) using (94),

(193)

From [2, Proposition 2.37] we obtain that Ψ is additive on mutually singular Radon measures μ,ν, i.e., if |μ|⊥|ν|, then

(194)

for any Borel set BΩ. This holds in particular for the three measures in (193), therefore

(195)

Since Du⌞(E)1≪|Du⌞(E)1|=|Du|⌞(E)1, we conclude Ψ(Dw)⌞(E)1=Ψ(Du)⌞(E)1 and Ψ(Dw)⌞(E)0=Ψ(Dv)⌞(E)0. Substitution into (192) proves the remaining assertion,

(196)

 □

Proof of Proposition 6

We first show (98). It suffices to show that

(197)

This can be seen by considering the precise representative \(\widetilde{1_{E}}\) of 1 E [2, Definition 3.63]: Starting with the definition,

(198)

the fact that \(\lim_{\delta\searrow0} \frac{| \varOmega\cap\mathcal {B}_{\delta} (x) |}{|\mathcal{B}_{\delta} (x) |} = 1\) implies

(199)
(200)
(201)

Substituting E by ΩE, the same equivalence shows that \(x \in(E)^{0} \Leftrightarrow\widetilde{1_{\varOmega\setminus E}} (x) = 1 \Leftrightarrow\widetilde{1_{E}} (x) = 0\). As \(\mathcal{L}^{d} (\varOmega \setminus((E)^{0} \cup(E)^{1})) = 0\), this shows that \(1_{E^{1}} = \widetilde{1_{E}}\) \(\mathcal{L}^{d}\)-a.e. Using the fact that \(\widetilde {1_{E}} = 1_{E}\) [2, Proposition 3.64], we conclude that \(1_{(E)^{1}} = 1_{E}\) \(\mathcal{L}^{d}\)-a.e., which proves (197) and therefore the assertion (98).

Since the measure-theoretic interior (E)1 is defined over \(\mathcal{L}^{d}\)-integrals, it is invariant under \(\mathcal{L}^{d}\)-negligible modifications of E. Together with (197) this implies

(202)

To show the relation (Du)⌞(E)1=(Dv)⌞(E)1, consider

(203)
(204)

The equality (∗) holds due to the assumption (96), and due to the fact that Df=Dg if f=g \(\mathcal{L}^{d}\)-a.e. (see, e.g., [2, Proposition 3.2]). We continue from (204) via

(205)
(206)
(207)
(208)
(209)

Therefore Du⌞(E)1=Dv⌞(E)1. Then,

(210)
(211)

In the equality (∗) we used the additivity of Ψ on mutually singular Radon measures [2, Proposition 2.37]. By definition of the total variation, |μA|=|μ|⌞A holds for any measure μ, therefore |Du⌞(Ω∖(E)1)|=|Du|⌞(Ω∖(E)1) and |Du⌞(Ω∖(E)1)|((E)1)=0, which together with (again by definition) Ψ(μ)≪|μ| implies that the second term in (211) vanishes. Since all observations equally hold for v instead of u, we conclude

(212)
(213)
(214)

Equation (97) follows immediately. □

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lellmann, J., Lenzen, F. & Schnörr, C. Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem. J Math Imaging Vis 47, 239–257 (2013). https://doi.org/10.1007/s10851-012-0390-7

Download citation

Keywords

  • Convex relaxation
  • Multiclass labeling
  • Approximation bound
  • Combinatorial optimization
  • Total variation
  • Linear programming relaxation