Journal of Mathematical Imaging and Vision

, Volume 46, Issue 1, pp 143–159 | Cite as

Modelling Convex Shape Priors and Matching Based on the Gromov-Wasserstein Distance

  • Bernhard Schmitzer
  • Christoph Schnörr


We present a novel convex shape prior functional with potential for application in variational image segmentation. Starting point is the Gromov-Wasserstein Distance which is successfully applied in shape recognition and classification tasks but involves solving a non-convex optimization problem and which is non-convex as a function of the involved shape representations. In two steps we derive a convex approximation which takes the form of a modified transport problem and inherits the ability to incorporate vast classes of geometric invariances beyond rigid isometries. We propose ways to counterbalance the loss of descriptiveness induced by the required approximations and to process additional (non-geometric) feature information. We demonstrate combination with a linear appearance term and show that the resulting functional can be minimized by standard linear programming methods and yields a bijective registration between a given template shape and the segmented foreground image region. Key aspects of the approach are illustrated by numerical experiments.


Shape prior Wasserstein distance Convex relaxation Image segmentation 



This work was supported by the German National Science Foundation (DFG) grant GRK 1653.


  1. 1.
    Aherne, F., Thacker, N.A., Rockett, P.: Optimal pairwise geometric histograms. In: Proc. British Machine Vision Conf. (BMVC), pp. 480–490 (1997) Google Scholar
  2. 2.
    Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24, 509–522 (2002) CrossRefGoogle Scholar
  3. 3.
    Bronstein, A., Bronstein, M., Kimmel, R., Mahmoudi, M., Sapiro, G.: A Gromov-Hausdorff framework with diffusion geometry for topologically-robust non-rigid shape matching. Int. J. Comput. Vis. 89, 266–286 (2010) CrossRefGoogle Scholar
  4. 4.
    Burger, M., Franek, M., Schönlieb, C.-B.: Regularised regression and density estimation based on optimal transport. Appl. Math. Res. eXpress, March (2012) Google Scholar
  5. 5.
    Burkard, R.E., Çela, P.P., Pitsoulis, L.: The Quadratic Assignment Problem. Handbook of Combinatorial Optimization. Kluwer Academic, Dordrecht (1998) Google Scholar
  6. 6.
    Burkard, R., Karisch, S., Rendl, F.: QAPLIB—a quadratic assignment problem library. J. Glob. Optim. 10, 391–403 (1997) MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66(5), 1632–1648 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Charpiat, G., Faugeras, O., Keriven, R.: Shape statistics for image segmentation with prior. In: Computer Vision and Pattern Recognition (CVPR 2007), pp. 1–6 (2007) CrossRefGoogle Scholar
  9. 9.
    Cremers, D., Kohlberger, T., Schnörr, C.: Shape statistics in kernel space for variational image segmentation. Pattern Recognit. 36(9), 1929–1943 (2003) zbMATHCrossRefGoogle Scholar
  10. 10.
    Cremers, D., Rousson, M., Deriche, R.: A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape. Int. J. Comput. Vis. 72(2), 195–215 (2007) CrossRefGoogle Scholar
  11. 11.
    Elad, A., Kimmel, R.: On bending invariant signatures for surfaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1285–1295 (2003) CrossRefGoogle Scholar
  12. 12.
    Gorelick, L., Galun, M., Sharon, E., Basri, R., Brandt, A.: Shape representation and classification using the Poisson equation. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 1991–2005 (2006) CrossRefGoogle Scholar
  13. 13.
    Gromov, M.: Metric Structures for Riemannian and Non-Riemannian Spaces. Birkhäuser, Boston (2007) zbMATHGoogle Scholar
  14. 14.
    Haker, S., Zhu, L., Tannenbaum, A., Angenent, S.: Optimal mass transport for registration and warping. Int. J. Comput. Vis. 60, 225–240 (2004) CrossRefGoogle Scholar
  15. 15.
    Korte, B., Vygen, J.: Combinatorial Optimization, 4th edn. Springer, Berlin (2008) Google Scholar
  16. 16.
    Lellmann, J., Schnörr, C.: Continuous multiclass labeling approaches and algorithms. SIAM J. Imaging Sci. 4(4), 1049–1096 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    McAuley, J.J., de Campos, T., Caetano, T.S.: Unified graph matching in Euclidean spaces. In: Computer Vision and Pattern Recognition (CVPR 2010), pp. 1871–1878 (2010) Google Scholar
  18. 18.
    Mémoli, F.: Gromov-Wasserstein distances and the metric approach to object matching. Found. Comput. Math. 11, 417–487 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Mémoli, F.: A spectral notion of the Gromov-Wasserstein distance and related methods. Appl. Comput. Harmon. Anal. 30(3), 363–401 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Mémoli, F., Sapiro, G.: A theoretical and computational framework for isometry invariant recognition of point cloud data. Found. Comput. Math. 5, 313–347 (2005) MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Pock, T., Chambolle, A., Cremers, D., Bischof, H.: A convex relaxation approach for computing minimal partitions. In: Computer Vision and Pattern Recognition (CVPR 2009), pp. 810–817 (2009) CrossRefGoogle Scholar
  22. 22.
    Rabin, J., Peyré, G., Cohen, L.: Geodesic shape retrieval via optimal mass transport. In: Computer Vision—ECCV 2010. LNCS, vol. 6315, pp. 771–784. Springer, Berlin (2010) CrossRefGoogle Scholar
  23. 23.
    Reuter, M., Wolter, F.E., Peinecke, N.: Laplace-Beltrami spectra as “shape-dna” of surfaces and solids. Comput. Aided Des. 38(4), 342–366 (2006) CrossRefGoogle Scholar
  24. 24.
    Schellewald, C., Roth, S., Schnörr, C.: Evaluation of a convex relaxation to a quadratic assignment matching approach for relational object views. Image Vis. Comput. 25, 1301–1314 (2007) CrossRefGoogle Scholar
  25. 25.
    Schmitzer, B., Schnörr, C.: Convex coupling continuous cuts and shape priors. In: Bruckstein, A.M., ter Haar Romeny, B.M., Bronstein, A.M., Bronstein, M.M. (eds.) Scale Space and Variational Methods in Computer Vision (SSVM 2011). LNCS, vol. 6667, pp. 423–434. Springer, Berlin (2011) CrossRefGoogle Scholar
  26. 26.
    Sethian, J.A.: A fast marching level set method for monotonically advancing fronts. In: Proc. Nat. Acad. Sci., pp. 1591–1595 (1995) Google Scholar
  27. 27.
    Sundaramoorthi, G., Mennucci, A., Soatto, S., Yezzi, A.: A new geometric metric in the space of curves, and applications to tracking deforming objects by prediction and filtering. SIAM J. Imaging Sci. 4(1), 109–145 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Van Loan, C.: The ubiquitous Kronecker product. J. Comput. Appl. Math. 123, 85–100 (2000) MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Villani, C.: Optimal Transport: Old and New. Springer, Berlin (2009) zbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Heidelberg UniversityHeidelbergGermany

Personalised recommendations