Total roto-translational variation

  • Antonin ChambolleEmail author
  • Thomas Pock


We consider curvature depending variational models for image regularization, such as Euler’s elastica. These models are known to provide strong priors for the continuity of edges and hence have important applications in shape- and image processing. We consider a lifted convex representation of these models in the roto-translation space: in this space, curvature depending variational energies are represented by means of a convex functional defined on divergence free vector fields. The line energies are then easily extended to any scalar function. It yields a natural generalization of the total variation to curvature-dependent energies. As our main result, we show that the proposed convex representation is tight for characteristic functions of smooth shapes. We also discuss cases where this representation fails. For numerical solution, we propose a staggered grid discretization based on an averaged Raviart–Thomas finite elements approximation. This discretization is consistent, up to minor details, with the underlying continuous model. The resulting non-smooth convex optimization problem is solved using a first-order primal-dual algorithm. We illustrate the results of our numerical algorithm on various problems from shape- and image processing.

Mathematics Subject Classification

53A04 49Q20 26A45 35J35 53A40 65K10 



The authors would like to thank Prof. R. Duits (T.U. Eindhoven), as well as the referees for their accurate reading of the paper and their helpful and positive comments. They also wish to thank the Isaac Newton Institute for Mathematical Sciences, Cambridge, for support and hospitality during the programme “Variational methods, new optimisation techniques and new fast numerical algorithms” (Sept.–Oct., 2017), when this paper was completed. This work was supported by: EPSRC Grant No. EP/K032208/1. The work of A.C. was also partially supported by a Grant of the Simons Foundation. T.P. acknowledges support by the Austrian science fund (FWF) under the project EANOI, No. I1148 and the ERC starting Grant HOMOVIS, No. 640156.


  1. 1.
    Acerbi, E., Mucci, D.: Curvature-dependent energies. Milan J. Math. 85(1), 41–69 (2017)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Acerbi, E., Mucci, D.: Curvature-dependent energies: the elastic case. Nonlinear Anal. 153, 7–34 (2017)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Ambrosio, L., Fusco, N., Pallara, D.: Functions of Bounded Variation and Free Discontinuity Problems. The Clarendon Press/Oxford University Press, New York (2000)zbMATHGoogle Scholar
  4. 4.
    Ambrosio, L., Masnou, S.: A direct variational approach to a problem arising in image reconstruction. Interfaces Free Bound. 5(1), 63–81 (2003)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Anzellotti, G.: Functionals depending on curvatures. Rend. Sem. Mat. Univ. Politec. Torino (Special Issue):47–62 (1990), 1989. Conference on Partial Differential Equations and Geometry (Torino, 1988)Google Scholar
  6. 6.
    Anzellotti, G., Serapioni, R., Tamanini, I.: Curvatures, functionals, currents. Indiana Univ. Math. J. 39(3), 617–669 (1990)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Anzellotti, G., Delladio, S.: Minimization of functionals of curvatures and the Willmore problem. In: Advances in Geometric Analysis and Continuum Mechanics (Stanford, CA, 1993), pp. 33–43. International Press Press, Cambridge (1995)Google Scholar
  8. 8.
    Bae, E., Tai, X.-C., Zhu, W.: Augmented Lagrangian method for an Euler’s elastica based segmentation model that promotes convex contours. Inverse Probl. Imaging 11(1), 1–23 (2017)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. Image Process. 10(8), 1200–1211 (2001)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Ballester, C., Caselles, V., Verdera, J.: Disocclusion by joint interpolation of vector fields and gray levels. Multiscale Model. Simul. 2(1), 80–123 (2003)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Bekkers, E.J., Duits, R., Mashtakov, A., Sanguinetti, G.R.: A PDE approach to data-driven sub-Riemannian geodesics in \(SE(2)\). SIAM J. Imaging Sci. 8(4), 2740–2770 (2015)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Bellettini, G., Dal Maso, G., Paolini, M.: Semicontinuity and relaxation properties of a curvature depending functional in 2D. Ann. Scuola Norm. Sup. Pisa Cl. Sci. 20(2), 247–297 (1993)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Bellettini, G., Mugnai, L.: Characterization and representation of the lower semicontinuous envelope of the elastica functional. Ann. Inst. H. Poincaré Anal. Non Linéaire 21(6), 839–880 (2004)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Bellettini, G., Mugnai, L.: On the approximation of the elastica functional in radial symmetry. Calc. Var. Partial Differ. Equ. 24(1), 1–20 (2005)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Bellettini, G., Mugnai, L.: A varifolds representation of the relaxed elastica functional. J. Convex Anal. 14(3), 543–564 (2007)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Bellettini, G., March, R.: An image segmentation variational model with free discontinuities and contour curvature. Math. Models Methods Appl. Sci. 14(1), 1–45 (2004)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Bellettini, G., March, R.: Asymptotic properties of the Nitzberg–Mumford variational model for segmentation with depth. In: Free Boundary Problems. Volume 154 of International Series of Numerical Mathematics, pp. 75–84. Birkhäuser, Basel (2007)Google Scholar
  18. 18.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417–424. ACM Press/Addison-Wesley Publishing Co. (2000)Google Scholar
  19. 19.
    Boscain, U., Chertovskih, R.A., Gauthier, J.P., Remizov, A.O.: Hypoelliptic diffusion and human vision: a semidiscrete new twist. SIAM J. Imaging Sci. 7(2), 669–695 (2014)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Boscain, U., Duits, R., Rossi, F., Sachkov, Y.: Curve cuspless reconstruction via sub-Riemannian geometry. ESAIM Control Optim. Calc. Var. 20(3), 748–770 (2014)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Bouchitté, G., Valadier, M.: Integral representation of convex functionals on a space of measures. J. Funct. Anal. 80(2), 398–420 (1988)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Bredies, K., Pock, T., Wirth, B.: Convex relaxation of a class of vertex penalizing functionals. J. Math. Imaging Vis. 47(3), 278–302 (2013)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Bredies, K., Pock, T., Wirth, B.: A convex, lower semicontinuous approximation of Euler’s elastica energy. SIAM J. Math. Anal. 47(1), 566–613 (2015)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Chambolle, A., Pock, T.: On the ergodic convergence rates of a first-order primal-dual algorithm. Math. Program. 159(1–2, Ser. A), 253–287 (2016)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Chan, T.F., Kang, S.H., Shen, J.: Euler’s elastica and curvature-based inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2002)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Chan, T.F., Shen, J.: Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 62(3), 1019–1043 (2001/2002)Google Scholar
  28. 28.
    Chen, D., Mirebeau, J.-M., Cohen, L.D.: Global minimum for a Finsler elastica minimal path approach. Int. J. Comput. Vis. 122(3), 458–483 (2017)MathSciNetGoogle Scholar
  29. 29.
    Citti, G., Franceschiello, B., Sanguinetti, G., Sarti, A.: Sub-Riemannian mean curvature flow for image processing. SIAM J. Imaging Sci. 9(1), 212–237 (2016)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Dayrens, F., Masnou, S., Novaga, M.: Existence, regularity and structure of confined elasticæ. In: ESAIM:COCV, 2017 (to appear) Google Scholar
  31. 31.
    Delladio, S.: Minimizing functionals depending on surfaces and their curvatures: a class of variational problems in the setting of generalized Gauss graphs. Pac. J. Math. 179(2), 301–323 (1997)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Delladio, S.: Special generalized Gauss graphs and their application to minimization of functionals involving curvatures. J. Reine Angew. Math. 486, 17–43 (1997)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Duits, R., Boscain, U., Rossi, F., Sachkov, Y.: Association fields via cuspless sub-Riemannian geodesics in SE(2). J. Math. Imaging Vis. 49(2), 384–417 (2014)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Duits, R., Meesters, S.P.L., Mirebeau, J.-M., Portegies, J. M.: Optimal Paths for Variants of the 2D and 3D Reeds-Shepp Car with Applications in Image Analysis. ArXiv e-prints (2016)Google Scholar
  35. 35.
    Ekeland, I., Témam, R.: Convex Analysis and Variational Problems, Volume 28 of Classics in Applied Mathematics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, english edition (1999). Translated from the FrenchGoogle Scholar
  36. 36.
    El-Zehiry, N., Grady, L.: Optimization of weighted curvature for image segmentation. arXiv preprint arXiv:1006.4175 (2010)
  37. 37.
    El-Zehiry, N.Y., Grady, L.: Fast global optimization of curvature. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3257–3264. IEEE (2010)Google Scholar
  38. 38.
    El-Zehiry, N.Y., Grady, L.: Contrast driven elastica for image segmentation. IEEE Trans. Image Process. 25(6), 2508–2518 (2016)MathSciNetzbMATHGoogle Scholar
  39. 39.
    Esedoglu, S., March, R.: Segmentation with depth but without detecting junctions. J. Math. Imaging Vis. 18(1), 7–15 (2003). Special issue on imaging science (Boston, MA, 2002)Google Scholar
  40. 40.
    Esedoglu, S., Shen, J.: Digital inpainting based on the Mumford–Shah–Euler image model. Eur. J. Appl. Math. 13(4), 353–370 (2002)MathSciNetzbMATHGoogle Scholar
  41. 41.
    Franken, E., Duits, R.: Crossing-preserving coherence-enhancing diffusion on invertible orientation scores. Int. J. Comput. Vis. 85(3), 253 (2009)Google Scholar
  42. 42.
    Glowinski, R.: ADMM and non-convex variational problems. In: Splitting Methods in Communication, Imaging, Science, and Engineering, Science, and Engineering, pp. 251–299. Springer, Cham (2016)Google Scholar
  43. 43.
    Glowinski, R., Pan, T.-W., Tai, X.-C.: Some facts about operator-splitting and alternating direction methods. In: Splitting Methods in Communication, Imaging, Science, and Engineering. Sci. Comput., pp. 19–94. Springer, Cham (2016)Google Scholar
  44. 44.
    Hubel, D.H., Wiesel, T.N.: Receptive fields of single neurones in the cat’s striate cortex. J. Physiol. 148(3), 574–591 (1959)Google Scholar
  45. 45.
    Hutchinson, J.E.: Second fundamental form for varifolds and the existence of surfaces minimising curvature. Indiana Univ. Math. J. 35(1), 45–71 (1986)MathSciNetzbMATHGoogle Scholar
  46. 46.
    Kanizsa, G.: Grammatica del vedere: saggi su percezione e gestalt. Biblioteca: Mulino. Il Mulino (1980), 2nd edition 1997Google Scholar
  47. 47.
    Kanizsa, G.: Organization in Vision : Essays on Gestalt Perception. Praeger, New York (1979). Foreword by Paolo Legrenzi and Paolo BozziGoogle Scholar
  48. 48.
    Koenderink, J.J., van Doorn, A.J.: Representation of local geometry in the visual system. Biol. Cybernet. 55(6), 367–375 (1987)MathSciNetzbMATHGoogle Scholar
  49. 49.
    Krueger, M., Delmas, P., Gimel’farb, G.: Efficient Image Segmentation Using Weighted Pseudo-Elastica, pp. 59–67. Springer, Berlin (2011)Google Scholar
  50. 50.
    Mantegazza, C.: Curvature varifolds with boundary. J. Differ. Geom. 43(4), 807–843 (1996)MathSciNetzbMATHGoogle Scholar
  51. 51.
    Masnou, S., Nardi, G.: A coarea-type formula for the relaxation of a generalized elastica functional. J. Convex Anal. 20(3), 617–653 (2013)MathSciNetzbMATHGoogle Scholar
  52. 52.
    Masnou, S., Morel, J.-M.: Level lines based disocclusion. In: International Conference on Image Processing, 1998. ICIP 98. Proceedings, pp. 259–263. IEEE (1998)Google Scholar
  53. 53.
    Masnou, S., Morel, J.-M.: On a variational theory of image amodal completion. Rend. Sem. Mat. Univ. Padova 116, 211–252 (2006)MathSciNetzbMATHGoogle Scholar
  54. 54.
    Mercier, G.: Continuity results for TV-minimizers. Indiana Univ. Math. J. 67(4), 1499–1545 (2018)MathSciNetzbMATHGoogle Scholar
  55. 55.
    Mirebeau, J.-M.: Anisotropic fast-marching on Cartesian grids using lattice basis reduction. SIAM J. Numer. Anal. 52(4), 1573–1599 (2014)MathSciNetzbMATHGoogle Scholar
  56. 56.
    Mirebeau, J.-M.: Fast Marching Methods for Curvature Penalized Shortest Paths, preprint (2017)Google Scholar
  57. 57.
    Moiseev, I., Sachkov, Y.L.: Maxwell strata in sub-Riemannian problem on the group of motions of a plane. ESAIM Control Optim. Calc. Var. 16(2), 380–399 (2010)MathSciNetzbMATHGoogle Scholar
  58. 58.
    Nitzberg, M., Mumford, D., Shiota, T.: Filtering, segmentation and depth. Lecture Notes in Computer Science, vol. 662. Springer, Berlin (1993)Google Scholar
  59. 59.
    Petitot, J., Tondut, Y.: Vers une neurogéométrie. Fibrations corticales, structures de contact et contours subjectifs modaux. Math. Inform. Sci. Hum. 145, 5–101 (1999)Google Scholar
  60. 60.
    Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: Proceedings of the 2011 International Conference on Computer Vision, ICCV’11, pp. 1762–1769, Washington, DC, USA, IEEE Computer Society (2011)Google Scholar
  61. 61.
    Prandi, D., Boscain, U., Gauthier, J.-P.: Image processing in the semidiscrete group of rototranslations. In: Geometric science of information. Volume 9389 of Lecture Notes in Computer Science, pp. 627–634. Springer, Cham (2015)Google Scholar
  62. 62.
    Raviart, P.-A., Thomas, J.M.: A mixed finite element method for 2nd order elliptic problems. In: Mathematical Aspects of Finite Element Methods (Proceedings of the International Conference on Consiglio Nazionale delle Ricerche (C.N.R.), Rome, 1975). Lecture Notes in Mathematics, Vol. 606, pp. 292–315. Springer, Berlin (1977)Google Scholar
  63. 63.
    Rockafellar, R.T.: Integrals which are convex functionals. II. Pac. J. Math. 39, 439–469 (1971)MathSciNetzbMATHGoogle Scholar
  64. 64.
    Rockafellar R.T.: Convex analysis. Princeton Landmarks in Mathematics. Princeton University Press, Princeton (1997). Reprint of the 1970 original, Princeton paperbacksGoogle Scholar
  65. 65.
    Sarti, A., Citti, G.: Subjective surfaces and Riemannian mean curvature flow of graphs. Acta Math. Univ. Comen. (N.S.) 70(1), 85–103 (2000)MathSciNetzbMATHGoogle Scholar
  66. 66.
    Sarti, A., Citti, G., Petitot, J.: The symplectic structure of the primary visual cortex. Biol. Cybernet. 98(1), 33–48 (2008)MathSciNetzbMATHGoogle Scholar
  67. 67.
    Schmaltz, C., Peter, P., Mainberger, M., Ebel, F., Weickert, J., Bruhn, A.: Understanding, optimising, and extending data compression with anisotropic diffusion. Int. J. Comput. Vis. 108(3), 222–240 (2014)MathSciNetGoogle Scholar
  68. 68.
    Schoenemann, T., Cremers, D.: Introducing curvature into globally optimal image segmentation: minimum ratio cycles on product graphs. In: IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, pp. 1–6. IEEE (2007)Google Scholar
  69. 69.
    Schoenemann, T., Kahl, F., Masnou, S., Cremers, D.: A linear framework for region-based image segmentation and inpainting involving curvature penalization. Int. J. Comput. Vis. 99(1), 53–68 (2012)MathSciNetzbMATHGoogle Scholar
  70. 70.
    Schoenemann, T., Masnou, S., Cremers, D.: The elastic ratio: introducing curvature into ratio-based image segmentation. IEEE Trans. Image Process. 20(9), 2565–2581 (2011)MathSciNetzbMATHGoogle Scholar
  71. 71.
    Schoenemann, T., Masnou, S., Cremers, D.: On a linear programming approach to the discrete Willmore boundary value problem and generalizations. In: Curves and Surfaces. Volume 6920 of Lecture Notes in Computer Science, pp. 629–646. Springer, Heidelberg (2012)Google Scholar
  72. 72.
    Sharma, U., Duits, R.: Left-invariant evolutions of wavelet transforms on the similitude group. Appl. Comput. Harmon. Anal. 39(1), 110–137 (2015)MathSciNetzbMATHGoogle Scholar
  73. 73.
    Smirnov, S.K.: Decomposition of solenoidal vector charges into elementary solenoids, and the structure of normal one-dimensional flows. Algebra i Analiz 5(4), 206–238 (1993)MathSciNetzbMATHGoogle Scholar
  74. 74.
    Tai, X.-C., Hahn, J., Chung, G.J.: A fast algorithm for Euler’s elastica model using augmented Lagrangian method. SIAM J. Imaging Sci. 4(1), 313–344 (2011)MathSciNetzbMATHGoogle Scholar
  75. 75.
    Weickert, J.: Theoretical Foundations of Anisotropic Diffusion in Image Processing, pp. 221–236. Springer, Vienna (1996)Google Scholar
  76. 76.
    Weickert, J.: Mathematische Bildverarbeitung mit Ideen aus der Natur. Mitt. Dtsch. Math.-Ver. 20(2), 82–90 (2012)MathSciNetzbMATHGoogle Scholar
  77. 77.
    Willmore, T.J.: A survey on Willmore immersions. In: Geometry and Topology of Submanifolds, IV (Leuven, 1991), pp. 11–16. World Science Publishing, River Edge (1992)Google Scholar
  78. 78.
    Yashtini, M., Kang, S.H.: Alternating direction method of multiplier for Euler’s elastica-based denoising. In: Aujol, J.-F., Nikolova, M., Papadakis, N. (eds.) Scale Space and Variational Methods in Computer Vision: 5th International Conference, SSVM 2015, Lège-Cap Ferret, France, May 31–June 4, 2015, Proceedings, pp. 690–701. Springer International Publishing, Cham (2015)Google Scholar
  79. 79.
    Yashtini, M., Kang, S.H.: A fast relaxed normal two split method and an effective weighted TV approach for Euler’s elastica image inpainting. SIAM J. Imaging Sci. 9(4), 1552–1581 (2016)MathSciNetzbMATHGoogle Scholar
  80. 80.
    Zhu, W., Tai, X.-C., Chan, T.: Augmented Lagrangian method for a mean curvature based image denoising model. Inverse Probl. Imaging 7(4), 1409–1432 (2013)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.CMAP, Ecole PolytechniqueCNRSPalaiseauFrance
  2. 2.Institute for Computer Graphics and VisionGraz University of TechnologyGrazAustria
  3. 3.Center for Vision, Automation and ControlAIT Austrian Institute of Technology GmbHViennaAustria

Personalised recommendations