Journal of Mathematical Imaging and Vision

, Volume 60, Issue 9, pp 1459–1481 | Cite as

Priors with Coupled First and Second Order Differences for Manifold-Valued Image Processing

  • Ronny Bergmann
  • Jan Henrik Fitschen
  • Johannes Persch
  • Gabriele Steidl


We generalize discrete variational models involving the infimal convolution (IC) of first and second order differences and the total generalized variation (TGV) to manifold-valued images. We propose both extrinsic and intrinsic approaches. The extrinsic models are based on embedding the manifold into an Euclidean space of higher dimension with manifold constraints. An alternating direction methods of multipliers can be employed for finding the minimizers. However, the components within the extrinsic IC or TGV decompositions live in the embedding space which makes their interpretation difficult. Therefore, we investigate two intrinsic approaches: for Lie groups, we employ the group action within the models; for more general manifolds, our IC model is based on recently developed absolute second order differences on manifolds, while our TGV approach uses an approximation of the parallel transport by the pole ladder. For computing the minimizers of the intrinsic models, we apply gradient descent algorithms. Numerical examples demonstrate that our approaches work well for certain manifolds.


Infimal convolution Total generalized variation Higher order differences Manifold-valued images Optimization on manifolds 

Mathematics Subject Classification

49M15 49M25 49Q20 68U10 56Y99 



R. Bergmann wants to thank B. Wirth (University of Münster) for fruitful discussions on Schild’s ladder TGV. Funding by the German Research Foundation (DFG) within the Project STE 571/13-1 and BE 5888/2-1 and within the Research Training Group 1932, project area P3 and also Bundesministerium für Bildung und Forschung (Grant No. 05M13UKA) is gratefully acknowledged.


  1. 1.
    Absil, P.-A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)CrossRefGoogle Scholar
  2. 2.
    Alouges, F.: A new algorithm for computing liquid crystal stable configurations: the harmonic mapping case. SIAM J. Numer. Anal. 34(5), 1708–1726 (1997)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Arsigny, V., Pennec, X., Ayache, N.: Bi-invariant means in Lie groups. Application to left-invariant polyaffine transformations. HAL Preprint, 00071383 (2006)Google Scholar
  4. 4.
    Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss–Seidel methods. Math. Program. 137(1), 91–129 (2013)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Bačák, M.: Convex Analysis and Optimization in Hadamard Spaces. De Gruyter Series in Nonlinear Analysis and Applications, vol. 22. De Gruyter, Berlin (2014)zbMATHGoogle Scholar
  6. 6.
    Bačák, M., Bergmann, R., Steidl, G., Weinmann, A.: A second order non-smooth variational model for restoring manifold-valued images. SIAM J. Sci. Comput. 38(1), A567–A597 (2016)CrossRefGoogle Scholar
  7. 7.
    Bachmann, F., Hielscher, R.: MTEX—MATLAB toolbox for quantitative texture analysis., 2005–2016
  8. 8.
    Bachmann, F., Hielscher, R., Jupp, P.E., Pantleon, W., Schaeben, H., Wegert, E.: Inferential statistics of electron backscatter diffraction data from within individual crystalline grains. J. Appl. Crystallogr. 43, 1338–1355 (2010)CrossRefGoogle Scholar
  9. 9.
    Balle, F., Beck, T., Eifler, D., Fitschen, J.H., Schuff, S., Steidl, G.: Strain analysis by a total generalized variation regularized optical flow model. Inverse Probl. Sci. Eng. (2018). CrossRefGoogle Scholar
  10. 10.
    Balle, F., Eifler, D., Fitschen, J.H., Schuff, S., Steidl, G.: Computation and visualization of local deformation for multiphase metallic materials by infimal convolution of TV-type functionals. In: SSVM 2015, Lecture Notes in Computer Science, pp. 385–396. Springer (2015)Google Scholar
  11. 11.
    Bamler, R., Hartl, P.: Synthetic aperture radar interferometry. Inverse Probl. 14(4), R1–R54 (1998)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Bergmann, R., Chan, R.H., Hielscher, R., Persch, J., Steidl, G.: Restoration of manifold-valued images by half-quadratic minimization. Inverse Probl. Imaging 10(2), 281–304 (2016)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Bergmann, R., Fitschen, J.H., Persch, J., Steidl, G.: Infimal convolution coupling of first and second order differences on manifold-valued images. In: Lauze, F., Dong, Y., Dahl, A.B. (eds) Scale Space and Variational Methods in Computer Vision: 6th International Conference, SSVM 2017, Kolding, Denmark, 4–8 June 2017, Proceedings, pp. 447–459. Springer, Cham (2017)Google Scholar
  14. 14.
    Bergmann, R., Laus, F., Steidl, G., Weinmann, A.: Second order differences of cyclic data and applications in variational denoising. SIAM J. Imaging Sci. 7(4), 2916–2953 (2014)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Bergmann, R., Tenbrinck, D.: A graph framework for manifold-valued data. arXiv Preprint arXiv:1702.05293 (2017)
  16. 16.
    Bergmann, R., Weinmann, A.: Inpainting of cyclic data using first and second order differences. In: Tai, X.C., Bae, E., Chan, T.F., Lysaker, M. (eds.) Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 155–168. Springer, Cham (2015)Google Scholar
  17. 17.
    Bergmann, R., Weinmann, A.: A second order TV-type approach for inpainting and denoising higher dimensional combined cyclic and vector space data. J. Math. Imaging Vis. 55(3), 401–427 (2016)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Bredies, K.: Recovering piecewise smooth multichannel images by minimization of convex functionals with total generalized variation penalty. In: Bruhn, A., Pock, T., Tai, X.-C. (eds.) Efficient Algorithms for Global Optimization Methods in Computer Vision, pp. 44–77. Springer, Berlin (2014)CrossRefGoogle Scholar
  19. 19.
    Bredies, K., Holler, M.: Regularization of linear inverse problems with total generalized variation. J. Inverse Ill-posed Probl. 22(6), 871–913 (2014)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Bredies, K., Holler, M., Storath, M., Weinmann, A.: Total generalized variation for manifold-valued data. Preprint arXiv:1709.01616 (2017)
  21. 21.
    Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Bredies, K., Sun, H.P.: Preconditioned Douglas–Rachford algorithms for TV- and TGV-regularized variational imaging problems. J. Math. Imaging Vis. 52(3), 317–344 (2015)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Bredies, K., Valkonen, T.: Inverse problems with second order total generalized variation constraints. In: International Conference on Sampling Theory and Applications (2011)Google Scholar
  24. 24.
    Burger, M., Sawatzky, A., Steidl, G.: First order algorithms in variational image processing. In: Glowinski, R., Osher, S., Yin, W. (eds.) Operator Splittings and Alternating Direction Methods. Springer, Cham (2016)Google Scholar
  25. 25.
    Bürgmann, R., Rosen, P.A., Fielding, E.J.: Synthetic aperture radar interferometry to measure Earth’s surface topography and its deformation. Annu. Rev. Earth Planet. Sci. 28(1), 169–209 (2000)CrossRefGoogle Scholar
  26. 26.
    Chambolle, A., Lions, P.-L.: Image recovery via total variation minimization and related problems. Numer. Math. 76(2), 167–188 (1997)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Cremers, D., Strekalovskiy, E.: Total cyclic variation and generalizations. J. Math. Imaging Vis. 47(3), 258–277 (2013)MathSciNetCrossRefGoogle Scholar
  28. 28.
    do Carmo, M.P.: Riemannian Geometry. Translated by F. Flatherty, vol. 115. Birkhäuser, Basel (1992)CrossRefGoogle Scholar
  29. 29.
    Ehlers, J., Pirani, F.A.E., Schild, A.: The geometry of free fall and light propagation. In: O’Reifeartaigh, L. (ed.) General Relativity, pp. 63–84. Oxford University Press, Oxford (1972)Google Scholar
  30. 30.
    Fitschen J.H.: Variational models in image processing with applications in the materials sciences. Dissertation, University of Kaiserslautern, 2017. Similarly: Verlag Dr. Hut, ISBN 978-3843932455 (2017)Google Scholar
  31. 31.
    Fletcher, P., Joshi, S.: Riemannian geometry for the statistical analysis of diffusion tensor data. Signal Process. 87, 250–262 (2007)CrossRefGoogle Scholar
  32. 32.
    Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2, 17–40 (1976)CrossRefGoogle Scholar
  33. 33.
    Gallier, J., Quaintance, J.: Notes on Differential Geometry and Lie Groups (2017)Google Scholar
  34. 34.
    Giaquinta, M., Modica, G., Souček, J.: Variational problems for maps of bounded variation with values in \(S^1\). Calc. Var. 1(1), 87–121 (1993)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Giaquinta, M., Mucci, D.: The BV-energy of maps into a manifold: relaxation and density results. Ann. Sc. Norm. Super. Pisa Cl. Sci. 5(4), 483–548 (2006)MathSciNetzbMATHGoogle Scholar
  36. 36.
    Giaquinta, M., Mucci, D.: Maps of bounded variation with values into a manifold: total variation and relaxed energy. Pure Appl. Math. Q. 3(2), 513–538 (2007)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Glowinski, R., Marroco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. Revue française d’automatique, informatique, recherche opérationnelle. Anal. Numer. 9(2), 41–76 (1975)MathSciNetzbMATHGoogle Scholar
  38. 38.
    Gräf, M.: A unified approach to scattered data approximation on \(\mathbb{S}^{3}\) and SO(3). Adv. Comput. Math. 37(3), 379–392 (2012)MathSciNetCrossRefGoogle Scholar
  39. 39.
    Gupta, V.K., Agnew, S.R.: A simple algorithm to eliminate ambiguities in EBSD orientation map visualization and analyses: application to fatigue crack-tips/wakes in aluminum alloys. Microsc. Microanal. 16, 831 (2010)CrossRefGoogle Scholar
  40. 40.
    Holler, M., Kunisch, K.: On infimal convolution of TV-type functionals and applications to video and image reconstruction. SIAM J. Imaging Sci. 7(4), 2258–2300 (2014)MathSciNetCrossRefGoogle Scholar
  41. 41.
    Hopf, H., Rinow, W.: Ueber den begriff der vollständigen differentialgeometrischen fläche. Comment. Math. Helv. 3(1), 209–225 (1931)MathSciNetCrossRefGoogle Scholar
  42. 42.
    Hosseini, S., Uschmajew, A.: A Riemannian gradient sampling algorithm for nonsmooth optimization on manifolds. SIAM J. Optim. 27(1), 173–189 (2017)MathSciNetCrossRefGoogle Scholar
  43. 43.
    Jarre, F.: Convex analysis on symmetric matrices. In: Wolkowicz, H., Saigal, R., Vandenberghe, L. (eds.) Handbook of Semidefinite Programming. Kluwer Academic Publishers, Alphen aan den Rijn (2000)zbMATHGoogle Scholar
  44. 44.
    Kheyfets, A., Miller, W.A., Newton, G.A.: Schild’s ladder parallel transport procedure for an arbitrary connection. Int. J. Theor. Phys. 39(12), 2891–2898 (2000)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Laus, F., Nikolova, M., Persch, J., Steidl, G.: A nonlocal denoising algorithm for manifold-valued images using second order statistics. SIAM J. Imaging Sci. 10(1), 416–448 (2017)MathSciNetCrossRefGoogle Scholar
  46. 46.
    Lellmann, J., Strekalovskiy, E., Koetter, S., Cremers, D.: Total variation regularization for functions with values in a manifold. IEEE ICCV 2013, 2944–2951 (2013)Google Scholar
  47. 47.
    Li, G., Pong, T.K.: Global convergence of splitting methods for nonconvex composite optimization. Preprint arXiv:1407.0753 (2014)
  48. 48.
    Lorenzi, M., Pennec, X.: Efficient parallel transport of deformations in time series of images: from Schild’s to pole ladder. J. Math. Imaging Vis. 50(1), 5–17 (2014)CrossRefGoogle Scholar
  49. 49.
    Nash, J.: The imbedding problem for Riemannian manifolds. Ann. Math. 63(1), 20–63 (1956)MathSciNetCrossRefGoogle Scholar
  50. 50.
    Papafitsoros, K., Schönlieb, C.B.: A combined first and second order variational approach for image reconstruction. J. Math. Imaging Vis. 2(48), 308–338 (2014)MathSciNetCrossRefGoogle Scholar
  51. 51.
    Pennec, X.: Parallel transport with pole ladder: a third order scheme in affine connection spaces which is exact in affine symmetric spaces. Preprint arXiv:1805.11436 (2018)
  52. 52.
    Persch, J.: Optimization methods in manifold-valued image processing. Ph.D. thesis, TU Kaiserslautern (2018)Google Scholar
  53. 53.
    Rentmeesters, Q.: A gradient method for geodesic data fitting on some symmetric Riemannian manifolds. In: 50th IEEE Conference on Decision and Control and European Control Conference 2011, pp. 7141–7146 (2011)Google Scholar
  54. 54.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)CrossRefGoogle Scholar
  55. 55.
    Rosman, G., Tai, X.-C., Kimmel, R., Bruckstein, A.M.: Augmented-Lagrangian regularization of matrix-valued maps. Methods Appl. Anal. 21(1), 121–138 (2014)MathSciNetzbMATHGoogle Scholar
  56. 56.
    Rosman, G., Wang, Y., Tai, X.-C., Kimmel, R., Bruckstein, A.M.: Fast regularization of matrix-valued images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) Efficient Algorithms for Global Optimization Methods in Computer Vision, pp. 19–43. Springer, Berlin (2014)CrossRefGoogle Scholar
  57. 57.
    Rossmann, W.: Lie Groups. Oxford Science Publications, Oxford (2003)zbMATHGoogle Scholar
  58. 58.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenom. 60(1), 259–268 (1992)MathSciNetCrossRefGoogle Scholar
  59. 59.
    Sasaki, S.: On the differential geometry of tangent bundles of Riemannian manifolds. Tohoku Math. J. Second Ser. 10(3), 338–354 (1958)MathSciNetCrossRefGoogle Scholar
  60. 60.
    Setzer, S., Steidl, G.: Variational methods with higher order derivatives in image processing. Approx. XII San Antonio 2007, 360–385 (2008)MathSciNetzbMATHGoogle Scholar
  61. 61.
    Setzer, S., Steidl, G., Teuber, T.: Infimal convolution regularizations with discrete \(\ell _1\)-type functionals. Commun. Math. Sci. 9(3), 797–827 (2011)MathSciNetCrossRefGoogle Scholar
  62. 62.
    Steidl, G., Setzer, S., Popilka, B., Burgeth, B.: Restoration of matrix fields by second order cone programming. Computing 81, 161–178 (2007)MathSciNetCrossRefGoogle Scholar
  63. 63.
    Strekalovskiy, E., Cremers, D.: Total variation for cyclic structures: convex relaxation and efficient minimization. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1905–1911 (2011)Google Scholar
  64. 64.
    Sun, S., Adams, B., King, W.: Observation of lattice curvature near the interface of a deformed aluminium bicrystal. Philos. Mag. A 80, 9–25 (2000)CrossRefGoogle Scholar
  65. 65.
    Valkonen, T., Bredies, K., Knoll, F.: Total generalized variation in diffusion tensor imaging. SIAM J. Imaging Sci. 6(1), 487–525 (2013)MathSciNetCrossRefGoogle Scholar
  66. 66.
    Wang, Y., Yin, W., Zeng, J.: Global convergence of ADMM in nonconvex nonsmooth optimization. ArXiv preprint arXiv:1511.06324 (2015)
  67. 67.
    Weinmann, A., Demaret, L., Storath, M.: Total variation regularization for manifold-valued data. SIAM J. Imaging Sci. 7(4), 2226–2257 (2014)MathSciNetCrossRefGoogle Scholar
  68. 68.
    Whitney, H.: Differentiable manifolds. Ann. Math. 37(3), 645–680 (1936)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Ronny Bergmann
    • 1
  • Jan Henrik Fitschen
    • 1
  • Johannes Persch
    • 1
  • Gabriele Steidl
    • 1
    • 2
  1. 1.Departement of MathematicsTechnische Universität KaiserslauternKaiserslauternGermany
  2. 2.Fraunhofer ITWMKaiserslauternGermany

Personalised recommendations