Functional Lifting for Variational Problems with Higher-Order Regularization

  • Benedikt Loewenhauser
  • Jan LellmannEmail author
Conference paper
Part of the Mathematics and Visualization book series (MATHVISUAL)


Variational approaches are an established paradigm in the field of image processing. The non-convexity of the functional can be addressed by functional lifting and convex relaxation techniques, which aim to solve a convex approximation of the original energy on a larger space. However, so far these approaches have been limited to first-order , gradient-based regularizers such as the total variation . In this work, we propose a way to extend functional lifting to a second-order regularizer derived from the Laplacian. We prove that it can be represented efficiently and thus allows numerical optimization. We experimentally demonstrate the usefulness on a synthetic convex denoising problem and on synthetic as well as real-world image registration problems.



This work was partially funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)—394737018 “Functional Lifting 2.0: Efficient Convexifications for Imaging and Vision”. We would like to thank Emanuel Laude and Thomas Möllenhoff for providing their library prost, which was used to solve the saddle-point formulation of the problems.


  1. 1.
    L. Ambrosio, N. Fusco, D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems (Clarendon Press, Oxford, 2000)zbMATHGoogle Scholar
  2. 2.
    K. Bredies, K. Kunisch, T. Pock, Total generalized variation. J. Imag. Sci. 3(3), 294–526 (2010)MathSciNetzbMATHGoogle Scholar
  3. 3.
    T. Brox, A. Bruhn, N. Papenberg, J. Weickert, High accuracy optical flow estimation based on a theory for warping, in European Conference on Computer Vision (Springer, Berlin, 2004), pp. 25–36zbMATHGoogle Scholar
  4. 4.
    A. Chambolle, An algorithm for total variation minimization and applications. J. Math. Imag. Vis. 20, 89–97 (2004)MathSciNetCrossRefGoogle Scholar
  5. 5.
    A. Chambolle, T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imag. Vis. 40(1), 120–145 (2011)MathSciNetCrossRefGoogle Scholar
  6. 6.
    A. Chambolle, D. Cremers, T. Pock, A convex approach to minimal partitions. J. Imag. Sci. 5(4), 1113–1158 (2012)MathSciNetCrossRefGoogle Scholar
  7. 7.
    T. Chan, A. Marquina, P. Mulet, Higher-order total variation-based image restoration. J. Sci. Comput. 22(2), 503–516 (2000)MathSciNetzbMATHGoogle Scholar
  8. 8.
    T. Chan, S. Esedoḡlu, F. Park, A. Yip, Total variation image restoration: overview and recent developments, in The Handbook of Mathematical Models in Computer Vision (Springer, Berlin, 2005)Google Scholar
  9. 9.
    F. Demengel, Fonctions à Hessien borné. Ann. Inst. Fourier 34, 155–190 (1985)CrossRefGoogle Scholar
  10. 10.
    B. Fischer, J. Modersitzki, Curvature based image registration. J. Math. Imag. Vis. 18(1), 81–85 (2003)MathSciNetCrossRefGoogle Scholar
  11. 11.
    W.J. Fu, Penalized regressions: the bridge versus the lasso. J. Comput. Graph. Stat. 7(3), 397–416 (1998)MathSciNetGoogle Scholar
  12. 12.
    M. Grant, S. Boyd, Graph implementations for nonsmooth convex programs, in Recent Advances in Learning and Control (Springer, Berlin, 2008), pp. 95–110CrossRefGoogle Scholar
  13. 13.
    M. Grant, S. Boyd, CVX: matlab software for disciplined convex programming (2014).
  14. 14.
    S. Henn, A full curvature based algorithm for image registration. J. Math. Imag. Vis. 24(2), 195–208 (2006)MathSciNetCrossRefGoogle Scholar
  15. 15.
    W. Hinterberger, O. Scherzer, Variational methods on the space of functions of bounded Hessian for convexification and denoising. Computing 76(1), 109–133 (2006)MathSciNetCrossRefGoogle Scholar
  16. 16.
    B.K.P. Horn, B.G. Schunck, Determining optical flow. Artif. Intell. 17, 185–203 (1981)CrossRefGoogle Scholar
  17. 17.
    H. Ishikawa, Exact optimization for Markov random fields with convex priors. Pattern. Anal. Mach. Intell. 25(10), 1333–1336 (2003)CrossRefGoogle Scholar
  18. 18.
    E. Laude, T. Möllenhoff, M. Moeller, J. Lellmann, D. Cremers, Sublabel-accurate convex relaxation of vectorial multilabel energies, in European Conference on Computer Vision (Springer, Berlin, 2016), pp. 614–627Google Scholar
  19. 19.
    J. Lellmann, C. Schnörr, Continuous multiclass labeling approaches and algorithms. SIAM J. Imag. Sci. 4(4), 1049–1096 (2011)MathSciNetCrossRefGoogle Scholar
  20. 20.
    J. Lellmann, J. Kappes, J. Yuan, F. Becker, C. Schnörr, Convex multi-class image labeling by simplex-constrained total variation, in Scale Space and Variational Methods in Computer Vision. Lecture Notes in Computer Science, vol. 5567 (2009), pp. 150–162Google Scholar
  21. 21.
    J. Lellmann, E. Strekalovskiy, S. Koetter, D. Cremers, Total variation regularization for functions with values in a manifold, in International Conference on Computer Vision (2013), pp. 2944–2951Google Scholar
  22. 22.
    M. Lysaker, X.C. Tai, Iterative image restoration combining total variation minimization and a second-order functional. Int. J. Comput. Vis. 66(1), 5–18 (2006)CrossRefGoogle Scholar
  23. 23.
    M. Lysaker, A. Lundervold, X.C. Tai, Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time. IEEE Trans. Image Process. 12(12), 1579–1590 (2003)CrossRefGoogle Scholar
  24. 24.
    J. Modersitzki, Numerical Methods for Image Registration (Oxford University Press on Demand, Oxford, 2004)zbMATHGoogle Scholar
  25. 25.
    J. Modersitzki, FAIR: Flexible Algorithms for Image Registration (SIAM, Philadelphia, 2009)CrossRefGoogle Scholar
  26. 26.
    T. Möllenhoff, E. Strekalovskiy, M. Moeller, D. Cremers, The primal-dual hybrid gradient method for semiconvex splittings. SIAM J. Imag. Sci. 8(2), 827–857 (2015)MathSciNetCrossRefGoogle Scholar
  27. 27.
    T. Möllenhoff, E. Strekalovskiy, M. Möller, D. Cremers, Low rank priors for color image regularization, in International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition (Springer, Berlin, 2015), pp. 126–140Google Scholar
  28. 28.
    T. Möllenhoff, E. Laude, M. Moeller, J. Lellmann, D. Cremers, Sublabel-accurate relaxation of nonconvex energies, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 3948–3956Google Scholar
  29. 29.
    M. Nikolova, Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares. Multiscale Model. Simul. 4(3), 960–991 (2005)MathSciNetCrossRefGoogle Scholar
  30. 30.
    J. Nocedal, S.J. Wright, Numerical Optimization (Springer, Berlin, 2006)zbMATHGoogle Scholar
  31. 31.
    K. Papafitsoros, C.B. Schönlieb, A combined first and second order variational approach for image reconstruction. J. Math. Imag. Vision 48(2), 308–338 (2014)MathSciNetCrossRefGoogle Scholar
  32. 32.
    T. Pock, A. Chambolle, D. Cremers, H. Bischof, A convex relaxation approach for computing minimal partitions, in Computer Vision and Pattern Recognition (2009)Google Scholar
  33. 33.
    T. Pock, D. Cremers, H. Bischof, A. Chambolle, Global solutions of variational models with convex regularization. J. Imag. Sci. 3(4), 1122–1145 (2010)MathSciNetCrossRefGoogle Scholar
  34. 34.
    R.T. Rockafellar, Convex Analysis (Princeton University Press, Princeton, 1970)CrossRefGoogle Scholar
  35. 35.
    L. Rudin, S. Osher, E. Fatemi, Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)MathSciNetCrossRefGoogle Scholar
  36. 36.
    S. Setzer, G. Steidl, T. Teuber, Infimal convolution regularizations with discrete l1-type functionals. Commun. Math. Sci 9(3), 797–872 (2011)MathSciNetCrossRefGoogle Scholar
  37. 37.
    P. Suquet, Existence et régularité des solutions des équations de la plasticité parfaite. C. R. Acad. Sci. Paris, Ser. A 286, 1201–1204 (1978)Google Scholar
  38. 38.
    J. Yuan, E. Bae, X.C. Tai, Y. Boykov, A continuous max-flow approach to Potts model, in European Conference on Computer Vision (2010), pp. 379–392Google Scholar
  39. 39.
    C. Zach, D. Gallup, J.M. Frahm, M. Niethammer, Fast global labeling for real-time stereo using multiple plane sweeps, in Vision, Modeling, and Visualization (2008)Google Scholar
  40. 40.
    C. Zach, C. Häne, M. Pollefeys, What is optimized in convex relaxations for multilabel problems: connecting discrete and continuously inspired map inference. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 157–170 (2014)CrossRefGoogle Scholar
  41. 41.
    W.P. Ziemer, Weakly Differentiable Functions (Springer, Berlin, 1989)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute of Mathematics and Image Computing (MIC)University of LübeckLübeckGermany
  2. 2.Technical University of MunichGarchingGermany
  3. 3.Institute of Mathematics and Image Computing (MIC)University of LübeckLübeckGermany

Personalised recommendations