Journal of Mathematical Imaging and Vision

, Volume 61, Issue 5, pp 571–601 | Cite as

Unified Models for Second-Order TV-Type Regularisation in Imaging: A New Perspective Based on Vector Operators

  • Eva-Maria BrinkmannEmail author
  • Martin Burger
  • Joana Sarah Grah


We introduce a novel regulariser based on the natural vector field operations gradient, divergence, curl and shear. For suitable choices of the weighting parameters contained in our model, it generalises well-known first- and second-order TV-type regularisation methods including TV, ICTV and TGV\(^2\) and enables interpolation between them. To better understand the influence of each parameter, we characterise the nullspaces of the respective regularisation functionals. Analysing the continuous model, we conclude that it is not sufficient to combine penalisation of the divergence and the curl to achieve high-quality results, but interestingly it seems crucial that the penalty functional includes at least one component of the shear or suitable boundary conditions. We investigate which requirements regarding the choice of weighting parameters yield a rotational invariant approach. To guarantee physically meaningful reconstructions, implying that conservation laws for vectorial differential operators remain valid, we need a careful discretisation that we therefore discuss in detail.


Variational methods Sparse regularisation Natural differential operators Helmholtz decomposition (Higher-order) total variation (TV) regularisation Denoising 



The authors thank Kristian Bredies, Martin Holler (both University of Graz) and Christoph Schnörr (University of Heidelberg) for useful discussions and links to literature.


  1. 1.
    Ambrosio, L., Fusco, N., Pallara, D.: Functions of Bounded Variation and Free Discontinuity Problems, vol. 254. Clarendon Press, Oxford (2000)zbMATHGoogle Scholar
  2. 2.
    Attouch, H., Brezis, H.: Duality for the sum of convex functions in general Banach spaces. In: Barroso, J.A. (eds.) North-Holland Mathematical Library, vol. 34, pp. 125–133. Elsevier (1986)Google Scholar
  3. 3.
    Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, vol. 147. Springer, Berlin (2006)zbMATHGoogle Scholar
  4. 4.
    Benning, M., Brune, C., Burger, M., Müller, J.: Higher-order TV methods—enhancement via Bregman iteration. J. Sci. Comput. 54(2–3), 269–310 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Benning, M., Burger, M.: Ground states and singular vectors of convex variational regularization methods. Methods Appl. Anal. 20(4), 295–334 (2013)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Benning, M., Burger, M.: Modern regularization methods for inverse problems. Acta Numerica 27, 1–111 (2018)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Bergounioux, M.: Poincaré-wirtinger inequalities in bounded variation function spaces. Control Cybern. 40, 921–929 (2011)zbMATHGoogle Scholar
  8. 8.
    Braides, A.: Gamma-Convergence for Beginners, vol. 22. Clarendon Press, Oxford (2002)CrossRefzbMATHGoogle Scholar
  9. 9.
    Bredies, K.: Symmetric tensor fields of bounded deformation. Annali di Matematica Pura ed Applicata 192(5), 815–851 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Bredies, K., Holler, M.: Regularization of linear inverse problems with total generalized variation. J. Inverse Ill-posed Probl. 22(6), 871–913 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Bredies, K., Valkonen, T.: Inverse problems with second-order total generalized variation constraints. Proc. SampTA 201 (2011)Google Scholar
  13. 13.
    Brinkmann, E.M., Burger, M., Grah, J.: Regularization with sparse vector fields: from image compression to TV-type reconstruction. In: Aujol, J.-F., Nikolova, M., Papadakis, N. (eds.) Scale Space and Variational Methods in Computer Vision, pp. 191–202. Springer (2015)Google Scholar
  14. 14.
    Brinkmann, E.M., Burger, M., Rasch, J., Sutour, C.: Bias reduction in variational regularization. J. Math. Imaging Vis. 59(3), 1–33 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Burger, M., Osher, S.: A guide to the TV zoo. In: Burger, M., Osher, S. (eds.) Level Set and PDE Based Reconstruction Methods in Imaging, pp. 1–70. Springer (2013)Google Scholar
  16. 16.
    Chambolle, A., Lions, P.L.: Image recovery via total variation minimization and related problems. Numerische Mathematik 76(2), 167–188 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Chan, T.F., Esedoglu, S., Park, F.: A fourth order dual method for staircase reduction in texture extraction and image restoration problems. In: 17th IEEE International Conference on Image Processing (ICIP), 2010, pp. 4137–4140. IEEE (2010)Google Scholar
  19. 19.
    Dal Maso, G.: An Introduction to \(\Gamma \)-Convergence. Springer, Berlin (2012)Google Scholar
  20. 20.
    Deledalle, C.A., Papadakis, N., Salmon, J.: On debiasing restoration algorithms: applications to total-variation and nonlocal-means. In: International Conference on Scale Space and Variational Methods in Computer Vision, pp. 129–141. Springer (2015)Google Scholar
  21. 21.
    Deledalle, C.A., Papadakis, N., Salmon, J., Vaiter, S.: CLEAR: Covariant least-square refitting with applications to image restoration. SIAM J. Imaging Sci. 10(1), 243–284 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Ekeland, I., Temam, R.: Convex Analysis and Variational Problems, vol. 28. SIAM, Philadelphia (1999)CrossRefzbMATHGoogle Scholar
  23. 23.
    Evans, L.C.: Partial Differential Equations. American Mathematical Society, Providence (1998)zbMATHGoogle Scholar
  24. 24.
    Goldstein, T., Li, M., Yuan, X., Esser, E., Baraniuk, R.: Adaptive primal-dual hybrid gradient methods for saddle-point problems (2013). arXiv:1305.0546
  25. 25.
    Haber, E.: Computational Methods in Geophysical Electromagnetics. SIAM, Philadelphia (2014)CrossRefzbMATHGoogle Scholar
  26. 26.
    Hyman, J.M., Shashkov, M.: Adjoint operators for the natural discretizations of the divergence, gradient and curl on logically rectangular grids. Appl. Numer. Math. 25(4), 413–442 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Hyman, J.M., Shashkov, M.: Natural discretizations for the divergence, gradient, and curl on logically rectangular grids. Comput. Math. Appl. 33(4), 81–104 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Mainberger, M., Bruhn, A., Weickert, J., Forchhammer, S.: Edge-based compression of cartoon-like images with homogeneous diffusion. Pattern Recognit. 44(9), 1859–1873 (2011)CrossRefGoogle Scholar
  29. 29.
    Mainberger, M., Weickert, J.: Edge-based image compression with homogeneous diffusion. In: Jiang, X., Petkov, N. (eds.) Computer Analysis of Images and Patterns, pp. 476–483. Springer (2009)Google Scholar
  30. 30.
    Osher, S., Burger, M., Goldfarb, D., Xu, J., Yin, W.: An iterative regularization method for total variation-based image restoration. Multiscale Model. Simul. 4(2), 460–489 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Raviart, P.A., Thomas, J.M.: A mixed finite element method for 2-nd order elliptic problems. In: Galligani, I., Magenes, E. (eds.) Mathematical Aspects of Finite Element Methods, pp. 292–315. Springer (1977)Google Scholar
  32. 32.
    Rockafellar, R.T.: Convex Analysis. Princeton, Princeton University Press (1972)Google Scholar
  33. 33.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1), 259–268 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Scherzer, O.: Denoising with higher order derivatives of bounded variation and an application to parameter estimation. Computing 60(1), 1–27 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Schnörr, C.: Segmentation of visual motion by minimizing convex non-quadratic functionals. In: 12th International Conference on Pattern Recognition, Jerusalem, Israel (1994), pp. 661–663 (1994)Google Scholar
  36. 36.
    Yuan, J., Schörr, C., Steidl, G.: Simultaneous higher-order optical flow estimation and decomposition. SIAM J. Sci. Comput. 29(6), 2283–2304 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Zhang, L., Wu, X., Buades, A., Li, X.: Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J. Electron. imaging 20(2), 023,016 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Applied Mathematics: Institute for Analysis and NumericsWestfälische Wilhelms-Universität MünsterMünsterGermany
  2. 2.Department MathematikFriedrich-Alexander Universität Erlangen-NürnbergErlangenGermany
  3. 3.Institute of Computer Graphics and VisionGraz University of TechnologyGrazAustria

Personalised recommendations