Skip to main content
Log in

A Combined First and Second Order Variational Approach for Image Reconstruction

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

In this paper we study a variational problem in the space of functions of bounded Hessian. Our model constitutes a straightforward higher-order extension of the well known ROF functional (total variation minimisation) to which we add a non-smooth second order regulariser. It combines convex functions of the total variation and the total variation of the first derivatives. In what follows, we prove existence and uniqueness of minimisers of the combined model and present the numerical solution of the corresponding discretised problem by employing the split Bregman method. The paper is furnished with applications of our model to image denoising, deblurring as well as image inpainting. The obtained numerical results are compared with results obtained from total generalised variation (TGV), infimal convolution and Euler’s elastica, three other state of the art higher-order models. The numerical discussion confirms that the proposed higher-order model competes with models of its kind in avoiding the creation of undesirable artifacts and blocky-like structures in the reconstructed images—a known disadvantage of the ROF model—while being simple and efficiently numerically solvable.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill-posed problems. In: Inverse Problems, pp. 1217–1229 (1994)

    Google Scholar 

  2. Alvarez, L., Mazorra, L.: Signal and image restoration using shock filters and anisotropic diffusion. SIAM J. Numer. Anal. 31(2), 590–605 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  3. Amar, M., Cicco, V.: Relaxation of quasi-convex integrals of arbitrary order. Proc. R. Soc. Edinb., Sect. A, Math. 124(05), 927–946 (1994)

    Article  MATH  Google Scholar 

  4. Ambrosio, L., Fusco, N., Pallara, D.: Functions of Bounded Variation and Free Discontinuity Problems. Oxford University Press, Oxford (2000)

    MATH  Google Scholar 

  5. Benning, M.: Singular Regularization of Inverse Problems (2011)

    MATH  Google Scholar 

  6. Benning, M., Brune, C., Burger, M., Müller, J.: Higher-order TV methods—Enhancement via Bregman iteration. J. Sci. Comput. 54(2–3), 269–310 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  7. Benning, M., Burger, M.: Error estimates for general fidelities. Electron. Trans. Numer. Anal. 38(44–68), 77 (2011)

    MathSciNet  Google Scholar 

  8. Bergounioux, M., Piffet, L.: A second-order model for image denoising. Set-Valued Var. Anal. 18(3–4), 277–306 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  9. Bertozzi, A., Esedoglu, S., Gillette, A.: Analysis of a two-scale Cahn-Hilliard model for binary image inpainting. Multiscale Model. Simul. 6(3), 913–936 (2008)

    Article  MathSciNet  Google Scholar 

  10. Bertozzi, A.L., Greer, J.B.: Low-curvature image simplifiers: global regularity of smooth solutions and Laplacian limiting schemes. Commun. Pure Appl. Math. 57(6), 764–790 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  11. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  12. Braides, A.: Γ-Convergence for Beginners. Oxford Lecture Series in Mathematics and Its Applications. (2002)

    Book  MATH  Google Scholar 

  13. Bredies, K.: Recovering piecewise smooth multichannel images by minimization of convex functionals with total generalized variation penalty. Preprint (2012). http://math.uni-graz.at/mobis/publications/SFB-Report-2012-006.pdf

  14. Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3, 1–42 (2009)

    MathSciNet  Google Scholar 

  15. Bredies, K., Lorenz, D.: Mathematische Bildverarbeitung: Einführung in Grundlagen und moderne Theorie. Vieweg+Teubner, Wiesbaden (2011)

    Book  Google Scholar 

  16. Bredies, K., Valkonen, T.: Inverse problems with second-order total generalized variation constraints. In: Proceedings of SampTA 2011—9th International Conference on Sampling Theory and Applications, Singapore (2011)

    Google Scholar 

  17. Burger, M., He, L., Schönlieb, C.: Cahn-Hilliard inpainting and a generalization for gray value images. SIAM J. Imaging Sci. 2(4), 1129–1167 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  18. Burger, M., Osher, S.: Convergence rates of convex variational regularization. Inverse Probl. 20, 1411 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  19. Burger, M., Resmerita, E., He, L.: Error estimation for Bregman iterations and inverse scale space methods in image restoration. Computing 81(2), 109–135 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  20. Buttazzo, G., Freddi, L.: Functionals defined on measures and applications to non equi-uniformly elliptic problems. Ann. Mat. Pura Appl. 159(1), 133–149 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  21. Cai, J., Chan, R., Shen, Z.: A framelet-based image inpainting algorithm. Appl. Comput. Harmon. Anal. 24(2), 131–149 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  22. Chambolle, A., Lions, P.: Image recovery via total variation minimization and related problems. Numer. Math. 76, 167–188 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  23. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  24. Chan, T., Esedoglu, S.: Aspects of total variation regularized L 1 function approximation. SIAM J. Appl. Math. 65(5), 1817–1837 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  25. Chan, T., Kang, S., Shen, J.: Euler’s elastica and curvature-based inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2002)

    MATH  MathSciNet  Google Scholar 

  26. Chan, T., Marquina, A., Mulet, P.: High-order total variation-based image restoration. SIAM J. Sci. Comput. 22(2), 503–516 (2001)

    Article  MathSciNet  Google Scholar 

  27. Chan, T.F., Esedoglu, S., Park, F.: Image decomposition combining staircase reduction and texture extraction. J. Vis. Commun. Image Represent. 18(6), 464–486 (2007)

    Article  Google Scholar 

  28. Charbonnier, P., Blanc-Féraud, L., Aubert, G., Barlaud, M.: Two deterministic half-quadratic regularization algorithms for computed imaging. In: ICIP (2)’94, pp. 168–172 (1994)

    Google Scholar 

  29. Combettes, P., Wajs, V., et al.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2006)

    Article  MathSciNet  Google Scholar 

  30. Dal Maso, G.: Introduction to Γ-Convergence. Birkhäuser, Basel (1993)

    Book  Google Scholar 

  31. Dal Maso, G., Fonseca, I., Leoni, G., Morini, M.: A higher order model for image restoration: the one dimensional case. SIAM J. Math. Anal. 40(6), 2351–2391 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  32. Demengel, F.: Fonctions à Hessien borné. Ann. Inst. Fourier 34, 155–190 (1985)

    Article  MathSciNet  Google Scholar 

  33. Demengel, F., Temam, R.: Convex functions of a measure and applications. Indiana Univ. Math. J. 33, 673–709 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  34. Didas, S., Weickert, J., Burgeth, B.: Properties of higher order nonlinear diffusion filtering. J. Math. Imaging Vis. 35, 208–226 (2009)

    Article  MathSciNet  Google Scholar 

  35. Dong, Y., Hintermüller, M., Knoll, F., Stollberger, R.: Total variation denoising with spatially dependent regularization. In: ISMRM 18th Annual Scientific Meeting and Exhibition Proceedings, p. 5088 (2010)

    Google Scholar 

  36. Duval, V., Aujol, J., Gousseau, Y.: The TVL1 model: a geometric point of view. Multiscale Model. Simul. 8(1), 154–189 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  37. Eckstein, J.: Splitting methods for monotone operators with applications to parallel optimisation. Ph.D. Thesis (1989)

  38. Ekeland, I., Témam, R.: Convex Analysis and Variational Problems. Classics in Applied Mathematics. SIAM, SIAM (1999)

    Book  MATH  Google Scholar 

  39. Esser, E., Zhang, X., Chan, T.: A general framework for a class of first order primal-dual algorithms for TV minimization. UCLA CAM report 09-67 (2009)

  40. Evans, L.: Partial Differential Equations, 2nd edn. Graduate Studies in Mathematics, vol. 19. AMS, Providence (2010)

    MATH  Google Scholar 

  41. Feng, X., Prohl, A.: Analysis of total variation flow and its finite element approximations. Modél. Math. Anal. Numér. 37(3), 533 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  42. Frick, K., Marnitz, P., Munk, A.: Statistical multiresolution estimation in imaging: fundamental concepts and algorithmic framework. Electron. J. Stat. 6, 231–268 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  43. Frohn-Schauf, C., Henn, S., Witsch, K.: Nonlinear multigrid methods for total variation image denoising. Comput. Vis. Sci. 7(3), 199–206 (2004)

    MATH  MathSciNet  Google Scholar 

  44. Gabay, D.: Chapter IX applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems. Studies in Mathematics and Its Applications, vol. 15, pp. 299–331. Elsevier, Amsterdam (1983)

    Chapter  Google Scholar 

  45. Gilboa, G., Sochen, N., Zeevi, Y.: Image enhancement and denoising by complex diffusion processes. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 1020–1036 (2004)

    Article  Google Scholar 

  46. Goldstein, T., Osher, S.: The split Bregman method for L1 regularized problems. SIAM J. Imaging Sci. 2, 323 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  47. Green, P.J.: Bayesian reconstructions from emission tomography data using a modified EM algorithm. IEEE Trans. Med. Imaging 9(1), 84–93 (1990)

    Article  Google Scholar 

  48. Hinterberger, W., Scherzer, O.: Variational methods on the space of functions of bounded Hessian for convexification and denoising. Computing 76, 109–133 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  49. Kanizsa, G.: La Grammaire du Voir. Diderot (1996)

  50. Lai, R., Tai, X., Chan, T.: A ridge and corner preserving model for surface restoration. UCLA CAM report 11-55 (2011)

  51. Lefkimmiatis, S., Bourquard, A., Unser, M.: Hessian-based norm regularization for image restoration with biomedical applications. IEEE Trans. Image Process. 21(3), 983–995 (2012)

    Article  MathSciNet  Google Scholar 

  52. Lysaker, M., Lundervold, A., Tai, X.C.: Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time. IEEE Trans. Image Process. 12(12), 1579–1590 (2003)

    Article  Google Scholar 

  53. Lysaker, M., Tai, X.C.: Iterative image restoration combining total variation minimization and a second-order functional. Int. J. Comput. Vis. 66(1), 5–18 (2006)

    Article  Google Scholar 

  54. Nikolova, M.: Minimizers of cost-functions involving nonsmooth data-fidelity terms. Application to the processing of outliers. SIAM J. Numer. Anal. 40(3), 965–994 (2003)

    Article  MathSciNet  Google Scholar 

  55. Nikolova, M.: A variational approach to remove outliers and impulse noise. J. Math. Imaging Vis. 20(1), 99–120 (2004)

    Article  MathSciNet  Google Scholar 

  56. Osher, S., Burger, M., Goldfarb, D., Xu, J., Yin, W.: An iterative regularization method for total variation-based image restoration. Multiscale Model. Simul. 4, 460–489 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  57. Papafitsoros, K., Sengul, B., Schönlieb, C.: Combined first and second order total variation inpainting using split Bregman (2013, to appear). http://www.maths.cam.ac.uk/postgrad/cca/files/ipol.pdf

  58. Pock, T., Cremers, D., Bischof, H., Chambolle, A.: Global solutions of variational models with convex regularization. SIAM J. Imaging Sci. 3(4), 1122–1145 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  59. Pöschl, C.: Tikhonov regularization with general residual term. Ph.D. Thesis, Leopold-Franzens-Universität Innsbruck, Austria (2008)

  60. Pöschl, C., Scherzer, O.: Characterization of minimisers of convex regularization functionals. Contemp. Math. 451, 219–248 (2008)

    Article  Google Scholar 

  61. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)

    Article  MATH  Google Scholar 

  62. Scherzer, O.: Denoising with higher order derivatives of bounded variation and an application to parameter estimation. Computing 60(1), 1–27 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  63. Schönlieb, C., Bertozzi, A., Burger, M., He, L.: Image inpainting using a fourth-order total variation flow. In: Proc. Int. Conf. SampTA09, Marseilles (2009)

    Google Scholar 

  64. Setzer, S.: Split Bregman algorithm, Douglas-Rachford splitting and frame shrinkage. In: Scale Space and Variational Methods in Computer Vision, vol. 5567, pp. 464–476 (2009)

    Chapter  Google Scholar 

  65. Setzer, S., Steidl, G.: Variational methods with higher order derivatives in image processing. In: Approximation XII, pp. 360–386 (2008)

    Google Scholar 

  66. Setzer, S., Steidl, G., Teuber, T.: Infimal convolution regularizations with discrete 1-type functionals. Commun. Math. Sci. 9, 797–872 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  67. Tai, X., Hahn, J., Chung, G.: A fast algorithm for Euler’s elastica model using augmented Lagrangian method. SIAM J. Imaging Sci. 4(1), 313–344 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  68. Tikhonov, A.: Solution of incorrectly formulated problems and the regularization method. Sov. Math. Dokl. 5, 1035 (1963)

    MATH  Google Scholar 

  69. Vassilevski, P., Wade, J.G.: A comparison of multilevel methods for total variation regularization. Electron. Trans. Numer. Anal. 6, 255–270 (1997)

    MATH  MathSciNet  Google Scholar 

  70. Vese, L.: A study in the BV space of a denoising-deblurring variational problem. Appl. Math. Optim. 44(2), 131–161 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  71. Vogel, C.: A multigrid method for total variation-based image denoising. Prog. Syst. Control Theory 20, 323 (1995)

    Google Scholar 

  72. Wang, Y., Yin, W., Zhang, Y.: A fast algorithm for image deblurring with total variation regularization. CAAM technical report TR07-10, Rice University, Houston, Tex, USA (2007)

  73. Wang, Z., Bovik, A.: Mean squared error: love it or leave it? A new look at signal fidelity measures. IEEE Signal Process. Mag. 26(1), 98–117 (2009)

    Article  Google Scholar 

  74. Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  75. Whittaker, E.T.: On a new method of graduation. Proc. Edinb. Math. Soc. 41, 63–75 (1923)

    Google Scholar 

  76. Wu, C., Tai, X.: Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM J. Imaging Sci. 3(3), 300–339 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  77. Yin, W., Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for 1-minimisation with applications to compressed sensing. SIAM J. Imaging Sci. 1, 142–168 (2008)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors acknowledge the financial support provided by the Cambridge Centre for Analysis (CCA), the Royal Society International Exchanges Award IE110314 for the project “High-order Compressed Sensing for Medical Imaging”, the EPSRC/Isaac Newton Trust Small Grant “Non-smooth geometric reconstruction for high resolution MRI imaging of fluid transport in bed reactors” and the EPSRC first grant Nr. EP/J009539/1 “Sparse & Higher-order Image Restoration”. Further, this publication is based on work supported by Award No. KUK-I1-007-43, made by King Abdullah University of Science and Technology (KAUST). We thank Clarice Poon for providing us with the Euler’s elastica code. Finally, we would like to thank the referees for their very useful comments and suggestions that improved the presentation of the paper.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to K. Papafitsoros.

Appendix: Some Useful Theorems

Appendix: Some Useful Theorems

Proposition A.1

Suppose that \(g:\mathbb{R}^{m}\to\mathbb{R}\) is a continuous function, positively homogeneous of degree 1 and let \(\mu\in[\mathcal{M}(\varOmega )]^{m}\). Then for every positive measure Radon measure ν such that μ is absolutely continuous with respect to ν, we have

$$g(\mu)=g \biggl(\frac{\mu}{\nu} \biggr)\nu. $$

Moreover, if g is a convex function, then \(g:[\mathcal{M}(\varOmega )]^{m}\to\mathcal{M}(\varOmega)\) is a convex function as well.

Proof

Since μν, we have that |μ|≪ν. Using the fact that g is positively homogeneous and the fact that |μ|/ν is a positive function, we get

Assuming that g is convex and using the first part of the proposition we get for 0≤λ≤1, μ, \(\nu\in[\mathcal{M}(\varOmega)]^{m}\):

 □

The following theorem which is a special case of a theorem that was proved in [20] and can be also found in [4] establishes the lower semicontinuity of convex functionals of measures with respect to the weak convergence.

Theorem A.2

(Buttazzo and Freddi [20])

Let Ω be an open subset of \(\mathbb{R}^{n}\), ν, \((\nu _{k})_{k\in\mathbb{N}}\) be \(\mathbb{R}^{m}\)-valued finite Radon measures and μ, \((\mu_{k})_{k\in\mathbb{N}}\) be positive Radon measures in Ω. Let \(g:\mathbb{R}^{m}\to\mathbb{R}\) be a convex function and suppose that ν k ν and μ k μ weakly in Ω. Consider the Lebesgue decompositions ν=(ν/μ)μ+ν s, \(\nu_{k}=(\nu_{k}/\mu_{k})\mu_{k}+\nu_{k}^{s}\), \(k\in\mathbb{N}\). Then

In particular, if \(\mu=\mu_{k}=\mathcal{L}^{n}\) for all \(k\in\mathbb {N}\) then according to the definition (2.1) the above inequality can be written as follows:

$$g(\nu) (\varOmega)\le\liminf_{k\to\infty}g(\nu_{k}) ( \varOmega). $$

The following theorem is a special case of Theorem 2.3 in [33].

Theorem A.3

(Demengel and Temam [33])

Suppose that \(\varOmega\subseteq\mathbb{R}^{n}\) is open, with Lipschitz boundary and let g be a convex function from \(\mathbb{R}^{n\times n}\) to \(\mathbb{R}\) with at most linear growth at infinity. Then for every u∈BH(Ω) there exists a sequence \((u_{k})_{k\in\mathbb{N}}\subseteq C^{\infty }(\varOmega)\cap W^{2,1}(\varOmega)\) such that

Lemma A.4

(Kronecker’s lemma)

Suppose that \((a_{n})_{n\in\mathbb{N}}\) and \((b_{n})_{n\in\mathbb{N}}\) are two sequences of real numbers such that \(\sum_{n=1}^{\infty }a_{n}<\infty\) and 0<b 1b 2≤⋯ with b n →∞. Then

$$\frac{1}{b_{n}}\sum_{k=1}^{n}b_{k}a_{k} \to0,\quad\textit{as } n\to\infty. $$

In particular, if \((c_{n})_{n\in\mathbb{N}}\) is a decreasing positive real sequence such that \(\sum_{n=1}^{\infty}c_{n}^{2}<\infty\), then

$$c_{n}\sum_{k=1}^{n}c_{k} \to0,\quad\textit{as } n\to\infty. $$

Rights and permissions

Reprints and permissions

About this article

Cite this article

Papafitsoros, K., Schönlieb, C.B. A Combined First and Second Order Variational Approach for Image Reconstruction. J Math Imaging Vis 48, 308–338 (2014). https://doi.org/10.1007/s10851-013-0445-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-013-0445-4

Keywords

Navigation