Skip to main content
SpringerLink
Log in

Preconditioned Douglas–Rachford Algorithms for TV- and TGV-Regularized Variational Imaging Problems

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

The recently introduced preconditioned Douglas–Rachford iteration (PDR) for convex–concave saddle-point problems is studied with respect to convergence rates and applied to variational imaging problems with total variation (TV) and total generalized variation (TGV) penalty. A rate of \({\mathcal {O}}(1/k)\) for restricted primal–dual gaps evaluated for ergodic sequences generated by the PDR iteration is established. Based on PDR, new fast iterative algorithms for TV-denoising, TV-deblurring, and TGV-denoising of second order with \(L^2\) and \(L^1\) discrepancy are proposed. While for denoising, symmetric (block) Red–Black Gauss–Seidel preconditioners are effective, fast Fourier transform-based preconditioners are employed for the deblurring problems. Finally, for the \(L^2\)-TGV-denoising problem, an effective modified primal–dual gap is developed which may serve as a stopping criterion. All algorithms are tested and compared in numerical experiments. In particular, for problems where strong convexity does not hold, it turns out that the proposed preconditioning techniques are beneficial and lead to competitive results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Boţ, R.I., Hendrich, C.: A Douglas-Rachford type primal–dual method for solving inclusions with mixtures of composite and parallel-sum type monotone operators. SIAM J. Optim. 23(4), 2541–2565 (2013a)

    Article  MATH  MathSciNet  Google Scholar 

  2. Boţ, R.I., Hendrich, C.: Solving monotone inclusions involving parallel sums of linear composed maximal monotone operators. arXiv:1306.3191 (2013b)

  3. Bredies, K.: Recovering piecewise smooth multichannel images by minimization of convex functionals with total generalized variation penalty. In: Bruhn, A., Pock, T., Tai, X.C. (eds.) Efficient Algorithms for Global Optimization Methods in Computer Vision. Lecture Notes in Computer Science, pp. 44–77. Springer, Berlin (2014)

    Chapter  Google Scholar 

  4. Bredies, K., Holler, M.: Artifact-free JPEG decompression with total generalized variation. In: Proceedings of VISAPP 2012—International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, pp. 12–21 (2012)

  5. Bredies, K., Sun, H.P.: Preconditioned Douglas–Rachford splitting methods for convex–concave saddle-point problems. SIAM J. Numer. Anal. (2014, in press)

  6. Bredies, K., Valkonen, T.: Inverse problems with second-order total generalized variation constraints. In: Proceedings of SampTA 2011—9th International Conference on Sampling Theory and Applications, Singapore (2011)

  7. Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  8. Briceño-Arias, L., Combettes, P.L.: A monotone+skew splitting model for composite monotone inclusions in duality. SIAM J. Optim. 21(4), 1230–1250 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  9. Chambolle, A., Pock, T.: A first-order primal–dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  10. Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55(1–3), 293–318 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  11. Ekeland, I., Témam, R.: Convex Analysis and Variational Problems. No. 1 in Classics in Applied Mathematics, Society for Industrial and Applied Mathematics (1999)

  12. Esser, E.: Applications of Lagrangian-based alternating direction methods and connections to split Bregman. Tech. rep., CAM report 09–2009, UCLA (2009)

  13. Farrall, A.: Human brain MRI. http://www.anatomy.mvm.ed.ac.uk/museum/explore-anatomy.php (2011). Accessed 07 Feb 2014

  14. Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, Studies in Mathematics and its Applications, vol. 15, pp. 299–331. Elsevier, Amsterdam (1983)

    Google Scholar 

  15. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)

    Article  MATH  Google Scholar 

  16. Goldstein, T., Osher, S.: The split Bregman method for \(L^1\)-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

  17. Ito, K., Kunisch, K.: Lagrange Multiplier Approach to Variational Problems and Applications. Society for Industrial and Applied Mathematics, Advances in Design and Control (2008)

  18. Knoll, K., Bredies, K., Pock, T., Stollberger, R.: Second order total generalized variation (TGV) for MRI. Magn. Reson. Med. 65(2), 480–491 (2011)

    Article  Google Scholar 

  19. Lions, P., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  20. Lorenz, D.A., Pock, T.: An inertial forward-backward algorithm for monotone inclusions. arXiv:1403.3522 (2014)

  21. Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal–dual algorithms in convex optimization. In: IEEE International Conference on Computer Vision (ICCV), 2011, pp. 1762–1769 (2011)

  22. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  23. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)

    Article  MATH  Google Scholar 

  24. Temam, R.: Mathematical Problems in Plasticity. Bordas (1985)

  25. Thomas, J.W.: Numerical Partial Differential Equations: Conservation Laws and Elliptic Equations, Texts in Applied Mathematics, vol. 33. Springer, New York (1999)

  26. Uzawa, H.: Iterative methods in concave programming. In: Arrow, K., Hurwicz, L., Uzawa, H. (eds.) Studies in Linear and Nonlinear Programming, pp. 154–165. Stanford University Press, Palo Alto (1958)

  27. Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38(3), 667–681 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  28. Valkonen, T., Bredies, K., Knoll, F.: Total generalized variation in diffusion tensor imaging. SIAM J. Imaging Sci. 6(1), 487–525 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  29. Zhang, X.Q., Burger, M., Osher, S.: A unified primal–dual algorithm framework based on Bregman iteration. J. Sci. Comput. 46(1), 20–46 (2011)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

The work of Kristian Bredies and Hongpeng Sun is supported by the Austrian Science Fund (FWF) under grant SFB32 (SFB “Mathematical Optimization and Applications in the Biomedical Sciences”). The Institute for Mathematics and Scientific Computing at the University of Graz is a member of NAWI Graz (http://www.nawigraz.at/).

Author information

Authors and Affiliations

  1. Institute for Mathematics and Scientific Computing, University of Graz, Heinrichstr. 36, 8010, Graz, Austria

    Kristian Bredies & Hong Peng Sun

Authors
  1. Kristian Bredies
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hong Peng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kristian Bredies.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bredies, K., Sun, H.P. Preconditioned Douglas–Rachford Algorithms for TV- and TGV-Regularized Variational Imaging Problems. J Math Imaging Vis 52, 317–344 (2015). https://doi.org/10.1007/s10851-015-0564-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-015-0564-1

Keywords

Search

Navigation