Journal of Mathematical Imaging and Vision

, Volume 52, Issue 3, pp 317–344 | Cite as

Preconditioned Douglas–Rachford Algorithms for TV- and TGV-Regularized Variational Imaging Problems

  • Kristian BrediesEmail author
  • Hong Peng Sun


The recently introduced preconditioned Douglas–Rachford iteration (PDR) for convex–concave saddle-point problems is studied with respect to convergence rates and applied to variational imaging problems with total variation (TV) and total generalized variation (TGV) penalty. A rate of \({\mathcal {O}}(1/k)\) for restricted primal–dual gaps evaluated for ergodic sequences generated by the PDR iteration is established. Based on PDR, new fast iterative algorithms for TV-denoising, TV-deblurring, and TGV-denoising of second order with \(L^2\) and \(L^1\) discrepancy are proposed. While for denoising, symmetric (block) Red–Black Gauss–Seidel preconditioners are effective, fast Fourier transform-based preconditioners are employed for the deblurring problems. Finally, for the \(L^2\)-TGV-denoising problem, an effective modified primal–dual gap is developed which may serve as a stopping criterion. All algorithms are tested and compared in numerical experiments. In particular, for problems where strong convexity does not hold, it turns out that the proposed preconditioning techniques are beneficial and lead to competitive results.


Preconditioned Douglas–Rachford iteration Primal–dual algorithms Variational image denoising and deblurring Total generalized variation Block preconditioner 



The work of Kristian Bredies and Hongpeng Sun is supported by the Austrian Science Fund (FWF) under grant SFB32 (SFB “Mathematical Optimization and Applications in the Biomedical Sciences”). The Institute for Mathematics and Scientific Computing at the University of Graz is a member of NAWI Graz (


  1. 1.
    Boţ, R.I., Hendrich, C.: A Douglas-Rachford type primal–dual method for solving inclusions with mixtures of composite and parallel-sum type monotone operators. SIAM J. Optim. 23(4), 2541–2565 (2013a)CrossRefzbMATHMathSciNetGoogle Scholar
  2. 2.
    Boţ, R.I., Hendrich, C.: Solving monotone inclusions involving parallel sums of linear composed maximal monotone operators. arXiv:1306.3191 (2013b)
  3. 3.
    Bredies, K.: Recovering piecewise smooth multichannel images by minimization of convex functionals with total generalized variation penalty. In: Bruhn, A., Pock, T., Tai, X.C. (eds.) Efficient Algorithms for Global Optimization Methods in Computer Vision. Lecture Notes in Computer Science, pp. 44–77. Springer, Berlin (2014)CrossRefGoogle Scholar
  4. 4.
    Bredies, K., Holler, M.: Artifact-free JPEG decompression with total generalized variation. In: Proceedings of VISAPP 2012—International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, pp. 12–21 (2012)Google Scholar
  5. 5.
    Bredies, K., Sun, H.P.: Preconditioned Douglas–Rachford splitting methods for convex–concave saddle-point problems. SIAM J. Numer. Anal. (2014, in press)Google Scholar
  6. 6.
    Bredies, K., Valkonen, T.: Inverse problems with second-order total generalized variation constraints. In: Proceedings of SampTA 2011—9th International Conference on Sampling Theory and Applications, Singapore (2011)Google Scholar
  7. 7.
    Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Briceño-Arias, L., Combettes, P.L.: A monotone+skew splitting model for composite monotone inclusions in duality. SIAM J. Optim. 21(4), 1230–1250 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Chambolle, A., Pock, T.: A first-order primal–dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55(1–3), 293–318 (1992)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Ekeland, I., Témam, R.: Convex Analysis and Variational Problems. No. 1 in Classics in Applied Mathematics, Society for Industrial and Applied Mathematics (1999)Google Scholar
  12. 12.
    Esser, E.: Applications of Lagrangian-based alternating direction methods and connections to split Bregman. Tech. rep., CAM report 09–2009, UCLA (2009)Google Scholar
  13. 13.
    Farrall, A.: Human brain MRI. (2011). Accessed 07 Feb 2014
  14. 14.
    Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, Studies in Mathematics and its Applications, vol. 15, pp. 299–331. Elsevier, Amsterdam (1983)Google Scholar
  15. 15.
    Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)CrossRefzbMATHGoogle Scholar
  16. 16.
    Goldstein, T., Osher, S.: The split Bregman method for \(L^1\)-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)Google Scholar
  17. 17.
    Ito, K., Kunisch, K.: Lagrange Multiplier Approach to Variational Problems and Applications. Society for Industrial and Applied Mathematics, Advances in Design and Control (2008)Google Scholar
  18. 18.
    Knoll, K., Bredies, K., Pock, T., Stollberger, R.: Second order total generalized variation (TGV) for MRI. Magn. Reson. Med. 65(2), 480–491 (2011)CrossRefGoogle Scholar
  19. 19.
    Lions, P., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)CrossRefzbMATHMathSciNetGoogle Scholar
  20. 20.
    Lorenz, D.A., Pock, T.: An inertial forward-backward algorithm for monotone inclusions. arXiv:1403.3522 (2014)
  21. 21.
    Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal–dual algorithms in convex optimization. In: IEEE International Conference on Computer Vision (ICCV), 2011, pp. 1762–1769 (2011)Google Scholar
  22. 22.
    Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14(5), 877–898 (1976)CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)CrossRefzbMATHGoogle Scholar
  24. 24.
    Temam, R.: Mathematical Problems in Plasticity. Bordas (1985)Google Scholar
  25. 25.
    Thomas, J.W.: Numerical Partial Differential Equations: Conservation Laws and Elliptic Equations, Texts in Applied Mathematics, vol. 33. Springer, New York (1999)Google Scholar
  26. 26.
    Uzawa, H.: Iterative methods in concave programming. In: Arrow, K., Hurwicz, L., Uzawa, H. (eds.) Studies in Linear and Nonlinear Programming, pp. 154–165. Stanford University Press, Palo Alto (1958)Google Scholar
  27. 27.
    Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38(3), 667–681 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  28. 28.
    Valkonen, T., Bredies, K., Knoll, F.: Total generalized variation in diffusion tensor imaging. SIAM J. Imaging Sci. 6(1), 487–525 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  29. 29.
    Zhang, X.Q., Burger, M., Osher, S.: A unified primal–dual algorithm framework based on Bregman iteration. J. Sci. Comput. 46(1), 20–46 (2011)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Institute for Mathematics and Scientific ComputingUniversity of GrazGrazAustria

Personalised recommendations