Skip to main content
SpringerLink
Log in

Douglas–Rachford Splitting for the Sum of a Lipschitz Continuous and a Strongly Monotone Operator

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

The Douglas–Rachford method is a popular splitting technique for finding a zero of the sum of two subdifferential operators of proper, closed, and convex functions and, more generally, two maximally monotone operators. Recent results concerned with linear rates of convergence of the method require additional properties of the underlying monotone operators, such as strong monotonicity and cocoercivity. In this paper, we study the case, when one operator is Lipschitz continuous but not necessarily a subdifferential operator and the other operator is strongly monotone. This situation arises in optimization methods based on primal–dual approaches. We provide new linear convergence results in this setting.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. This property also means that \(R_A\) is hypomonotone; see [11, Example 12.28].

  2. This is also known as negative averagedness of the operator \(T_2T_1\) [20].

  3. A closed function is also known as lower semicontinuous.

  4. Here and elsewhere, we use \(f^{*}\) to denote the convex conjugate (this is also known as the Fenchel or Legendre conjugate) of f defined at \(u\in X\) as \(f^*(u) =\sup _{x\in X}\left\{ \langle u,x\rangle -f(x)\right\} \).

References

  1. Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  2. Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, pp. 299–331. North-Holland, Amsterdam (1983)

    Chapter  Google Scholar 

  3. Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Prog. 55, 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  4. O’Connor, D., Vandenberghe, L.: On the equivalence of the primal-dual hybrid gradient method and Douglas–Rachford splitting. Math. Prog. Ser. A. https://doi.org/10.1007/s10107-018-1321-1

  5. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edn. Springer, New York (2017)

    Book  MATH  Google Scholar 

  6. Brezis, H.: Opérateurs Maximaux Monotones et Semi-Groupes de Contractions dans les Espaces de Hilbert. North-Holland/Elsevier, Amsterdam (1973)

    MATH  Google Scholar 

  7. Burachik, R.S., Iusem, A.N.: Set-Valued Mappings and Enlargements of Monotone Operators. Springer, New York (2008)

    MATH  Google Scholar 

  8. Combettes, P.L.: The convex feasibility problem in image recovery. Adv. Imag. Electron Phys. 25, 155–270 (1995)

    Google Scholar 

  9. Simons, S.: Minimax and Monotonicity. Springer, Berlin (1998)

    Book  MATH  Google Scholar 

  10. Simons, S.: From Hahn–Banach to Monotonicity. Springer, Dordrecht (2008)

    MATH  Google Scholar 

  11. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis, 3rd edn. Springer, Berlin (2009)

    MATH  Google Scholar 

  12. Zeidler, E.: Nonlinear Functional Analysis and Its Applications II/A: Linear Monotone Operators. Springer, New York (1990)

    Book  MATH  Google Scholar 

  13. Zeidler, E.: Nonlinear Functional Analysis and Its Applications II/B: Nonlinear Monotone Operators. Springer, New York (1990)

    Book  MATH  Google Scholar 

  14. Borwein, J.M.: Fifty years of maximal monotonicity. Optim. Lett. 4, 473–490 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  15. Ryu, E.K., Boyd, S.: A primer on monotone operator methods. Appl. Comput. Math. 15, 3–43 (2016)

    MathSciNet  MATH  Google Scholar 

  16. Bauschke, H.H., Cruz, J.Y.B., Nghia, T.T.A., Phan, H.M., Wang, X.: The rate of linear convergence of the Douglas–Rachford algorithm for subspaces is the cosine of the Friedrichs angle. J. Approx. Theory 185, 63–79 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  17. Bauschke, H.H., Moursi, W.M.: The Douglas–Rachford algorithm for two (not necessarily intersecting) affine subspaces. SIAM J. Optim. 26, 968–985 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  18. Phan, H.M.: Linear convergence of the Douglas–Rachford method for two closed sets. Optimization 65, 36–385 (2016)

    Article  MathSciNet  Google Scholar 

  19. Lorenz, D.A., Tran-Dinh, Q.: Non-stationary Douglas–Rachford and alternating direction method of multipliers: adaptive stepsizes and convergence. arXiv:1801.03765

  20. Giselsson, P.: Tight global linear convergence rate bounds for Douglas–Rachford splitting. J. Fixed Point Theory Appl. 19, 2241–2270 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  21. Giselsson, P., Boyd, S.: Linear convergence and metric selection for Douglas–Rachford splitting and ADMM. IIEEE Trans. Autom. Control 62, 532–544 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  22. Demanet, L., Zhang, X.: Eventual linear convergence of the Douglas–Rachford iteration for basis pursuit. Math. Comput. AMS 85, 209–238 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  23. Liang, J., Fadili, J., Peyré, G.: Local convergence properties of Douglas–Rachford and alternating direction method of multipliers. J. Optim. Theory Appl. 172, 874–913 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  24. Aspelmeier, T., Charitha, C., Luke, D.R.: Local linear convergence of the ADMM/Douglas–Rachford algorithms without strong convexity and application to statistical imaging. SIAM J. Imag. Sci. 9, 842–868 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  25. Hesse, R., Luke, D.R.: Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems. SIAM J. Optim. 23, 2397–2419 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  26. Hesse, R., Luke, D.R., Neumann, P.: Alternating projections and Douglas–Rachford for sparse affine feasibility. IEEE Trans. Signal Proc. 62, 4868–4881 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  27. Boţ, R.I., Hendrich, C.: A Douglas–Rachford type primal-dual method for solving inclusions with mixtures of composite and parallel-sum type monotone operators. SIAM J. Optim. 23, 2541–2565 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  28. Briceño-Arias, L.M., Combettes, P.L.: A monotone + skew splitting model for composite monotone inclusions in duality. SIAM J. Optim. 21, 1230–1250 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  29. Combettes, P.L., Pesquet, J.-C.: Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set Valued Var. Anal. 20, 307–330 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  30. Condat, L.: A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158, 460–479 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  31. O’Connor, D., Vandenberghe, L.: Primal-dual decomposition by operator splitting and applications to image deblurring. SIAM J. Imag. Sc. 7, 1724–1754 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  32. Combettes, P.L.: Solving monotone inclusions via compositions of nonexpansive averaged operators. Optimization 53, 475–504 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  33. Svaiter, B.F.: On weak convergence of the Douglas–Rachford method. SIAM J. Control Optim. 49, 280–287 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  34. Goebel, K., Kirk, W.A.: Topics in Metric Fixed Point Theory. Cambridge University Press, Cambridge (1990)

    Book  MATH  Google Scholar 

  35. Wolfram Research, Inc., Mathematica, Version 11.3, Champaign, IL (2018)

  36. Rockafellar, R.T.: On the maximal monotonicity of subdifferential mappings. Pac. J. Math. 33, 209–216 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  37. Baillon, J.-B., Haddad, G.: Quelques propriétés des opérateurs angle-bornés et n-cycliquement monotones. Israel J. Math. 26, 137–150 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  38. Bauschke, H.H., Combettes, P.L.: The Baillon–Haddad theorem revisited. J. Convex Anal. 17, 781–787 (2010)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was done while the authors were visiting the Simons Institute for the Theory of Computing. It was partially supported by the DIMACS/Simons Collaboration on Bridging Continuous and Discrete Optimization through NSF Grant # CCF-1740425.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Stanford University, Stanford, CA, 94305, USA

    Walaa M. Moursi

  2. Mathematics Department, Faculty of Science, Mansoura University, Mansoura, 35516, Egypt

    Walaa M. Moursi

  3. Electrical and Computer Engineering Department, University of California Los Angeles, Los Angeles, CA, 90095, USA

    Lieven Vandenberghe

Authors
  1. Walaa M. Moursi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lieven Vandenberghe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Walaa M. Moursi.

Additional information

Communicated by Jalal Fadili.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moursi, W.M., Vandenberghe, L. Douglas–Rachford Splitting for the Sum of a Lipschitz Continuous and a Strongly Monotone Operator. J Optim Theory Appl 183, 179–198 (2019). https://doi.org/10.1007/s10957-019-01517-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-019-01517-8

Keywords

Mathematics Subject Classification

Search

Navigation