Skip to main content
SpringerLink
Log in

A Lyapunov-type approach to convergence of the Douglas–Rachford algorithm for a nonconvex setting

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

The Douglas–Rachford projection algorithm is an iterative method used to find a point in the intersection of closed constraint sets. The algorithm has been experimentally observed to solve various nonconvex feasibility problems; an observation which current theory cannot sufficiently explain. In this paper, we prove convergence of the Douglas–Rachford algorithm in a potentially nonconvex setting. Our analysis relies on the existence of a Lyapunov-type functional whose convexity properties are not tantamount to convexity of the original constraint sets. Moreover, we provide various nonconvex examples in which our framework proves global convergence of the algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Aragón Artacho, F.J., Borwein, J.M.: Global convergence of a nonconvex Douglas–Rachford iteration. J. Glob. Optim. 57(3), 753–769 (2013)

    Article  MATH  Google Scholar 

  2. Aragón Artacho, F.J., Borwein, J.M., Tam, M.K.: Global behavior of the Douglas–Rachford method for a nonconvex feasibility problem. J. Glob. Optim. 65(2), 309–327 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  3. Barnsley, M.F.: Fractals Everywhere, 2nd edn. Morgen Kaufman, Burlington (1993)

    MATH  Google Scholar 

  4. Bauschke, H.H., Bello Cruz, J.Y., Nghia, T.T.A., Phan, H.M., Wang, X.: The rate of linear convergence of the Douglas–Rachford algorithm for subspaces is the cosine of the Friedrichs angle. J. Approx. Theory 185, 63–79 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011)

    Book  MATH  Google Scholar 

  6. Bauschke, H.H., Combettes, P.L., Luke, D.R.: Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization. JOSA A 19(7), 1334–1345 (2002)

    Article  MathSciNet  Google Scholar 

  7. Bauschke, H.H., Combettes, P.L., Luke, D.R.: Finding best approximation pairs relative to two closed convex sets in Hilbert spaces. J. Approx. Theory 127(2), 178–192 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bauschke, H.H., Combettes, P.L., Noll, D.: Joint minimization with alternating Bregman proximity operators. Pac. J. Optim. 2(3), 401–424 (2006)

    MathSciNet  MATH  Google Scholar 

  9. Bauschke, H.H., Dao, M.N.: On the finite convergence of the Douglas–Rachford algorithm for solving (not necessarily convex) feasibility problems in Euclidean spaces. SIAM J. Optim. 27(1), 507–537 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  10. Bauschke, H.H., Dao, M.N., Moursi, W.M.: On Fejér monotone sequences and nonexpansive mappings. Linear Nonlinear Anal. 1(2), 287–295 (2015)

    MathSciNet  MATH  Google Scholar 

  11. Bauschke, H.H., Dao, M.N., Moursi, W.M.: The Douglas–Rachford algorithm in the affine-convex case. Oper. Res. Lett. 44(3), 379–382 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  12. Bauschke, H.H., Dao, M.N., Noll, D., Phan, H.M.: Proximal point algorithm, Douglas–Rachford algorithm and alternating projections: a case study. J. Convex Anal. 23(1), 237–261 (2016)

    MathSciNet  MATH  Google Scholar 

  13. Bauschke, H.H., Dao, M.N., Noll, D., Phan, H.M.: On Slater’s condition and finite convergence of the Douglas–Rachford algorithm for solving convex feasibility problems in Euclidean spaces. J. Glob. Optim. 65(2), 329–349 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  14. Bauschke, H.H., Moursi, W.M.: On the order of the operators in the Douglas–Rachford algorithm. Optim. Lett. 10(3), 447–455 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  15. Bauschke, H.H., Moursi, W.M.: On the Douglas–Rachford algorithm. Math. Program. A 164(1), 263–284 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  16. Bauschke, H.H., Wang, C., Wang, X., Xu, J.: On subgradient projections. SIAM J. Optim. 25, 1064–1082 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  17. Bauschke, H.H., Wang, C., Wang, X., Xu, J.: Subgradient projectors: extensions, theory, and characterizations. Set-Valued Var. Anal. (2017). https://doi.org/10.1007/s11228-017-0415-x

    Article  MATH  Google Scholar 

  18. Benoist, J.: The Douglas–Rachford algorithm for the case of the sphere and the line. J. Glob. Optim. 63(2), 363–380 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  19. Borwein, J.M., Sims, B.: The Douglas–Rachford algorithm in the absence of convexity. In: Bauschke, H.H., Burachik, R., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 93–109. Springer, New York (2011)

    Chapter  Google Scholar 

  20. Dao, M.N., Phan, H.M.: Linear convergence of projection algorithms. Math. Oper. Res. (to appear). arXiv:1609.00341

  21. Dao, M.N., Phan, H.M.: Linear convergence of the generalized Douglas–Rachford algorithm for feasibility problems. J. Glob. Optim. (2018). https://doi.org/10.1007/s10898-018-0654-x

    Article  MathSciNet  MATH  Google Scholar 

  22. Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  23. Drusvyatskiy, D., Ioffe, A.D., Lewis, A.S.: Transversality and alternating projections for nonconvex sets. Found. Comput. Math. 15(6), 1637–1651 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  24. Giladi, O.: A remark on the convergence of the Douglas–Rachford iteration in a non-convex setting. Set-Valued Var. Anal. 26(2), 207–225 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  25. Hesse, R., Luke, D.R.: Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems. SIAM J. Optim. 23(4), 2397–2419 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  26. Lewis, A.S., Luke, D.R., Malick, J.: Local linear convergence for alternating and averaged nonconvex projections. Found. Comput. Math. 9(4), 485–513 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  27. Lindstrom, S.B., Sims, B., Skerritt, M.: Computing intersections of implicitly specified plane curves. J. Nonlinear Convex Anal. 18(3), 347–359 (2017)

    MathSciNet  MATH  Google Scholar 

  28. Lions, P.-L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  29. Mordukhovich, B.: Variational Analysis and Generalized Differentiation I. Basic Theory. Springer, Berlin (2006)

    Book  Google Scholar 

  30. Phan, H.M.: Linear convergence of the Douglas–Rachford method for two closed sets. Optimization 65(2), 369–385 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  31. Noll, D., Rondepierre, A.: On local convergence of the method of alternating projections. Found. Comput. Math. 16(2), 425–455 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  32. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (1998)

    Book  MATH  Google Scholar 

  33. Sherman, J., Morrison, W.J.: Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. Ann. Math. Stat. 21(1), 124–127 (1950)

    Article  MathSciNet  MATH  Google Scholar 

  34. Svaiter, B.F.: On weak convergence of the Douglas–Rachford method. SIAM J. Control Optim. 49(1), 280–287 (2011)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This paper is dedicated to the memory of Jonathan M. Borwein and his enthusiasm for the Douglas–Rachford algorithm. MND was partially supported by the Australian Research Council Discovery Project DP160101537 and a Startup Research Grant from the University of Newcastle. He wishes to acknowledge the hospitality and the support of D. Russell Luke during his visit to Universität Göttingen. MKT was partially supported by the Deutsche Forschungsgemeinschaft RTG 2088 and a Postdoctoral Fellowship from the Alexander von Humboldt Foundation. The authors wish to thank the three anonymous referees for their comments and suggestions.

Author information

Authors and Affiliations

  1. CARMA, University of Newcastle, Callaghan, NSW, 2308, Australia

    Minh N. Dao

  2. Institut für Numerische und Angewandte Mathematik, Universität Göttingen, 37083, Göttingen, Germany

    Matthew K. Tam

Authors
  1. Minh N. Dao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew K. Tam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Minh N. Dao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dao, M.N., Tam, M.K. A Lyapunov-type approach to convergence of the Douglas–Rachford algorithm for a nonconvex setting. J Glob Optim 73, 83–112 (2019). https://doi.org/10.1007/s10898-018-0677-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-018-0677-3

Keywords

Mathematics Subject Classification

Search

Navigation