Skip to main content
SpringerLink
Log in

The Douglas–Rachford algorithm for convex and nonconvex feasibility problems

  • Original Article
  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

The Douglas–Rachford algorithm is an optimization method that can be used for solving feasibility problems. To apply the method, it is necessary that the problem at hand is prescribed in terms of constraint sets having efficiently computable nearest points. Although the convergence of the algorithm is guaranteed in the convex setting, the scheme has demonstrated to be a successful heuristic for solving combinatorial problems of different type. In this self-contained tutorial, we develop the convergence theory of projection algorithms within the framework of fixed point iterations, explain how to devise useful feasibility problem formulations, and demonstrate the application of the Douglas–Rachford method to said formulations. The paradigm is then illustrated on two concrete problems: a generalization of the “eight queens puzzle” known as the “(m, n)-queens problem”, and the problem of constructing a probability distribution with prescribed moments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  • Alwadani S, Bauschke HH, Moursi WM, Wang X (2018) On the asymptotic behaviour of the Aragón Artacho-Campoy algorithm. Oper Res Lett 46(6):585–587

    MathSciNet  MATH  Google Scholar 

  • Aragón Artacho FJ, Borwein JM (2013) Global convergence of a non-convex Douglas–Rachford iteration. J Glob Optim 57(3):753–769

    MathSciNet  MATH  Google Scholar 

  • Aragón Artacho FJ, Campoy R (2018a) A new projection method for finding the closest point in the intersection of convex sets. Comput Optim Appl 69(1):99–132

    MathSciNet  MATH  Google Scholar 

  • Aragón Artacho FJ, Campoy R (2018b) Solving graph coloring problems with the Douglas–Rachford algorithm. Set-Valued Var Anal 26(2):277–304

    MathSciNet  MATH  Google Scholar 

  • Aragón Artacho FJ, Borwein JM, Tam MK (2014a) Douglas–Rachford feasibility methods for matrix completion problems. ANZIAM J 55(4):299–326

    MathSciNet  MATH  Google Scholar 

  • Aragón Artacho FJ, Borwein JM, Tam MK (2014b) Recent results on Douglas–Rachford methods for combinatorial optimization problem. J Optim Theory Appl 163(1):1–30

    MathSciNet  MATH  Google Scholar 

  • Aragón Artacho FJ, Borwein JM, Tam MK (2016) Global behavior of the Douglas–Rachford method for a nonconvex feasibility problem. J Glob Optim 65(2):309–327

    MathSciNet  MATH  Google Scholar 

  • Aragón Artacho FJ, Campoy R, Kotsireas IS, Tam MK (2018b) A feasibility approach for constructing combinatorial designs of circulant type. J Comb Optim 35(4):1061–1085

    MathSciNet  MATH  Google Scholar 

  • Aragón Artacho FJ, Censor Y, Gibali A (2019) The cyclic Douglas-Rachford algorithm with \(r\)-sets-Douglas-Rachford operators. Optim Methods Softw 34(4):875–889

    MathSciNet  MATH  Google Scholar 

  • Aragón Artacho FJ, Campoy R, Elser V (2018a) An enhanced formulation for successfully solving graph coloring problems with the Douglas–Rachford algorithm. arXiv e-prints arXiv:1808.01022

  • Baillon JB, Bruck RE, Reich S (1978) On the asymptotic behavior of nonexpansive mappings and semigroups in Banach spaces. Houston J Math 4(1):1–9

    MathSciNet  MATH  Google Scholar 

  • Banach S (1922) Sur les opérations dans les ensembles abstraits et leur application aux équations intégrales. Fund math 3(1):133–181

    MathSciNet  MATH  Google Scholar 

  • Bauschke HH (2013) New demiclosedness principles for (firmly) nonexpansive operators. In: Computational and analytical mathematics, Springer, pp 19–28

  • Bauschke HH, Combettes PL (2017) Convex analysis and monotone operator theory in hilbert spaces, 2nd edn. Springer, New York

    MATH  Google Scholar 

  • Bauschke HH, Dao MN (2017) On the finite convergence of the Douglas–Rachford algorithm for solving (not necessarily convex) feasibility problems in Euclidean spaces. SIAM J Optim 27(1):507–537

    MathSciNet  MATH  Google Scholar 

  • Bauschke HH, Moursi WM (2017) On the Douglas–Rachford algorithm. Math Program, Ser A 164(1–2):263–284

    MathSciNet  MATH  Google Scholar 

  • Bauschke HH, Noll D (2014) On the local convergence of the Douglas–Rachford algorithm. Arch Math 102(6):589–600

    MathSciNet  MATH  Google Scholar 

  • Bauschke HH, Combettes PL, Luke DR (2002) Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization. J Opt Soc Am A: 19(7):1334–1345

    MathSciNet  Google Scholar 

  • Bauschke HH, Combettes PL, Luke DR (2004) Finding best approximation pairs relative to two closed convex sets in Hilbert spaces. J Approx Theory 127(2):178–192

    MathSciNet  MATH  Google Scholar 

  • Bauschke HH, Bello Cruz JY, Nghia TT, Phan HM, Wang X (2014) The rate of linear convergence of the Douglas–Rachford algorithm for subspaces is the cosine of the Friedrichs angle. J Approx Theory 185:63–79

    MathSciNet  MATH  Google Scholar 

  • Bauschke HH, Noll D, Phan HM (2015) Linear and strong convergence of algorithms involving averaged nonexpansive operators. J Math Anal Appl 421(1):1–20

    MathSciNet  MATH  Google Scholar 

  • Bauschke HH, Lukens B, Moursi WM (2017) Affine nonexpansive operators, Attouch-Théra duality and the Douglas–Rachford algorithm. Set-Valued Var Anal 25(3):481–505

    MathSciNet  MATH  Google Scholar 

  • Bauschke HH, Dao MN, Lindstrom SB (2019) The Douglas–Rachford algorithm for a hyperplane and a doubleton. J Glob Optim 74(1):79–93

    MathSciNet  MATH  Google Scholar 

  • Behling R, Bello Cruz JY, Santos L (2018) Circumcentering the Douglas–Rachford method. Numer Algor 78(3):759–776

    MathSciNet  MATH  Google Scholar 

  • Benoist J (2015) The Douglas–Rachford algorithm for the case of the sphere and the line. J Global Optim 63(2):363–380

    MathSciNet  MATH  Google Scholar 

  • Borwein J, Lewis A (2010) Convex analysis and nonlinear optimization: theory and examples. Springer, New York

    Google Scholar 

  • Borwein JM, Sims B (2011) The Douglas–Rachford algorithm in the absence of convexity. In: Bauschke H, Burachik R, Combettes P, Elser V, Luke D, Wolkowicz H (eds) Fixed-point algorithms for inverse problems in science and engineering, springer optimization and its applications, vol 49. Springer, New York, pp 93–109

    Google Scholar 

  • Borwein JM, Tam MK (2014) A cyclic Douglas–Rachford iteration scheme. J Optim Theory Appl 160(1):1–29

    MathSciNet  MATH  Google Scholar 

  • Borwein JM, Tam MK (2015) The cyclic Douglas–Rachford method for inconsistent feasibility problems. J Nonlinear Convex Anal 16(4):573–584

    MathSciNet  MATH  Google Scholar 

  • Borwein JM, Tam MK (2017) Reflection methods for inverse problems with applications to protein conformation determination. Forum for interdisciplinary mathematics. In: Aussel D, Lalitha C (eds) Generalized nash equilibrium problems, bilevel programming and MPEC. Springer, Singapore, pp 83–100

    Google Scholar 

  • Borwein JM, Sims B, Tam MK (2015) Norm convergence of realistic projection and reflection methods. Optimization 64(1):161–178

    MathSciNet  MATH  Google Scholar 

  • Borwein JM, Lindstrom SB, Sims B, Schneider A, Skerritt MP (2018) Dynamics of the Douglas–Rachford method for ellipses and \(p\)-spheres. Set-Valued Var Anal 26(2):385–403

    MathSciNet  MATH  Google Scholar 

  • Bregman LM (1965) The method of successive projection for finding a common point of convex sets. Soviet Math Dokl 162(3):688–692

    MATH  Google Scholar 

  • Cegielski A (2012) Iterative methods for fixed point problems in hilbert spaces, Lecture Notes in Mathematics, vol 2057. Springer

  • Censor Y (1984) Iterative methods for convex feasibility problems. Ann Discrete Math 20:83–91

    MathSciNet  MATH  Google Scholar 

  • Censor Y, Cegielski A (2015) Projection methods: an annotated bibliography of books and reviews. Optimization 64(11):2343–2358

    MathSciNet  MATH  Google Scholar 

  • Censor Y, Mansour R (2016) New Douglas–Rachford algorithmic structures and their convergence analyses. SIAM J Optim 26:474–487

    MathSciNet  MATH  Google Scholar 

  • Cheney W, Goldstein A (1959) Proximity maps for convex sets. Proc Amer Math Soc 10(3):448–450

    MathSciNet  MATH  Google Scholar 

  • Dao MN, Tam MK (2019b) Union averaged operators with applications to proximal algorithms for min-convex functions. J Optim Theory and Appl pp 1–34

  • Dao MN, Phan HM (2018) Linear convergence of the generalized Douglas–Rachford algorithm for feasibility problems. J Glob Optim 72(3):443–474

    MathSciNet  MATH  Google Scholar 

  • Dao MN, Tam MK (2019a) A Lyapunov-type approach to convergence of the Douglas–Rachford algorithm for a nonconvex setting. J Glob Optim 73(1):83–112

    MathSciNet  MATH  Google Scholar 

  • Deutsch F (2001) Best approximation in inner product spaces, CMS books in mathematics/ouvrages de mathématiques de la SMC, vol 7. Springer, New York

    Google Scholar 

  • Douglas J, Rachford HH (1956) On the numerical solution of heat conduction problems in two and three space variables. Trans Amer Math Soc 82:421–439

    MathSciNet  MATH  Google Scholar 

  • Eckstein J, Bertsekas DP (1992) On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math Program 55(1):293–318

    MathSciNet  MATH  Google Scholar 

  • Elser V (2003) Phase retrieval by iterated projections. J Opt Soc Am A: 20(1):40–55

    Google Scholar 

  • Elser V (2018) The complexity of bit retrieval. IEEE Trans Inf Theory 64(1):412–428

    MathSciNet  MATH  Google Scholar 

  • Elser V, Rankenburg I, Thibault P (2007) Searching with iterated maps. Proc Natl Acad Sci 104(2):418–423

    MathSciNet  MATH  Google Scholar 

  • Halperin I (1962) The product of projection operators. Acta Sci Math 23:96–99

    MathSciNet  MATH  Google Scholar 

  • Hesse R, Luke DR (2013) Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems. SIAM J Optim 23(4):2397–2419

    MathSciNet  MATH  Google Scholar 

  • Hesse R, Luke DR, Neumann P (2014) Alternating projections and Douglas–Rachford for sparse affine feasibility. IEEE Trans Signal Process 62(18):4868–4881

    MathSciNet  MATH  Google Scholar 

  • Hundal HS (2004) An alternating projection that does not converge in norm. Nonlin Anal Theory Methods Appl 57(1):35–61

    MathSciNet  MATH  Google Scholar 

  • Kaczmarz S (1937) Angenäherte Auflösung von Systemen linearer Gleichungen. Bull Int Acad Sci Pologne A 35:355–357

    MATH  Google Scholar 

  • Lamichhane BP, Lindstrom SB, Sims B (2019) Application of projection algorithms to differential equations: boundary value problems. ANZIAM J 61(1):23–46

    MathSciNet  MATH  Google Scholar 

  • Li G, Pong TK (2016) Douglas–Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems. Math Prog 159:371–401

    MathSciNet  MATH  Google Scholar 

  • Lindstrom SB, Sims B (2018) Survey: Sixty years of Douglas–Rachford. J AustMS (to appear)

  • Lions PL, Mercier B (1979) Splitting algorithms for the sum of two nonlinear operators. SIAM J Numer Anal 16(6):964–979

    MathSciNet  MATH  Google Scholar 

  • Luke DR (2008) Finding best approximation pairs relative to a convex and a prox-regular set in a Hilbert space. SIAM J Optim 19(2):714–739

    MathSciNet  MATH  Google Scholar 

  • Luke DR, Nguyen HT, Tam MK (2018) Quantitative convergence analysis of iterated expansive, set-valued mappings. Math Oper Res 43(4):1143–1176

    MathSciNet  MATH  Google Scholar 

  • Maohua L, Weixuan L, Wang E (1990) A generalization of the \(n\)-queen problem. J Systems Sci Math Sci 3(2):183–191

    MathSciNet  MATH  Google Scholar 

  • Opial Z (1967) Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bull Am Math Soc 73(4):591–597

    MathSciNet  MATH  Google Scholar 

  • Pazy A (1971) Asymptotic behavior of contractions in Hilbert space. Israel J Math 9:235–240

    MathSciNet  MATH  Google Scholar 

  • Phan HM (2016) Linear convergence of the Douglas–Rachford method for two closed sets. Optimization 65(2):369–385

    MathSciNet  MATH  Google Scholar 

  • Pierra G (1984) Decomposition through formalization in a product space. Math Program 28:96–115

    MathSciNet  MATH  Google Scholar 

  • Schaad J (2010) Modeling the \(8\)-queens problem and Sudoku using an algorithm based on projections onto nonconvex sets. Master’s thesis, University of British Columbia

  • Svaiter BF (2011) On weak convergence of the Douglas–Rachford method. SIAM J Control Optim 49(1):280–287

    MathSciNet  MATH  Google Scholar 

  • Tam MK (2018) Algorithms based on unions of nonexpansive maps. Optim Lett 12(5):1019–1027

    MathSciNet  MATH  Google Scholar 

  • Thao NH (2018) A convergent relaxation of the Douglas–Rachford algorithm. Comput Optim Appl 70(3):841–863

    MathSciNet  MATH  Google Scholar 

  • von Neumann J (1950) Functional operators II: the geometry of orthogonal spaces. Princeton University Press, Princeton

    MATH  Google Scholar 

Download references

Acknowledgements

We are grateful to two anonymous reviewers for their constructive comments, which helped to improve the paper.

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Alicante, Alicante, Spain

    Francisco J. Aragón Artacho & Rubén Campoy

  2. Institute for Numerical and Applied Mathematics, University of Göttingen, Göttingen, Germany

    Matthew K. Tam

Authors
  1. Francisco J. Aragón Artacho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rubén Campoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew K. Tam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francisco J. Aragón Artacho.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

FJAA and RC were partially supported by Ministerio de Economía, Industria y Competitividad (MINECO) and European Regional Development Fund (ERDF), grant MTM2014-59179-C2-1-P. FJAA was supported by the Ramón y Cajal program by MINECO and ERDF (RYC-2013-13327) and by the Ministerio de Ciencia, Innovación y Universidades and ERDF, grant PGC2018-097960-B-C22. RC was supported by MINECO and European Social Fund (BES-2015-073360) under the program “Ayudas para contratos predoctorales para la formación de doctores 2015”.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aragón Artacho, F.J., Campoy, R. & Tam, M.K. The Douglas–Rachford algorithm for convex and nonconvex feasibility problems. Math Meth Oper Res 91, 201–240 (2020). https://doi.org/10.1007/s00186-019-00691-9

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00186-019-00691-9

Keywords

Mathematics Subject Classification

Search

Navigation