A Cyclic Douglas–Rachford Iteration Scheme

  • Jonathan M. Borwein
  • Matthew K. TamEmail author


In this paper, we present two Douglas–Rachford inspired iteration schemes which can be applied directly to N-set convex feasibility problems in Hilbert space. Our main results are weak convergence of the methods to a point whose nearest point projections onto each of the N sets coincide. For affine subspaces, convergence is in norm. Initial results from numerical experiments, comparing our methods to the classical (product-space) Douglas–Rachford scheme, are promising.


Douglas–Rachford method Convex feasibility problem Projections Firmly nonexpansive map Nonexpansive map Asymptotic regularity Fixed points Parallelization 



The authors wish to acknowledge Francisco J. Aragón Artacho, Brailey Sims, Simeon Reich, and the two anonymous referees for their helpful comments and suggestions.

Jonathan M. Borwein’s research is supported in part by the Australian Research Council.

Matthew K. Tam’s research is supported in part by an Australian Postgraduate Award.


  1. 1.
    von Neumann, J.: Functional Operators, vol. II. The Geometry of Orthogonal Spaces vol. 22. Princeton University Press, Princeton (1950) zbMATHGoogle Scholar
  2. 2.
    Halperin, I.: The product of projection operators. Acta Sci. Math. (Szeged) 23, 96–99 (1962) zbMATHMathSciNetGoogle Scholar
  3. 3.
    Bregman, L.: The method of successive projection for finding a common point of convex sets. J. Sov. Math. 6, 688–692 (1965) zbMATHGoogle Scholar
  4. 4.
    Bauschke, H., Borwein, J.: On the convergence of von Neumann’s alternating projection algorithm for two sets. Set-Valued Anal. 1(2), 185–212 (1993) CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Bauschke, H., Borwein, J., Lewis, A.: The method of cyclic projections for closed convex sets in Hilbert space. Contemp. Math. 204, 1–38 (1997) CrossRefMathSciNetGoogle Scholar
  6. 6.
    Kopecká, E., Reich, S.: A note on the von Neumann alternating projections algorithm. J. Nonlinear Convex Anal. 5(3), 379–386 (2004) zbMATHMathSciNetGoogle Scholar
  7. 7.
    Kopecká, E., Reich, S.: Another note on the von Neumann alternating projections algorithm. J. Nonlinear Convex Anal. 11, 455–460 (2010) zbMATHMathSciNetGoogle Scholar
  8. 8.
    Pustylnik, E., Reich, S., Zaslavski, A.: Convergence of non-periodic infinite products of orthogonal projections and nonexpansive operators in Hilbert space. J. Approx. Theory 164(5), 611–624 (2012) CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Douglas, J., Rachford, H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82(2), 421–439 (1956) CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Lions, P., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979) CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Bauschke, H., Combettes, P., Luke, D.: Finding best approximation pairs relative to two closed convex sets in Hilbert spaces. J. Approx. Theory 127(2), 178–192 (2004) CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    Dykstra, R.: An algorithm for restricted least squares regression. J. Am. Stat. Assoc. 78(384), 837–842 (1983) CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Boyle, J., Dykstra, R.: A method for finding projections onto the intersection of convex sets in Hilbert spaces. In: Advances in Order Restricted Statistical Inference. Lecture Notes in Statistics, vol. 37, pp. 28–47. Springer, Berlin (1986) CrossRefGoogle Scholar
  14. 14.
    Bauschke, H., Borwein, J.: Dykstra’s alternating projection algorithm for two sets. J. Approx. Theory 79(3), 418–443 (1994) CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    Bauschke, H.: Projection algorithms: results and open problems. Stud. Comput. Math. 8, 11–22 (2001) CrossRefMathSciNetGoogle Scholar
  16. 16.
    Bauschke, H., Borwein, J.: On projection algorithms for solving convex feasibility problems. SIAM Rew. 38(3), 367–426 (1996) CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Deutsch, F.: The method of alternating orthogonal projections. In: Approximation Theory, Spline Functions and Applications, pp. 105–121. Kluwer Academic, Dordrecht (1992) CrossRefGoogle Scholar
  18. 18.
    Tam, M.: The method of alternating projections. Honours thesis, Univ. of Newcastle (2012)
  19. 19.
    Escalante, R., Raydan, M.: Alternating Projection Methods. Fundamentals of Algorithms. Society for Industrial and Applied Mathematics, Philadelphia (2011) CrossRefGoogle Scholar
  20. 20.
    Borwein, J.: Maximum entropy and feasibility methods for convex and nonconvex inverse problems. Optimization 61(1), 1–33 (2012) CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Bauschke, H., Combettes, P., Luke, D.: Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization. J. Opt. Soc. Am. A 19(7), 1334–1345 (2002) CrossRefMathSciNetGoogle Scholar
  22. 22.
    Bauschke, H., Combettes, P., Luke, D.: Hybrid projection–reflection method for phase retrieval. J. Opt. Soc. Am. A 20(6), 1025–1034 (2003) CrossRefGoogle Scholar
  23. 23.
    Elser, V., Rankenburg, I., Thibault, P.: Searching with iterated maps. Proc. Natl. Acad. Sci. 104(2), 418–423 (2007) CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    Gravel, S., Elser, V.: Divide and concur: a general approach to constraint satisfaction. Phys. Rev. E 78(3), 036,706 (2008) CrossRefGoogle Scholar
  25. 25.
    Schaad, J.: Modeling the 8-queens problem and sudoku using an algorithm based on projections onto nonconvex sets. Master’s thesis, Univ. of British Columbia (2010) Google Scholar
  26. 26.
    Lewis, A., Luke, D., Malick, J.: Local linear convergence for alternating and averaged nonconvex projections. Found. Comput. Math. 9(4), 485–513 (2009) CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Bauschke, H., Luke, D., Phan, H., Wang, X.: Restricted normal cones and the method of alternating projections. Set-Valued Var. Anal. To appear (2013).
  28. 28.
    Hesse, R., Luke, D.: Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems. Preprint (2012).
  29. 29.
    Drusvyatskiy, D., Ioffe, A., Lewis, A.: Alternating projections: a new approach. In preparation Google Scholar
  30. 30.
    Borwein, J., Sims, B.: The Douglas–Rachford algorithm in the absence of convexity. In: Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 93–109 (2011) CrossRefGoogle Scholar
  31. 31.
    Aragón Artacho, F., Borwein, J.: Global convergence of a non-convex Douglas–Rachford iteration. J. Glob. Optim. (2012). doi: 10.1007/s10898-012-9958-4 zbMATHGoogle Scholar
  32. 32.
    Bauschke, H., Combettes, P.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Canadian Mathematical Society Societe Mathematique Du Canada. Springer, New York (2011) CrossRefzbMATHGoogle Scholar
  33. 33.
    Reich, S., Shafrir, I.: The asymptotic behavior of firmly nonexpansive mappings. Proc. Am. Math. Soc. 101(2), 246–250 (1987) CrossRefzbMATHMathSciNetGoogle Scholar
  34. 34.
    Bruck, R., Reich, S.: Nonexpansive projections and resolvents of accretive operators in Banach space. Houst. J. Math. 4 (1977) Google Scholar
  35. 35.
    Bauschke, H., Martín-Márquez, V., Moffat, S., Wang, X.: Compositions and convex combinations of asymptotically regular firmly nonexpansive mappings are also asymptotically regular. Fixed Point Theory Appl. 2012(53), 1–11 (2012) Google Scholar
  36. 36.
    Opial, Z.: Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bull. Am. Math. Soc. 73(4), 591–597 (1967) CrossRefzbMATHMathSciNetGoogle Scholar
  37. 37.
    Netyanun, A., Solmon, D.: Iterated products of projections in Hilbert space. Am. Math. Mon. 113(7), 644–648 (2006) CrossRefzbMATHMathSciNetGoogle Scholar
  38. 38.
    Borwein, J., Reich, S., Shafrir, I.: Krasnoselski–Mann iterations in normed spaces. Can. Math. Bull. 35(1), 21–28 (1992) CrossRefzbMATHMathSciNetGoogle Scholar
  39. 39.
    Cheney, W., Goldstein, A.: Proximity maps for convex sets. Proc. Am. Math. Soc. 10(3), 448–450 (1959) CrossRefzbMATHMathSciNetGoogle Scholar
  40. 40.
    Wolpert, D., Macready, W.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997) CrossRefGoogle Scholar
  41. 41.
    Aragón Artacho, F., Borwein, J., Tam, M.: 2013, Recent results on Douglas–Rachford methods for combinatorial optimization problems. Preprint. arXiv:1305.2657v1

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.CARMA CentreUniversity of NewcastleCallaghanAustralia

Personalised recommendations