A Cyclic Douglas–Rachford Iteration Scheme



In this paper, we present two Douglas–Rachford inspired iteration schemes which can be applied directly to N-set convex feasibility problems in Hilbert space. Our main results are weak convergence of the methods to a point whose nearest point projections onto each of the N sets coincide. For affine subspaces, convergence is in norm. Initial results from numerical experiments, comparing our methods to the classical (product-space) Douglas–Rachford scheme, are promising.


Douglas–Rachford method Convex feasibility problem Projections Firmly nonexpansive map Nonexpansive map Asymptotic regularity Fixed points Parallelization 


  1. 1.
    von Neumann, J.: Functional Operators, vol. II. The Geometry of Orthogonal Spaces vol. 22. Princeton University Press, Princeton (1950) MATHGoogle Scholar
  2. 2.
    Halperin, I.: The product of projection operators. Acta Sci. Math. (Szeged) 23, 96–99 (1962) MATHMathSciNetGoogle Scholar
  3. 3.
    Bregman, L.: The method of successive projection for finding a common point of convex sets. J. Sov. Math. 6, 688–692 (1965) MATHGoogle Scholar
  4. 4.
    Bauschke, H., Borwein, J.: On the convergence of von Neumann’s alternating projection algorithm for two sets. Set-Valued Anal. 1(2), 185–212 (1993) CrossRefMATHMathSciNetGoogle Scholar
  5. 5.
    Bauschke, H., Borwein, J., Lewis, A.: The method of cyclic projections for closed convex sets in Hilbert space. Contemp. Math. 204, 1–38 (1997) CrossRefMathSciNetGoogle Scholar
  6. 6.
    Kopecká, E., Reich, S.: A note on the von Neumann alternating projections algorithm. J. Nonlinear Convex Anal. 5(3), 379–386 (2004) MATHMathSciNetGoogle Scholar
  7. 7.
    Kopecká, E., Reich, S.: Another note on the von Neumann alternating projections algorithm. J. Nonlinear Convex Anal. 11, 455–460 (2010) MATHMathSciNetGoogle Scholar
  8. 8.
    Pustylnik, E., Reich, S., Zaslavski, A.: Convergence of non-periodic infinite products of orthogonal projections and nonexpansive operators in Hilbert space. J. Approx. Theory 164(5), 611–624 (2012) CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    Douglas, J., Rachford, H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82(2), 421–439 (1956) CrossRefMATHMathSciNetGoogle Scholar
  10. 10.
    Lions, P., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979) CrossRefMATHMathSciNetGoogle Scholar
  11. 11.
    Bauschke, H., Combettes, P., Luke, D.: Finding best approximation pairs relative to two closed convex sets in Hilbert spaces. J. Approx. Theory 127(2), 178–192 (2004) CrossRefMATHMathSciNetGoogle Scholar
  12. 12.
    Dykstra, R.: An algorithm for restricted least squares regression. J. Am. Stat. Assoc. 78(384), 837–842 (1983) CrossRefMATHMathSciNetGoogle Scholar
  13. 13.
    Boyle, J., Dykstra, R.: A method for finding projections onto the intersection of convex sets in Hilbert spaces. In: Advances in Order Restricted Statistical Inference. Lecture Notes in Statistics, vol. 37, pp. 28–47. Springer, Berlin (1986) CrossRefGoogle Scholar
  14. 14.
    Bauschke, H., Borwein, J.: Dykstra’s alternating projection algorithm for two sets. J. Approx. Theory 79(3), 418–443 (1994) CrossRefMATHMathSciNetGoogle Scholar
  15. 15.
    Bauschke, H.: Projection algorithms: results and open problems. Stud. Comput. Math. 8, 11–22 (2001) CrossRefMathSciNetGoogle Scholar
  16. 16.
    Bauschke, H., Borwein, J.: On projection algorithms for solving convex feasibility problems. SIAM Rew. 38(3), 367–426 (1996) CrossRefMATHMathSciNetGoogle Scholar
  17. 17.
    Deutsch, F.: The method of alternating orthogonal projections. In: Approximation Theory, Spline Functions and Applications, pp. 105–121. Kluwer Academic, Dordrecht (1992) CrossRefGoogle Scholar
  18. 18.
    Tam, M.: The method of alternating projections. http://docserver.carma.newcastle.edu.au/id/eprint/1463. Honours thesis, Univ. of Newcastle (2012)
  19. 19.
    Escalante, R., Raydan, M.: Alternating Projection Methods. Fundamentals of Algorithms. Society for Industrial and Applied Mathematics, Philadelphia (2011) CrossRefGoogle Scholar
  20. 20.
    Borwein, J.: Maximum entropy and feasibility methods for convex and nonconvex inverse problems. Optimization 61(1), 1–33 (2012) CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    Bauschke, H., Combettes, P., Luke, D.: Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization. J. Opt. Soc. Am. A 19(7), 1334–1345 (2002) CrossRefMathSciNetGoogle Scholar
  22. 22.
    Bauschke, H., Combettes, P., Luke, D.: Hybrid projection–reflection method for phase retrieval. J. Opt. Soc. Am. A 20(6), 1025–1034 (2003) CrossRefGoogle Scholar
  23. 23.
    Elser, V., Rankenburg, I., Thibault, P.: Searching with iterated maps. Proc. Natl. Acad. Sci. 104(2), 418–423 (2007) CrossRefMATHMathSciNetGoogle Scholar
  24. 24.
    Gravel, S., Elser, V.: Divide and concur: a general approach to constraint satisfaction. Phys. Rev. E 78(3), 036,706 (2008) CrossRefGoogle Scholar
  25. 25.
    Schaad, J.: Modeling the 8-queens problem and sudoku using an algorithm based on projections onto nonconvex sets. Master’s thesis, Univ. of British Columbia (2010) Google Scholar
  26. 26.
    Lewis, A., Luke, D., Malick, J.: Local linear convergence for alternating and averaged nonconvex projections. Found. Comput. Math. 9(4), 485–513 (2009) CrossRefMATHMathSciNetGoogle Scholar
  27. 27.
    Bauschke, H., Luke, D., Phan, H., Wang, X.: Restricted normal cones and the method of alternating projections. Set-Valued Var. Anal. To appear (2013). http://arxiv.org/pdf/1205.0318v1
  28. 28.
    Hesse, R., Luke, D.: Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems. Preprint (2012). http://arxiv.org/pdf/1205.0318v1
  29. 29.
    Drusvyatskiy, D., Ioffe, A., Lewis, A.: Alternating projections: a new approach. In preparation Google Scholar
  30. 30.
    Borwein, J., Sims, B.: The Douglas–Rachford algorithm in the absence of convexity. In: Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 93–109 (2011) CrossRefGoogle Scholar
  31. 31.
    Aragón Artacho, F., Borwein, J.: Global convergence of a non-convex Douglas–Rachford iteration. J. Glob. Optim. (2012). doi:10.1007/s10898-012-9958-4 MATHGoogle Scholar
  32. 32.
    Bauschke, H., Combettes, P.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Canadian Mathematical Society Societe Mathematique Du Canada. Springer, New York (2011) CrossRefMATHGoogle Scholar
  33. 33.
    Reich, S., Shafrir, I.: The asymptotic behavior of firmly nonexpansive mappings. Proc. Am. Math. Soc. 101(2), 246–250 (1987) CrossRefMATHMathSciNetGoogle Scholar
  34. 34.
    Bruck, R., Reich, S.: Nonexpansive projections and resolvents of accretive operators in Banach space. Houst. J. Math. 4 (1977) Google Scholar
  35. 35.
    Bauschke, H., Martín-Márquez, V., Moffat, S., Wang, X.: Compositions and convex combinations of asymptotically regular firmly nonexpansive mappings are also asymptotically regular. Fixed Point Theory Appl. 2012(53), 1–11 (2012) Google Scholar
  36. 36.
    Opial, Z.: Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bull. Am. Math. Soc. 73(4), 591–597 (1967) CrossRefMATHMathSciNetGoogle Scholar
  37. 37.
    Netyanun, A., Solmon, D.: Iterated products of projections in Hilbert space. Am. Math. Mon. 113(7), 644–648 (2006) CrossRefMATHMathSciNetGoogle Scholar
  38. 38.
    Borwein, J., Reich, S., Shafrir, I.: Krasnoselski–Mann iterations in normed spaces. Can. Math. Bull. 35(1), 21–28 (1992) CrossRefMATHMathSciNetGoogle Scholar
  39. 39.
    Cheney, W., Goldstein, A.: Proximity maps for convex sets. Proc. Am. Math. Soc. 10(3), 448–450 (1959) CrossRefMATHMathSciNetGoogle Scholar
  40. 40.
    Wolpert, D., Macready, W.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997) CrossRefGoogle Scholar
  41. 41.
    Aragón Artacho, F., Borwein, J., Tam, M.: 2013, Recent results on Douglas–Rachford methods for combinatorial optimization problems. Preprint. arXiv:1305.2657v1

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.CARMA CentreUniversity of NewcastleCallaghanAustralia

Personalised recommendations