Skip to main content
SpringerLink
Log in

A Cyclic Douglas–Rachford Iteration Scheme

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this paper, we present two Douglas–Rachford inspired iteration schemes which can be applied directly to N-set convex feasibility problems in Hilbert space. Our main results are weak convergence of the methods to a point whose nearest point projections onto each of the N sets coincide. For affine subspaces, convergence is in norm. Initial results from numerical experiments, comparing our methods to the classical (product-space) Douglas–Rachford scheme, are promising.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Kakutani had earlier proven weak convergence for finitely many subspaces [37]. Von Neumann’s original two-set proof does not seem to generalize.

  2. See http://www.cinderella.de/.

References

  1. von Neumann, J.: Functional Operators, vol. II. The Geometry of Orthogonal Spaces vol. 22. Princeton University Press, Princeton (1950)

    MATH  Google Scholar 

  2. Halperin, I.: The product of projection operators. Acta Sci. Math. (Szeged) 23, 96–99 (1962)

    MATH  MathSciNet  Google Scholar 

  3. Bregman, L.: The method of successive projection for finding a common point of convex sets. J. Sov. Math. 6, 688–692 (1965)

    MATH  Google Scholar 

  4. Bauschke, H., Borwein, J.: On the convergence of von Neumann’s alternating projection algorithm for two sets. Set-Valued Anal. 1(2), 185–212 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  5. Bauschke, H., Borwein, J., Lewis, A.: The method of cyclic projections for closed convex sets in Hilbert space. Contemp. Math. 204, 1–38 (1997)

    Article  MathSciNet  Google Scholar 

  6. Kopecká, E., Reich, S.: A note on the von Neumann alternating projections algorithm. J. Nonlinear Convex Anal. 5(3), 379–386 (2004)

    MATH  MathSciNet  Google Scholar 

  7. Kopecká, E., Reich, S.: Another note on the von Neumann alternating projections algorithm. J. Nonlinear Convex Anal. 11, 455–460 (2010)

    MATH  MathSciNet  Google Scholar 

  8. Pustylnik, E., Reich, S., Zaslavski, A.: Convergence of non-periodic infinite products of orthogonal projections and nonexpansive operators in Hilbert space. J. Approx. Theory 164(5), 611–624 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  9. Douglas, J., Rachford, H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82(2), 421–439 (1956)

    Article  MATH  MathSciNet  Google Scholar 

  10. Lions, P., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16(6), 964–979 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  11. Bauschke, H., Combettes, P., Luke, D.: Finding best approximation pairs relative to two closed convex sets in Hilbert spaces. J. Approx. Theory 127(2), 178–192 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  12. Dykstra, R.: An algorithm for restricted least squares regression. J. Am. Stat. Assoc. 78(384), 837–842 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  13. Boyle, J., Dykstra, R.: A method for finding projections onto the intersection of convex sets in Hilbert spaces. In: Advances in Order Restricted Statistical Inference. Lecture Notes in Statistics, vol. 37, pp. 28–47. Springer, Berlin (1986)

    Chapter  Google Scholar 

  14. Bauschke, H., Borwein, J.: Dykstra’s alternating projection algorithm for two sets. J. Approx. Theory 79(3), 418–443 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  15. Bauschke, H.: Projection algorithms: results and open problems. Stud. Comput. Math. 8, 11–22 (2001)

    Article  MathSciNet  Google Scholar 

  16. Bauschke, H., Borwein, J.: On projection algorithms for solving convex feasibility problems. SIAM Rew. 38(3), 367–426 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  17. Deutsch, F.: The method of alternating orthogonal projections. In: Approximation Theory, Spline Functions and Applications, pp. 105–121. Kluwer Academic, Dordrecht (1992)

    Chapter  Google Scholar 

  18. Tam, M.: The method of alternating projections. http://docserver.carma.newcastle.edu.au/id/eprint/1463. Honours thesis, Univ. of Newcastle (2012)

  19. Escalante, R., Raydan, M.: Alternating Projection Methods. Fundamentals of Algorithms. Society for Industrial and Applied Mathematics, Philadelphia (2011)

    Book  Google Scholar 

  20. Borwein, J.: Maximum entropy and feasibility methods for convex and nonconvex inverse problems. Optimization 61(1), 1–33 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  21. Bauschke, H., Combettes, P., Luke, D.: Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization. J. Opt. Soc. Am. A 19(7), 1334–1345 (2002)

    Article  MathSciNet  Google Scholar 

  22. Bauschke, H., Combettes, P., Luke, D.: Hybrid projection–reflection method for phase retrieval. J. Opt. Soc. Am. A 20(6), 1025–1034 (2003)

    Article  Google Scholar 

  23. Elser, V., Rankenburg, I., Thibault, P.: Searching with iterated maps. Proc. Natl. Acad. Sci. 104(2), 418–423 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  24. Gravel, S., Elser, V.: Divide and concur: a general approach to constraint satisfaction. Phys. Rev. E 78(3), 036,706 (2008)

    Article  Google Scholar 

  25. Schaad, J.: Modeling the 8-queens problem and sudoku using an algorithm based on projections onto nonconvex sets. Master’s thesis, Univ. of British Columbia (2010)

  26. Lewis, A., Luke, D., Malick, J.: Local linear convergence for alternating and averaged nonconvex projections. Found. Comput. Math. 9(4), 485–513 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  27. Bauschke, H., Luke, D., Phan, H., Wang, X.: Restricted normal cones and the method of alternating projections. Set-Valued Var. Anal. To appear (2013). http://arxiv.org/pdf/1205.0318v1

  28. Hesse, R., Luke, D.: Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems. Preprint (2012). http://arxiv.org/pdf/1205.0318v1

  29. Drusvyatskiy, D., Ioffe, A., Lewis, A.: Alternating projections: a new approach. In preparation

  30. Borwein, J., Sims, B.: The Douglas–Rachford algorithm in the absence of convexity. In: Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 93–109 (2011)

    Chapter  Google Scholar 

  31. Aragón Artacho, F., Borwein, J.: Global convergence of a non-convex Douglas–Rachford iteration. J. Glob. Optim. (2012). doi:10.1007/s10898-012-9958-4

    MATH  Google Scholar 

  32. Bauschke, H., Combettes, P.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Canadian Mathematical Society Societe Mathematique Du Canada. Springer, New York (2011)

    Book  MATH  Google Scholar 

  33. Reich, S., Shafrir, I.: The asymptotic behavior of firmly nonexpansive mappings. Proc. Am. Math. Soc. 101(2), 246–250 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  34. Bruck, R., Reich, S.: Nonexpansive projections and resolvents of accretive operators in Banach space. Houst. J. Math. 4 (1977)

  35. Bauschke, H., Martín-Márquez, V., Moffat, S., Wang, X.: Compositions and convex combinations of asymptotically regular firmly nonexpansive mappings are also asymptotically regular. Fixed Point Theory Appl. 2012(53), 1–11 (2012)

    Google Scholar 

  36. Opial, Z.: Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bull. Am. Math. Soc. 73(4), 591–597 (1967)

    Article  MATH  MathSciNet  Google Scholar 

  37. Netyanun, A., Solmon, D.: Iterated products of projections in Hilbert space. Am. Math. Mon. 113(7), 644–648 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  38. Borwein, J., Reich, S., Shafrir, I.: Krasnoselski–Mann iterations in normed spaces. Can. Math. Bull. 35(1), 21–28 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  39. Cheney, W., Goldstein, A.: Proximity maps for convex sets. Proc. Am. Math. Soc. 10(3), 448–450 (1959)

    Article  MATH  MathSciNet  Google Scholar 

  40. Wolpert, D., Macready, W.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)

    Article  Google Scholar 

  41. Aragón Artacho, F., Borwein, J., Tam, M.: 2013, Recent results on Douglas–Rachford methods for combinatorial optimization problems. Preprint. arXiv:1305.2657v1

Download references

Acknowledgements

The authors wish to acknowledge Francisco J. Aragón Artacho, Brailey Sims, Simeon Reich, and the two anonymous referees for their helpful comments and suggestions.

Jonathan M. Borwein’s research is supported in part by the Australian Research Council.

Matthew K. Tam’s research is supported in part by an Australian Postgraduate Award.

Author information

Authors and Affiliations

  1. CARMA Centre, University of Newcastle, Callaghan, NSW, 2308, Australia

    Jonathan M. Borwein & Matthew K. Tam

Authors
  1. Jonathan M. Borwein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew K. Tam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew K. Tam.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Borwein, J.M., Tam, M.K. A Cyclic Douglas–Rachford Iteration Scheme. J Optim Theory Appl 160, 1–29 (2014). https://doi.org/10.1007/s10957-013-0381-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-013-0381-x

Keywords

Search

Navigation