Skip to main content
SpringerLink
Log in

Optimal Rates of Linear Convergence of Relaxed Alternating Projections and Generalized Douglas-Rachford Methods for Two Subspaces

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

We systematically study the optimal linear convergence rates for several relaxed alternating projection methods and the generalized Douglas-Rachford splitting methods for finding the projection on the intersection of two subspaces. Our analysis is based on a study on the linear convergence rates of the powers of matrices. We show that the optimal linear convergence rate of powers of matrices is attained if and only if all subdominant eigenvalues of the matrix are semisimple. For the convenience of computation, a nonlinear approach to the partially relaxed alternating projection method with at least the same optimal convergence rate is also provided. Numerical experiments validate our convergence analysis

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aragón Artacho, F.J., Borwein, J.M., Tam, M.K.: Recent results on Douglas-Rachford methods for combinatorial optimization problems. J. Optim. Theory Appl. 163, 1–30 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38, 367–426 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bauschke, H.H., Combettes, P.L.: Convex analysis and monotone operator theory in Hilbert spaces. Springer (2011)

  4. Bauschke, H.H., Combettes, P.L., Kruk, S.G.: Extrapolation algorithm for affine-convex feasibility problems. Numer. Algorithms 4, 239–274 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bauschke, H.H., Bello Cruz, J.Y., Nghia, T.T.A., Phan, H.M., Wang, X.: The rate of linear convergence of the Douglas-Rachford algorithm for subspaces is the cosine of the Friedrichs angle. J. Approx. Theory 185, 63–79 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  6. Bauschke, H.H., Deutsch, F., Hundal, H., Park, S.-H.: Accelerating the convergence of the method of alternating projections. Trans. Amer. Math. Soc. 355, 3433–3461 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bauschke, H.H., Kruk, S.G.: Reflection-projection method for convex feasibility problems with an obtuse cone. J. Optim. Theory Appl. 120, 503–531 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bauschke, H.H., Dao, M.N., Noll, D., Phan, H.M.: Proximal point algorithm, Douglas-Rachford algorithm and alternating projections: a case study. J Convex Anal. (to appear)

  9. Björck, A., Golub, G.H.: Numerical methods for computing angles between linear subspaces. Math. Comp. 27, 579–594 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  10. Berman, A., Plemmons, R.J.: Nonnegative matrices in the mathematical sciences. SIAM, Philadelphia (1994)

    Book  MATH  Google Scholar 

  11. Cegielski, A.: Iterative methods for fixed point problems in Hilbert spaces. Springer (2012)

  12. Cegielski, A., Suchocka, A.: Relaxed alternating projection methods. SIAM J. Optim. 19, 1093–1106 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  13. Censor, Y., Elfving, T., Herman, G.T., Nikazad, T.: On diagonally relaxed orthogonal projection methods. SIAM J. Sci. Comput. 30, 473–504 (2007/08)

  14. Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing, Fixed-point algorithms for inverse problems in science and engineering, 185–212, Springer Optim. Appl., 49. Springer, New York (2011)

    Google Scholar 

  15. Combettes, P.L.: Solving monotone inclusions via compositions of nonexpansive averaged operators. Optimization 53, 475–504 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  16. Demanet, L., Zhang, X.: Eventual linear convergence of the Douglas-Rachford iteration for basis pursuit. Math. Comput. 85, 209–238 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  17. Deutsch, F.: The angle between subspaces of a Hilbert space. In: Singh, S.P. (ed.) Approximation theory, wavelets and applications, pp 107–130. Kluwer (1995)

  18. Deutsch, F.: Best approximations in inner product spaces. Springer (2001)

  19. Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Amer. Math. Soc. 82, 421–439 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  20. Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  21. Elfving, T., Nikazad, T., Hansen, P.C.: Semi-convergence and relaxation parameters for a class of SIRT algorithms. Electron. Trans. Numer. Anal. 37, 321–336 (2010)

    MathSciNet  MATH  Google Scholar 

  22. Escalante, R., Raydan, M.: Alternating projection methods, fundamentals of algorithms 8. SIAM, Philadelphia (2011)

    Book  MATH  Google Scholar 

  23. Gearhart, W.B., Koshy, M.: Acceleration schemes for the method of alternating projections. J. Comp. Appl. Math. 26, 235–249 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  24. GNU Plot: http://sourceforge.net/projects/gnuplot

  25. Gubin, L.G., Polyak, B.T., Raik, E.V.: The method of projections for finding the common point of convex sets. USSR Comp. Math. Math. Phys. 7, 1–24 (1967)

    Article  Google Scholar 

  26. Hensel, K.: Über Potenzreihen von Matrizen. J. Reine Angew. Math. 155, 107–110 (1926)

    MathSciNet  MATH  Google Scholar 

  27. Kayalar, S., Weinert, H.: Error bounds for the method of alternating projections. Math. Control Signals Syst. 1, 43–59 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  28. Kirkland, S.: A cycle-based bound for subdominant eigenvalues of stochastic matrices. Linear Multilinear Algebra 57, 247–266 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  29. Hesse, R., Luke, D.R., Neumann, P.: Alternating projections and Douglas-Rachford for sparse affine feasibility. IEEE Trans. Signal Process. 62, 4868–4881 (2014)

    Article  MathSciNet  Google Scholar 

  30. Hesse, R., Luke, D.R.: Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems. SIAM J. Optim. 23, 2397–2419 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  31. The Julia language: http://julialang.org/

  32. Lions, P.-L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  33. Luke, D.R.: Finding best approximation pairs relative to a convex and prox-regular set in a Hilbert space. SIAM J. Optim. 19, 714–739 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  34. Marek, I.: On square roots of M-operators. Linear Algebra Appl. 223(/224), 501–520 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  35. Marek, I., Szyld, D.B.: Comparison theorems for the convergence factor of iterative methods for singular matrices. Linear Algebra Appl. 316, 67–87 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  36. Marek, I., Szyld, D.B.: Comparison of convergence of general stationary iterative methods for singular matrices. SIAM J. Matrix Anal. Appl. 24, 68–77 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  37. Meyer, C.D.: Matrix analysis and applied linear algebra. SIAM, Philadelphia (2000)

    Book  Google Scholar 

  38. Meyer, C.D., Plemmons, R.J.: Convergent powers of a matrix with applications to iterative methods for singular linear systems. SIAM J. Numer. Anal. 14, 699–705 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  39. Miao, J., Ben-Israel, A.: On principal angles between subspaces in ℝn. Linear Algebra Appl. 171, 81–98 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  40. Nelson, S., Neumann, M.: Generalizations of the projection method with applications to SOR theory for Hermitian positive semidefinite linear systems. Numer. Math. 51, 123–141 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  41. Oldenburger, R.: Infinite powers of matrices and characteristic roots. Duke Math. J. 6, 357–361 (1940)

    Article  MathSciNet  MATH  Google Scholar 

  42. Saad, Y.: Iterative methods for sparse linear systems. SIAM, Philadelphia (2003)

    Book  MATH  Google Scholar 

  43. Stewart, G.W.: Matrix algorithms, II: eigensystems. SIAM, Philadelphia (2001)

Download references

Author information

Authors and Affiliations

  1. Mathematics, University of British Columbia, Kelowna, BC, V1V 1V7, Canada

    Heinz H. Bauschke & Xianfu Wang

  2. IME, Federal University of Goias, Goiania, GO, 74001-970, Brazil

    J. Y. Bello Cruz

  3. Mathematics & Statistics, Oakland University, Rochester, MI, 48309, USA

    Tran T. A. Nghia

  4. Department of Mathematical Sciences, University of Massachusetts Lowell, 265 Riverside St. Olney Hall 428, Lowell, MA, 01854, USA

    Hung M. Pha

Authors
  1. Heinz H. Bauschke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Y. Bello Cruz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tran T. A. Nghia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hung M. Pha
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xianfu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianfu Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bauschke, H.H., Bello Cruz, J.Y., Nghia, T.T.A. et al. Optimal Rates of Linear Convergence of Relaxed Alternating Projections and Generalized Douglas-Rachford Methods for Two Subspaces. Numer Algor 73, 33–76 (2016). https://doi.org/10.1007/s11075-015-0085-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-015-0085-4

Keywords

Mathematics Subject Classification (2010)

Search

Navigation