Numerical Explorations of Feasibility Algorithms for Finding Points in the Intersection of Finite Sets

  • Heinz H. BauschkeEmail author
  • Sylvain Gretchko
  • Walaa M. Moursi


Projection methods are popular algorithms for iteratively solving feasibility problems in Euclidean or even Hilbert spaces. They employ (selections of) nearest point mappings to generate sequences that are designed to approximate a point in the intersection of a collection of constraint sets. Theoretical properties of projection methods are fairly well understood when the underlying constraint sets are convex; however, convergence results for the nonconvex case are more complicated and typically only local. In this paper, we explore the perhaps simplest instance of a feasibility algorithm, namely when each constraint set consists of only finitely many points. We numerically investigate four constellations: either few or many constraint sets, with either few or many points. Each constellation is tackled by four popular projection methods each of which features a tuning parameter. We examine the behaviour for a single and for a multitude of orbits, and we also consider local and global behaviour. Our findings demonstrate the importance of the choice of the algorithm and that of the tuning parameter.


Cyclic Douglas–Rachford algorithm Douglas–Rachford algorithm Extrapolated parallel projection method Method of cyclic projections Nonconvex feasibility problem Optimization algorithm Projection 

AMS 2010 Subject Classification

49M20 49M27 49M37 65K05 65K10 90C25 90C26 90C30 



We thank the referee for constructive comments and suggestions. The research of HHB was partially supported by NSERC.


  1. 1.
    Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38, 367–426 (1996)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, second edition. Springer, Cham (2017)CrossRefGoogle Scholar
  3. 3.
    Bauschke, H.H., Combettes, P.L., Kruk, S.G.: Extrapolation algorithm for affine-convex feasibility problems. Numer. Algorithms 41, 239–274 (2006)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Bauschke, H.H., Koch, V.R.: Projection methods: Swiss army knives for solving feasibility and best approximation problems with halfspaces. Contemp. Math. 636, 1–40 (2015) doi: 10.1090/conm/636/12726Google Scholar
  5. 5.
    Bauschke, H.H., Dao, M.N., Lindstrom, S.B.: The Douglas–Rachford algorithm for a hyperplane and a doubleton. J. Glob. Optim. 74, 79–93 (2019)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Bauschke, H.H., Moursi, W.M.: On the Douglas–Rachford algorithm. Math. Prog. (Ser. A) 164, 263–284 (2017)Google Scholar
  7. 7.
    Bauschke, H.H., Noll, D., Phan, H.M.: Linear and strong convergence of algorithms involving averaged nonexpansive operators. J. Math. Anal. Appl. 421, 1–20 (2015)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Borwein, J.M., Lindstrom, S.B., Sims, B., Schneider, A., Skerritt, M.P.: Dynamics of the Douglas–Rachford method for ellipses and p-spheres. Set-Valued Var. Anal. 26, 385–403 (2018)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Borwein, J.M., Tam, M.K.: A cyclic Douglas–Rachford iteration scheme. J. Optim. Th. Appl. 160, 1–29 (2014)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Cegielski, A.: Iterative methods for fixed point problems in Hilbert spaces. Springer, Heidelberg (2012)zbMATHGoogle Scholar
  11. 11.
    Censor, Y., Chen, W., Combettes, P.L., Davidi, R., Herman, G.T.: On the effectiveness of projection methods for convex feasibility problems Extrapolation algorithm for affine-convex feasibility problems. Numer. Algorithms 41, 239–274 (2006)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Censor, Y., Zaknoon, M.: Algorithms and convergence results of projection methods for inconsistent feasibility problems: a review. Pure Appl. Funct. Anal. 3, 565–586 (2018) [math.OC] (2018)
  13. 13.
    Censor, Y., Zenios, S.A.: Parallel Optimization. Oxford University Press (1997)Google Scholar
  14. 14.
    Combettes, P.L.: Convex set theoretic image recovery by extrapolated iterations of parallel subgradient projections. IEEE Trans. Image Process. 6, 493–506 (1997)CrossRefGoogle Scholar
  15. 15.
    Combettes, P.L.: Hilbertian convex feasibility problems: convergence of projection methods. Appl. Math. Optim. 35, 311–330 (1997)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Dao, M., Phan, H.M.: Linear convergence of the generalized Douglas–Rachford algorithm for feasibility problems. J. Glob. Optim. 72, 443–474 (2018)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Prog. (Ser. A) 55, 293–318 (1992)Google Scholar
  18. 18.
    Elser, V., Rankenburg, I., Thibault, P.: Searching with iterated maps. Proc. Natl. Acad. Sci. U.S.A. 104, 418–423 (2007)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Lions, P.-L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Luke, D.R.: Finding best approximation pairs relative to a convex and a prox-regular set in a Hilbert space. SIAM J. Optim. 19, 714–739 (2008)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Luke, D.R., Sabach, S., Teboulle, M.: Optimization on spheres: models and proximal algorithms with computational performance comparisons. [math.OC] (2018)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Heinz H. Bauschke
    • 1
    Email author
  • Sylvain Gretchko
    • 2
  • Walaa M. Moursi
    • 3
    • 4
  1. 1.Department of MathematicsUniversity of British ColumbiaKelownaCanada
  2. 2.Mathematics, UBCOKelownaCanada
  3. 3.Electrical EngineeringStanford UniversityStanfordUSA
  4. 4.Faculty of Science, Mathematics DepartmentMansoura UniversityMansouraEgypt

Personalised recommendations