Numerical Algorithms

, Volume 41, Issue 3, pp 239–274 | Cite as

Extrapolation algorithm for affine-convex feasibility problems

  • Heinz H. Bauschke
  • Patrick L. Combettes
  • Serge G. Kruk

The convex feasibility problem under consideration is to find a common point of a countable family of closed affine subspaces and convex sets in a Hilbert space. To solve such problems, we propose a general parallel block-iterative algorithmic framework in which the affine subspaces are exploited to introduce extrapolated over-relaxations. This framework encompasses a wide range of projection, subgradient projection, proximal, and fixed point methods encountered in various branches of applied mathematics. The asymptotic behavior of the method is investigated and numerical experiments are provided to illustrate the benefits of the extrapolations.


affinite sets convex feasibility problem convex sets extrapolation Hilbert space Projection method 

AMS subject classification

90C25 47J25 47N10 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. Auslender, Méthodes Numériques pour la Résolution des Problèmes d’Optimisation avec Contraintes, Thèse, Faculté des Sciences, Grenoble, France (1969).Google Scholar
  2. [2]
    H.H. Bauschke, Projection algorithms and monotone operators, PhD Thesis, Department of Mathematics, Simon Fraser University, Burnaby, British Columbia, Canada (August 1996). Available at
  3. [3]
    H.H. Bauschke and J.M. Borwein, On projection algorithms for solving convex feasibility problems, SIAM Rev. 38 (1996) 367–426.CrossRefMATHMathSciNetGoogle Scholar
  4. [4]
    H.H. Bauschke and P.L. Combettes, A weak-to-strong convergence principle for Fejér-monotone methods in Hilbert spaces, Math. Oper. Res. 26 (2001) 248–264.CrossRefMATHMathSciNetGoogle Scholar
  5. [5]
    H.H. Bauschke, F. Deutsch, H. Hundal and S.-H. Park, Fejér monotonicity and weak convergence of an accelerated method of projections, in: Constructive, Experimental, and Nonlinear Analysis, ed. M. Théra (CMS Conference Proceedings 27, 2000) pp. 1–6.Google Scholar
  6. [6]
    H.H. Bauschke, F. Deutsch, H. Hundal, and S.-H. Park, Accelerating the convergence of the method of alternating projections, Trans. Am. Math. Soc. 355 (2003) 3433–3461.CrossRefMATHMathSciNetGoogle Scholar
  7. [7]
    H.H. Bauschke and S.G. Kruk, Reflection-projection method for convex feasibility problems with an obtuse cone, J. Optim. Theory Appl. 120 (2004) 503–531.CrossRefMATHMathSciNetGoogle Scholar
  8. [8]
    E. Blum and W. Oettli, From optimization and variational inequalities to equilibrium problems, Math. Stud. 63 (1994) 123–145.MATHMathSciNetGoogle Scholar
  9. [9]
    L.M. Bregman, The method of successive projection for finding a common point of convex sets, Sov. Math., Dokl. 6 (1965) 688–692.MATHGoogle Scholar
  10. [10]
    F.E. Browder, Convergence theorems for sequences of nonlinear operators in Banach spaces, Math. Z. 100 (1967) 201–225.CrossRefMATHMathSciNetGoogle Scholar
  11. [11]
    D. Butnariu and Y. Censor, On the behavior of a block-iterative projection method for solving convex feasibility problems, Int. J. Comput. Math. 34 (1990) 79–94.CrossRefGoogle Scholar
  12. [12]
    D. Butnariu, Y. Censor and S. Reich, eds., Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications (Elsevier, New York, 2001).MATHGoogle Scholar
  13. [13]
    A. Cauchy, Méthode générale pour la résolution des systèmes d’équations simultanées, C. R. Acad. Sci. Paris 25 (1847) 536–538.Google Scholar
  14. [14]
    Y. Censor, Iterative methods for the convex feasibility problem, Ann. Discrete Math. 20 (1984) 83–91.MathSciNetGoogle Scholar
  15. [15]
    Y. Censor and A. Lent, Cyclic subgradient projections, Math. Program. 24 (1982) 233–235.CrossRefMATHMathSciNetGoogle Scholar
  16. [16]
    Y. Censor and S.A. Zenios, Parallel Optimization: Theory, Algorithms, and Applications (Oxford University Press, New York, 1997).MATHGoogle Scholar
  17. [17]
    G. Cimmino, Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari, La Ricerca Scientifica (Roma) 1 (1938) 326–333.Google Scholar
  18. [18]
    P.L. Combettes, The foundations of set theoretic estimation, Proc. IEEE 81 (1993) 182–208.CrossRefGoogle Scholar
  19. [19]
    P.L. Combettes, Construction d’un point fixe commun à une famille de contractions fermes, C. R. Acad. Sci. Paris Sér. I Math. 320 (1995) 1385–1390.MATHMathSciNetGoogle Scholar
  20. [20]
    P.L. Combettes, The convex feasibility problem in image recovery, in: Advances in Imaging and Electron Physics, ed. P. Hawkes, Vol. 95, (Academic, New York, 1996) pp. 155–270.Google Scholar
  21. [21]
    P.L. Combettes, Convex set theoretic image recovery by extrapolated iterations of parallel subgradient projections, IEEE Trans. Image Process. 6 (1997) 493–506.CrossRefGoogle Scholar
  22. [22]
    P.L. Combettes, Hilbertian convex feasibility problem: Convergence of projection methods, Appl. Math. Optim. 35 (1997) 311–330.CrossRefMATHMathSciNetGoogle Scholar
  23. [23]
    P.L. Combettes, Quasi-Fejérian analysis of some optimization algorithms, in: Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications, eds. D. Butnariu, Y. Censor and S. Reich (Elsevier, New York, 2001) pp. 115–152.Google Scholar
  24. [24]
    P.L. Combettes, Solving monotone inclusions via compositions of nonexpansive averaged operators, Optimization 53 (2004) 475–504.CrossRefMATHMathSciNetGoogle Scholar
  25. [25]
    P.L. Combettes and S.A. Hirstoaga, Equilibrium programming in Hilbert spaces, J. Nonlinear Convex Anal. 6 (2005) 117–136.MATHMathSciNetGoogle Scholar
  26. [26]
    G. Crombez, Image recovery by convex combinations of projections, J. Math. Anal. Appl. 155 (1991) 413–419.CrossRefMATHMathSciNetGoogle Scholar
  27. [27]
    G. Crombez, Viewing parallel projection methods as sequential ones in convex feasibility problems, Trans. Am. Math. Soc. 347 (1995) 2575–2583.MATHCrossRefMathSciNetGoogle Scholar
  28. [28]
    A.R. De Pierro and A.N. Iusem, A parallel projection method for finding a common point of a family of convex sets, Pesqui. Oper. 5 (1985) 1–20.Google Scholar
  29. [29]
    F. Deutsch, The method of alternating orthogonal projections, in: Approximation Theory, Spline Functions and Applications, ed. S.P. Singh (Kluwer, The Netherlands, 1992) pp. 105–121.Google Scholar
  30. [30]
    F. Deutsch, Best Approximation in Inner Product Spaces (Springer, Berlin Heidelberg New York, 2001).MATHGoogle Scholar
  31. [31]
    L.T. Dos Santos, A parallel subgradient projections method for the convex feasibility problem, J. Comput. Appl. Math. 18 (1987) 307–320.CrossRefMATHMathSciNetGoogle Scholar
  32. [32]
    I.I. Eremin, Generalization of the relaxation method of Motzkin–Agmon, Uspekhi Mat. Nauk 20 (1965) 183–187.MATHMathSciNetGoogle Scholar
  33. [33]
    S.D. Flåm and J. Zowe, Relaxed outer projections, weighted averages, and convex feasibility, BIT 30 (1990) 289–300.CrossRefMATHMathSciNetGoogle Scholar
  34. [34]
    U. García-Palomares, Parallel-projected aggregation methods for solving the convex feasibility problem, SIAM J. Optim. 3 (1993) 882–900.CrossRefMATHMathSciNetGoogle Scholar
  35. [35]
    U.M. García-Palomares and F.J. González-Castaño, Incomplete projection algorithms for solving the convex feasibility problem, Numer. Algorithms 18 (1998) 177–193.CrossRefMATHMathSciNetGoogle Scholar
  36. [36]
    K. Goebel, and S. Reich, Uniform Convexity, Hyperbolic Geometry, and Nonexpansive Mappings (Marcel Dekker, New York, 1984).MATHGoogle Scholar
  37. [37]
    K. Goebel, and W.A. Kirk, Topics in Metric Fixed Point Theory (Cambridge University Press, Cambridge, 1990).MATHGoogle Scholar
  38. [38]
    F.J. González-Castaño, U.M. García-Palomares, J.L. Alba-Castro and J.M. Pousada-Carballo, Fast image recovery using dynamic load balancing in parallel architectures, by means of incomplete projections, IEEE Trans. Image Process. 10 (2001) 493–499.CrossRefMATHGoogle Scholar
  39. [39]
    A. Göpfert, H. Riahi, C. Tammer and C. Zaălinescu, Variational Methods in Partially Ordered Spaces (Springer, Berlin Heidelberg New York, 2003).MATHGoogle Scholar
  40. [40]
    L.G. Gubin, B.T. Polyak and E.V. Raik, The method of projections for finding the common point of convex sets, USSR Comput. Math. Math. Phys. 7 (1967) 1–24.CrossRefGoogle Scholar
  41. [41]
    S. Kaczmarz, Angenäherte Auflösung von Systemen linearer Gleichungen, Bull. Acad. Sci. Pologne A35 (1937) 355–357.Google Scholar
  42. [42]
    K.C. Kiwiel, Block-iterative surrogate projection methods for convex feasibility problems, Linear Algebra Appl. 215 (1995) 225–259.CrossRefMATHMathSciNetGoogle Scholar
  43. [43]
    K.C. Kiwiel and B. opuch, Surrogate projection methods for finding fixed points of firmly nonexpansive mappings, SIAM J. Optim. 7 (1997) 1084–1102.CrossRefMATHMathSciNetGoogle Scholar
  44. [44]
    N. Lehdili and B. Lemaire, The barycentric proximal method, Comm. Appl. Nonlinear Anal. 6 (1999) 29–47.MATHMathSciNetGoogle Scholar
  45. [45]
    Y.I. Merzlyakov, On a relaxation method of solving systems of linear inequalities, USSR Comput. Math. Math. Phys. 2 (1963) 504–510.CrossRefGoogle Scholar
  46. [46]
    W. Oettli, A remark on vector-valued equilibria and generalized monotonicity, Acta Math. Vietnam. 22 (1997) 215–221.MathSciNetGoogle Scholar
  47. [47]
    N. Ottavy, Strong convergence of projection-like methods in Hilbert spaces, J. Optim. Theory Appl. 56 (1988) 433–461.CrossRefMATHMathSciNetGoogle Scholar
  48. [48]
    G. Pierra, Éclatement de contraintes en parallèle pour la minimisation d’une forme quadratique, in: Lecture Notes in Computer Science, Vol. 41, (Springer, Berlin Heidelberg New York, 1976) pp. 200–218.Google Scholar
  49. [49]
    G. Pierra, Decomposition through formalization in a product space, Math. Programming 28 (1984) 96–115.MATHCrossRefMathSciNetGoogle Scholar
  50. [50]
    B.T. Polyak, Minimization of unsmooth functionals, USSR Comput. Math. Math. Phys. 9 (1969) 14–29.CrossRefGoogle Scholar
  51. [51]
    S. Reich, A limit theorem for projections, Linear Multilinear Algebra 13 (1983) 281–290.MATHCrossRefGoogle Scholar
  52. [52]
    R.T. Rockafellar, Monotone operators and the proximal point algorithm, SIAM J. Control Optim. 14 (1976) 877–898.CrossRefMATHMathSciNetGoogle Scholar
  53. [53]
    P. Tseng, On the convergence of products of firmly nonexpansive mappings, SIAM J. Optim. 2 (1992) 425–434.CrossRefMATHMathSciNetGoogle Scholar
  54. [54]
    I. Yamada, K. Slavakis and K. Yamada, An efficient robust adaptive filtering algorithm based on parallel subgradient projection techniques, IEEE Trans. Signal Process. 50 (2002) 1091–1101.CrossRefGoogle Scholar
  55. [55]
    I. Yamada and N. Ogura, Adaptive projected subgradient method for asymptotic minimization of sequence of nonnegative convex functions, Numer. Funct. Anal. Optim. 25 (2004) 593–617.CrossRefMATHMathSciNetGoogle Scholar
  56. [56]
    E. Zeidler, Nonlinear Functional Analysis and Its Applications II/B – Nonlinear Monotone Operators (Springer, Berlin Heidelberg New York, 1990).MATHGoogle Scholar

Copyright information

© Springer Science+Business Media, Inc. 2005

Authors and Affiliations

  • Heinz H. Bauschke
    • 1
  • Patrick L. Combettes
    • 2
  • Serge G. Kruk
    • 3
  1. 1.Mathematics, Irving K. Barber SchoolUBC OkanaganKelownaCanada
  2. 2.Laboratoire Jacques-Louis LionsUniversité Pierre et Marie Curie, Paris 6ParisFrance
  3. 3.Department of Mathematics and StatisticsOakland UniversityRochesterUSA

Personalised recommendations