Applied Mathematics and Optimization

, Volume 35, Issue 3, pp 311–330 | Cite as

Hilbertian convex feasibility problem: Convergence of projection methods

  • P. L. Combettes


The classical problem of finding a point in the intersection of countably many closed and convex sets in a Hilbert space is considered. Extrapolated iterations of convex combinations of approximate projections onto subfamilies of sets are investigated to solve this problem. General hypotheses are made on the regularity of the sets and various strategies are considered to control the order in which the sets are selected. Weak and strong convergence results are established within thisbroad framework, which provides a unified view of projection methods for solving hilbertian convex feasibility problems.

Key Words

Alternating projections Boundedly regular sets Chaotic iterations Convergence Convex feasibility problem Convex sets Extrapolated projections Fejér-monotone sequences Hilbert spaces Parallel projections Relaxations Successive projections 

AMS Classification

90C25 65J05 52A41 40A05 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agmon S (1954) The relaxation method for linear inequalities. Canad J Math 6:382–392zbMATHGoogle Scholar
  2. 2.
    Aharoni R, Berman A, Censor Y (1983) An interior points algorithm for the convex feasibility problem. Adv in Appl Math 4:479–489zbMATHCrossRefGoogle Scholar
  3. 3.
    Aharoni R, Censor Y (1989) Block-iterative methods for parallel computation of solutions to convex feasibility problems. Linear Algebra Appl 120:165–175zbMATHCrossRefGoogle Scholar
  4. 4.
    Alber Y, Butnariu D (1997) Convergence of Brègman-projection methods for solving consistent convex feasibility problems in reflexive Banach spaces. J Optim Theory Appl 92:33–61zbMATHCrossRefGoogle Scholar
  5. 5.
    Amemiya I, Ando T (1965) Convergence of random products of contractions in Hilbert space. Acta Sci Math (Szeged) 26:239–244zbMATHGoogle Scholar
  6. 6.
    Auslender A (1969) Méthodes Numériques pour la Résolution des Problèmes d’Optimisation avec Contraintes. Thèse, Faculté des Sciences, GrenobleGoogle Scholar
  7. 7.
    Bauschke HH (1995) A norm convergence result on random products of relaxed projections in Hilbert space. Trans Amer Math Soc 347:1365–1373zbMATHCrossRefGoogle Scholar
  8. 8.
    Bauschke HH, Borwein JM (1996) On projection algorithms for solving convex feasibility problems. SIAM Rev 38:367–426zbMATHCrossRefGoogle Scholar
  9. 9.
    Bauschke HH, Borwein JM, Lewis AS (1996) On the method of cyclic projections for closed convex sets in Hilbert space. To appear in Contemporary Mathematics: Optimization and Nonlinear Analysis (Censor Y, Reich S, editors). American Mathematical Society, Providence, RIGoogle Scholar
  10. 10.
    Brègman LM (1965) The method of successive projection for finding a common point of convex sets. Soviet Math Dokl 6:688–692zbMATHGoogle Scholar
  11. 11.
    Brézis H (1983) Analyse Fonctionnelle. Masson, PariszbMATHGoogle Scholar
  12. 12.
    Browder FE (1958) On some approximation methods for solutions of the Dirichlet problem for linear elliptic equations of arbitrary order. J Math Mech 7:69–80zbMATHGoogle Scholar
  13. 13.
    Browder FE (1967) Convergence theorems for sequences of nonlinear operators in Banach spaces. Math Z 100:201–225zbMATHCrossRefGoogle Scholar
  14. 14.
    Bruck RE (1982) Random products of contractions in metric and Banach spaces. J Math Anal Appl 88:319–332zbMATHCrossRefGoogle Scholar
  15. 15.
    Butnariu D, Flåm SD (1995) Strong convergence of expected-projection methods in Hilbert spaces. Numer Funct Anal Optim 16:601–636zbMATHGoogle Scholar
  16. 16.
    Censor Y (1981) Row-action methods for huge and sparse systems and their applications. SIAM Rev 23:444–466zbMATHCrossRefGoogle Scholar
  17. 17.
    Censor Y, Lent A (1982) Cyclic subgradient projections. Math Programming 24:233–235zbMATHCrossRefGoogle Scholar
  18. 18.
    Cheney W, Goldstein AA (1959) Proximity maps for convex sets. Proc Amer Math Soc 10:448–450zbMATHCrossRefGoogle Scholar
  19. 19.
    Cimmino G (1938) Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari. Ricerca Sci (Roma) 1:326–333Google Scholar
  20. 20.
    Combettes PL (1993) The foundations of set theoretic estimation. Proc IEEE 81:182–208CrossRefGoogle Scholar
  21. 21.
    Combettes PL (1995) Construction d’un point fixe commun à une famille de contractions fermes. C R Acad Sci Paris Sér I Math 320:1385–1390zbMATHGoogle Scholar
  22. 22.
    Combettes PL (1996) The convex feasibility problem in image recovery. Advances in Imaging and Electron Physics (Hawkes P, editor), vol 95, pp 155–270. Academic Press, New YorkGoogle Scholar
  23. 23.
    Combettes PL, Puh H (1993) Extrapolated projection method for the euclidean convex feasibility problem. Research report, City University of New YorkGoogle Scholar
  24. 24.
    Combettes PL, Puh H (1994) Iterations of parallel convex projections in Hilbert spaces. Numer Funct Anal Optim 15:225–243zbMATHCrossRefGoogle Scholar
  25. 25.
    Combettes PL, Trussell HJ (1990) Method of successive projections for finding a common point of sets in metric spaces. J Optim Theory Appl 67:487–507zbMATHCrossRefGoogle Scholar
  26. 26.
    Crombez G (1991) Image recovery by convex combinations of projections. J Math Anal Appl 155:413–419zbMATHCrossRefGoogle Scholar
  27. 27.
    De Pierro AR, Iusem AN (1985) A parallel projection method for finding a common point of a family of convex sets. Pesqui Oper 5:1–20Google Scholar
  28. 28.
    De Pierro AR, Iusem AN (1988) A finitely convergent “row-action” method for the convex feasibility problem. Appl Math Optim 17:225–235zbMATHCrossRefGoogle Scholar
  29. 29.
    Deutsch F (1992) The method of alternating orthogonal projections. In Approximation Theory, Spline Functions and Application (Singh SP, editor), pp 105–121. Kluwer, The NetherlandsGoogle Scholar
  30. 30.
    Dye JM, Reich S (1992) Unrestricted iterations of nonexpansive mappings in Hilbert space. Nonlinear Anal 18:199–207zbMATHCrossRefGoogle Scholar
  31. 31.
    Eremin II (1965) Generalization of the relaxation method of Motzkin-Agmon. Uspekhi Mat Nauk 20:183–187zbMATHGoogle Scholar
  32. 32.
    Flåm SD, Zowe J (1990) Relaxed outer projections, weighted averages, and convex feasibility. BIT 30:289–300zbMATHCrossRefGoogle Scholar
  33. 33.
    Gurin (Gubin) LG, Polyak BT, Raik EV (1967) The method of projections for finding the common point of convex sets. USSR Comput Math and Math Phys 7:1–24CrossRefGoogle Scholar
  34. 34.
    Halperin I (1962) The product of projection operators. Acta Sci Math (Szeged) 23:96–99zbMATHGoogle Scholar
  35. 35.
    Iusem AN, De Pierro AR (1986) Convergence results for an accelerated nonlinear Cimmino algorithm. Numer Math 49:367–378zbMATHCrossRefGoogle Scholar
  36. 36.
    Iusem AN, Moledo L (1986) A finitely convergent method of simultaneous subgradient projections for the convex feasibility problem. Mat Apl Comput 5:169–184zbMATHGoogle Scholar
  37. 37.
    Kaczmarz S (1937) Angenäherte Auflösung von Systemen linearer Gleichungen. Bull Acad Sci Pologne A35:355–357.Google Scholar
  38. 38.
    Kiwiel KC (1995) Block-iterative surrogate projection methods for convex feasibility problems. Linear Algebra Appl 215:225–259zbMATHCrossRefGoogle Scholar
  39. 39.
    Levitin ES, Polyak BT (1966) Convergence of minimizing sequences in conditional extremum problems. Soviet Math Dokl 7:764–767zbMATHGoogle Scholar
  40. 40.
    Merzlyakov YI (1963) On a relaxation method of solving systems of linear inequalities. USSR Comput Math and Math Phys 2:504–510CrossRefGoogle Scholar
  41. 41.
    Motzkin TS, Schoenberg IJ (1954) The relaxation method for linear inequalities. Canad J Math 6:393–404zbMATHGoogle Scholar
  42. 42.
    Nashed MZ (1981) Continuous and semicontinuous analogues of iterative methods of Cimmino and Kaczmarz with applications to the inverse Radon transform. Lectures Notes in Medical Informatics (Herman GT, Natterer F, editors), vol 8, pp 160–178. Springer-Verlag, New YorkGoogle Scholar
  43. 43.
    Ottavy N (1988) Strong convergence of projection-like methods in Hilbert spaces. J Optim Theory Appl 56:433–461zbMATHCrossRefGoogle Scholar
  44. 44.
    Pierra G (1975) Méthodes de projections parallèles extrapolées relatives à une intersection de convexes. Rapport de Recherche, INPG, GrenobleGoogle Scholar
  45. 45.
    Pierra G (1984) Decomposition through formalization in a product space. Math Programming 28:96–115zbMATHCrossRefGoogle Scholar
  46. 46.
    Pierra G, Ottavy N (1988) New parallel projection methods for linear varieties and applications. Presented at the XIIth International Symposium on Mathematical Programming, TokyoGoogle Scholar
  47. 47.
    Poincaré H (1890) Sur les équations aux dérivées partielles de la physique mathématique. Amer J Math 12:211–294CrossRefGoogle Scholar
  48. 48.
    Polyak BT (1969) Minimization of unsmooth functionals. USSR Comput Math and Math Phys 9:14–29CrossRefGoogle Scholar
  49. 49.
    Práger M (1960) On a principle of convergence in Hilbert space. Czechoslovak Math J 10:271–282Google Scholar
  50. 50.
    Reich S (1983) A limit theorem for projections. Linear and Multilinear Algebra 13:281–290zbMATHCrossRefGoogle Scholar
  51. 51.
    Stiles WJ (1965) Closest point maps and their product II. Nieuw Arch Wisk 13:212–225Google Scholar
  52. 52.
    Tseng P (1992) On the convergence of products of firmly nonexpansive mappings. SIAM J Optim 2:425–434zbMATHCrossRefGoogle Scholar
  53. 53.
    Tsent P, Bertsekas DP (1987) Relaxation methods for problems with strictly convex separable costs and linear constraints. Math Programming 38:303–321CrossRefGoogle Scholar
  54. 54.
    Von Neumann J (1949) On rings of operators. Reduction theory. Ann of Math 50:401–485 (th result of interest first appeared in 1933 in lecture notes)CrossRefGoogle Scholar
  55. 55.
    Zarantonello EH (1971) Projections on convex sets in Hilbert space and spectral theory. In Contributions to Nonlinear Functional Analysis (Zarantonello EH editor), pp 237–424. Academic Press, New YorkGoogle Scholar

Copyright information

© Springer-Verlag New York Inc 1997

Authors and Affiliations

  • P. L. Combettes
    • 1
  1. 1.Department of Electrical Engineering, City College and Graduate SchoolCity University of New YorkNew YorkUSA

Personalised recommendations