Computational Optimization and Applications

, Volume 51, Issue 3, pp 1065–1088 | Cite as

On the effectiveness of projection methods for convex feasibility problems with linear inequality constraints

  • Yair Censor
  • Wei Chen
  • Patrick L. Combettes
  • Ran Davidi
  • Gabor T. Herman


The effectiveness of projection methods for solving systems of linear inequalities is investigated. It is shown that they often have a computational advantage over alternatives that have been proposed for solving the same problem and that this makes them successful in many real-world applications. This is supported by experimental evidence provided in this paper on problems of various sizes (up to tens of thousands of unknowns satisfying up to hundreds of thousands of constraints) and by a discussion of the demonstrated efficacy of projection methods in numerous scientific publications and commercial patents (dealing with problems that can have over a billion unknowns and a similar number of constraints).


Projection methods Convex feasibility problems Numerical evaluation Optimization Linear inequalities Sparse matrices 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agmon, S.: The relaxation method for linear inequalities. Can. J. Math. 6, 382–392 (1954) MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Aharoni, R., Censor, Y.: Block-iterative projection methods for parallel computation of solutions to convex feasibility problems. Linear Algebra Appl. 120, 165–175 (1989) MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Andersen, E.D., Andersen, K.D.: The MOSEK interior point optimizer for linear programming: an implementation of the homogeneous algorithm. In: Frenk, H., Roos, K., Terlaky, T., Zhang, S. (eds.) High Performance Optimization, pp. 197–232. Kluwer, Boston (2000) Google Scholar
  4. 4.
    Andrews, H.C., Hunt, B.R.: Digital Image Restoration. Englewood Cliffs, Prentice-Hall (1977) Google Scholar
  5. 5.
    Auslender, A.: Méthodes Numériques pour la Résolution des Problèmes d’Optimisation avec Contraintes. Thèse, Faculté des Sciences, Grenoble (1969) Google Scholar
  6. 6.
    Auslender, A.: Optimisation – Méthodes Numériques. Masson, Paris (1976) zbMATHGoogle Scholar
  7. 7.
    Bauschke, H.H.: The approximation of fixed points of compositions of nonexpansive mappings in Hilbert space. J. Math. Anal. Appl. 202, 150–159 (1996) MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38, 367–426 (1996) MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Bauschke, H.H., Combettes, P.L.: A weak-to-strong convergence principle for Fejér-monotone methods in Hilbert spaces. Math. Oper. Res. 26, 248–264 (2001) MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Bauschke, H.H., Combettes, P.L., Kruk, S.G.: Extrapolation algorithm for affine-convex feasibility problems. Numer. Algorithms 41, 239–274 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Bauschke, H.H., Deutsch, F., Hundal, H., Park, S.-H.: Accelerating the convergence of the method of alternating projections. Trans. Am. Math. Soc. 355, 3433–3461 (2003) MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Bauschke, H.H., Matoušková, E., Reich, S.: Projection and proximal point methods: convergence results and counterexamples. Nonlinear Anal. 56, 715–738 (2004) MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Bilbao-Castro, J.R., Marabini, R., Sorzano, C.O.S., García, I., Carazo, J.M., Fernández, J.J.: Exploiting desktop supercomputing for three-dimensional electron microscopy reconstructions using ART with blobs. J. Struct. Biol. 65, 19–26 (2009) CrossRefGoogle Scholar
  14. 14.
    Bioucas-Dias, J., Figueiredo, M.: A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Image Process. 16, 2992–3004 (2007) MathSciNetCrossRefGoogle Scholar
  15. 15.
    Blatt, D., Hero, A.O., III: Energy based sensor network source localization via projection onto convex sets (POCS). IEEE Trans. Signal Process. 54, 3614–3619 (2006) CrossRefGoogle Scholar
  16. 16.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004) zbMATHGoogle Scholar
  17. 17.
    Brègman, L.M.: The method of successive projection for finding a common point of convex sets. Sov. Math. Dokl. 6, 688–692 (1965) zbMATHGoogle Scholar
  18. 18.
    Butnariu, D., Censor, Y.: On the behavior of a block-iterative projection method for solving convex feasibility problems. Int. J. Comput. Math. 34, 79–94 (1990) zbMATHCrossRefGoogle Scholar
  19. 19.
    Butnariu, D., Censor, Y., Reich, S. (eds.): Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications. Elsevier, Amsterdam (2001) zbMATHGoogle Scholar
  20. 20.
    Butnariu, D., Davidi, R., Herman, G.T., Kazansev, I.G.: Stable convergence behavior under summable perturbations of a class of projection methods for convex feasibility and optimization problems. IEEE J. Sel. Top. Signal Process. 1, 540–547 (2007) CrossRefGoogle Scholar
  21. 21.
    Carazo, J.M., Herman, G.T., Sorzano, C.O.S., Marabini, R.: Algorithms for three-dimensional reconstruction from imperfect projection data provided by electron microscopy. In: Frank, J. (ed.) Electron Tomography: Methods for Three-Dimensional Visualization of Structures in the Cell, 2nd edn., pp. 217–243. Springer, New York (2006) Google Scholar
  22. 22.
    Censor, Y.: Row-action methods for huge and sparse systems and their applications. SIAM Rev. 23, 444–466 (1981) MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Censor, Y., Altschuler, M.D., Powlis, W.D.: On the use of Cimmino’s simultaneous projections method for computing a solution of the inverse problem in radiation therapy treatment planning. Inverse Probl. 4, 607–623 (1988) MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Censor, Y., Davidi, R., Herman, G.T.: Perturbation resilience and superiorization of iterative algorithms. Inverse Probl. 26, 065008 (2010) MathSciNetCrossRefGoogle Scholar
  25. 25.
    Censor, Y., Elfving, T., Herman, G.T.: Averaging strings of sequential iterations for convex feasibility problems. In: Butnariu, D., Censor, Y., Reich, S. (eds.) Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications, pp. 101–114. Elsevier, Amsterdam (2001) Google Scholar
  26. 26.
    Censor, Y., Gordon, D., Gordon, R.: BICAV: A block-iterative, parallel algorithm for sparse systems with pixel-related weighting. IEEE Trans. Med. Imaging 20, 1050–1060 (2001) CrossRefGoogle Scholar
  27. 27.
    Censor, Y., Segal, A.: On the string averaging method for sparse common fixed points problems. Int. Trans. Oper. Res. 16, 481–494 (2009) MathSciNetzbMATHGoogle Scholar
  28. 28.
    Censor, Y., Tom, E.: Convergence of string-averaging projection schemes for inconsistent convex feasibility problems. Optim. Methods Softw. 18, 543–554 (2003) MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Censor, Y., Zenios, S.A.: Parallel Optimization: Theory, Algorithms, and Applications. Oxford University Press, New York (1997) zbMATHGoogle Scholar
  30. 30.
    Cetin, A.E., Ozaktas, H., Ozaktas, H.M.: Resolution enhancement of low resolution wavefields with POCS algorithm. Electron. Lett. 39, 1808–1810 (2003) CrossRefGoogle Scholar
  31. 31.
    Chen, W., Herman, G.T.: Effcient controls for finitely convergent sequential algorithms. ACM Trans. Math. Softw. 37, Article No. 14 (2010) MathSciNetCrossRefGoogle Scholar
  32. 32.
    Chen, W., Craft, D., Madden, T.M., Zhang, K., Kooy, H.M., Herman, G.T.: A fast optimization algorithm for multi-criteria intensity modulated proton therapy planning. Med. Phys. 7, 4938–4945 (2010) CrossRefGoogle Scholar
  33. 33.
    Chiang, S., Cardi, C., Matej, S., Zhuang, H., Newberg, A., Alavi, A., Karp, J.S.: Clinical validation of fully 3-D versus 2.5-D RAMLA reconstruction on the Phillips-ADAC CPET PET scanner. Nucl. Med. Commun. 25, 1103–1107 (2004) CrossRefGoogle Scholar
  34. 34.
    Choi, H., Baraniuk, R.G.: Multiple wavelet basis image denoising using Besov ball projections. IEEE Signal Process. Lett. 11, 717–720 (2004) CrossRefGoogle Scholar
  35. 35.
    Cimmino, G.: Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari. Ric. Sci. (Roma) 1, 326–333 (1938) Google Scholar
  36. 36.
    Combettes, P.L.: The foundations of set theoretic estimation. Proc. IEEE 81, 182–208 (1993) CrossRefGoogle Scholar
  37. 37.
    Combettes, P.L.: The convex feasibility problem in image recovery. Adv. Imaging Electron Phys. 95, 155–270 (1996) CrossRefGoogle Scholar
  38. 38.
    Combettes, P.L.: Hilbertian convex feasibility problem: Convergence of projection methods. Appl. Math. Optim. 35, 311–330 (1997) MathSciNetzbMATHGoogle Scholar
  39. 39.
    Combettes, P.L.: Convex set theoretic image recovery by extrapolated iterations of parallel subgradient projections. IEEE Trans. Image Process. 6, 493–506 (1997) CrossRefGoogle Scholar
  40. 40.
    Combettes, P.L., Hirstoaga, S.A.: Equilibrium programming in Hilbert spaces. J. Nonlinear Convex Anal. 6, 117–136 (2005) MathSciNetzbMATHGoogle Scholar
  41. 41.
    Combettes, P.L., Trussell, H.J.: Methods for digital restoration of signals degraded by a stochastic impulse response. IEEE Trans. Acoust. Speech Signal Process. 37, 393–401 (1989) CrossRefGoogle Scholar
  42. 42.
    Cottle, R.W., Pang, J.-S.: On solving linear complementarity problems as linear programs. Math. Program. Stud. 7, 88–107 (1978) MathSciNetzbMATHCrossRefGoogle Scholar
  43. 43.
    Crombez, G.: Improving the speed of convergence in the method of projections onto convex sets. Publ. Math. (Debr.) 58, 29–48 (2001) MathSciNetzbMATHGoogle Scholar
  44. 44.
    Crombez, G.: Finding common fixed points of strict paracontractions by averaging strings of sequential iterations. J. Nonlinear Convex Anal. 3, 345–351 (2002) MathSciNetzbMATHGoogle Scholar
  45. 45.
    Davidi, R., Herman, G.T., Censor, Y.: Perturbation-resilient block-iterative projection methods with application to image reconstruction from projections. Int. Trans. Oper. Res. 16, 505–524 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  46. 46.
    Davidi, R., Herman, G.T., Klukowska, J.: SNARK09: A programming system for the reconstruction of 2D images from 1D projections. (2011). Accessed 25 February 2011
  47. 47.
    Deutsch, F.: Best Approximation in Inner Product Spaces. Springer, New York (2001) zbMATHGoogle Scholar
  48. 48.
    Eckstein, J., Svaiter, B.F.: General projective splitting methods for sums of maximal monotone operators. SIAM J. Control Optim. 48, 787–811 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  49. 49.
    Eggermont, P.P.B., Herman, G.T., Lent, A.: Iterative algorithms for large partitioned linear systems, with applications to image reconstruction. Linear Algebra Appl. 40, 37–67 (1981) MathSciNetzbMATHCrossRefGoogle Scholar
  50. 50.
    Goldstein, T., Osher, S.: The split Bregman method for L1 regularized problems. SIAM J. Imaging Sci. 2, 323–343 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  51. 51.
    Goldstein, T., Osher, S.: Personal communication (2010) Google Scholar
  52. 52.
    Golub, G.H., van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins University Press, Baltimore (1996) zbMATHGoogle Scholar
  53. 53.
    González-Castaño, F.J., García-Palomares, U.M., Alba-Castro, J.L., Pousada-Carballo, J.M.: Fast image recovery using dynamic load balancing in parallel architectures, by means of incomplete projections. IEEE Trans. Image Process. 10, 493–499 (2001) zbMATHCrossRefGoogle Scholar
  54. 54.
    Gordon, R., Bender, R., Herman, G.T.: Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography. J. Theor. Biol. 29, 471–482 (1970) CrossRefGoogle Scholar
  55. 55.
    Gould, N.I.M.: How good are projection methods for convex feasibility problems? Comput. Optim. Appl. 40, 1–12 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  56. 56.
    Gu, J., Stark, H., Yang, Y.: Wide-band smart antenna design using vector space projection methods. IEEE Trans. Antennas Propag. 52, 3228–3236 (2004) CrossRefGoogle Scholar
  57. 57.
    Gubin, L.G., Polyak, B.T., Raik, E.V.: The method of projections for finding the common point of convex sets. U.S.S.R. Comput. Math. Math. Phys. 7, 1–24 (1967) CrossRefGoogle Scholar
  58. 58.
    Herman, G.T.: Fundamentals of Computerized Tomography: Image Reconstruction from Projections, 2nd edn. Springer, London (2009) Google Scholar
  59. 59.
    Herman, G.T., Chen, W.: A fast algorithm for solving a linear feasibility problem with application to intensity-modulated radiation therapy. Linear Algebra Appl. 428, 1207–1217 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  60. 60.
    Hounsfield, G.N.: A method and apparatus for examination of a body by radiation such as X or gamma radiation. UK Patent No. 1283915 (1968/72) Google Scholar
  61. 61.
    Isola, A.A., Ziegler, A., Koehler, T., Niessen, W.J., Grass, M.: Motion compensated iterative cone-beam CT image reconstruction with adapted blobs as basis functions. Phys. Med. Biol. 53, 6777–6797 (2008) CrossRefGoogle Scholar
  62. 62.
    Kaczmarz, S.: Angenäherte Auflösung von Systemen linearer Gleichungen. Bull. Acad. Sci. Pol. A 35, 355–357 (1937) Google Scholar
  63. 63.
    Kazantsev, I.G., Schmidt, S., Poulsen, H.F.: A discrete spherical X-ray transform of orientation distribution functions using bounding cubes. Inverse Probl. 25, 105009 (2009) MathSciNetCrossRefGoogle Scholar
  64. 64.
    Kiwiel, K.C.: Monotone Gram matrices and deepest surrogate inequalities in accelerated relaxation methods for convex feasibility problems. Linear Algebra Appl. 252, 27–33 (1997) MathSciNetzbMATHCrossRefGoogle Scholar
  65. 65.
    Kiwiel, K.C., Łopuch, B.: Surrogate projection methods for finding fixed points of firmly nonexpansive mappings. SIAM J. Optim. 7, 1084–1102 (1997) MathSciNetzbMATHCrossRefGoogle Scholar
  66. 66.
    Lee, S.-H., Kwon, K.-R.: Mesh watermarking based on projection onto two convex sets. Multimed. Syst. 13, 323–330 (2008) CrossRefGoogle Scholar
  67. 67.
    Lewitt, R.M.: Multidimensional digital image representation using generalized Kaiser-Bessel window functions. J. Opt. Soc. Am. A 7, 1834–1846 (1990) CrossRefGoogle Scholar
  68. 68.
    Liew, A.W.-C., Yan, H., Law, N.-F.: POCS-based blocking artifacts suppression using a smoothness constraint set with explicit region modeling. IEEE Trans. Circuits Syst. Video Technol. 15, 795–800 (2005) CrossRefGoogle Scholar
  69. 69.
    Lu, Y.M., Karzand, M., Vetterli, M.: Demosaicking by alternating projections: theory and fast one-step implementation. IEEE Trans. Image Process. 19, 2085–2098 (2010) MathSciNetCrossRefGoogle Scholar
  70. 70.
    Merzlyakov, Y.I.: On a relaxation method of solving systems of linear inequalities. U.S.S.R. Comput. Math. Math. Phys. 2, 504–510 (1963) CrossRefGoogle Scholar
  71. 71.
    Motzkin, T.S., Schoenberg, I.J.: The relaxation method for linear inequalities. Can. J. Math. 6, 393–404 (1954) MathSciNetzbMATHCrossRefGoogle Scholar
  72. 72.
    Natterer, F., Wübbeling, F.: Mathematical Methods in Image Reconstruction. SIAM, Philadelphia (2001) zbMATHCrossRefGoogle Scholar
  73. 73.
    Ottavy, N.: Strong convergence of projection-like methods in Hilbert spaces. J. Optim. Theory Appl. 56, 433–461 (1988) MathSciNetzbMATHCrossRefGoogle Scholar
  74. 74.
    Penfold, S.N., Schulte, R.W., Censor, Y., Bashkirov, V., McAllister, S., Schubert, K.E., Rosenfeld, A.B.: Block-iterative and string-averaging projection algorithms in proton computed tomography image reconstruction. In: Censor, Y., Jiang, M., Wang, G. (eds.) Biomedical Mathematics: Promising Directions in Imaging, Therapy Planning and Inverse Problems, pp. 347–367. Medical Physics Publishing, Madison (2010) Google Scholar
  75. 75.
    Pierra, G.: Éclatement de contraintes en parallèle pour la minimisation d’une forme quadratique. In: Proc. 7th IFIP Conf. on Optimization Techniques: Modeling and Optimization in the Service of Man. Lecture Notes in Comput. Sci., vol. 41, pp. 200–218. Springer, London (1976) CrossRefGoogle Scholar
  76. 76.
    Pierra, G.: Decomposition through formalization in a product space. Math. Program. 28, 96–115 (1984) MathSciNetzbMATHCrossRefGoogle Scholar
  77. 77.
    Rhee, H.: An application of the string averaging method to one-sided best simultaneous approximation. J. Korea Soc. Math. Educ. Ser. B Pure Appl. Math. 10, 49–56 (2003) MathSciNetzbMATHGoogle Scholar
  78. 78.
    Samsonov, A.A., Kholmovski, E.G., Parker, D.L., Johnson, C.R.: POCSENSE: POCS-based reconstruction for sensitivity encoded magnetic resonance imaging. Magn. Reson. Med. 52, 1397–1406 (2004) CrossRefGoogle Scholar
  79. 79.
    Shaked, N.T., Rosen, J.: Multiple-viewpoint projection holograms synthesized by spatially incoherent correlation with broadband functions. J. Opt. Soc. Am. A 25, 2129–2138 (2008) CrossRefGoogle Scholar
  80. 80.
    Sharma, G.: Set theoretic estimation for problems in subtractive color. Color Res. Appl. 25, 333–348 (2000) CrossRefGoogle Scholar
  81. 81.
    Stark, H., Yang, Y.: Vector Space Projections: A Numerical Approach to Signal and Image Processing, Neural Nets, and Optics. Wiley-Interscience, New York (1998) zbMATHGoogle Scholar
  82. 82.
    van Wyk, B.J., van Wyk, M.A.: A POCS-based graph matching algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1526–1530 (2004) CrossRefGoogle Scholar
  83. 83.
    Youla, D.C., Webb, H.: Image restoration by the method of convex projections: Part 1—theory. IEEE Trans. Med. Imaging 1, 81–94 (1982) CrossRefGoogle Scholar
  84. 84.
    Yukawa, M., Yamada, I.: Pairwise optimal weight realization—acceleration technique for set-theoretic adaptive parallel subgradient projection algorithm. IEEE Trans. Signal Process. 54, 4557–4571 (2006) CrossRefGoogle Scholar
  85. 85.
    Zhang, T., Hong, H.: Restoration algorithms for turbulence-degraded images based on optimized estimation of discrete values of overall point spread functions. Opt. Eng. 44, 017005 (2005) CrossRefGoogle Scholar
  86. 86.
    Ziegler, A., Grass, M., Koehler, T.: Method and device for the iterative reconstruction of cardiac images. US Patent Number 7596204 (2009) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Yair Censor
    • 1
  • Wei Chen
    • 2
  • Patrick L. Combettes
    • 3
  • Ran Davidi
    • 2
  • Gabor T. Herman
    • 2
  1. 1.Department of MathematicsUniversity of HaifaHaifaIsrael
  2. 2.Department of Computer Science, The Graduate CenterCity University of New YorkNew YorkUSA
  3. 3.Laboratoire Jacques-Louis Lions – UMR CNRS 7598UPMC Université Paris 06ParisFrance

Personalised recommendations