Mathematical Programming

, Volume 39, Issue 1, pp 93–116 | Cite as

Projected gradient methods for linearly constrained problems

  • Paul H. Calamai
  • Jorge J. Moré


The aim of this paper is to study the convergence properties of the gradient projection method and to apply these results to algorithms for linearly constrained problems. The main convergence result is obtained by defining a projected gradient, and proving that the gradient projection method forces the sequence of projected gradients to zero. A consequence of this result is that if the gradient projection method converges to a nondegenerate point of a linearly constrained problem, then the active and binding constraints are identified in a finite number of iterations. As an application of our theory, we develop quadratic programming algorithms that iteratively explore a subspace defined by the active constraints. These algorithms are able to drop and add many constraints from the active set, and can either compute an accurate minimizer by a direct method, or an approximate minimizer by an iterative method of the conjugate gradient type. Thus, these algorithms are attractive for large scale problems. We show that it is possible to develop a finite terminating quadratic programming algorithm without non-degeneracy assumptions.

Key words

Linearly constrained problems projected gradients bound constrained problems large scale problems convergence theory 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. P. Bertsekas, “On the Goldstein-Levitin-Polyak gradient projection method,”IEEE Transactions on Automatic Control 21 (1976) 174–184.zbMATHCrossRefMathSciNetGoogle Scholar
  2. D.P. Bertsekas, “Projected Newton methods for optimization problems with simple constraints,”SIAM Journal on Control and Optimization 20 (1982) 221–246.zbMATHCrossRefMathSciNetGoogle Scholar
  3. J.V. Burke and J.J. Moré, “On the identification of active constraints,” Argonne National Laboratory, Mathematics and Computer Science Division Report ANL/MCS-TM-82, (Argonne, IL, 1986).Google Scholar
  4. R.S. Dembo and U. Tulowitzki, “On the minimization of quadratic functions subject to box constraints,” Working Paper Series B#71, School of Organization and Management, Yale University, (New Haven, CT, 1983).Google Scholar
  5. R.S. Dembo and U. Tulowitzki, “Local convergence analysis for successive inexact quadratic programming methods,” Working Paper Series B#78, School of Organization and Management, Yale University, (New Haven, CT, 1984).Google Scholar
  6. R.S. Dembo and U. Tulowitzki, “Sequential truncated quadratic programming methods,”, in: P.T. Boggs, R.H. Byrd and R.B. Schnabel, eds.,Numerical Optimization 1984 (Society of Industrial and Applied Mathematics, Philadelphia, 1985) pp. 83–101.Google Scholar
  7. J.C. Dunn, “Global and asymptotic convergence rate estimates for a class of projected gradient processes,”SIAM Journal on Control and Optimization 19 (1981), 368–400.zbMATHCrossRefMathSciNetGoogle Scholar
  8. J.C. Dunn, “On the convergence of projected gradient processes to singular critical points,”Journal of Optimization Theory and Applications (1986) to appear.Google Scholar
  9. R. Fletcher,Practical Methods of Optimization Volume 2: Constrained Optimization (John Wiley & Sons, New York, 1981).Google Scholar
  10. E.M. Gafni and D.P. Bertsekas, “Convergence of a gradient projection method,” Massachusetts Institute of Technology, Laboratory for Information and Decision Systems Report LIDS-P-1201 (Cambridge, Massachusetts, 1982).Google Scholar
  11. E.M. Gafni and D.P. Bertsekas, “Two-metric projection methods for constrained optimization,”SIAM Journal on Control and Optimization 22 (1984) 936–964.zbMATHCrossRefMathSciNetGoogle Scholar
  12. P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, New York, 1981).zbMATHGoogle Scholar
  13. A.A. Goldstein, “Convex programming in Hilbert space,”Bulletin of the American Mathematical Society 70 (1964) 709–710.zbMATHCrossRefMathSciNetGoogle Scholar
  14. E.S. Levitin and B.T. Polyak, “Constrained minimization problems”,USSR Computational Mathematics and Mathematical Physics 6 (1966), 1–50.CrossRefGoogle Scholar
  15. G.P. McCormick and R.A. Tapia, “The gradient projection method under mild differentiability conditions,”SIAM Journal on Control 10 (1972) 93–98.zbMATHCrossRefMathSciNetGoogle Scholar
  16. D.P. O'Leary, “A generalized conjugate gradient algorithm for solving a class of quadratic programming problems,”Linear Algebra and its Applications 34 (1980) 371–399.zbMATHCrossRefMathSciNetGoogle Scholar
  17. R.R. Phelps “The gradient projection method using Curry's steplength,”SIAM Journal on Control and Optimization 24 (1986) 692–699.zbMATHCrossRefMathSciNetGoogle Scholar
  18. B.T. Polyak, “The conjugate gradient method in extremal problems,”USSR Computational Mathematics and Mathematical Physics 9 (1969) 94–112.CrossRefGoogle Scholar
  19. E.H. Zarantonello, “Projections on convex sets in Hilbert space and spectral theory,” in: E.H. Zarantonello, ed.,Contributions to Nonlinear Functional Analysis (Academic Press, New York, 1971).Google Scholar

Copyright information

© The Mathematical Programming Society, Inc. 1987

Authors and Affiliations

  • Paul H. Calamai
    • 1
  • Jorge J. Moré
    • 2
  1. 1.University of WaterlooCanada
  2. 2.Argonne National LaboratoryArgonneUSA

Personalised recommendations