# Projected gradient methods for linearly constrained problems

- 1.3k Downloads
- 268 Citations

## Abstract

The aim of this paper is to study the convergence properties of the gradient projection method and to apply these results to algorithms for linearly constrained problems. The main convergence result is obtained by defining a projected gradient, and proving that the gradient projection method forces the sequence of projected gradients to zero. A consequence of this result is that if the gradient projection method converges to a nondegenerate point of a linearly constrained problem, then the active and binding constraints are identified in a finite number of iterations. As an application of our theory, we develop quadratic programming algorithms that iteratively explore a subspace defined by the active constraints. These algorithms are able to drop and add many constraints from the active set, and can either compute an accurate minimizer by a direct method, or an approximate minimizer by an iterative method of the conjugate gradient type. Thus, these algorithms are attractive for large scale problems. We show that it is possible to develop a finite terminating quadratic programming algorithm without non-degeneracy assumptions.

## Key words

Linearly constrained problems projected gradients bound constrained problems large scale problems convergence theory## Preview

Unable to display preview. Download preview PDF.

## References

- P. Bertsekas, “On the Goldstein-Levitin-Polyak gradient projection method,”
*IEEE Transactions on Automatic Control*21 (1976) 174–184.zbMATHCrossRefMathSciNetGoogle Scholar - D.P. Bertsekas, “Projected Newton methods for optimization problems with simple constraints,”
*SIAM Journal on Control and Optimization*20 (1982) 221–246.zbMATHCrossRefMathSciNetGoogle Scholar - J.V. Burke and J.J. Moré, “On the identification of active constraints,” Argonne National Laboratory, Mathematics and Computer Science Division Report ANL/MCS-TM-82, (Argonne, IL, 1986).Google Scholar
- R.S. Dembo and U. Tulowitzki, “On the minimization of quadratic functions subject to box constraints,” Working Paper Series B#71, School of Organization and Management, Yale University, (New Haven, CT, 1983).Google Scholar
- R.S. Dembo and U. Tulowitzki, “Local convergence analysis for successive inexact quadratic programming methods,” Working Paper Series B#78, School of Organization and Management, Yale University, (New Haven, CT, 1984).Google Scholar
- R.S. Dembo and U. Tulowitzki, “Sequential truncated quadratic programming methods,”, in: P.T. Boggs, R.H. Byrd and R.B. Schnabel, eds.,
*Numerical Optimization 1984*(Society of Industrial and Applied Mathematics, Philadelphia, 1985) pp. 83–101.Google Scholar - J.C. Dunn, “Global and asymptotic convergence rate estimates for a class of projected gradient processes,”
*SIAM Journal on Control and Optimization*19 (1981), 368–400.zbMATHCrossRefMathSciNetGoogle Scholar - J.C. Dunn, “On the convergence of projected gradient processes to singular critical points,”
*Journal of Optimization Theory and Applications*(1986) to appear.Google Scholar - R. Fletcher,
*Practical Methods of Optimization Volume 2: Constrained Optimization*(John Wiley & Sons, New York, 1981).Google Scholar - E.M. Gafni and D.P. Bertsekas, “Convergence of a gradient projection method,” Massachusetts Institute of Technology, Laboratory for Information and Decision Systems Report LIDS-P-1201 (Cambridge, Massachusetts, 1982).Google Scholar
- E.M. Gafni and D.P. Bertsekas, “Two-metric projection methods for constrained optimization,”
*SIAM Journal on Control and Optimization*22 (1984) 936–964.zbMATHCrossRefMathSciNetGoogle Scholar - P.E. Gill, W. Murray and M.H. Wright,
*Practical Optimization*(Academic Press, New York, 1981).zbMATHGoogle Scholar - A.A. Goldstein, “Convex programming in Hilbert space,”
*Bulletin of the American Mathematical Society*70 (1964) 709–710.zbMATHCrossRefMathSciNetGoogle Scholar - E.S. Levitin and B.T. Polyak, “Constrained minimization problems”,
*USSR Computational Mathematics and Mathematical Physics*6 (1966), 1–50.CrossRefGoogle Scholar - G.P. McCormick and R.A. Tapia, “The gradient projection method under mild differentiability conditions,”
*SIAM Journal on Control*10 (1972) 93–98.zbMATHCrossRefMathSciNetGoogle Scholar - D.P. O'Leary, “A generalized conjugate gradient algorithm for solving a class of quadratic programming problems,”
*Linear Algebra and its Applications*34 (1980) 371–399.zbMATHCrossRefMathSciNetGoogle Scholar - R.R. Phelps “The gradient projection method using Curry's steplength,”
*SIAM Journal on Control and Optimization*24 (1986) 692–699.zbMATHCrossRefMathSciNetGoogle Scholar - B.T. Polyak, “The conjugate gradient method in extremal problems,”
*USSR Computational Mathematics and Mathematical Physics*9 (1969) 94–112.CrossRefGoogle Scholar - E.H. Zarantonello, “Projections on convex sets in Hilbert space and spectral theory,” in: E.H. Zarantonello, ed.,
*Contributions to Nonlinear Functional Analysis*(Academic Press, New York, 1971).Google Scholar