Abstract
A polyhedral active set algorithm PASA is developed for solving a nonlinear optimization problem whose feasible set is a polyhedron. Phase one of the algorithm is the gradient projection method, while phase two is any algorithm for solving a linearly constrained optimization problem. Rules are provided for branching between the two phases. Global convergence to a stationary point is established, while asymptotically PASA performs only phase two when either a nondegeneracy assumption holds, or the active constraints are linearly independent and a strong second-order sufficient optimality condition holds.
References
Bertsekas D P. On the Goldstein-Levitin-Polyak gradient projection method. IEEE Trans Automat Control, 1976, 21: 174–184
Bertsekas D P. Projected Newton methods for optimization problems with simple constraints. SIAM J Control Optim, 1982, 20: 221–246
Branch M, Coleman T, Li Y. A subspace, interior, and conjugate gradient method for large-scale bound-constrained minimization problems. SIAM J Sci Comput, 1999, 21: 1–23
Burke JV, Moré J J. On the identification of active constraints. SIAM J Numer Anal, 1988, 25: 1197–1211
Burke JV, Moré J J. Exposing constraints. SIAM J Optim, 1994, 25: 573–595
Burke JV, Moré J J, Toraldo G. Convergence properties of trust region methods for linear and convex constraints. Math Program, 1990, 47: 305–336
Calamai P, Moré J. Projected gradient for linearly constrained problems. Math Program, 1987, 39: 93–116
Coleman T F, Li Y. On the convergence of interior-reflective Newton methods for nonlinear minimization subject to bounds. Math Program, 1994, 67: 189–224
Coleman T F, Li Y. An interior trust region approach for nonlinear minimization subject to bounds. SIAM J Optim, 1996, 6: 418–445
Coleman T F, Li Y. A trust region and affine scaling interior point method for nonconvex minimization with linear inequality constraints. Math Program, 2000, 88: 1–31
Conn A R, Gould N I M, Toint P L. Global convergence of a class of trust region algorithms for optimization with simple bounds. SIAM J Numer Anal, 1988, 25: 433–460
Conn A R, Gould N I M, Toint P L. Trust-Region Methods. Philadelphia: SIAM, 2000
Dembo R S, Tulowitzki U. On the minimization of quadratic functions subject to box constraints. Working paper. New Haven: Yale University, 1983
Dennis J E, Heinkenschloss M, Vicente L N. Trust-region interior-point algorithms for a class of nonlinear programming problems. SIAM J Control Optim, 1998, 36: 1750–1794
Dostál Z. Box constrained quadratic programming with proportioning and projections. SIAM J Optim, 1997, 7: 871–887
Dostál Z. A proportioning based algorithm for bound constrained quadratic programming with the rate of convergence. Numer Algorithms, 2003, 34: 293–302
Dostál Z, Friedlander A, Santos S A. Augmented Lagrangians with adaptive precision control for quadratic programming with simple bounds and equality constraints. SIAM J Optim, 2003, 13: 1120–1140
Facchinei F, Júdice J, Soares J. An active set Newton’s algorithm for large-scale nonlinear programs with box constraints. SIAM J Optim, 1998, 8: 158–186
Facchinei F, Lucidi S, Palagi L. A truncated Newton algorithm for large-scale box constrained optimization. SIAM J Optim, 2002, 4: 1100–1125
Forsgren A, Gill P E, Wong E. Active-set methods for convex quadratic programming. ArXiv:1503.08349, 2015
Friedlander A, Martínez J M, Santos S A. A new trust region algorithm for bound constrained minimization. Appl Math Optim, 1994, 30: 235–266
Friedlander M P, Leyffer S. Global and finite termination of a two-phase augmented Lagrangian filter method for general quadratic programs. SIAM J Sci Comput, 2008, 30: 1706–1729
Gill P E, Wong E. Methods for convex and general quadratic programming. Math Prog Comp, 2014, 7: 71–112
Goldberg N, Leyffer S. An active-set method for second-order conic-constrained quadratic programming. SIAM J Optim, 2015, 25: 1455–1477
Goldstein A A. Convex programming in Hilbert space. Bull Amer Math Soc, 1964, 70: 709–710
Hager W W, Phan D T, Zhang H. Gradient-based methods for sparse recovery. SIAM J Imaging Sci, 2011, 4: 146–165
Hager W W, Zhang H. A new active set algorithm for box constrained optimization. SIAM J Optim, 2006, 17: 526–557
Hager W W, Zhang H. An affine scaling method for optimization problems with polyhedral constraints. Comput Optim Appl, 2014, 59: 163–183
Hager W W, Zhang H. Projection on a polyhedron that exploits sparsity. SIAM J Optim, 2015, in press
Heinkenschloss M, Ulbrich M, Ulbrich S. Superlinear and quadratic convergence of affine-scaling interior-point Newton methods for problems with simple bounds without strict complementarity assumption. Math Program, 1999, 86: 615–635
Hoffman A J. On approximate solutions of systems of linear inequalities. J Res National Bureau Standards, 1952, 49: 263–265
Izmailov A, Solodov M V. Newton-Type Methods for Optimization and Variational Problems. New York: Springer, 2014
Kanzow C, Klug A. On affine-scaling interior-point Newton methods for nonlinear minimization with bound constraints. Comput Optim Appl, 2006, 35: 177–197
Lescrenier M. Convergence of trust region algorithms for optimization with bounds when strict complementarity does not hold. SIAM J Numer Anal, 1991, 28: 476–495
Levitin E S, Polyak B T. Constrained minimization problems. USSR Comput Math Math Phys, 1966, 6: 1–50
Lin C-J, Moré J J. Newton’s method for large bound-constrained optimization problems. SIAM J Optim, 1999, 9: 1100–1127
McCormick G P, Tapia R A. The gradient projection method under mild differentiability conditions. SIAM J Control, 1972, 10: 93–98
Moré J J, Toraldo G. On the solution of large quadratic programming problems with bound constraints. SIAM J Optim, 1991, 1: 93–113
Polyak B T. The conjugate gradient method in extremal problems. USSR Comp Math Math Phys, 1969, 9: 94–112
Robinson S M. Some continuity properties of polyhedral multifunctions. Math Prog Study, 1981, 14: 206–214
Robinson S M. Generalized equations and their solutions, part II: applications to nonlinear programming. Math Prog Study, 1982, 19: 200–221
Schwartz A, Polak E. Family of projected descent methods for optimization problems with simple bounds. J Optim Theory Appl, 1997, 92: 1–31
Ulbrich M, Ulbrich S, Heinkenschloss M. Global convergence of trust-region interior-point algorithms for infinitedimensional nonconvex minimization subject to pointwise bounds. SIAM J Control Optim, 1999, 37: 731–764
Wen Z, Yin W, Goldfarb D, Zhang Y. A fast algorithm for sparse reconstruction based on shrinkage subspace optimization and continuation. SIAM J Sci Comp, 2010, 32: 1832–1857
Wen Z, Yin W, Zhang H, Goldfarb D. On the convergence of an active set method for L 1 minimization. Optim Methods Softw, 2012, 27: 1127–1146
Wright S J, Nowak R D, Figueiredo M A T. Sparse reconstruction by separable approximation. IEEE Trans Signal Process, 2009, 57: 2479–2493
Yang E K, Tolle J W. A class of methods for solving large convex quadratic programs subject to box constraints. Math Program, 1991, 51: 223–228
Zhang H, Hager W W. A nonmonotone line search technique and its application to unconstrained optimization. SIAM J Optim, 2004, 14: 1043–1056
Zhang Y. Interior-point gradient methods with diagonal-scalings for simple-bound constrained optimization. Tech. Rep. TR04-06. Houston: Rice University, 2004
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hager, W.W., Zhang, H. An active set algorithm for nonlinear optimization with polyhedral constraints. Sci. China Math. 59, 1525–1542 (2016). https://doi.org/10.1007/s11425-016-0300-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11425-016-0300-6
Keywords
- polyhedral constrained optimization
- active set algorithm
- PASA
- gradient projection algorithm
- local and global convergence