Abstract
A new active set algorithm (ASA) for large-scale box constrained optimization is introduced. The algorithm consists of a nonmonotone gradient projection step, an unconstrained optimization step, and a set of rules for switching between the two steps. Numerical experiments and comparisons are presented using box constrained problems in the CUTEr and MINPACK test problem libraries.
This paper is based upon work supported by the National Science Foundation under Grant No. 0203270.
Chapter PDF
Similar content being viewed by others
keywords
References
L. Armijo. Minimization of functions having Lipschitz continuous first partial derivatives. Pacific J. Math., 16:1–3, 1966.
B. M. Averick, R. G. Carter, J. J. Moré, and G. L. Xue. The MINPACK-2 test problem collection. Technical report, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL, 1992.
J. Barzilai and J. M. Borwein. Two point step size gradient methods. IMA J. Numer. Anal., 8:141–148, 1988.
D. P. Bertsekas. On the Goldstein-Levitin-Polyak gradient projection method. IEEE Trans. Automatic Control, 21:174–184, 1976.
D. P. Bertsekas. Projected Newton methods for optimization problems with simple constraints. SIAM J. Control Optim., 20:221–246, 1982.
E. G. Birgin and J. M. MartÃnez. Large-scale active-set box-constrained optimization method with spectral projected gradients. Comput. Optim. Appl., 23:101–125, 2002.
E. G. Birgin, J. M. MartÃnez, and M. Raydan. Nonmonotone spectral projected gradient methods for convex sets. SIAM J. Optim., 10:1196–1211, 2000.
E. G. Birgin, J. M. MartÃnez, and M. Raydan. Algorithm 813: SPG-software for convex-constrained optimization. ACM Trans. Math. Software, 27:340–349, 2001.
I. Bongartz, A. R. Conn, N. I. M. Gould, and P. L. Toint. CUTE: constrained and unconstrained testing environments. ACM Trans. Math. Software, 21:123–160, 1995.
M.A. Branch, T.F. Coleman, and Y. Li. A subspace, interior, and conjugate gradient method for large-scale bound-constrained minimization problems. SIAM J. Sci. Comput., 21:1–23, 1999.
J. V. Burke and J. J. Moré. On the identification of active constraints. SIAM J. Numer. Anal, 25:1197–1211, 1988.
J. V. Burke and J. J. Moré. Exposing constraints. SIAM J. Optim., 25:573–595, 1994.
J. V. Burke, J. J. Moré, and G. Toraldo. Convergence properties of trust region methods for linear and convex constraints. Math. Prog., 47:305–336, 1990.
P. Calamai and J. Moré. Projected gradient for linearly constrained problems. Math. Prog., 39:93–116, 1987.
T. F. Coleman and Y. Li. On the convergence of interior-reflective Newton methods for nonlinear minimization subject to bounds. Math. Prog., 67:189–224, 1994.
T. F. Coleman and Y. Li. An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim., 6:418–445, 1996.
T. F. Coleman and Y. Li. A trust region and affine scaling interior point method for nonconvex minimization with linear inequality constraints. Technical report, Cornell University, Ithaca, NY, 1997.
A. R. Conn, N. I. M. Gould, and Ph. L. Toint. Global convergence of a class of trust region algorithms for optimization with simple bounds. SIAM J. Numer. Anal., 25:433–460, 1988.
A. R. Conn, N. I. M. Gould, and Ph. L. Toint. A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J. Numer. Anal., 28:545–572, 1991.
Y. H. Dai, W. W. Hager, K. Schittkowski, and H. Zhang. The cyclic Barzilai-Borwein method for unconstrained optimization. IMA J. Numer. Anal, submitted, 2005.
R. S. Dembo and U. Tulowitzki. On the minimization of quadratic functions subject to box constraints. Technical report, School of Organization and Management, Yale University, New Haven, CT, 1983.
J. E. Dennis, M. Heinkenschloss, and L. N. Vicente. Trust-region interior-point algorithms for a class of nonlinear programming problems. SIAM J. Control Optim., 36:1750–1794, 1998.
E. D. Dolan and J. J. Moré. Benchmarking optimization software with performance profiles. Math. Program., 91:201–213, 2002.
Z. Dostál. Box constrained quadratic programming with proportioning and projections. SIAM J. Optim., 7:871–887, 1997.
Z. Dostál. A proportioning based algorithm for bound constrained quadratic programming with the rate of convergence. Numer. Algorithms, 34:293–302, 2003.
Z. Dostál, A. Friedlander, and S. A. Santos. Solution of coercive and semicoercive contact problems by FETI domain decomposition. Contemp. Math., 218:82–93, 1998.
Z. Dostál, A. Friedlander, and S. A. Santos. Augmented Lagrangians with adaptive precision control for quadratic programming with simple bounds and equality constraints. SIAM J. Optim., 13:1120–1140, 2003.
A. S. El-Bakry, R. A. Tapia, T. Tsuchiya, and Y. Zhang. On the formulation and theory of the primal-dual Newton interior-point method for nonlinear programming. J. Optim. Theory Appl., 89:507–541, 1996.
F. Facchinei, J. Júdice, and J. Soares. An active set Newton’s algorithm for large-scale nonlinear programs with box constraints. SIAM J. Optim., 8:158–186, 1998.
F. Facchinei and S. Lucidi. A class of penalty functions for optimization problems with bound constraints. Optimization, 26:239–259, 1992.
F. Facchinei, S. Lucidi, and L. Palagi. A truncated Newton algorithm for large-scale box constrained optimization. SIAM J. Optim., 4:1100–1125, 2002.
A. Friedlander, J. M. MartÃnez, and S. A. Santos. A new trust region algorithm for bound constrained minimization. Appl. Math. Optim., 30:235–266, 1994.
A. A. Goldstein. Convex programming in Hilbert space. Bull. Amer. Math. Soc, 70:709–710, 1964.
L. Grippo, F. Lampariello, and S. Lucidi. A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal, 23:707–716, 1986.
W. W. Hager and H. Zhang. CG_DESCENT user’s guide. Technical report, Dept. Math., Univ. Florida, 2004.
W. W. Hager and H. Zhang. A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim., 16:170–192, 2005.
W. W. Hager and H. Zhang. A new active set algorithm for box constrained optimization. SIAM J. Optim., submitted, 2005.
W. W. Hager and H. Zhang. CG_DESCENT, a conjugate gradient method with guaranteed descent. ACM Trans. Math. Software, to appear 2006.
W. W. Hager and H. Zhang. A survey of nonlinear conjugate gradient methods. Pacific J. Optim., to appear 2006.
M. Heinkenschloss, M. Ulbrich, and S. Ulbrich. Superlinear and quadratic convergence of affine-scaling interior-point Newton methods for problems with simple bounds without strict complementarity assumption. Math. Prog., 86:615–635, 1999.
C. Kanzow and A. Klug. On affine-scaling interior-point Newton methods for nonlinear minimization with bound constraints. Comput. Optim. Appl., 2006, to appear.
D. Kinderlehrer and G. Stampacchia. An introduction to variational inequalities and their applications, volume 31 of Classics in Applied Mathematics. SIAM, Philadelphia, PA, 2000.
M. Lescrenier. Convergence of trust region algorithms for optimization with bounds when strict complementarity does not hold. SIAM J. Numer. Anal., 28:476–495, 1991.
E. S. Levitin and B. T. Polyak. Constrained minimization problems. USSR Comput. Math. Math. Physics, 6:1–50, 1966.
C. J. Lin and J. J. Moré. Incomplete cholesky factorizations with limited memory. SIAM J. Sci. Comput., 21:24–45, 1999.
C. J. Lin and J. J. Moré. Newton’s method for large bound-constrained optimization problems. SIAM J. Optim., 9:1100–1127, 1999.
J. M. MartÃnez. BOX-QUACAN and the implementation of augmented Lagrangian algorithms for minimization with inequality constraints. J. Comput. Appl. Math., 19:31–56, 2000.
G. P. McCormick and R. A. Tapia. The gradient projection method under mild differentiability conditions. SIAM J. Control, 10:93–98, 1972.
J. J. Moré and G. Toraldo. On the solution of large quadratic programming problems with bound constraints. SIAM J. Optim., 1:93–113, 1991.
B. T. Polyak. The conjugate gradient method in extremal problems. USSR Comp. Math. Math. Phys., 9:94–112, 1969.
S. M. Robinson. Strongly regular generalized equations. Math. Oper. Res., 5:43–62, 1980.
M. Ulbrich, S. Ulbrich, and M. Heinkenschloss. Global convergence of affine-scaling interior-point Newton methods for infinite-dimensional nonlinear problems with pointwise bounds. SIAM J. Control Optim., 37:731–764, 1999.
S. J. Wright. Implementing proximal point methods for linear programming. J. Optim. Theory Appl., 65:531–554, 1990.
H. Yamashita and H. Yabe. Superlinear and quadratic convergence of some primal-dual interior-point methods for constrained optimization. Math. Prog., 75:377–397, 1996.
E. K. Yang and J. W. Tolle. A class of methods for solving large convex quadratic programs subject to box constraints. Math. Prog., 51:223–228, 1991.
Y. Zhang. Interior-point gradient methods with diagonal-scalings for simple-bound constrained optimization. Technical Report TR04-06, Department of Computational and Applied Mathematics, Rice University, Houston, Texas, 2004.
C. Zhu, R. H. Byrd, and J. Nocedal. Algorithm 778: L-BFGS-B, Fortran subroutines for large-scale bound-constrained optimization. ACM Trans. Math. Software, 23:550–560, 1997.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 International Federation for Information Processing
About this paper
Cite this paper
Hager, W.W., Zhang, H. (2006). Recent Advances in Bound Constrained Optimization. In: Ceragioli, F., Dontchev, A., Futura, H., Marti, K., Pandolfi, L. (eds) System Modeling and Optimization. CSMO 2005. IFIP International Federation for Information Processing, vol 199. Springer, Boston, MA. https://doi.org/10.1007/0-387-33006-2_7
Download citation
DOI: https://doi.org/10.1007/0-387-33006-2_7
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-32774-7
Online ISBN: 978-0-387-33006-8
eBook Packages: Computer ScienceComputer Science (R0)