Abstract
We show how a direct active set method for solving definite and indefinite quadratic programs with simple bounds can be efficiently implemented for large sparse problems. All of the necessary factorizations can be carried out in a static data structure that is set up before the numeric computation begins. The space required for these factorizations is no larger than that required for a single sparse Cholesky factorization of the Hessian of the quadratic. We propose several improvements to this basic algorithm: a new way to find a search direction in the indefinite case that allows us to free more than one variable at a time and a new heuristic method for finding a starting point. These ideas are motivated by the two-norm trust region problem. Additionally, we also show how projection techniques can be used to add several constraints to the active set at each iteration. Our experimental results show that an algorithm with these improvements runs much faster than the basic algorithm for positive definite problems and finds local minima with lower function values for indefinite problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
J.L. Barlow, “The use of three different rank detection methods in the solution of sparse weighted and equality constrained least squares problems,” Technical Report CS-88-18, The Pennsylvania State University (University Park, PA, 1988).
D.P. Bertsekas, “Projected Newton methods for optimization problems with simple constraints,”SIAM Journal on Control and Optimization 20 (1982) 221–246.
A. Björck, “A direct method for sparse least squares problems with lower and upper bounds,”Numerische Mathematik 54 (1988) 19–32.
P.H. Calamai and J.J. Moré, “Projected gradient methods for linearly constrained problems,”Mathematical Programming 39 (1987) 93–116.
T.F. Coleman, “A chordal preconditioner for large scale optimization,”Mathematical Programming 40 (1988) 265–287.
A.R. Conn, N.I.M. Gould and Ph.L. Toint, “Global convergence of a class of trust region algorithms for optimization with simple bounds,”SIAM Journal on Numerical Analysis 25 (1988) 433–460.
A.R. Conn, N.I.M. Gould and Ph.L. Toint, “Testing a class of methods for solving minimization problems with simple bounds on the variables,”Mathematics of Computation 50 (1988) 399–430.
R.S. Dembo and U. Tulowitzki, “On the minimization of quadratic functions subject to box constraints,” Technical Report B 71, Yale University (New Haven, CT, 1983).
J.J. Dongarra, J.R. Bunch, C.B. Moler and G.W. Stewart,LINPACK Users' Guide (SIAM, Philadelphia, PA, 1979).
G.C. Everstine, “A comparison of three resequencing algorithms for the reduction of matrix profile and wave front“,International Journal for Numerical Methods in Engineering 14 (1979) 837–853.
R. Fletcher, “A general quadratic programming algorithm,”Journal of the Institute of Mathematics and its Applications 7 (1971) 76–91.
R. Fletcher and M.P. Jackson, “Minimization of a quadratic function of many variables subject only to lower and upper bounds,”Journal of the Institute of Mathematics and Its Applications 14 (1974) 159–174.
A. George and J.W.H. Liu,Computer Solution of Large Sparse Positive Definite Systems (Prentice-Hall, Englewood Cliffs, NY, 1981).
J.R. Gilbert and T. Peierls, “Sparse partial pivoting in time proportional to arithmetic operations,”SIAM Journal on Scientific and Statistical Computing 9 (1988) 862–874.
P.E. Gill and W. Murray, “Numerically stable methods for quadratic programming,Mathematical Programming 14 (1978) 349–372.
P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, New York, 1981).
G.H. Golub and C.F. Van Loan,Matrix Computations (The Johns Hopkins University Press, Baltimore, MD, 1983).
N.J. Higham, “Fortran codes for estimating the one-norm of a real or complex matrix, with applications to condition estimation,” Technical Report Numerical Analysis Report No. 135, University of Manchester (Manchester, UK, 1987).
J.J. Moré, “Numerical solution of bound constrained problems,” Technical Report 96, Argonne National Laboratory (Argonne, IL, 1987).
J.J. Moré and D.C. Sorensen, “Computing a trust region step,”SIAM Journal on Scientific and Statistical Computing (1983) 553–572.
J.J. Moré and G. Toraldo, “Algorithms for bound constrained quadratic programming problems,” Technical Report 117, Argonne National Laboratory (Argonne, IL, 1988).
K.G. Murty and S.N. Kabadi, “Some NP-complete problems in quadratic and nonlinear programming,”Mathematical Programming 39 (1987) 117–129.
S. Parter, “The use of linear graphs in Gauss elimination,”SIAM Review 3 (1961) 119–130.
D.J. Rose, R.E. Tarjan and G.S. Leuker, “Algorithmic aspects of vertex elimination on graphs,”SIAM Journal on Computing 5 (1976) 266–283.
M. Yannakakis, “Computing the minimum fill-in is NP-complete,”SIAM Journal on algebraic and Discrete Methods 2 (1981) 77–79.
Y. Ye and E. Tse, “A polynomial-time algorithm for convex quadratic programming,” Technical Report, Stanford University (Stanford, CA, 1986).
Author information
Authors and Affiliations
Additional information
Research partially supported by the Applied Mathematical Sciences Research Program (KC-04-02) of the Office of Energy Research of the U.S. Department of Energy under grant DE-FG02-86ER25013.A000.
Rights and permissions
About this article
Cite this article
Coleman, T.F., Hulbert, L.A. A direct active set algorithm for large sparse quadratic programs with simple bounds. Mathematical Programming 45, 373–406 (1989). https://doi.org/10.1007/BF01589112
Issue Date:
DOI: https://doi.org/10.1007/BF01589112