Representations of quasi-Newton matrices and their use in limited memory methods
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
We derive compact representations of BFGS and symmetric rank-one matrices for optimization. These representations allow us to efficiently implement limited memory methods for large constrained optimization problems. In particular, we discuss how to compute projections of limited memory matrices onto subspaces. We also present a compact representation of the matrices generated by Broyden's update for solving systems of nonlinear equations.
- J. Barnes, “An algorithm for solving nonlinear equations based on the secant method,”Computer Journal 8 (1965) 66–67.
- L. Biegler, J. Nocedal and C. Schmid, “Reduced Hessian methods for large scale constrained optimization,” Technical Report, Department of Electrical Engineering and Computer Science, Northwestern University (Evanston, IL, 1993).
- C.G. Broyden, “A class of methods for solving nonlinear simultaneous equations,”Mathematics of Computation 19 (1965) 577–593.
- A. Buckley and A. LeNir, “QN-like variable storage conjugate gradients,”Mathematical Programming 27 (1983) 103–119.
- R.H. Byrd, P. Lu and J. Nocedal, “A limited memory algorithm for bound constrained optimization,” Technical Report, Department of Electrical Engineering and Computer Science, Northwestern University (Evanston, IL, 1993).
- A.R. Conn, N.I.M. Gould and Ph.L. Toint, “Testing a class of methods for solving minimization problems with simple bounds on the variables,”Mathematics of Computation 50/182 (1988) 399–430.
- J.E. Dennis Jr. and R.B. Schnabel,Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Prentice-Hall, Englewood Cliffs, NJ, 1983).
- R. Fletcher,Practical Methods of Optimization (Wiley, Chichester, 1987, 2nd ed.).
- D.M. Gay and R.B. Schnabel, “Solving systems of nonlinear equations by Broyden's method with projected updates,” in: O.L. Mangasarian, R.R. Meyer and S.M. Robinson, eds.,Nonlinear Programming 3 (Academic Press, New York, 1978) pp. 245–281.
- J.C. Gilbert and C. Lemaréchal, “Some numerical experiments with variable storage quasi-Newton algorithms,”Mathematical Programming 45 (1989) 407–436.
- J.C. Gilbert and J. Nocedal, “The limited memory step computation and automatic differentiation,”Applied Math Letters 6(3) (1993) 47–50.
- A. Griewank, “On automatic differentiation,” in: M. Iri and K. Tanabe, eds.,Mathematical Programming (Kluwer Academic Publishers, Tokyo, 1989) pp. 83–107.
- H. Fayez Khalfan, “Topics in quasi-Newton methods for unconstrained optimization,” Ph.D. thesis, Department of Mathematics, University of Colorado (Boulder, CO, 1989).
- H. Fayez Khalfan, R.H. Byrd, and R.B. Schnabel, “A theoretical and experimental study of the symmetric rank one update,”SIAM Journal on Optimization 3 (1993) 1–24.
- D.C. Liu and J. Nocedal, “On the limited memory BFGS method for large scale optimization,”Mathematical Programming 45 (1989) 503–528.
- D.Q. Mahidhara and L. Lasdon, “An SQP algorithm for large sparse nonlinear programs,” Technical report, MSIS Department, School of Business Administration, University of Texas (Austin, TX, 1990).
- H. Matthies and G. Strang, “The solution of nonlinear finite element equations,”International Journal of Numerical Methods in Engineering 14 (1979) 1613–1626.
- J. Nocedal, “Updating quasi-Newton matrices with limited storage,”Mathematics of Computation 35 (1980) 773–782.
- J.M. Ortega and W.C. Rheinboldt,Iterative Solution of Nonlinear Equations in Several Variables (Academic Press, New York, 1970).
- R.B. Schnabel, “Quasi-Newton methods using multiple secant equations,” Technical Report CU-CS-247-83, Department of Computer Science, University of Colorado (Boulder, CO, 1983).
- H.F. Walker, “Implementation of the GMRES method using Householder transformations,”SIAM Journal on Scientific and Statistical Computing 9(1) (1988) 152–163.
- Representations of quasi-Newton matrices and their use in limited memory methods
Volume 63, Issue 1-3 , pp 129-156
- Cover Date
- Print ISSN
- Online ISSN
- Additional Links
- Quasi-Newton method
- constrained optimization
- limited memory method
- large-scale optimization
- Industry Sectors