Quasi-Newton Algorithms for Large-Scale Nonlinear Least-Squares
Low storage quasi-Newton algorithms for large-scale nonlinear least-squares problems are considered with “better” modified Hessian approximations defined implicitly in terms of a set of vector pairs. The modification technique replaces one vector of each pair, namely the difference in the gradients of the objective function, by a superior choice in various ways. These vectors introduce information about the true Hessian of this function by exploiting information about the Jacobian matrix of the residual vector of the problem. The proposed technique is also based on a new safeguarded scheme for enforcing the positive definiteness of Hessian approximations. It is shown, in particular, that this technique enhances the quality of the limited memory (L-)BFGS Hessian, maintains the simplicity formulation of the L-BFGS algorithm and improves its performance substantially.
Keywordsnonlinear least-squares problems limited memory BFGS algorithm
Unable to display preview. Download preview PDF.
- M. Al-Baali. (1984), “Methods for nonlinear least squares,” PhD thesis, Dept. of Mathematics and Computer Science, Univ. of Dundee, Scotland.Google Scholar
- C. Bischof, A. Carle, G. Corliss, A. Griewank and P. Hovland (1992), “ADIFOR: Generating derivative codes from Fortran programs,” Scientific Programming, 1, 11–29.Google Scholar
- A.R. Conn, N.I.M. Gould and Ph.L. Toint. (1992), LANCELOT: a FORTRAN package for large-scale nonlinear optimization (Release A), Number 17, Springer Series in Computational Mathematics, Springer-Verlag, New York, USA.Google Scholar
- J.E. Dennis (1977), “Non-linear least squares and equations,” In D.A.H. Jacobs, editor, The State of the Art in Numerical Analysis,Academic Press, 269–312.Google Scholar
- R. Fletcher (1990), “Low storage methods for unconstrained optimization,” In E.L. Allgower and K. George, editors, Computational Solution of Nonlinear Systems of Equations, Lectures in Applied Mathematics, AMS, Providence, RI, 165–179.Google Scholar
- J. Nocedal (1996), “Large scale unconstrained optimization,” Tech. Report, Dept. of Electrical Engineering and Computer Science, Northwestern Univ., Evanston, USA.Google Scholar
- M.J.D. Powell (1976), “Some global convergence properties of a variable metric algorithm for minimization without exact line searches,” In R.W. Cottle and C.E. Lemke, editors, Nonlinear Programming, SIAM-AMS Proceedings, SIAM Publications, vol. IX, 53–72.Google Scholar