Abstract
A new method for solving large nonlinear optimization problems is outlined. It attempts to combine the best properties of the discrete-truncated Newton method and the limited memory BFGS method, to produce an algorithm that is both economical and capable of handling ill-conditioned problems. The key idea is to use the curvature information generated during the computation of the discrete Newton step to improve the limited memory BFGS approximations. The numerical performance of the new method is studied using a family of functions whose nonlinearity and condition number can be controlled.
This author was supported by NSF grant CCR-9101795, ARO grant DAAL 03-91-G-0151, and AFOSR grant AFOSR-90-0109.
These authors were supported by National Science Foundation Grants CCR-940081 and ASC-9213149, and by Department of Energy Grant DE-FG02-87ER25047-A004.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
R.S. Dembo, S.C. Eisenstat and T. Steihaug, “Inexact Newton methods”. SIAM J. Numer. Anal. 19, pp. 400–408, 1982.
J. E. Dennis, Jr. and R.B. Schnabel, “Numerical Methods for Unconstrained Optimization and Nonlinear Equations”. Englewood Cliffs, N.J., Prentice-Hall, 1983.
J.C. Gilbert and C. Lemaréchal, “Some numerical experiments with variable storage quasi-Newton algorithms.” Mathematical Programming 45, pp. 407–436, 1989.
P.E. Gill, W Murray and M.H. Wright, “Practical Optimization”. London, Academic Press, 1981.
G.H. Golub and C.F. Van Loan, “Matrix Computations”. (Second Edition), The John Hopkins University Press, Baltimore and London, 1989.
A. Griewank, “On automatic differentiation.” In “Mathematical Programming” (M. Iri and K. Tanabe, eds.), Kluwer Academic Publishers, pp. 83-107, Tokyo, 1989.
D.C. Liu and J. Nocedal, “On the limited memory BFGS method for large scale optimization methods.” Mathematical Programming 45 pp. 503–528, 1989.
J.J. Moré and D.J. Thuente, “Line search algorithms with guaranteed sufficient decrease.” ACM Transactions on Mathematical Software 20, no. 3, pp. 286–307, 1994.
S.G. Nash, “User’s guide for TN/TNBC: FORTRAN routines for nonlinear optimization”. Report 397, Mathematical Sciences Dept., The Johns Hopkins University, 1984.
S.G. Nash, “Preconditioning of truncated-Newton methods.” SIAM Journal on Scientific and Statistical Computing 6, pp. 599–616, 1985.
S.G. Nash and J. Nocedal, “A Numerical Study of the Limited Memory BFGS Method and the Truncated-Newton Method for Large Scale Optimization.” SIAM Journal on Optimization 1, no. 3, pp. 358–372, 1991.
J. Nocedal, “Updating quasi-Newton matrices with limited storage.” Mathematics of Computation 35, pp. 773–782, 1980.
D.P. O’Leary, “A discrete Newton algorithm for minimizing a function of many variables.” Mathematical Programming 23, pp. 20–33, 1982.
T. Schlick and A. Fogelson, “TNPACK-A truncated Newton package for large-scale problems: I. Algorithms and usage.” ACM Transactions on Mathematical Software 18, no. 1, pp. 46–70, 1992.
T. Steihaug (1983), “The conjugate gradient method and trust regions in large scale optimization.” SIAM J. Num. Anal. 20, pp. 626–637, 1983.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1996 Springer Science+Business Media New York
About this chapter
Cite this chapter
Byrd, R.H., Nocedal, J., Zhu, C. (1996). Towards a Discrete Newton Method with Memory for Large-Scale Optimization. In: Di Pillo, G., Giannessi, F. (eds) Nonlinear Optimization and Applications. Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-0289-4_1
Download citation
DOI: https://doi.org/10.1007/978-1-4899-0289-4_1
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4899-0291-7
Online ISBN: 978-1-4899-0289-4
eBook Packages: Springer Book Archive