Abstract
In this paper, low-order Newton methods are proposed that make use of previously obtained second-derivative information by suitable preconditioning. When applied to a particular 2-dimensional Newton method (the LS method), it is shown that a member of the Broyden family of quasi-Newton methods is obtained. Algorithms based on this preconditioned LS model are tested against some variations of the BFGS method and shown to be much superior in terms of number of iterations and function evaluations, but not so effective in terms of number of gradient evaluations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Fletcher, R.,Practical Methods of Optimization, John Wiley and Sons, Chichester, England, 1987.
Liu, Y., andStorey, C.,Efficient Generalized Conjugate Gradient Algorithms, Part 1: Theory, Journal of Optimization Theory and Applications, Vol. 69, pp. 129–137, 1991.
Hu, Y. F., andStorey, C.,Efficient Generalized Conjugate Gradient Algorithms, Part 2: Implementation, Journal of Optimization Theory and Applications, Vol. 69, pp. 139–152, 1991.
Nazareth, J. L.,An Algorithm Based upon Successive Affine Reduction and Newton's Method, Computing Methods in Applied Sciences and Engineering, Edited by R. Glowinski and J. L. Lions, Elsevier Science Publishers, Amsterdam, Holland, Vol. 7, pp. 641–646, 1986.
Hu, Y. F.,New Conjugate Gradient Methods and Their Connections with Quasi-Newton and Low-Dimensional Newton Methods, PhD. Thesis, Department of Mathematical Sciences, Loughborough University of Technology, Loughborough, Leicestershire, England, 1992.
Nazareth, J. L.,Conjugate Gradient Methods Less Dependent on Conjugacy, SIAM Review, Vol. 28, pp. 501–511, 1986.
Nazareth, J. L.,The Method of Successive Affine Reduction for Nonlinear Minimization, Mathematical Programming, Vol. 35, pp. 97–109, 1986.
Hu, Y. F., andStorey, C.,A Family of Optimally Conditioned Quasi-Newton Updates for Unconstrained Optimization, Mathematics Report A143, Department of Mathematical Sciences, Loughborough University of Technology, Loughborough, Leicestershire, England, 1991.
Hu, Y. F., andStorey, C.,Motivating Quasi-Newton Updates by Preconditioned Conjugate Gradient Methods, Mathematics Report A150, Department of Mathematical Sciences, Loughborough University of Technology, Loughborough, Leicestershire, England, 1991.
Shanno, D. F., andPhua, K. H.,Matrix Conditioning and Nonlinear Optimization, Mathematical Programming, Vol. 14, pp. 149–160, 1978.
Moré, J. J., Garbow, B. S., andHillstrom, K. E.,Testing Unconstrained Optimization Software, ACM Transactions on Mathematical Software, Vol. 7, pp. 17–41, 1978.
Hu, Y. F., andStorey, C.,Preconditioned Low-Order Newton Methods, Mathematics Report No. A159, Department of Mathematical Sciences, Loughborough University of Technology, Loughborough, Leicestershire, England, 1992.
Griewank, A.,On Automatic Differentiation, Mathematical Programming: Recent Developments and Applications, Edited by M. Iri and K. Tanabe, Kluwer Academic Publishers, Amsterdam, Holland, pp. 83–108, 1989.
Author information
Authors and Affiliations
Additional information
Communicated by L. C. W. Dixon
Rights and permissions
About this article
Cite this article
Hu, Y.F., Storey, C. Preconditioned low-order Newton methods. J Optim Theory Appl 79, 311–331 (1993). https://doi.org/10.1007/BF00940583
Issue Date:
DOI: https://doi.org/10.1007/BF00940583