Abstract
Davidon's new quasi-Newton optimization algorithm selects the new inverse Hessian approximation\(\bar H\) at each step to be the “optimally conditioned” member of a certain one-parameter class of rank two updates to the last inverse Hessian approximationH. In this paper we show that virtually the same goals of conditioning can be achieved while restricting\(\bar H\) to the convex class of updates, which are bounded by the popular DFP and BFGS updates. This suggests the computational testing of alternatives to the “optimal conditioning” strategy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
K.W. Brodlie, “Unconstrained minimization”, in: D. Jacobs, ed.,Modern numerical analysis (Academic Press, London) to appear (Presented at the York State-of-the-Art in Numerical Analysis Conference, University of York, April 1976.)
K.W. Brodlie, A.R. Gourlay and J. Greenstadt, “Rank-one and rank-two corrections to positive definite matrices expressed in product form”,Journal of the Institute of Mathematics and its Applications 11 (1973) 73–82.
C.G. Broyden, “The convergence of a class of double-rank minimization algorithms”,Journal of the Institute of Mathematics and its Applications 6 (1970) 76–90.
C.G. Broyden, J.E. Dennis and J.J. Moré, “On the local and superlinear convergence of quasi-Newton methods”,Journal of the Institute of Mathematics and its Applications 12 (1973) 223–246.
W.C. Davidon, “Variable metric method for minimization”, Argonne National Laboratory Report ANL-5990 Rev. (1959).
W.C. Davidon, “Optimally conditioned optimization algorithms without line searches”,Mathematical Programming 9 (1975) 1–30.
J.E. Dennis and H.H.W. Mei, “An unconstrained optimization algorithm which uses function and gradient values”,Journal of Optimization Theory and its Applications, to appear.
R. Fletcher, “A new approach to variable metric algorithms”,The computer Journal 13 (1970) 317–322.
R. Fletcher and M.J.D. Powell, “A rapidly convergent descent method for minimization”,The Computer Journal 6 (1963) 163–168.
D. Goldfarb, “A family of variable metric updates defined by variational means”,Mathematics of Computation 24 (1970) 23–26.
H.H.W. Mei, “On the conditioning of DFP and BFGS updates for unconstrained optimization”, Tech. Rept. 75-265, Department of Computer Science, Cornell University, Ithaca, N.Y. (1975).
H.H.W. Mei, “An analysis and implementation of Davidon's techniques for unconstrained optimization”, Ph.D. thesis, Department of Computer Science, Cornell University, Ithaca, N.Y. (1977).
S.S. Oren and E. Spedicato, “Optimal conditioning of self-scaling variable metric algorithms”,Mathematical Programming 10 (1976) 70–90.
M.J.D. Powell, “Some global convergence properties of a variable metric algorithm for minimization without exact line searches”, in: R.W. Cottle and C.E. Lemke, eds.,Nonlinear programming (SIAM-AMS proceedings, vol. IX, 1976).
R.B. Schnabel, “Analyzing and improving quasi-Newton methods for unconstrained optimization”, Ph.D. thesis, Department of Computer Science, Cornell University, Ithaca, N.Y. (1977).
D.F. Shanno, “Conditioning of quasi-Newton methods for function minimization”,Mathematics of Computation 24 (1970) 647–656.
E. Spedicato, “A bound to the condition number of canonical rank-two corrections and applications to the variable metric method”,Calcolo 12 (1975) 185–200.
E. Specicato, “On condition numbers of matrices in rank-two minimization algorithms”, in: L.C.W. Dixon and G.P. Szegö, eds.,Towards global minimization (North-Holland, 1975).
J. Stoer, “On the convergence rate of imperfect minimization algorithms in Broyden'sβ-class”,Mathematical Programming 9 (1975) 313–335.
Author information
Authors and Affiliations
Additional information
This research supported by NSF grant 73-03413, contract P04361 of the National Bureau of Economic Research, Cambridge, Massachusetts, and a National Science Foundation Graduate Fellowship, forms a portion of the author's doctoral thesis at Cornell University directed by Professor J.E. Dennis.
Rights and permissions
About this article
Cite this article
Schnabel, R.B. Optimal conditioning in the convex class of rank two updates. Mathematical Programming 15, 247–260 (1978). https://doi.org/10.1007/BF01609030
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01609030