Abstract
Many iterative algorithms for optimization calculations form positive definite second derivative approximations,B say, automatically, butB is not stored explicitly because of the need to solve equations of the formBd--g. We consider working with matricesZ, whose columns satisfy the conjugacy conditionsZ 1 BZ=1. Particular attention is given to updatingZ in a way that corresponds to revisingB by the BFGS formula. A procedure is proposed that seems to be much more stable than the direct use of a product formula [1]. An extension to this procedure provides some automatic rescaling of the columns ofZ, which avoids some inefficiencies due to a poor choice of the initial second derivative approximation. Our work is also relevant to active set methods for linear inequality constraints, to updating the Cholesky factorization ofB, and to explaining some properties of the BFGS algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
K.W. Brodlie, A.R. Gourlay and J. Greenstadt, “Rank-one and rank-two corrections to positive definite matrices expressed in product form”,Journal of the Institute of Mathematics and its Applications 11 (1973) 73–82.
W.C. Davidon, “Optimally conditioned optimization algorithms without line searches”,Mathematical Programming 9 (1975) 1–30.
R. Fletcher,Practical Methods of Optimization, Vol. 1: Unconstrained Optimization (John Wiley & Sons, Chichester, 1980).
R. Fletcher and M.J.D. Powell, “On the modification ofLDL T factorizations”,Mathematics of Computation 28 (1974) 1067–1087.
W.M. Gentleman, “Least squares computations by Givens transformations without square roots,”Journal of the Institute of Mathematics and its Applications 12 (1973) 329–336.
P.E. Gill and W. Murray, “Quasi-Newton methods for unconstrained optimization”,Journal of the Institute of Mathematics and its Applications 9 (1972) 91–108.
D. Goldfarb, “Factorized variable metric methods for unconstrained optimization”,Mathematics of Computation 30 (1976) 796–811.
D. Goldfarb and A. Idnani, “A numerically stable dual method for solving strictly convex quadratic programs”,Mathematical Programming 27 (1983) 1–33.
S-P. Han, “Optimization by updated conjugated subspaces,” in: D.F. Griffiths and G.A. Watson, eds.,Numerical Analysis: Pitman Research Notes in Mathematics Series 140 (Longman Scientific & Technical, Burnt Mill, England) pp. 82–97.
W. Murray, “An algorithm for finding a local minimum of an indefinite quadratic program,” Report NAC 1, National Physical Laboratory, Teddington (1971).
M.J.D. Powell, “On the quadratic programming algorithm of Goldfarb and Idnani,”Mathematical Programming Study 25 (1985) 46–61.
Author information
Authors and Affiliations
Additional information
Dedicated to Martin Beale, whose achievements, advice and encouragement were of great value to my research, especially in the field of conjugate direction methods.
Rights and permissions
About this article
Cite this article
Powell, M.J.D. Updating conjugate directions by the BFGS formula. Mathematical Programming 38, 29–46 (1987). https://doi.org/10.1007/BF02591850
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02591850