Abstract
Metric-based SR1 updates which are stabilized by a variationalrelaxation of the quasi-Newton relation are examined. Thisinvestigation reveals an interesting and surprising connection to theorigin of quasi-Newton methods as first formulated by Davidon [1]. Anextended version of Davidon's original direct prediction SR1 updateis shown to be self-complementary and to possess a finite terminationproperty on quadratics, and limited-memory versions of the update areshown to be globally convergent. Variants of this update are testednumerically and compared to several other metric-based SR1 variantsand the BFGS update. Finally, metric-based stabilizations of the SR1update are critiqued in general, and a promising new model-basedstrategy recently developed is briefly described.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
H.L. Anderson and W.C. Davidon, "Machine analysis of pion scattering by the maximum likelihood method," Il Nuovo Cimento, Serie X, vol. 5, pp. 1238–1255, 1957.
R.K. Brayton and J. Cullum, "An algorithm for minimizing a differentiable function subject to box constraints and errors," J.O.T.A., vol. 29, pp. 521–558, 1979.
C.G. Broyden, "Quasi-Newton methods and their application to function minimization," Mathematics of Computation, pp. 368–381, 1967.
R.H. Byrd, "A multiobjective characterization of the Broyden class," presented at ORSA/TIMS, New York, 1990.
A.R. Conn, N.I.M. Gould, and Ph.L. Toint, "Convergence of quasi-Newton matrices generated by the symmetric rank one update," Mathematical Programing, vol. 50, pp. 177–195, 1991.
W.C. Davidon, "Variable metric methods for minimization," Research and Development Report ANL-5990 (Rev.), Argonne National Laboratory, Argonne, Illinois, 1959. (Reprinted in SIAM J. Optimization, vol. 1, 1991.)
W.C. Davidon, "Variance algorithm for minimization," Computer Journal, vol. 10, pp. 406–410, 1968.
W.C. Davidon, "Optimally conditioned optimization algorithms without line searches," Mathematical Programming, vol. 9, pp. 1–30, 1975.
W.C. Davidon, R.B. Mifflin, and J.L. Nazareth, "Some comments on notation for quasi-Newton methods," Optima, vol. 32, pp. 3–4, 1991.
J.E. Dennis and H. Wolkowicz, "Sizing and least change secant updates," CORR Report 90-02, Department of Combinatorics and Optimization, University of Waterloo, Ontario, Canada, 1990.
A.V. Fiacco and G.P. McCormick, Nonlinear Programming: Sequential Unconstrained Minimization Techniques, John Wiley: N.Y., 1968.
H. Khalfan, R.H. Byrd, and R.B. Schnabel, "A theoretical and experimental study of the symmetric rank one update," SIAM J. Optimization, vol. 3, pp. 1–24, 1993.
R.B. Mifflin and J.L. Nazareth, "The least prior deviation quasi-Newton update," Mathematical Programming, vol. 65, pp. 247–261, 1994.
J.J. Moré, B.S. Garbow, and K.E. Hillstrom, "Testing unconstrained optimization software," ACM Trans. on Mathematical Software, vol. 7, pp. 17–41, 1981.
J.J. Moré and D.J. Thuente, "On linesearch algorithms with guaranteed sufficient decrease," MCS-P153-0590, Argonne National Laboratory, Argonne, IL, 1990.
J.L. Nazareth, "An alternative variational principle for variable metric updating," Mathematical Programming, vol. 30, pp. 99–104, 1984.
J.L. Nazareth, "The Newton and Cauchy perspectives on computational nonlinear optimization," SIAM Review, vol. 36, pp. 215–225, 1994.
J. Nocedal, "Theory of algorithms for unconstrained optimization," Acta Numerica, vol. 1, pp. 199–242, 1991.
M.R. Osborne and L.P. Sun, "A new approach to the Symmetric-Rank-One updating algorithm," Report NMO/01, Department of Statistics, Australian National University, Canberra, Australia, 1988.
J.D. Pearson, "Variable metric methods of minimization," Computer Journal, vol. 12, pp. 171–178, 1969.
B. Smith, "Metric and model based line search symmetric rank one updates," Ph.D. Thesis, Department of Pure and Applied Mathematics, Washington State University, 1995.
P. Wolfe, "Convergence conditions for ascent methods," SIAM Review, vol. 11, pp. 226–2235, 1969.
P. Wolfe, "Convergence conditions for ascent methods II: Some corrections,"SIAM Review, vol. 13, pp. 185–188, 1971.
H. Wolkowicz, "Measures for symmetric rank-one updates," Mathematics of Operations Research, vol. 19, pp. 815–830, 1994.
H. Wolkowicz and Q. Zhao, "An all-inclusive efficient region of updates for least change secant methods," SIAM J. Optimization, vol. 5, pp. 171–191, 1995.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Smith, B., Nazareth, J. Metric-Based Symmetric Rank-One Updates. Computational Optimization and Applications 8, 219–244 (1997). https://doi.org/10.1023/A:1008645321602
Issue Date:
DOI: https://doi.org/10.1023/A:1008645321602