Abstract
It is shown that algorithms for minimizing an unconstrained functionF(x), x ∈ E n, which are solely methods of conjugate directions can be expected to exhibit only ann or (n−1) step superlinear rate of convergence to an isolated local minimizer. This is contrasted with quasi-Newton methods which can be expected to exhibit every step superlinear convergence. Similar statements about a quadratic rate of convergence hold when a Lipschitz condition is placed on the second derivatives ofF(x).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A. Cohen, “Convergence rates for root finding algorithms,” dissertation. Department of Electrical Engineering, University of California, Berkeley, California, 1970.
J.W. Daniel, “The conjugate gradient method for linear and nonlinear programming,”SIAM Journal of Numerical Analysis 4 (1967) 10–25.
W.C. Davidon, “Variance algorithm for minimization,”The Computer Journal 10 (1968).
J. Dieudonné,Foundations of modern analysis (Academic Press, New York, 1960).
R. Fletcher, “A new approach to variable metric algorithms,” Report No. T.P. 383, A.E.R.E. Harwell, 1969.
A.A. Goldstein and J.F. Price, “An effective algorithm for minimization,”Numerische Mathematik 10 (1967) 184–189.
M. Hestenes and E. Stiefel, “Method of conjugate gradients for solving linear systems,” Report 1659, National Bureau of Standards, 1952.
G.P. McCormick and J.D. Pearson, “Variable metric methods and unconstrained optimization,” in:Optimization, Ed. R. Fletcher (Academic Press, New York — London, 1969) pp. 307–326.
G.P. McCormick and K. Ritter, “On the convergence and rate of convergence of the conjugate gradient method,” MRC Report No. 1118, Mathematics Research Center, University of Wisconsin, Madison, Wisconsin, December 1970.
G.P. McCormick and K. Ritter, “The projection method for unconstrained optimization,” RAC-TP-425, Research Analysis Corporation, McLean, Virginia, June 1971.
E. Polak,Computational methods in optimization (Academic Press, New York, 1971).
B.T. Polyak, “The method of conjugate gradients in extremum problems,”USSR Computational Mathematics and Mathematical Physics 9, no.4 (1969) 807–821 (in Russian).
M.J.D. Powell, “On the convergence of the variable metric algorithm,” Report No. T.P. 382, Theoretical Physics Division, A.E.R.E. Harwell, 1969.
M.J.D. Powell, “A new algorithm for unconstrained optimization,” Report No. T.P. 393, A.E.R.E. Harwell, 1970.
K. Ritter, “A superlinearly convergent method for unconstrained minimization,” in:Nonlinear programming, Eds. J.B. Rosen, O.L. Mangasarian and K. Ritter (Academic Press, New York — London, 1970) pp. 177–206.
M.M. Vainberg,Variational methods for the study of nonlinear operators (Holden—Day, San Francisco, California, 1964).
G. Zoutendijk, “Some algorithms based on the principle of feasible directions,” in:Nonlinear programming, Eds. J.B. Rosen, O.L. Mangasarian and K. Ritter (Academic Press, New York — London, 1970) pp. 93–122.
Author information
Authors and Affiliations
Additional information
Research was supported in part by Army Research Office, Contract Number DAHC 19-69-C-0017 and the Office of Naval Research, Contract Number N00014-71-C-0116 (NR 047-99).
Rights and permissions
About this article
Cite this article
McCormick, G.P., Ritter, K. Methods of conjugate directions versus quasi-Newton methods. Mathematical Programming 3, 101–116 (1972). https://doi.org/10.1007/BF01584978
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01584978