Abstract
For the problem of minimizing an unconstrained function, the conjugate-gradient method is shown to be convergent. If the function is uniformly strictly convex, the ultimate rate of convergence is shown to ben-step superlinear. If the Hessian matrix is Lipschitz continuous, the rate of convergence is shown to ben-step quadratic. All results are obtained for the reset version of the method and with a relaxed requirement on the solution of the stepsize problem. In addition to obtaining sharper results, the paper differs from previously published ones in the mode of proof which contains as a corollary the proof of finiteness of the conjugate-gradient method when applied to a quadratic problem rather than assuming that result.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Hestenes, M. R., andStiefel, E.,Method of Conjugate Gradients for Solving Linear Systems, National Bureau of Standards, Report No. 1959, 1952.
Crowder, H., andWolfe, P.,Linear Convergence of the Conjugate Gradient Method, IBM Thomas J. Watson Research Center, Report No. RC-3330, 1971.
Cohen, A.,Rate of Convergence of Several Conjugate Gradient Algorithms, SIAM Journal on Numerical Analysis (to appear).
Polak, E.,Computational Methods in Optimization, Academic Press, New York, New York, 1971.
Polyak, B. T.,The Method of Conjugate-Gradients in Extremum Problems (in Russian), USSR Computational Mathematics and Mathematical Physics, Vol. 9, No. 4, 1969.
Polyak, B. T.,The Conjugate-Gradient Method inWorks of the Second Winter School on Mathematical Programming and Related Questions, Vol. 1, pp. 152–202, 1969.
Smoljan, S. A.,The Conjugate Gradient Method inWorks of the Third Winter School on Mathematical Programming and Related Questions, Vol. 3, pp. 558–574, 1970.
Klessig, R., andPolak, E.,Efficient Implementations of the Polak-Ribiere Conjugate Gradient Algorithm, University of California, Electronics Research Laboratory, Memorandum No. ERL-M279, 1970.
Polak, E., andRibiere, G., Note sur la Convergence de Méthodes de Directions Conjugées, Revue Française d'Informatique et de Recherche Opérationnelle, Vol. 16, pp. 35–43, 1969.
Dieudonnée, J.,Foundations of Modern Analysis, Academic Press, New York, New York, 1960.
Vainberg, M. M.,Variational Methods for the Study of Nonlinear Operators, Holden-Day, San Francisco, California, 1964.
Fletcher, R., andReeves, C. M.,Function Minimization by Conjugate Gradients, Computer Journal, Vol. 7, pp. 149–154, 1964.
Daniel, J. W.,Convergence of the Conjugate Gradient Method with Computationally Convenient Modifications, Numerische Mathematik, Vol. 10, pp. 125–131, 1967.
Zoutendijk, G.,Nonlinear Programming, Computational Methods, Integer and Nonlinear Programming, Edited by J. Abadie, North-Holland Publishing Company, Amsterdam, Holland, 1970.
Mangasarian, O. L., Private Communication, 1972.
Daniel, J. W.,A Correction Concerning the Convergence Rate for the Conjugate Gradient Method, SIAM Journal of Numerical Analysis, Vol. 7, No. 2, 1970.
Daniel, J. W.,The Conjugate Gradient Method for Linear and Nonlinear Programming, SIAM Journal on Numerical Analysis, Vol. 7, No. 2, 1970.
Author information
Authors and Affiliations
Additional information
Communicated by M. R. Hestenes
This research was sponsored by the United States Army under Contracts No. DA-31-124-ARO-D-462 and DAHC-04-73-C-0009.
Rights and permissions
About this article
Cite this article
McCormick, G.P., Ritter, K. Alternative proofs of the convergence properties of the conjugate-gradient method. J Optim Theory Appl 13, 497–518 (1974). https://doi.org/10.1007/BF00933041
Issue Date:
DOI: https://doi.org/10.1007/BF00933041