Abstract
The convergence rates of descent methods with different stepsize rules are compared. Among the stepsize rules considered are: constant stepsize, exact minimization along a line, Goldstein-Armijo rules, and stepsize equal to that which yields the minimum of certain interpolatory polynomials. One of the major results shown is that the rate of convergence of descent methods with the Goldstein-Armijo stepsize rules can be made as close as desired to the rate of convergence of methods that require exact minimization along a line. Also, a descent algorithm that combines a Goldstein-Armijo stepsize rule with a secant-type step is presented. It is shown that this algorithm has a convergence rate equal to the convergence of descent methods that require exact minimization along a line and that, eventually (i.e., near the minimum), it does not require a search to determine an acceptable stepsize.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ortega, J. M., andRheinbolt, W. C.,Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, New York, 1970.
Goldstein, A. A.,Constructive Real Analysis, Harper and Row, New York, New York, 1967.
Armijo, L.,Minimization of Functions Having Continuous Partial Derivatives, Pacific Journal of Mathematics, Vol. 16, pp. 1–3, 1966.
Wolfe, P.,Convergence Conditions for Ascent Methods, SIAM Review, Vol. 11, pp. 226–235, 1969.
Polak, E.,Computational Methods in Optimization: A Unified Approach, Academic Press, New York, New York, 1971.
Crockett, J. B., andChernoff, H.,Gradient Methods of Maximization, Pacific Journal of Mathematics, Vol. 5, pp. 33–50, 1955.
Goldstein, A.,Cauchy's Method of Minimization, Numerische Mathematik, Vol. 4, pp. 146–150, 1962.
Polyak, B. T.,Gradient Methods for the Minimization of Functionals, USSR Computational Mathematics and Mathematical Physics, Vol. 3, pp. 864–878, 1963.
Greenstadt, J.,On the Relative Efficiencies of Gradient Methods, Mathematics of Computation, Vol. 21, pp. 360–367, 1967.
Luenberger, D. G.,Introduction to Linear and Nonlinear Programming, Addison-Wesley Publishing Company, Reading, Massachusetts, 1973.
Akaike, H.,On the Successive Transformation of Probability Distributions and Its Application to the Analysis of the Optimum Gradient Method, Annals of Institute of Mathematical Statistics, Tokyo, Japan, Vol. 11, pp. 1–17, 1959.
Bauer, F. L., andHouseholder, A. S.,Some Inequalities Involving the Euclidean Condition of a Matrix, Numerische Mathematik, Vol. 2, pp. 308–311, 1960.
Author information
Authors and Affiliations
Additional information
Communicated by D. Q. Mayne
Rights and permissions
About this article
Cite this article
Cohen, A.I. Stepsize analysis for descent methods. J Optim Theory Appl 33, 187–205 (1981). https://doi.org/10.1007/BF00935546
Issue Date:
DOI: https://doi.org/10.1007/BF00935546