Summary
Many algorithms for solving minimization problems of the form
are devised such that they terminate with the optimal solution\(\bar x\) within at mostn steps, when applied to the minimization of strictly convex quadratic functionsf onR n. In this paper general conditions are given, which together with the quadratic termination property, will ensure that the algorithm locally converges at leastn-step quadratically to a local minimum\(\bar x\) for sufficiently smooth nonquadratic functionsf. These conditions apply to most algorithms with the quadratic termination property.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Broyden, C.G.: Quasi-Newton methods and their application to function minimization. Math. Comput.21, 368–381 (1967)
Broyden, C.G.: The convergence of a class of double-rank minimization algorithms. Part 1 and Part 2. J. Inst. Math. Appl.6, 76–90, 222–231 (1970)
Broyden, C.G., Dennis, J.E., Jr., Moré, J.J.: On the local and superlinear convergence of Quasi-Newton methods. J. Inst. Math. Appl.12, 223–245 (1973)
Burmeister, W.: Die Konvergenzordnung des Fletcher-Powell Algorithmus. Z. angew. Math. Mech.53, 696–699 (1973)
Cohen, A.: Rate of convergence of several conjugate gradient algorithms. SIAM J. numer. Analysis9, 248–259 (1972)
Daniel, J.W.: The conjugate gradient method for linear and nonlinear operator equations. SIAM J. numer. Analysis4, 10–26 (1967)
Davidon, W.C.: Variable metric method for minimization. Argonne Nat. Labs., Report ANL-5990, Rev. 1959
Dennis, J.E., Jr.: Toward a unified convergence theory for Newton-like methods. In: Non-linear functional analysis and applications (L.B. Rall, ed.). New York: Academic Press 1971
Dennis, J.E., Jr.: On some methods based on Broyden's secant approximation to the Hessian. In: Numerical methods for non-linear optimization (F.A. Lootsma, ed.). London: Academic Press 1972
Dennis, J.E., Jr., Moré, J.J.: A characterization of superlinear convergence and its applications to quasi-Newton methods. Math. Comput.28, 549–560 (1974)
Dennis, J.E., Jr., Moré, J.J.: QuasiNewton methods, motivation and theory. SIAM Review19, 46–89 (1977)
Dixon, L.C.W.: On quadratic termination and second order convergence: Two properties of unconstrained optimization algorithms. 1975
Fletcher, R.: A new approach to variable metric algorithms. Computer J.13, 317–322 (1970)
Fletcher, R., Powell, M.J.D.: A rapidly convergent descent method for minimization. Computer J.6, 163–168 (1963)
Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Computer J.7, 145–154 (1964)
Huang, H.Y.: Unified approach to quadratically convergent algorithms for function minimization. J. Optimization Theory Appl.5, 405–423 (1970)
Lenard, M.L.: Practical convergence conditions for unconstrained optimization. Math. Programming4, 309–323 (1973)
Lenard, M.L.: Practical convergence conditions for the Davidon-Fletcher-Powell method. Math. Programming9, 69–86 (1975)
Lenard, M.L.: Convergence conditions for restarted conjugate gradient methods with inaccurate line searches. Math. Programming10, 32–51 (1976)
McCormick, G.P., Ritter, K.: Methods of conjugate directions versus quasi-Newton methods. Math. Programming3, 101–116 (1972)
McCormick, G.P.: Alternative proofs of the convergence properties of the conjugate-gradient method. J. Optimization Theory Appl.13, 497–518 (1974)
Oren, S.S., Luenberger, D.G.: Self-scaling variable metric algorithms. Part I: Criteria and sufficient conditions for scaling a class of algorithms. Management Sci.20, 845–862 (1974)
Oren, S.S.: Self-scaling variable metric algorithms, Part II: Implementation and experiments. Management Sci.20, 863–874 (1974)
Pearson, J.D.: Variable metric methods of minimization. Computer J.12, 171–178 (1969)
Polak, E., Ribière, G.: Note sur la convergence de methodes de directions conjugées. Rev. Francaise Informat. Recherche Operationelle (16-R1), pp. 35–43, 1969
Powell, M.J.D.: On the convergence of the variable metric algorithm. J. Inst. Math. Appl.7, 21–36 (1971)
Powell, M.J.D.: Some properties of the variable metric algorithm. In: Numerical methods for nonlinear optimization (F.A. Lootsma, ed.). London: Academic Press 1972
Powell, M.J.D.: Some global convergence properties for a variable metric algorithm for minimization without exact line searches. Paper presented at AMS Symposium on non-linear programming, New York, 1975
Schuller, G., Stoer, J.: Über die Konvergenzordnung gewisser Rang-2 Verfahren zur Minimierung von Funktionen. In: International series of numerical mathematics, Vol. 23, pp. 125–147. Basel: Birkhäuser 1974
Stoer, J.: On the convergence rate of imperfect minimization algorithms in Broyden's ß-class. Math. Programming9, 313–335 (1975)
Author information
Authors and Affiliations
Additional information
This work was supported in part at Stanford University, Stanford, California, under Energy Research and Development Administration, Contract E(04-3) 326 PA No. 30, and National Science Foundation Grant DCR 71-01996 AO 4 and in part by the Deutsche Forschungsgemeinschaft
Rights and permissions
About this article
Cite this article
Stoer, J. On the relation between quadratic termination and convergence properties of minimization algorithms. Numer. Math. 28, 343–366 (1977). https://doi.org/10.1007/BF01389973
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF01389973