Mathematical Programming

, Volume 112, Issue 1, pp 159–181 | Cite as

Accelerating the cubic regularization of Newton’s method on convex problems

FULL LENGTH PAPER

Abstract

In this paper we propose an accelerated version of the cubic regularization of Newton’s method (Nesterov and Polyak, in Math Program 108(1): 177–205, 2006). The original version, used for minimizing a convex function with Lipschitz-continuous Hessian, guarantees a global rate of convergence of order \(O\big({1 \over k^2}\big)\), where k is the iteration counter. Our modified version converges for the same problem class with order \(O\big({1 \over k^3}\big)\), keeping the complexity of each iteration unchanged. We study the complexity of both schemes on different classes of convex problems. In particular, we argue that for the second-order schemes, the class of non-degenerate problems is different from the standard class.

Keywords

Convex optimization Unconstrained minimization Newton’s method Cubic regularization Worst-case complexity Global complexity bounds Non-degenerate problems Condition number 

Mathematics Subject Classification (2000)

49M15 49M37 58C15 90C25 90C30 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bennet A.A. (1916). Newton’s method in general analysis. Proc. Nat. Ac. Sci. USA. 2(10): 592–598 CrossRefGoogle Scholar
  2. 2.
    Conn A.B., Gould N.I.M. and Toint Ph.L. (2000). Trust Region Methods. SIAM, Philadelphia MATHGoogle Scholar
  3. 3.
    Dennis J.E.Jr. and Schnabel R.B. (1996). Numerical Methods for Unconstrained Optimization and Nonlinear Equations. SIAM, Philadelphia MATHGoogle Scholar
  4. 4.
    Kantorovich, L.V.: Functional analysis and applied mathematics. Uspehi Matem. Nauk. 3(1), 89–185 (1948), (in Russian). Translated as N.B.S. Report 1509, Washington (1952)Google Scholar
  5. 5.
    Nesterov Yu. (2004). Introductory Lectures on Convex Programming. A Basic Course.. Kluwer, Boston Google Scholar
  6. 6.
    Nesterov Yu. and Polyak B. (2006). Cubic regularization of Newton method and its global performance. Math. Program. 108(1): 177–205 MATHCrossRefGoogle Scholar
  7. 7.
    Ortega J.M. and Rheinboldt W.C. (1970). Iterative Solution of Nonlinear Equations in Several Variables. Academic, NY MATHGoogle Scholar
  8. 8.
    Vladimirov, A., Nesterov, Yu., Chekanov, Yu.: Uniformly convex functionals. Vestnik Moskovskogo universiteta, ser. Vychislit. Matem. i Kibern., 4, 18–27 (1978), (In Russian)Google Scholar

Copyright information

© Springer-Verlag 2007

Authors and Affiliations

  1. 1.Center for Operations Research and Econometrics (CORE)Catholic University of Louvain (UCL)Louvain-la-NeuveBelgium

Personalised recommendations