Skip to main content

Cubic regularization of Newton method and its global performance

Mathematical Programming volume 108, pages 177–205 (2006)Cite this article

Abstract

In this paper, we provide theoretical analysis for a cubic regularization of Newton method as applied to unconstrained minimization problem. For this scheme, we prove general local convergence results. However, the main contribution of the paper is related to global worst-case complexity bounds for different problem classes including some nonconvex cases. It is shown that the search direction can be computed by standard linear algebra technique.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. Bennet, A.A.: Newton's method in general analysis. Proc. Nat. Ac. Sci. USA. 2 (10), 592–598 (1916)

    Google Scholar 

  2. Conn, A.B., Gould, N.I.M., Toint, Ph.L.: Trust Region Methods. SIAM, Philadelphia, 2000

  3. Dennis, J.E., Jr., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. SIAM, Philadelphia, 1996

  4. Fletcher, R.: Practical Methods of Optimization, Vol. 1, Unconstrained Minimization. John Wiley, NY, 1980

  5. Goldfeld, S., Quandt, R., Trotter, H.: Maximization by quadratic hill climbing. Econometrica. 34, 541–551 (1966)

    MATH  MathSciNet  Google Scholar 

  6. Kantorovich, L.V.: Functional analysis and applied mathematics. Uspehi Matem. Nauk. 3 (1), 89–185 (1948), (in Russian). Translated as N.B.S. Report 1509, Washington D.C. (1952)

    MathSciNet  Google Scholar 

  7. Levenberg, K.: A method for the solution of certain problems in least squares. Quart. Appl. Math. 2, 164–168 (1944)

    MATH  MathSciNet  Google Scholar 

  8. Marquardt, D.: An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11, 431–441 (1963)

    Article  MATH  MathSciNet  Google Scholar 

  9. Nemirovsky, A., Yudin, D.: Informational complexity and efficient methods for solution of convex extremal problems. Wiley, New York, 1983

  10. Nesterov, Yu.: Introductory lectures on convex programming: a basic course. Kluwer, Boston, 2004

  11. Nesterov, Yu., Nemirovskii, A.: Interior-Point Polynomial Algorithms in Convex Programming. SIAM, Philadelphia, 1994

  12. Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, NY, 1970

  13. Polyak, B.T.: Gradient methods for minimization of functionals. USSR Comp. Math. Math. Phys. 3 (3), 643–653 (1963)

    Google Scholar 

  14. Polyak, B.T.: Convexity of quadratic transformations and its use in control and optimization. J. Optim. Theory and Appl. 99 (3), 553–583 (1998)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Operations Research and Econometrics (CORE), Catholic University of Louvain (UCL), 34 voie du Roman Pays, 1348, Louvain-la-Neuve, Belgium

    Yurii Nesterov

  2. Institute of Control Science, Profsojuznaya 65, Moscow, 117997, Russia

    B.T. Polyak

Authors
  1. Yurii Nesterov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. B.T. Polyak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yurii Nesterov.

Additional information

The research results presented in this paper have been supported by a grant ``Action de recherche concertè ARC 04/09-315'' from the ``Direction de la recherche scientifique - Communautè française de Belgique''. The scientific responsibility rests with the authors.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Nesterov, Y., Polyak, B. Cubic regularization of Newton method and its global performance. Math. Program. 108, 177–205 (2006). https://doi.org/10.1007/s10107-006-0706-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-006-0706-8

Keywords

  • General nonlinear optimization
  • Unconstrained optimization
  • Newton method
  • Trust-region methods
  • Global complexity bounds
  • Global rate of convergence

Mathematics Subject Classification (1991)

  • 49M15
  • 49M37
  • 58C15
  • 90C25
  • 90C30