Applied Mathematics and Optimization

, Volume 62, Issue 1, pp 27–46

Convergence Properties of the Regularized Newton Method for the Unconstrained Nonconvex Optimization



The regularized Newton method (RNM) is one of the efficient solution methods for the unconstrained convex optimization. It is well-known that the RNM has good convergence properties as compared to the steepest descent method and the pure Newton’s method. For example, Li, Fukushima, Qi and Yamashita showed that the RNM has a quadratic rate of convergence under the local error bound condition. Recently, Polyak showed that the global complexity bound of the RNM, which is the first iteration k such that ‖f(xk)‖≤ε, is O(ε−4), where f is the objective function and ε is a given positive constant. In this paper, we consider a RNM extended to the unconstrained “nonconvex” optimization. We show that the extended RNM (E-RNM) has the following properties. (a) The E-RNM has a global convergence property under appropriate conditions. (b) The global complexity bound of the E-RNM is O(ε−2) if 2f is Lipschitz continuous on a certain compact set. (c) The E-RNM has a superlinear rate of convergence under the local error bound condition.

Regularized Newton methods Global convergence Global complexity bound Local error bound Superlinear convergence 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dan, H., Yamashita, N., Fukushima, M.: Convergence properties of the inexact Levenberg-Marquardt method under local error bound conditions. Optim. Methods Softw. 17, 605–626 (2002) MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1985) MATHGoogle Scholar
  3. 3.
    Li, Y.J., Li, D.H.: Truncated regularized Newton method for convex minimizations. Comput. Optim. Appl. 43, 119–131 (2009) MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Li, D.H., Fukushima, M., Qil, L., Yamashita, N.: Regularized Newton methods for convex minimization problems with singular solutions. Comput. Optim. Appl. 28, 131–147 (2004) MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999) MATHCrossRefGoogle Scholar
  6. 6.
    Polyak, R.A.: Regularized Newton method for unconstrained convex optimization. Math. Program., Ser. B 120, 125–145 (2009) MATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg-Marguardt method. Comput., Suppl (Wien) 15, 227–238 (2001) MathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Department of Applied Mathematics and Physics, Graduate School of InformaticsKyoto UniversityKyotoJapan

Personalised recommendations