Skip to main content

Optimal step length for the Newton method: case of self-concordant functions

Abstract

The theoretical foundation of path-following methods is the performance analysis of the (damped) Newton step on the class of self-concordant functions. However, the bounds available in the literature and used in the design of path-following methods are not optimal. In this contribution we use methods of optimal control theory to compute the optimal step length of the Newton method on the class of self-concordant functions, as a function of the initial Newton decrement, and the resulting worst-case decrease of the decrement. The exact bounds are expressed in terms of solutions of ordinary differential equations which cannot be integrated explicitly. We provide approximate numerical and analytic expressions which are accurate enough for use in optimization methods. Consequently, the neighbourhood of the central path in which the iterates of path-following methods are required to stay can be enlarged, enabling faster progress along the central path during each iteration and hence fewer iterations to achieve a given accuracy.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Notes

  1. 1.

    In preparation, joint work with Anastasia S. Ivanova (Moscow Institute of Physics and Technology).

References

  1. Bellman R (2003) Dynamic programming. Dover

  2. Burdakov OP (1980) Some globally convergent modifications of Newton’s method for solving systems of linear equations. Soviet Math Dokl 22(2):376–379

  3. Gao W, Goldfarb D (2019) Quasi-Newton methods: superlinear convergence without line searches for self-concordant functions. Optim Methods Softw 34(1):194–217

    MathSciNet  Article  Google Scholar 

  4. Hildebrand R (2021) Optimal inequalities between distances in convex projective domains. J Geom Anal 31(11):11357–11385

    MathSciNet  Article  Google Scholar 

  5. de Klerk E, Glineur F, Taylor A (2017) On the worst-case complexity of the gradient method with exact line search for smooth strongly convex functions. Optim Lett 11:1185–1199

    MathSciNet  Article  Google Scholar 

  6. de Klerk E, Glineur F, Taylor A (2020) Worst-case convergence analysis of inexact gradient and Newton methods through semidefinite programming performance estimation. SIAM J Optim 30(3):2053–2082

  7. Nesterov Y (2018) Lectures on convex optimization, Springer optimization and its applications, 2nd edn, vol 137. Springer

  8. Nesterov Y, Nemirovskii A (1994) Interior-point polynomial algorithms in convex programming, SIAM studies in applied mathematics, vol 13. SIAM, Philadelphia

    Book  Google Scholar 

  9. Pontryagin L, Boltyanskii V, Gamkrelidze R, Mischchenko E (1962) The mathematical theory of optimal processes. Wiley, New York

    Google Scholar 

  10. Ralph D (1994) Global convergence of damped Newton’s method for nonsmooth equations via the path search. Math Oper Res 19(2):352–389

  11. Renegar J (2001) A mathematical view of interior-point methods in convex optimization. SIAM, Philadelphia

    Book  Google Scholar 

  12. Taylor AB, Hendrickx JM, Glineur F (2017) Exact worst-case performance of first-order methods for composite convex optimization. SIAM J Optim 27(3):1283–1313

    MathSciNet  Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Roland Hildebrand.

Ethics declarations

Conflict of interest

The author declares that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hildebrand, R. Optimal step length for the Newton method: case of self-concordant functions. Math Meth Oper Res 94, 253–279 (2021). https://doi.org/10.1007/s00186-021-00755-9

Download citation

Keywords

  • Newton method
  • Step length
  • Path-following methods
  • Optimal control

Mathematics Subject Classification

  • 90C51
  • 90C60