Abstract
The theoretical foundation of path-following methods is the performance analysis of the (damped) Newton step on the class of self-concordant functions. However, the bounds available in the literature and used in the design of path-following methods are not optimal. In this contribution we use methods of optimal control theory to compute the optimal step length of the Newton method on the class of self-concordant functions, as a function of the initial Newton decrement, and the resulting worst-case decrease of the decrement. The exact bounds are expressed in terms of solutions of ordinary differential equations which cannot be integrated explicitly. We provide approximate numerical and analytic expressions which are accurate enough for use in optimization methods. Consequently, the neighbourhood of the central path in which the iterates of path-following methods are required to stay can be enlarged, enabling faster progress along the central path during each iteration and hence fewer iterations to achieve a given accuracy.
Similar content being viewed by others
Notes
In preparation, joint work with Anastasia S. Ivanova (Moscow Institute of Physics and Technology).
References
Bellman R (2003) Dynamic programming. Dover
Burdakov OP (1980) Some globally convergent modifications of Newton’s method for solving systems of linear equations. Soviet Math Dokl 22(2):376–379
Gao W, Goldfarb D (2019) Quasi-Newton methods: superlinear convergence without line searches for self-concordant functions. Optim Methods Softw 34(1):194–217
Hildebrand R (2021) Optimal inequalities between distances in convex projective domains. J Geom Anal 31(11):11357–11385
de Klerk E, Glineur F, Taylor A (2017) On the worst-case complexity of the gradient method with exact line search for smooth strongly convex functions. Optim Lett 11:1185–1199
de Klerk E, Glineur F, Taylor A (2020) Worst-case convergence analysis of inexact gradient and Newton methods through semidefinite programming performance estimation. SIAM J Optim 30(3):2053–2082
Nesterov Y (2018) Lectures on convex optimization, Springer optimization and its applications, 2nd edn, vol 137. Springer
Nesterov Y, Nemirovskii A (1994) Interior-point polynomial algorithms in convex programming, SIAM studies in applied mathematics, vol 13. SIAM, Philadelphia
Pontryagin L, Boltyanskii V, Gamkrelidze R, Mischchenko E (1962) The mathematical theory of optimal processes. Wiley, New York
Ralph D (1994) Global convergence of damped Newton’s method for nonsmooth equations via the path search. Math Oper Res 19(2):352–389
Renegar J (2001) A mathematical view of interior-point methods in convex optimization. SIAM, Philadelphia
Taylor AB, Hendrickx JM, Glineur F (2017) Exact worst-case performance of first-order methods for composite convex optimization. SIAM J Optim 27(3):1283–1313
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author declares that there is no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Hildebrand, R. Optimal step length for the Newton method: case of self-concordant functions. Math Meth Oper Res 94, 253–279 (2021). https://doi.org/10.1007/s00186-021-00755-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00186-021-00755-9