Abstract
For a non-convex function \(f: R^{n} \rightarrow R\) with gradient g and Hessian H, define a step vector p(μ,x) as a function of scalar parameter μ and position vector x by the equation (H(x) + μI)p(μ,x) = −g(x). Under mild conditions on f, we construct criteria for selecting μ so as to ensure that the algorithm x := x + p(μ,x) descends to a second-order stationary point of f, and avoids saddle points.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Usually, but not always: for example, the hyperbola given by f(x) = (1 + x2)1/2 with x ∈ Rn is convex, but H− 1g = (1 + x2)x, so taking α = 1 oscillates for ∥x∥ = 1 and diverges for ∥x∥ > 1.
See for example Chapter F02 of the NAG Toolbox at www.nag.co.uk
References
Aitken, A.C.: Studies in practical mathematics II: the evaluation of latent roots and latent vectors of a matrix. Proc. R. Soc. Edinburgh 57, 269–304 (1937)
Bartholomew-Biggs, M., Beddiaf, S., Christianson, B.: Further developments of methods for traversing regions of non-convexity in optimization problems. Optimization Online. 8306 (2021)
Bartholomew-Biggs, M., Beddiaf, S., Christianson, B.: Comparison of methods for traversing regions of non-convexity in optimization problems. Numer. Algo. 85(2), 1–23 (2019)
Bartholomew-Biggs, M., Brown, S., Christianson, B., Dixon, L.: Automatic Differentiation of algorithms. J. Comput. Appl. Math. 124(1-2), 171–190 (2000)
Beddiaf, S.: Continuous Steepest Descent Path for Traversing Non-Convex Regions. Phd thesis University of Hertfordshire UK (2016)
Behrman, W.: An efficient gradient flow method for unconstrained optimization. PhD Thesis, Stanford University (1998)
Brown, A.A.: Optimization methods involving the solution of ordinary differential equations. PhD Thesis, Hatfield Polytechnic (1986)
Christianson, B.: Global Convergence using De-linked Goldstein or Wolfe Linesearch Conditions. Adv. Model. Optim. 11(1), 25–31 (2009)
Conn, AR, Gould, NIM, Toint, PT: Trust region methods. In: MPS-SIAM Series on Optimization, Philadelphia (2000)
Craig, I.J.D., Sneyd, A.D.: The acceleration of matrix power methods by cyclic variations of the shift parameter. Comput. Math. Appl. 17(7), 1149–1159 (1989)
Goldfeld, S.M., Quandt, R.E., Trotter, H.F.: Maximization by quadratc hill climbing. Econometrica 34, 541–551 (1966)
Hebden, M.D.: An Algorithm for Minimization Using Exact Second Derivatives, Atomic Energy Research Establishment Report TP515. Harwell, England (1973)
Higham, D.J.: Trust-region Algorithms and Time step selection. SIAM J. Numer. Anal. 37(1), 194–210 (1999)
Samadi, M.: Efficient trust region methods for nonconvex optimization. PhD Thesis. Lehigh University, Bethlehem (2019)
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Additional information
Data sharing
Data sharing is not applicable to this article, as no datasets were generated or analysed during the current study.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
‘Ah, you have said something true and so untidy,’ complained Constantine, ‘and what I said was not quite true, but so beautifully neat.’ Black Lamb and Grey Falcon Rebecca West
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Bartholomew-Biggs, M., Beddiaf, S. & Christianson, B. Global convergence of a curvilinear search for non-convex optimization. Numer Algor 92, 2025–2043 (2023). https://doi.org/10.1007/s11075-022-01375-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-022-01375-y