Skip to main content
SpringerLink
Log in

A comparison of methods for traversing regions of non-convexity in optimization problems

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

This paper considers the well-known problem of dealing with non-convexity during the minimization of a non-linear function f(x) by Newton-like methods. The proposal made here involves a curvilinear search along an approximation to the continuous steepest descent path defined by the solution of the differential equation

The algorithm we develop and describe has some features in common with trust-region methods and we present some numerical experiments in which its performance is compared with other ODE-based and trust-region methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Arrow, K.J., Hurwicz, L., Uzawa, H.: Studies in linear and Non-Linear Programming. Stanford University Press (1972)

  2. Bartholomew-Biggs, M.C., Beddiaf, S., Kane, S.J.: Traversing non-convex regions. Adv. Model. Optim. 15(2), 387–407 (2013)

    MATH  Google Scholar 

  3. Beddiaf, S.: Continuous steepest descent path for traversing non-convex regions. Adv. Model. Optim. 11(1), 3–24 (2009)

    MathSciNet  MATH  Google Scholar 

  4. Behrman, W.: An efficient gradient flow method for unconstrained optimization. Stanford University, PhD thesis (1998)

    Google Scholar 

  5. Botsaris, C.A.: A curvilinear optimization method based upon iterative estimation of the eigensystem of the hessian matrix. J. Maths. Anal. Appl. 63(2), 396–411 (1978)

    MathSciNet  MATH  Google Scholar 

  6. Botsaris, C.A.: Differential gradient methods. J. Maths. Anal. Appl. 63(1), 177–198 (1978)

    MathSciNet  MATH  Google Scholar 

  7. Branch, M.A., Coleman, T.F., Li, Y.: A subspace, interior, and conjugate gradient method for large-scale bound-constrained minimization problems. SIAM J. Sci. Comput. 21(1), 1–23 (1999)

    MathSciNet  MATH  Google Scholar 

  8. Brown, A.A.: Optimisation methods involving the solution of ordinary differential equations. PhD thesis, Hatfield Polytechnic (1986)

    Google Scholar 

  9. Brown, A.A., Bartholomew-Biggs, M.C.: Some effective methods for unconstrained optimization based on the solution of systems of ordinary differential equations. J. Optim Theory Appl. 62(2), 211–224 (1989)

    MathSciNet  MATH  Google Scholar 

  10. Broyden, C.G., Vespucci, M.T.: Krylov solvers for linear algebraic systems. Stud. Comput. Math., 11 (2004)

  11. Byrd, R.H., Schnabel, R.B., Shultz, G.A.: Approximate solution of the trust region problem by minimization over two dimensional subspaces. Math. Program. 40, 247–263 (1988)

    MathSciNet  MATH  Google Scholar 

  12. Christianson, B.: Global convergence using de-linked goldstein or wolfe linesearch conditions. Adv. Model. Optim. 11(1), 25–31 (2009)

    MathSciNet  MATH  Google Scholar 

  13. Conn, A.R., Gould, N.I.M., Toint, P.T.: Trust Region Methods. MPS-SIAM Series on Optimization, Philadelphia (2000)

    MATH  Google Scholar 

  14. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)

    MathSciNet  MATH  Google Scholar 

  15. Gould, N.I.M., Orban, D., Toint, P.h.L.: Cuter and sifdec: A constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)

    MATH  Google Scholar 

  16. Higham, D.J.: Trust region algorithms and timestep selection. SIAM J. Numer. Anal. 37(1), 194–210 (1999)

    MathSciNet  MATH  Google Scholar 

  17. MATLAB: Matlab, optimization toolbox, version 7.10.0 (R2010a), www.mathworks.com. The MathWorks Inc. Natick, Massachusetts (2010)

Download references

Author information

Authors and Affiliations

  1. School of Physics, Astronomy and Mathematics, University of Hertfordshire, Hatfield, UK

    Michael Bartholomew-Biggs & Salah Beddiaf

  2. School of Engineering and Computer Science, University of Hertfordshire, Hatfield, UK

    Bruce Christianson

Authors
  1. Michael Bartholomew-Biggs
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Salah Beddiaf
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bruce Christianson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Bartholomew-Biggs.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Some results using CUTEr test problems

Appendix: Some results using CUTEr test problems

Table 1 CUTEr test results from TR, Nimp1, Behrman, and Higham
Full size table

The following tables present results obtained with the methods discussed in this paper. The problems are drawn from the CUTEr test set [15] and the methods compared are the trust-region approach fminunc—denoted by TR—along wth Nimp1 and Behrman (as implemented in Algorithm 3) and the Higham method as implemented in Algorithm 4.

The stopping criteria used by all the methods were

$$ ||g(x_{k})||_{2} < 10^{-6} \text{ and } ||x_{k+1} - x_{k}||_{2} < 10^{-6}(1 + ||x_{k}||_{2}) $$

The values used for the controlling parameters for the iterations in Algorithms 3 and 4 are as follows:

$$ \alpha_{1} = 0.4, \alpha_{2} = 0.1, \eta_{2} = 0.9, \nu_{1} =0.5, \nu_{2} = 0.75. $$

It will be noted that the tables do not report on every one of the 139 test problems. We have omitted results for problems where it is clear from the counts of iterations and function calls that no interpolation or extrapolation has taken place. This allows us to observe more clearly the differences in performance in situations that are actually relevant to our investigation.

The following notation is used in the tables:

  • n: the number of variables

Its/Fcs: the number of iterations/function calls (The maximum number of iterations allowed was 10000 and a table entry of the form 10000/10001 indicates this limit has been reached.)

  1. “*”

    denotes the performance on each problem that is judged “best” by the criteria discussed in Section 5.1

  2. “F”

    indicates a method has suffered a numerical failure such as floating overflow in a function evauation, e.g. because of an exceptionally large correction step.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bartholomew-Biggs, M., Beddiaf, S. & Christianson, B. A comparison of methods for traversing regions of non-convexity in optimization problems. Numer Algor 85, 231–253 (2020). https://doi.org/10.1007/s11075-019-00811-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-019-00811-w

Keywords

Search

Navigation