Advertisement

On the use of third-order models with fourth-order regularization for unconstrained optimization

  • E. G. BirginEmail author
  • J. L. Gardenghi
  • J. M. Martínez
  • S. A. Santos
Original paper
First Online:
  • 23 Downloads

Abstract

In a recent paper (Birgin et al. in Math Program 163(1):359–368, 2017), it was shown that, for the smooth unconstrained optimization problem, worst-case evaluation complexity \(O(\epsilon ^{-(p+1)/p})\) may be obtained by means of algorithms that employ sequential approximate minimizations of p-th order Taylor models plus \((p+1)\)-th order regularization terms. The aforementioned result, which assumes Lipschitz continuity of the p-th partial derivatives, generalizes the case \(p=2\), known since 2006, which has already motivated efficient implementations. The present paper addresses the issue of defining a reliable algorithm for the case \(p=3\). With that purpose, we propose a specific algorithm and we show numerical experiments.

Keywords

Unconstrained minimization Third-order models Regularization Complexity 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

We are thankful to the anonymous reviewers, and to the Associate Editor, whose comments and suggestions improved the presentation of our work. This work has been partially supported by FAPESP (Grants 2013/03447-6, 2013/05475-7, 2013/07375-0, 2013/23494-9, 2016/01860-1, and 2017/03504-0) and CNPq (Grants 309517/2014-1, 303750/2014-6, and 302915/2016-8)

References

  1. 1.
    Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: Augmented Lagrangian methods under the constant positive linear dependence constraint qualification. Math. Program. 111(1), 5–32 (2008)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Andretta, M., Birgin, E.G., Martínez, J.M.: Practical active-set Euclidian trust-region method with spectral projected gradients for bound-constrained minimization. Optimization 54(3), 305–325 (2005)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Averbukh, V.Z., Figueroa, S., Schlick, T.: Remark on algorithm 566. ACM Trans. Math. Softw. 20(3), 282–285 (1994)zbMATHGoogle Scholar
  4. 4.
    Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont, Massachussets (1999)zbMATHGoogle Scholar
  5. 5.
    Birgin, E.G., Bueno, L.F., Martínez, J.M.: Assessing the reliability of general-purpose inexact restoration methods. J. Comput. Appl. Math. 282, 1–16 (2015)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Birgin, E.G., Bueno, L.F., Martínez, J.M.: Sequential equality-constrained optimization for nonlinear programming. Comput. Optim. Appl. 65(3), 699–721 (2016)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Birgin, E.G., Castillo, R.A., Martínez, J.M.: Numerical comparison of augmented Lagrangian algorithms for nonconvex problems. Comput. Optim. Appl. 31(1), 31–55 (2005)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Birgin, E.G., Gardenghi, J.L., Martínez, J.M., Santos, S.A.: Third-order derivatives of the Moré, Garbow, and Hillstrom test set problems. Technical Report MCDO010418, University of São Paulo (2018). http://www.ime.usp.br/~egbirgin/
  9. 9.
    Birgin, E.G., Gardenghi, J.L., Martínez, J.M., Santos, S.A., Toint, P.L.: Worst-case evaluation complexity for unconstrained nonlinear optimization using high-order regularized models. Math. Program. 163(1), 359–368 (2017)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Birgin, E.G., Gentil, J.M.: New and improved results for packing identical unitary radius circles within triangles, rectangles and strips. Comput. Oper. Res. 37(7), 1318–1327 (2010)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Birgin, E.G., Gentil, J.M.: Evaluating bound-constrained minimization software. Comput. Optim. Appl. 53(2), 347–373 (2012)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Birgin, E.G., Martínez, J.M.: A box-constrained optimization algorithm with negative curvature directions and spectral projected gradients. Computing [Suppl] 15, 49–60 (2001)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Birgin, E.G., Martínez, J.M.: Large-scale active-set box-constrained optimization method with spectral projected gradients. Comput. Optim. Appl. 23(1), 101–125 (2002)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Birgin, E.G., Martínez, J.M.: Structured minimal-memory inexact quasi-Newton method and secant preconditioners for augmented Lagrangian optimization. Comput. Optim. Appl. 39(1), 1–16 (2008)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Birgin, E.G., Martínez, J.M.: Augmented Lagrangian method with nonmonotone penalty parameters for constrained optimization. Comput. Optim. Appl. 51(3), 941–965 (2012)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Birgin, E.G., Martínez, J.M.: Practical Augmented Lagrangian Methods for Constrained Otimization, Fundamentals of Algorithms, vol. 10. SIAM, Philadelphia, PA (2014)Google Scholar
  17. 17.
    Birgin, E.G., Martínez, J.M.: The use of quadratic regularization with a cubic descent condition for unconstrained optimization. SIAM J. Optim. 27(2), 1049–1074 (2017)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Birgin, E.G., Martínez, J.M., Ronconi, D.P.: Optimizing the packing of cylinders into a rectangular container: a nonlinear approach. Eur. J. Oper. Res. 160(1), 19–33 (2005)zbMATHGoogle Scholar
  19. 19.
    Birgin, E.G., Sobral, F.N.C.: Minimizing the object dimensions in circle and sphere packing problems. Comput. Oper. Res. 35(7), 2357–2375 (2008)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Cartis, C., Gould, N.I.M., Toint, P.L.: On the complexity of steepest descent, Newton’s and regularized Newton’s methods for nonconvex unconstrained optimization problems. SIAM J. Optim. 20(6), 2833–2852 (2010)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Cartis, C., Gould, N.I.M., Toint, P.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. 127(2), 245–295 (2011)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Cartis, C., Gould, N.I.M., Toint, P.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity. Math. Program. 130(2), 295–319 (2011)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Curtis, F.E., Mitchell, T., Overton, M.L.: A BFGS-SQP method for nonsmooth, nonconvex, constrained optimization and its evaluation using relative minimization profiles. Optim. Methods Softw. 32(1), 148–181 (2017)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Curtis, F.E., Robinson, D.P., Samadi, M.: A trust region algorithm with a worst-case iteration complexity of \(\cal{O}(\epsilon ^{-3/2})\) for nonconvex optimization. Math. Program. 162(1), 1–32 (2017)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Dussault, J.P.: ARCq: a new adaptive regularization by cubics. Optim. Methods Softw. 33(2), 322–335 (2018)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Grapiglia, G.N., Yuan, J., Yuan, Y.: On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization. Math. Program. 152(1), 491–520 (2015)MathSciNetzbMATHGoogle Scholar
  28. 28.
    Griewank, A.: The modification of Newton’s method for unconstrained optimization by bounding cubic terms. Technical Report. NA/12, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, England (1981)Google Scholar
  29. 29.
    Martínez, J.M., Raydan, M.: Cubic-regularization counterpart of a variable-norm trust-region method for unconstrained minimization. J. Glob. Optim. 68(2), 367–385 (2017)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Martínez, L., Andrade, R., Birgin, E.G., Martínez, J.M.: PACKMOL: a package for building initial configurations for molecular dynamics simulations. J. Comput. Chem. 30(13), 2157–2164 (2009)Google Scholar
  31. 31.
    Mitchell, T.: Robust and efficient methods for approximation and optimization of stability measures. Ph.D. thesis, Department of Computer Science, New York University, New York (2014)Google Scholar
  32. 32.
    Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7(1), 17–41 (1981)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Nesterov, Y., Polyak, B.T.: Cubic regularization of Newton method and its global performance. Math. Program. 108(1), 177–205 (2006)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science, Institute of Mathematics and StatisticsUniversity of São PauloSão PauloBrazil
  2. 2.Department of Applied Mathematics, Institute of Mathematics, Statistics, and Scientific ComputingUniversity of CampinasCampinasBrazil

Personalised recommendations