Skip to main content
SpringerLink
Log in

A matrix-free trust-region newton algorithm for convex-constrained optimization

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

We describe a matrix-free trust-region algorithm for solving convex-constrained optimization problems that uses the spectral projected gradient method to compute trial steps. To project onto the intersection of the feasible set and the trust region, we reformulate and solve the dual projection problem as a one-dimensional root finding problem. We demonstrate our algorithm’s performance on various problems from data science and PDE-constrained optimization. Our algorithm shows superior performance when compared with five existing trust-region and spectral projected gradient methods, and has the added benefit that it is simple to implement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Andretta, M., Birgin, E.G., Martínez, J.M.: Practical active-set Euclidian trust-region method with spectral projected gradients for bound-constrained minimization. Optimization 54(3), 305–325 (2005)

    Article  MathSciNet  Google Scholar 

  2. Antil, H., Kouri, D.P., Lacasse, M.D., Ridzal, D.: Frontiers in PDE-constrained optimization, vol. 163. Springer, Cham (2018)

    Book  Google Scholar 

  3. Baker, C.G., Heroux, M.A.: Tpetra, and the use of generic programming in scientific computing. Sci. Program. 20(2), 115–128 (2012)

    Google Scholar 

  4. Bendsoe, M.P., Sigmund, O.: Topology optimization: theory, methods, and applications. Springer, Cham (2013)

    MATH  Google Scholar 

  5. Birgin, E.G., Martínez, J.M., Raydan, M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Opt. 10(4), 1196–1211 (2000)

    Article  MathSciNet  Google Scholar 

  6. Birgin, E.G., Martínez, J.M., Raydan, M.: Inexact spectral projected gradient methods on convex sets. IMA J. Numer. Anal. 23(4), 539–559 (2003)

    Article  MathSciNet  Google Scholar 

  7. Birgin, E.G., Martínez, J.M., Raydan, M.: Spectral projected gradient methods: review and perspectives. J. Stat. Softw 60(3), 1–21 (2014)

    Article  Google Scholar 

  8. Bochev, P., Edwards, H.C., Kirby, R.C., Peterson, K., Ridzal, D.: Solving PDEs with intrepid. Sci. Program. 20(2), 151–180 (2012)

    Google Scholar 

  9. Borrvall, T., Petersson, J.: Topology optimization of fluids in stokes flow. Int. J. Numer. Methods Fluids 41(1), 77–107 (2003)

    Article  MathSciNet  Google Scholar 

  10. Brent, R.P.: Algorithms for minimization without derivatives. Courier Corporation (2013)

  11. Burke, J.V., Moré, J.J., Toraldo, G.: Convergence properties of trust region methods for linear and convex constraints. Math. Program. 47(1–3), 305–336 (1990)

    Article  MathSciNet  Google Scholar 

  12. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 1–27 (2011)

    Article  Google Scholar 

  13. Conn, A.R., Gould, N.I.M., Sartenaer, A., Toint, P.L.: Global convergence of a class of trust region algorithms for optimization using inexact projections on convex constraints. SIAM J. Opt. 3(1), 164–221 (1993). https://doi.org/10.1137/0803009

    Article  MathSciNet  MATH  Google Scholar 

  14. Conn, A.R., Gould, N.I.M., Sartenaer, A., Toint, P.L.: Convergence properties of minimization algorithms for convex constraints using a structured trust region. SIAM J. Opt. 6(4), 1059–1086 (1996). https://doi.org/10.1137/S1052623492236481

    Article  MathSciNet  MATH  Google Scholar 

  15. Conn, A.R., Gould, N.I.M., Toint, P.L.: A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J. Numer. Anal. 28(2), 545–572 (1991). https://doi.org/10.1137/0728030

    Article  MathSciNet  MATH  Google Scholar 

  16. Dai, Y.H., Fletcher, R.: New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds. Math. Program. 106(3), 403–421 (2006)

    Article  MathSciNet  Google Scholar 

  17. Deng, Y., Liu, Z., Wu, J., Wu, Y.: Topology optimization of steady Navier-Stokes flow with body force. Comput. Methods Appl. Mech. Eng. 255, 306–321 (2013)

    Article  MathSciNet  Google Scholar 

  18. Dua, D., Graff, C.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml

  19. Hartung, J.: An extension of Sion’s minimax theorem with an application to a method for constrained games. Pacific J. Math. 103(2), 401–408 (1982)

    Article  MathSciNet  Google Scholar 

  20. Heinkenschloss, M.: Numerical solution of implicitly constrained optimization problems. Rice University, Tech. rep. (2008)

  21. Kouri, D.P., von Winckel, G., Ridzal, D.: ROL: Rapid Optimization Library. https://trilinos.org/packages/rol (2017)

  22. Lazarov, B.S., Sigmund, O.: Filters in topology optimization based on Helmholtz-type differential equations. Int. J. Numer. Methods Eng. 86(6), 765–781 (2011)

    Article  MathSciNet  Google Scholar 

  23. Lin, C.J., Moré, J.J.: Newton’s method for large bound-constrained optimization problems. SIAM J. Opt. 9(4), 1100–1127 (1999)

    Article  MathSciNet  Google Scholar 

  24. Maciel, M.C., Mendonça, M.G., Verdiell, A.B.: Monotone and nonmonotone trust-region-based algorithms for large scale unconstrained optimization problems. Comput. Opt. Appl. 54(1), 27–43 (2013)

    Article  MathSciNet  Google Scholar 

  25. Meyer, M., Vlachos, P.: StatLib—datasets archive. http://lib.stat.cmu.edu/datasets/. Accessed: 2021-05-10

  26. Pace, R.K., Barry, R.: Sparse spatial autoregressions. Stat. Probab. Lett. 33(3), 291–297 (1997)

    Article  Google Scholar 

  27. Sala, M., Stanley, K., Heroux, M.: Amesos: A set of general interfaces to sparse direct solver libraries. In: Proceedings of PARA’06 Conference, Umea, Sweden (2006)

  28. Schenk, O., Gärtner, K.: Solving unsymmetric sparse systems of linear equations with PARDISO. Future Gener. Comput. Syst. 20(3), 475–487 (2004)

    Article  Google Scholar 

  29. Schmidt, M., Berg, E., Friedlander, M., Murphy, K.: Optimizing costly functions with simple constraints: A limited-memory projected quasi-Newton algorithm. In: D. van Dyk, M. Welling (eds.) Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, vol. 5, pp. 456–463. PMLR, Hilton Clearwater Beach Resort, Clearwater Beach, Florida USA (2009)

  30. Toint, P.L.: Global convergence of a clas of trust-region methods for nonconvex minimization in Hilbert space. IMA J. Numer. Anal. 8(2), 231–252 (1988). https://doi.org/10.1093/imanum/8.2.231

    Article  MathSciNet  MATH  Google Scholar 

  31. Tröltzsch, F.: Optimal control of partial differential equations: theory, methods, and applications. Graduate studies in mathematics. American Mathematical Society (2010)

  32. Vapnik, V.N.: The nature of statistical learning theory. Springer, New York (2013)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Optimization and Uncertainty Quantification, Sandia National Laboratories, P.O. Box 5800, MS-1320, Albuquerque, NM, 87185-1320, United States

    D. P. Kouri

Authors
  1. D. P. Kouri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to D. P. Kouri.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research was sponsored by the U.S. Air Force Office of Scientific Research, Optimization Program under Award NO: F4FGA09135G001.

Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kouri, D.P. A matrix-free trust-region newton algorithm for convex-constrained optimization. Optim Lett 16, 983–997 (2022). https://doi.org/10.1007/s11590-021-01794-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-021-01794-1

Keywords

Search

Navigation