We describe a matrix-free trust-region algorithm for solving convex-constrained optimization problems that uses the spectral projected gradient method to compute trial steps. To project onto the intersection of the feasible set and the trust region, we reformulate and solve the dual projection problem as a one-dimensional root finding problem. We demonstrate our algorithm’s performance on various problems from data science and PDE-constrained optimization. Our algorithm shows superior performance when compared with five existing trust-region and spectral projected gradient methods, and has the added benefit that it is simple to implement.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price includes VAT (USA)
Tax calculation will be finalised during checkout.
Andretta, M., Birgin, E.G., Martínez, J.M.: Practical active-set Euclidian trust-region method with spectral projected gradients for bound-constrained minimization. Optimization 54(3), 305–325 (2005)
Antil, H., Kouri, D.P., Lacasse, M.D., Ridzal, D.: Frontiers in PDE-constrained optimization, vol. 163. Springer, Cham (2018)
Baker, C.G., Heroux, M.A.: Tpetra, and the use of generic programming in scientific computing. Sci. Program. 20(2), 115–128 (2012)
Bendsoe, M.P., Sigmund, O.: Topology optimization: theory, methods, and applications. Springer, Cham (2013)
Birgin, E.G., Martínez, J.M., Raydan, M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Opt. 10(4), 1196–1211 (2000)
Birgin, E.G., Martínez, J.M., Raydan, M.: Inexact spectral projected gradient methods on convex sets. IMA J. Numer. Anal. 23(4), 539–559 (2003)
Birgin, E.G., Martínez, J.M., Raydan, M.: Spectral projected gradient methods: review and perspectives. J. Stat. Softw 60(3), 1–21 (2014)
Bochev, P., Edwards, H.C., Kirby, R.C., Peterson, K., Ridzal, D.: Solving PDEs with intrepid. Sci. Program. 20(2), 151–180 (2012)
Borrvall, T., Petersson, J.: Topology optimization of fluids in stokes flow. Int. J. Numer. Methods Fluids 41(1), 77–107 (2003)
Brent, R.P.: Algorithms for minimization without derivatives. Courier Corporation (2013)
Burke, J.V., Moré, J.J., Toraldo, G.: Convergence properties of trust region methods for linear and convex constraints. Math. Program. 47(1–3), 305–336 (1990)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 1–27 (2011)
Conn, A.R., Gould, N.I.M., Sartenaer, A., Toint, P.L.: Global convergence of a class of trust region algorithms for optimization using inexact projections on convex constraints. SIAM J. Opt. 3(1), 164–221 (1993). https://doi.org/10.1137/0803009
Conn, A.R., Gould, N.I.M., Sartenaer, A., Toint, P.L.: Convergence properties of minimization algorithms for convex constraints using a structured trust region. SIAM J. Opt. 6(4), 1059–1086 (1996). https://doi.org/10.1137/S1052623492236481
Conn, A.R., Gould, N.I.M., Toint, P.L.: A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J. Numer. Anal. 28(2), 545–572 (1991). https://doi.org/10.1137/0728030
Dai, Y.H., Fletcher, R.: New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds. Math. Program. 106(3), 403–421 (2006)
Deng, Y., Liu, Z., Wu, J., Wu, Y.: Topology optimization of steady Navier-Stokes flow with body force. Comput. Methods Appl. Mech. Eng. 255, 306–321 (2013)
Dua, D., Graff, C.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml
Hartung, J.: An extension of Sion’s minimax theorem with an application to a method for constrained games. Pacific J. Math. 103(2), 401–408 (1982)
Heinkenschloss, M.: Numerical solution of implicitly constrained optimization problems. Rice University, Tech. rep. (2008)
Kouri, D.P., von Winckel, G., Ridzal, D.: ROL: Rapid Optimization Library. https://trilinos.org/packages/rol (2017)
Lazarov, B.S., Sigmund, O.: Filters in topology optimization based on Helmholtz-type differential equations. Int. J. Numer. Methods Eng. 86(6), 765–781 (2011)
Lin, C.J., Moré, J.J.: Newton’s method for large bound-constrained optimization problems. SIAM J. Opt. 9(4), 1100–1127 (1999)
Maciel, M.C., Mendonça, M.G., Verdiell, A.B.: Monotone and nonmonotone trust-region-based algorithms for large scale unconstrained optimization problems. Comput. Opt. Appl. 54(1), 27–43 (2013)
Meyer, M., Vlachos, P.: StatLib—datasets archive. http://lib.stat.cmu.edu/datasets/. Accessed: 2021-05-10
Pace, R.K., Barry, R.: Sparse spatial autoregressions. Stat. Probab. Lett. 33(3), 291–297 (1997)
Sala, M., Stanley, K., Heroux, M.: Amesos: A set of general interfaces to sparse direct solver libraries. In: Proceedings of PARA’06 Conference, Umea, Sweden (2006)
Schenk, O., Gärtner, K.: Solving unsymmetric sparse systems of linear equations with PARDISO. Future Gener. Comput. Syst. 20(3), 475–487 (2004)
Schmidt, M., Berg, E., Friedlander, M., Murphy, K.: Optimizing costly functions with simple constraints: A limited-memory projected quasi-Newton algorithm. In: D. van Dyk, M. Welling (eds.) Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, vol. 5, pp. 456–463. PMLR, Hilton Clearwater Beach Resort, Clearwater Beach, Florida USA (2009)
Toint, P.L.: Global convergence of a clas of trust-region methods for nonconvex minimization in Hilbert space. IMA J. Numer. Anal. 8(2), 231–252 (1988). https://doi.org/10.1093/imanum/8.2.231
Tröltzsch, F.: Optimal control of partial differential equations: theory, methods, and applications. Graduate studies in mathematics. American Mathematical Society (2010)
Vapnik, V.N.: The nature of statistical learning theory. Springer, New York (2013)
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This research was sponsored by the U.S. Air Force Office of Scientific Research, Optimization Program under Award NO: F4FGA09135G001.
Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government.
About this article
Cite this article
Kouri, D.P. A matrix-free trust-region newton algorithm for convex-constrained optimization. Optim Lett (2021). https://doi.org/10.1007/s11590-021-01794-1
- Nonconvex optimization
- Convex constraints
- Trust regions
- Spectral projected gradient
- Large-scale optimization
- Newton’s method