Computational Optimization and Applications

, Volume 74, Issue 3, pp 669–701 | Cite as

Large-scale quasi-Newton trust-region methods with low-dimensional linear equality constraints

  • Johannes J. BrustEmail author
  • Roummel F. Marcia
  • Cosmin G. Petra


We propose two limited-memory BFGS (L-BFGS) trust-region methods for large-scale optimization with linear equality constraints. The methods are intended for problems where the number of equality constraints is small. By exploiting the structure of the quasi-Newton compact representation, both proposed methods solve the trust-region subproblems nearly exactly, even for large problems. We derive theoretical global convergence results of the proposed algorithms, and compare their numerical effectiveness and performance on a variety of large-scale problems.


Linear equality constraints Quasi-Newton L-BFGS Trust-region algorithm Compact representation Eigendecomposition Shape-changing norm 



  1. 1.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2007)zbMATHGoogle Scholar
  2. 2.
    Brust, J.J., Burdakov, O., Erway, J.B., Marcia, R.F., Yuan, Y.X.: Shape-changing L-SR1 trust-region methods. Technical Report 2016-2, Department of Mathematics, Wake Forest University (2016)Google Scholar
  3. 3.
    Brust, J.J., Burdakov, O.P., Erway, J.B., Marcia, R.F.: Dense initializations for limited-memory quasi-Newton methods. Comput. Optim. Appl. 74(1), 121–142 (2019). MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Brust, J.J., Erway, J.B., Marcia, R.F.: On solving L-SR1 trust-region subproblems. Comput. Optim. Appl. 66(2), 245–266 (2017)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Burdakov, O., Gong, L., Yuan, Y.X., Zikrin, S.: On efficiently combining limited memory and trust-region techniques. Math. Program. Comput. 9, 101–134 (2016)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Burdakov, O., Martinez, J., Pilotta, E.: A limited-memory multipoint symmetric secant method for bound constrained optimization. Ann. Oper. Res. 117, 51–70 (2002)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Burke, J.V., Wiegmann, A., Xu, L.: Limited memory BFGS updating in a trust-region framework. Technical Report, University of Washington (1996)Google Scholar
  8. 8.
    Byrd, R.H., Gilbert, J.C., Nocedal, J.: A trust region method based on interior point techniques for nonlinear programming. Math. Program. Ser. A 89, 149–185 (2000)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Byrd, R.H., Hribar, M., Nocedal, J.: An interior point algorithm for large-scale nonlinear programming. SIAM J. Optim. 9, 877–900 (1999)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Byrd, R.H., Nocedal, J., Schnabel, R.B.: Representations of quasi-Newton matrices and their use in limited-memory methods. Math. Program. 63, 129–156 (1994)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Celis, M., Dennis Jr., J., Tapia, R.: A trust region strategy for equality constrained optimization. Technical Report 84-1, Mathematical Sciences Department, Rice University (1984)Google Scholar
  12. 12.
    Coleman, T., Branch, M.A., Grace, A.: Optimization Toolbox for Use with MATLAB. MathWorks, Natick (1999)Google Scholar
  13. 13.
    Coleman, T., Verma, A.: A preconditioned conjugate gradient approach to linear equality constrained minimization. Comput. Optim. Appl. 20, 61–72 (2001)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust-Region Methods. SIAM, Philadelphia (2000)CrossRefGoogle Scholar
  15. 15.
    DeGuchy, O., Erway, J.B., Marcia, R.F.: Compact representation of the full Broyden class of quasi-Newton updates. Numer Linear Algebra Appl 25(5), e2186 (2018)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Dolan, E., Moré, J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Erway, J.B., Marcia, R.F.: Algorithm 943: MSS: MATLAB software for L-BFGS trust-region subproblems for large-scale optimization. ACM Trans. Math. Softw. 40(4), 28:1–28:12 (2014). MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Hager, W.W.: Updating the inverse of a matrix. SIAM Rev. 31(2), 221–239 (1989)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Lalee, M., Nocedal, J., Plantenga, T.: On the implementation of an algorithm for large-scale equality constrained optimization. SIAM J. Optim. 8(3), 682–706 (1998)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Moré, J.J., Sorensen, D.C.: Computing a trust region step. SIAM J. Sci. Stat. Comput. 4, 553–572 (1983)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)zbMATHGoogle Scholar
  22. 22.
    Powell, M., Yuan, Y.: A trust region algorithm for equality constrained optimization. Math. Program. 49, 189–211 (1991)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Saunders, M.A.: PDCO: Primal-dual interior method for convex objectives (2002–2015). Accessed 21 June 2018
  24. 24.
    Steihaug, T.: The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal. 20, 626–637 (1983)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Vardi, A.: A trust region algorithm for equality constrained minimization: convergence properties and implementation. SIAM J. Numer. Anal. 22(3), 575–591 (1985)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106, 25–57 (2006)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Waltz, R., Morales, J., Nocedal, J., Orban, D.: An interior algorithm for nonlinear optimization that combines line search and trust region steps. SIAM. J. Optim. 9, 877–900 (1999)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Yuan, Y.X.: Trust region algorithms for constrained optimization. Technical report, State Key Laboratory of Scientific and Engineering Computing, BeijingGoogle Scholar
  29. 29.
    Zhijiang, S.: RSQP toolbox for MATLAB (2006). Accessed 21 June 2018

Copyright information

© This is a U.S. Government work and not under copyright protection in the US; foreign copyright protection may apply  2019

Authors and Affiliations

  1. 1.Argonne National LaboratoryLemontUSA
  2. 2.University of California MercedMercedUSA
  3. 3.Lawrence Livermore National LaboratoryLivermoreUSA

Personalised recommendations