Skip to main content
SpringerLink
Log in

On the Performance of SQP Methods for Nonlinear Optimization

  • Conference paper
Modeling and Optimization: Theory and Applications

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 147))

Abstract

This paper concerns some practical issues associated with the formulation of sequential quadratic programming (SQP) methods for large-scale nonlinear optimization. SQP methods find approximate solutions of a sequence of quadratic programming (QP) subproblems in which a quadratic model of the Lagrangian is minimized subject to the linearized constraints. Numerical results are given for 1153 problems from the CUTEst test collection. The results indicate that SQP methods based on maintaining a quasi-Newton approximation to the Hessian of the Lagrangian function are both reliable and efficient for general large-scale optimization problems. In particular, the results show that in some situations, quasi-Newton SQP methods are more efficient than interior methods that utilize the exact Hessian of the Lagrangian. The paper concludes with discussion of an SQP method that employs both approximate and exact Hessian information. In this approach the quadratic programming subproblem is either the conventional subproblem defined in terms of a positive-definite quasi-Newton approximate Hessian or a convexified subproblem based on the exact Hessian.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Here, the value of n df is taken as the number of degrees of freedom at a solution found by SNOPT7.

  2. 2.

    i.e., \(c_{j}^{}(x^{{\ast}})y_{j}^{{\ast}} = 0\) and \(c_{j}^{}(x^{{\ast}}) + y_{j}^{{\ast}}> 0\) at the optimal primal-dual pair \((x^{{\ast}},y^{{\ast}})\).

References

  1. Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16, 1190–1208 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  2. Byrd, R.H., Gould, N.I.M., Nocedal, J., Waltz, R.A.: An algorithm for nonlinear optimization using linear programming and equality constrained subproblems. Math. Program. 100(1, Ser. B), 27–48 (2004)

    Google Scholar 

  3. Byrd, R.H., Gould, N.I.M., Nocedal, J., Waltz, R.A.: On the convergence of successive linear-quadratic programming algorithms. SIAM J. Optim. 16(2), 471–489 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  4. Byrd, R., Nocedal, J., Waltz, R., Wu, Y.: On the use of piecewise linear models in nonlinear programming. Math. Program. 137, 289–324 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  5. Contesse, L.B.: Une caractérisation complète des minima locaux en programmation quadratique. Numer. Math. 34, 315–332 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  6. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with COPS. Technical Memorandum ANL/MCS-TM-246, Argonne National Laboratory, Argonne (2000)

    Google Scholar 

  7. Fletcher, R.: An ℓ 1 penalty method for nonlinear constraints. In: Boggs, P.T., Byrd, R.H., Schnabel, R.B. (eds.) Numerical Optimization 1984, pp. 26–40. SIAM, Philadelphia (1985)

    Google Scholar 

  8. Fletcher, R., Leyffer, S.: User manual for filterSQP. Tech. Rep. NA/181, Dept. of Mathematics, University of Dundee, Scotland (1998)

    Google Scholar 

  9. Fletcher, R., Sainz de la Maza, E.: Nonlinear programming and nonsmooth optimization by successive linear programming. Math. Program. 43, 235–256 (1989)

    Google Scholar 

  10. Forsgren, A.: Inertia-controlling factorizations for optimization algorithms. Appl. Num. Math. 43, 91–107 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  11. Forsgren, A., Gill, P.E.: Primal-dual interior methods for nonconvex nonlinear programming. SIAM J. Optim. 8, 1132–1152 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  12. Forsgren, A., Murray, W.: Newton methods for large-scale linear equality-constrained minimization. SIAM J. Matrix Anal. Appl. 14, 560–587 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  13. Forsgren, A., Gill, P.E., Murray, W.: On the identification of local minimizers in inertia-controlling methods for quadratic programming. SIAM J. Matrix Anal. Appl. 12, 730–746 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  14. Gill, P.E., Leonard, M.W.: Limited-memory reduced-Hessian methods for large-scale unconstrained optimization. SIAM J. Optim. 14, 380–401 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  15. Gill, P.E., Murray, W.: Newton-type methods for unconstrained and linearly constrained optimization. Math. Program. 7, 311–350 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  16. Gill, P.E., Robinson, D.P.: A globally convergent stabilized SQP method. SIAM J. Optim. 23(4), 1983–2010 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  17. Gill, P.E., Wong, E.: Sequential quadratic programming methods. In: Lee, J., Leyffer, S. (eds.) Mixed Integer Nonlinear Programming. The IMA Volumes in Mathematics and its Applications, vol. 154, pp. 147–224. Springer New York (2012). http://dx.doi.org/10.1007/978-1-4614-1927-3_6. 10.1007/978-1-4614-1927-3_6

  18. Gill, P.E., Wong, E.: User’s guide for SQIC: software for large-scale quadratic programming. Center for Computational Mathematics Report CCoM 14-02, Center for Computational Mathematics, University of California, San Diego, La Jolla (2014)

    Google Scholar 

  19. Gill, P.E., Wong, E.: Methods for convex and general quadratic programming. Math. Program. Comput. 7, 71–112 (2015). doi:10.1007/s12532-014-0075-x. http://dx.doi.org/10.1007/s12532-014-0075-x

    Google Scholar 

  20. Gill, P.E., Murray, W., Saunders, M.A., Wright, M.H.: User’s guide for NPSOL (Version 4.0): a Fortran package for nonlinear programming. Report SOL 86-2, Department of Operations Research, Stanford University, Stanford (1986)

    Google Scholar 

  21. Gill, P.E., Murray, W., Saunders, M.A., Wright, M.H.: Maintaining LU factors of a general sparse matrix. Linear Algebra Appl. 88/89, 239–270 (1987). doi:10.1016/0024-3795(87)90112-1. http://dx.doi.org/10.1016/0024-3795(87)90112-1

  22. Gill, P.E., Murray, W., Saunders, M.A., Wright, M.H.: Some theoretical properties of an augmented Lagrangian merit function. In: Pardalos, P.M. (ed.) Advances in Optimization and Parallel Computing, pp. 101–128. North Holland, Amsterdam (1992)

    Google Scholar 

  23. Gill, P.E., Murray, W., Saunders, M.A.: SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM Rev. 47, 99–131 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  24. Gill, P.E., Murray, W., Saunders, M.A.: User’s guide for SQOPT Version 7: software for large-scale linear and quadratic programming. Numerical Analysis Report 06-1, Department of Mathematics, University of California, San Diego, La Jolla (2006)

    Google Scholar 

  25. Gill, P.E., Saunders, M.A., Wong, E.: User’s Guide for DNOPT: a Fortran package for medium-scale nonlinear programming. Center for Computational Mathematics Report CCoM 14-05, Department of Mathematics, University of California, San Diego, La Jolla (2014)

    Google Scholar 

  26. Gould, N.I.M.: On modified factorizations for large-scale linearly constrained optimization. SIAM J. Optim. 9, 1041–1063 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  27. Gould, N.I.M., Robinson, D.P.: A second derivative SQP method with imposed descent. Numerical Analysis Report 08/09, Computational Laboratory, University of Oxford, Oxford (2008)

    Google Scholar 

  28. Gould, N.I.M., Robinson, D.P.: A second derivative SQP method: global convergence. SIAM J. Optim. 20(4), 2023–2048 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  29. Gould, N.I.M., Robinson, D.P.: A second derivative SQP method: local convergence and practical issues. SIAM J. Optim. 20(4), 2049–2079 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  30. Gould, N.I.M., Orban, D., Toint, P.L.: CUTEst: a constrained and unconstrained testing environment with safe threads. Technical report, Rutherford Appleton Laboratory, Chilton (2013). doi:10.1007/s10589-014-9687-3. http://dx.doi.org/10.1007/s10589-014-9687-3

  31. Greenstadt, J.: On the relative efficiencies of gradient methods. Math. Comput. 21, 360–367 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  32. Grippo, L., Lampariello, F., Lucidi, S.: Newton-type algorithms with nonmonotone line search for large-scale unconstrained optimization. In: System modelling and optimization (Tokyo, 1987). Lecture Notes in Control and Inform. Sci., vol. 113, pp. 187–196. Springer, Berlin (1988). doi:10.1007/BFb0042786. http://dx.doi.org/10.1007/BFb0042786

  33. Grippo, L., Lampariello, F., Lucidi, S.: A truncated Newton method with nonmonotone line search for unconstrained optimization. J. Optim. Theory Appl. 60(3), 401–419 (1989). doi:10.1007/BF00940345. http://dx.doi.org/10.1007/BF00940345

  34. Grippo, L., Lampariello, F., Lucidi, S.: A class of nonmonotone stabilization methods in unconstrained optimization. Numer. Math. 59(8), 779–805 (1991). doi:10.1007/BF01385810. http://dx.doi.org/10.1007/BF01385810

  35. Han, S.P.: A globally convergent method for nonlinear programming. J. Optim. Theory Appl. 22, 297–309 (1977)

    Article  MATH  Google Scholar 

  36. Kungurtsev, V.: Second-derivative sequential quadratic programming methods for nonlinear optimization. Ph.D. thesis, Department of Mathematics, University of California San Diego, La Jolla (2013)

    Google Scholar 

  37. Morales, J.L., Nocedal, J., Wu, Y.: A sequential quadratic programming algorithm with an additional equality constrained phase. IMA J. Numer. Anal. 32, 553–579 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  38. Paige, C.C., Saunders, M.A.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12, 617–629 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  39. Powell, M.J.D.: A fast algorithm for nonlinearly constrained optimization calculations. In: Watson, G.A. (ed.) Numerical Analysis, Dundee 1977, no. 630 in Lecture Notes in Mathematics, pp. 144–157. Springer, Heidelberg, Berlin, New York (1978)

    Google Scholar 

  40. Schittkowski, K.: The nonlinear programming method of Wilson, Han, and Powell with an augmented Lagrangian type line search function. I. Convergence analysis. Numer. Math. 38(1), 83–114 (1981/1982). doi:10.1007/BF01395810. http://dx.doi.org/10.1007/BF01395810

  41. Schittkowski, K.: NLPQL: a Fortran subroutine for solving constrained nonlinear programming problems. Ann. Oper. Res. 11, 485–500 (1985/1986)

    Google Scholar 

  42. Schnabel, R.B., Eskow, E.: A new modified Cholesky factorization. SIAM J. Sci. Stat. Comput. 11, 1136–1158 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  43. Spellucci, P.: An SQP method for general nonlinear programs using only equality constrained subproblems. Math. Program. 82, 413–448 (1998)

    MathSciNet  MATH  Google Scholar 

  44. Toint, P.L.: An assessment of nonmonotone linesearch techniques for unconstrained optimization. SIAM J. Sci. Comput. 17(3), 725–739 (1996). doi:10.1137/S106482759427021X. http://dx.doi.org/10.1137/S106482759427021X

  45. Wächter, A., Biegler, L.T.: Line search filter methods for nonlinear programming: local convergence. SIAM J. Optim. 16(1), 32–48 (electronic) (2005). doi:10.1137/S1052623403426544. http://dx.doi.org/10.1137/S1052623403426544

  46. Wächter, A., Biegler, L.T.: Line search filter methods for nonlinear programming: motivation and global convergence. SIAM J. Optim. 16(1), 1–31 (electronic) (2005). doi:10.1137/ S1052623403426556. http://dx.doi.org/10.1137/S1052623403426556

  47. Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1, Ser. A), 25–57 (2006)

    Google Scholar 

  48. Wächter, A., Biegler, L.T., Lang, Y.D., Raghunathan, A.: IPOPT: an interior point algorithm for large-scale nonlinear optimization (2002). https://projects.coin-or.org/Ipopt

  49. Zhang, H., Hager, W.W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 14(4), 1043–1056 (electronic) (2004). doi:10.1137/ S1052623403428208. http://dx.doi.org/10.1137/S1052623403428208

Download references

Acknowledgements

We would like to thank Nick Gould for providing the latest version of the CUTEst test collection. We are also grateful to the referees for constructive comments that resulted in significant improvements in the final manuscript.

The research of the author “Philip E. Gill” was supported in part by National Science Foundation grants DMS-1318480 and DMS-1361421. The research of the author “Michael A. Saunders” was supported in part by the National Institute of General Medical Sciences of the National Institutes of Health [award U01GM102098]. The research of the author “Elizabeth Wong” was supported in part by Northrop Grumman Aerospace Systems. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies.

Author information

Authors and Affiliations

  1. Department of Mathematics, UC San Diego, La Jolla, CA, 92093-0112, USA

    Philip E. Gill & Elizabeth Wong

  2. Systems Optimization Laboratory, Department of Management Science and Engineering, Stanford University, Stanford, CA, 94305-4121, USA

    Michael A. Saunders

Authors
  1. Philip E. Gill
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael A. Saunders
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elizabeth Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philip E. Gill .

Editor information

Editors and Affiliations

  1. Dept. of Industrial & Systems Eng., Lehigh University, Bethlehem, Pennsylvania, USA

    Boris Defourny

  2. Dept. of Industrial & Systems Eng., Lehigh University, Bethlehem, Pennsylvania, USA

    Tamás Terlaky

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Gill, P.E., Saunders, M.A., Wong, E. (2015). On the Performance of SQP Methods for Nonlinear Optimization. In: Defourny, B., Terlaky, T. (eds) Modeling and Optimization: Theory and Applications. Springer Proceedings in Mathematics & Statistics, vol 147. Springer, Cham. https://doi.org/10.1007/978-3-319-23699-5_5

Download citation

Publish with us

Policies and ethics

Search

Navigation