Skip to main content

A stabilized SQP method: superlinear convergence

Abstract

Stabilized sequential quadratic programming (sSQP) methods for nonlinear optimization generate a sequence of iterates with fast local convergence regardless of whether or not the active-constraint gradients are linearly dependent. This paper concerns the local convergence analysis of an sSQP method that uses a line search with a primal-dual augmented Lagrangian merit function to enforce global convergence. The method is provably well-defined and is based on solving a strictly convex quadratic programming subproblem at each iteration. It is shown that the method has superlinear local convergence under assumptions that are no stronger than those required by conventional stabilized SQP methods. The fast local convergence is obtained by allowing a small relaxation of the optimality conditions for the quadratic programming subproblem in the neighborhood of a solution. In the limit, the line search selects the unit step length, which implies that the method does not suffer from the Maratos effect. The analysis indicates that the method has the same strong first- and second-order global convergence properties that have been established for augmented Lagrangian methods, yet is able to transition seamlessly to sSQP with fast local convergence in the neighborhood of a solution. Numerical results on some degenerate problems are reported.

This is a preview of subscription content, access via your institution.

Notes

  1. 1.

    Andreani et al. [1, Definition 3.1].

  2. 2.

    Andreani et al. [2, page 532].

References

  1. 1.

    Andreani, R., Haeser, G., Schuverdt, M.L., Silva, P.J.S.: Two new weak constraint qualifications and applications. SIAM J. Optim. 22(3), 1109–1135 (2012)

    MathSciNet  Article  MATH  Google Scholar 

  2. 2.

    Andreani, R., Martínez, J.M., Schuverdt, M.L.: On second-order optimality conditions for nonlinear programming. Optimization 56, 529–542 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    Baumrucker, B.T., Renfro, J.G., Biegler, L.T.: MPEC problem formulations in chemical engineering applications. Comput. Chem. Eng. 32, 2903–2913 (2008)

  4. 4.

    Curtis, F.E., Nocedal, J.: Flexible penalty functions for nonlinear constrained optimization. IMA J. Numer. Anal. 28(4), 749–769 (2008)

    MathSciNet  Article  MATH  Google Scholar 

  5. 5.

    Dirkse, S.: MPECLib: a collection of mathematical programs with equilibrium constraints. http://www.gamsworld.eu/mpec/mpeclib.htm (2004)

  6. 6.

    Facchinei, F., Fischer, A., Herrich, M.: A family of Newton methods for nonsmooth constrained systems with nonisolated solutions. Math. Methods Oper. Res. 77(3), 433–443 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  7. 7.

    Facchinei, F., Fischer, A., Herrich, M.: An LP-Newton method: nonsmooth equations, KKT systems, and nonisolated solutions. Math. Program. 146(1), 1–36 (2014). doi:10.1007/s10107-013-0676-6

  8. 8.

    Facchinei, F., Fischer, A., Kanzow, C.: On the accurate identification of active constraints. SIAM J. Optim. 9(1), 14–32 (1998)

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Fernández, D., Solodov, M.: Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems. Math. Program. Ser. A 125, 47–73 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  10. 10.

    Fischer, A.: Modified Wilson’s method for nonlinear programs with nonunique multipliers. Math. Oper. Res. 24(3), 699–727 (1999)

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    Forsgren, A.: Inertia-controlling factorizations for optimization algorithms. Appl. Numer. Math. 43, 91–107 (2002)

    MathSciNet  Article  MATH  Google Scholar 

  12. 12.

    Forsgren, A., Gill, P.E.: Primal-dual interior methods for nonconvex nonlinear programming. SIAM J. Optim. 8, 1132–1152 (1998)

    MathSciNet  Article  MATH  Google Scholar 

  13. 13.

    Gill, P.E., Kungurtsev, V., Robinson, D.P.: Distance-to-solution estimates for optimization problems with constraints in standard form. Center for Computational Mathematics Report CCoM 16-01, University of California, San Diego (2016)

  14. 14.

    Gill, P.E., Kungurtsev, V., Robinson, D.P.: A stabilized SQP method: global convergence. IMA J. Numer. Anal. doi:10.1093/imanum/drw004. http://imajna.oxfordjournals.org/content/early/2016/05/04/imanum.drw004.abstract

  15. 15.

    Gill, P.E., Robinson, D.P.: A primal-dual augmented Lagrangian. Comput. Optim. Appl. 51, 1–25 (2012)

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    Gill, P.E., Robinson, D.P.: A globally convergent stabilized SQP method. SIAM J. Optim. 23(4), 1983–2010 (2013)

    MathSciNet  Article  MATH  Google Scholar 

  17. 17.

    Gill, P.E., Wong, E.: Sequential quadratic programming methods. In: Lee, J., Leyffer, S. (eds.) Mixed Integer Nonlinear Programming, Volume 154 of The IMA Volumes in Mathematics and its Applications, pp. 147–224. Springer, New York (2012). doi:10.1007/978-1-4614-1927-3_6

    Chapter  Google Scholar 

  18. 18.

    Gould, N.I.M., Orban, D., Toint, PhL: CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60(3), 545–557 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  19. 19.

    Hager, W.W.: Stabilized sequential quadratic programming. Comput. Optim. Appl. 12(1–3):253–273 (1999). Computational optimization—a tribute to Olvi Mangasarian, Part I

  20. 20.

    Izmailov, A.F.: Solution sensitivity for Karush–Kuhn–Tucker systems with non-unique Lagrange multipliers. Optimization 59(5), 747–775 (2010). doi:10.1080/02331930802434922

  21. 21.

    Izmailov, A.F., Solodov, M.V.: Newton-type methods for optimization problems without constraint qualifications. SIAM J. Optim. 15(1), 210–228 (2005)

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    Izmailov, A.F., Solodov, M.V.: Examples of dual behaviour of Newton-type methods on optimization problems with degenerate constraints. Comput. Optim. Appl. 42, 231–264 (2009). doi:10.1007/s10589-007-9074-4

    MathSciNet  Article  MATH  Google Scholar 

  23. 23.

    Izmailov, A.F., Solodov, M.V.: On attraction of linearly constrained Lagrangian methods and of stabilized and quasi-Newton SQP methods to critical multipliers. Math. Program. 126(2, Ser. A), 231–257 (2011)

    MathSciNet  Article  MATH  Google Scholar 

  24. 24.

    Izmailov, A.F., Solodov, M.V.: Stabilized SQP revisited. Math. Program. 133, 93–120 (2012)

    MathSciNet  Article  MATH  Google Scholar 

  25. 25.

    Izmailov, A.F., Solodov, M.V.: Critical Lagrange multipliers: what we currently know about them, how they spoil our lives, and what we can do about it. TOP 23(1), 1–26 (2015)

    MathSciNet  Article  MATH  Google Scholar 

  26. 26.

    Izmailov, A.F., Uskov, E.I.: On the influence of the critical Lagrange multipliers on the convergence rate of the multiplier method. Comput. Math. Math. Phys. 52(11), 1504–1519 (2012)

    MathSciNet  Article  Google Scholar 

  27. 27.

    Kungurtsev, V.: Second-derivative sequential quadratic programming methods for nonlinear optimization. Ph.D. thesis, Department of Mathematics, University of California San Diego, La Jolla, CA (2013)

  28. 28.

    Maratos, N.: Exact penalty function algorithms for finite-dimensional and control optimization problems. Ph.D. thesis, Department of Computing and Control, University of London (1978)

  29. 29.

    MathWorks Inc., Natick, Massachusetts. Matlab User’s Guide (1992)

  30. 30.

    Mostafa, E.-S., Vicente, L., Wright, S.: Numerical behavior of a stabilized SQP method for degenerate NLP problems. In: Bliek, C., Jermann, C., Neumaier, A. (eds.) Global Optimization and Constraint Satisfaction. Lecture Notes in Computer Science, vol. 2861, pp. 123–141. Springer, Berlin (2003)

  31. 31.

    Oberlin, C., Wright, S.J.: Active set identification in nonlinear programming. SIAM J. Optim. 17(2), 577–605 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  32. 32.

    Wright, S.J.: Superlinear convergence of a stabilized SQP method to a degenerate solution. Comput. Optim. Appl. 11(3), 253–275 (1998)

    MathSciNet  Article  MATH  Google Scholar 

  33. 33.

    Wright, S.J.: Modifying SQP for degenerate problems. SIAM J. Optim. 13(2), 470–497 (2002)

    MathSciNet  Article  MATH  Google Scholar 

  34. 34.

    Wright, S.J.: Constraint identification and algorithm stabilization for degenerate nonlinear programs. Math. Program. 95(1, Ser. B), 137–160 (2003). ISMP 2000, Part 3 (Atlanta, GA)

  35. 35.

    Wright, S.J.: An algorithm for degenerate nonlinear programming with rapid local convergence. SIAM J. Optim. 15(3), 673–696 (2005)

    MathSciNet  Article  MATH  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the referees for a number of suggestions that significantly improved the presentation.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Daniel P. Robinson.

Additional information

P. E. Gill: Research supported by the National Science Foundation Grants DMS-1318480 and DMS-1361421.

Vyacheslav Kungurtsev: Research supported by the European social fund within the framework of realizing the project “Support of inter-sectoral mobility and quality enhancement of research teams at the Czech Technical University in Prague”, CZ.1.07/2.3.00/30.0034.

Daniel P. Robinson: Research supported by National Science Foundation Grant DMS-1217153.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gill, P.E., Kungurtsev, V. & Robinson, D.P. A stabilized SQP method: superlinear convergence. Math. Program. 163, 369–410 (2017). https://doi.org/10.1007/s10107-016-1066-7

Download citation

Keywords

  • Nonlinear programming
  • Augmented Lagrangian
  • Sequential quadratic programming
  • SQP methods
  • Stabilized SQP
  • Primal-dual methods
  • Second-order optimality

Mathematics Subject Classification

  • 49J20
  • 49J15
  • 49M37
  • 49D37
  • 65F05
  • 65K05
  • 90C30