Mathematical Programming

, Volume 163, Issue 1–2, pp 369–410 | Cite as

A stabilized SQP method: superlinear convergence

  • Philip E. Gill
  • Vyacheslav Kungurtsev
  • Daniel P. Robinson
Full Length Paper Series A


Stabilized sequential quadratic programming (sSQP) methods for nonlinear optimization generate a sequence of iterates with fast local convergence regardless of whether or not the active-constraint gradients are linearly dependent. This paper concerns the local convergence analysis of an sSQP method that uses a line search with a primal-dual augmented Lagrangian merit function to enforce global convergence. The method is provably well-defined and is based on solving a strictly convex quadratic programming subproblem at each iteration. It is shown that the method has superlinear local convergence under assumptions that are no stronger than those required by conventional stabilized SQP methods. The fast local convergence is obtained by allowing a small relaxation of the optimality conditions for the quadratic programming subproblem in the neighborhood of a solution. In the limit, the line search selects the unit step length, which implies that the method does not suffer from the Maratos effect. The analysis indicates that the method has the same strong first- and second-order global convergence properties that have been established for augmented Lagrangian methods, yet is able to transition seamlessly to sSQP with fast local convergence in the neighborhood of a solution. Numerical results on some degenerate problems are reported.


Nonlinear programming Augmented Lagrangian Sequential quadratic programming SQP methods Stabilized SQP Primal-dual methods Second-order optimality 

Mathematics Subject Classification

49J20 49J15 49M37 49D37 65F05 65K05 90C30 



The authors would like to thank the referees for a number of suggestions that significantly improved the presentation.


  1. 1.
    Andreani, R., Haeser, G., Schuverdt, M.L., Silva, P.J.S.: Two new weak constraint qualifications and applications. SIAM J. Optim. 22(3), 1109–1135 (2012)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Andreani, R., Martínez, J.M., Schuverdt, M.L.: On second-order optimality conditions for nonlinear programming. Optimization 56, 529–542 (2007)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Baumrucker, B.T., Renfro, J.G., Biegler, L.T.: MPEC problem formulations in chemical engineering applications. Comput. Chem. Eng. 32, 2903–2913 (2008)Google Scholar
  4. 4.
    Curtis, F.E., Nocedal, J.: Flexible penalty functions for nonlinear constrained optimization. IMA J. Numer. Anal. 28(4), 749–769 (2008)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Dirkse, S.: MPECLib: a collection of mathematical programs with equilibrium constraints. (2004)
  6. 6.
    Facchinei, F., Fischer, A., Herrich, M.: A family of Newton methods for nonsmooth constrained systems with nonisolated solutions. Math. Methods Oper. Res. 77(3), 433–443 (2013)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Facchinei, F., Fischer, A., Herrich, M.: An LP-Newton method: nonsmooth equations, KKT systems, and nonisolated solutions. Math. Program. 146(1), 1–36 (2014). doi: 10.1007/s10107-013-0676-6
  8. 8.
    Facchinei, F., Fischer, A., Kanzow, C.: On the accurate identification of active constraints. SIAM J. Optim. 9(1), 14–32 (1998)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Fernández, D., Solodov, M.: Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems. Math. Program. Ser. A 125, 47–73 (2010)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Fischer, A.: Modified Wilson’s method for nonlinear programs with nonunique multipliers. Math. Oper. Res. 24(3), 699–727 (1999)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Forsgren, A.: Inertia-controlling factorizations for optimization algorithms. Appl. Numer. Math. 43, 91–107 (2002)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Forsgren, A., Gill, P.E.: Primal-dual interior methods for nonconvex nonlinear programming. SIAM J. Optim. 8, 1132–1152 (1998)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Gill, P.E., Kungurtsev, V., Robinson, D.P.: Distance-to-solution estimates for optimization problems with constraints in standard form. Center for Computational Mathematics Report CCoM 16-01, University of California, San Diego (2016)Google Scholar
  14. 14.
    Gill, P.E., Kungurtsev, V., Robinson, D.P.: A stabilized SQP method: global convergence. IMA J. Numer. Anal. doi: 10.1093/imanum/drw004.
  15. 15.
    Gill, P.E., Robinson, D.P.: A primal-dual augmented Lagrangian. Comput. Optim. Appl. 51, 1–25 (2012)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Gill, P.E., Robinson, D.P.: A globally convergent stabilized SQP method. SIAM J. Optim. 23(4), 1983–2010 (2013)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Gill, P.E., Wong, E.: Sequential quadratic programming methods. In: Lee, J., Leyffer, S. (eds.) Mixed Integer Nonlinear Programming, Volume 154 of The IMA Volumes in Mathematics and its Applications, pp. 147–224. Springer, New York (2012). doi: 10.1007/978-1-4614-1927-3_6 CrossRefGoogle Scholar
  18. 18.
    Gould, N.I.M., Orban, D., Toint, PhL: CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60(3), 545–557 (2015)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Hager, W.W.: Stabilized sequential quadratic programming. Comput. Optim. Appl. 12(1–3):253–273 (1999). Computational optimization—a tribute to Olvi Mangasarian, Part IGoogle Scholar
  20. 20.
    Izmailov, A.F.: Solution sensitivity for Karush–Kuhn–Tucker systems with non-unique Lagrange multipliers. Optimization 59(5), 747–775 (2010). doi: 10.1080/02331930802434922
  21. 21.
    Izmailov, A.F., Solodov, M.V.: Newton-type methods for optimization problems without constraint qualifications. SIAM J. Optim. 15(1), 210–228 (2005)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Izmailov, A.F., Solodov, M.V.: Examples of dual behaviour of Newton-type methods on optimization problems with degenerate constraints. Comput. Optim. Appl. 42, 231–264 (2009). doi: 10.1007/s10589-007-9074-4 MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Izmailov, A.F., Solodov, M.V.: On attraction of linearly constrained Lagrangian methods and of stabilized and quasi-Newton SQP methods to critical multipliers. Math. Program. 126(2, Ser. A), 231–257 (2011)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Izmailov, A.F., Solodov, M.V.: Stabilized SQP revisited. Math. Program. 133, 93–120 (2012)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Izmailov, A.F., Solodov, M.V.: Critical Lagrange multipliers: what we currently know about them, how they spoil our lives, and what we can do about it. TOP 23(1), 1–26 (2015)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Izmailov, A.F., Uskov, E.I.: On the influence of the critical Lagrange multipliers on the convergence rate of the multiplier method. Comput. Math. Math. Phys. 52(11), 1504–1519 (2012)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Kungurtsev, V.: Second-derivative sequential quadratic programming methods for nonlinear optimization. Ph.D. thesis, Department of Mathematics, University of California San Diego, La Jolla, CA (2013)Google Scholar
  28. 28.
    Maratos, N.: Exact penalty function algorithms for finite-dimensional and control optimization problems. Ph.D. thesis, Department of Computing and Control, University of London (1978)Google Scholar
  29. 29.
    MathWorks Inc., Natick, Massachusetts. Matlab User’s Guide (1992)Google Scholar
  30. 30.
    Mostafa, E.-S., Vicente, L., Wright, S.: Numerical behavior of a stabilized SQP method for degenerate NLP problems. In: Bliek, C., Jermann, C., Neumaier, A. (eds.) Global Optimization and Constraint Satisfaction. Lecture Notes in Computer Science, vol. 2861, pp. 123–141. Springer, Berlin (2003)Google Scholar
  31. 31.
    Oberlin, C., Wright, S.J.: Active set identification in nonlinear programming. SIAM J. Optim. 17(2), 577–605 (2006)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Wright, S.J.: Superlinear convergence of a stabilized SQP method to a degenerate solution. Comput. Optim. Appl. 11(3), 253–275 (1998)MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Wright, S.J.: Modifying SQP for degenerate problems. SIAM J. Optim. 13(2), 470–497 (2002)MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Wright, S.J.: Constraint identification and algorithm stabilization for degenerate nonlinear programs. Math. Program. 95(1, Ser. B), 137–160 (2003). ISMP 2000, Part 3 (Atlanta, GA)Google Scholar
  35. 35.
    Wright, S.J.: An algorithm for degenerate nonlinear programming with rapid local convergence. SIAM J. Optim. 15(3), 673–696 (2005)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2016

Authors and Affiliations

  1. 1.Department of MathematicsUniversity of California, San DiegoLa JollaUSA
  2. 2.Agent Technology Center, Department of Computer Science, Faculty of Electrical EngineeringCzech Technical University in PraguePragueCzech Republic
  3. 3.Department of Applied Mathematics and StatisticsJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations