Computational Optimization and Applications

, Volume 59, Issue 3, pp 475–509 | Cite as

Sequential quadratic programming methods for parametric nonlinear optimization



Sequential quadratic programming (SQP) methods are known to be efficient for solving a series of related nonlinear optimization problems because of desirable hot and warm start properties—a solution for one problem is a good estimate of the solution of the next. However, standard SQP solvers contain elements to enforce global convergence that can interfere with the potential to take advantage of these theoretical local properties in full. We present two new predictor–corrector procedures for solving a nonlinear program given a sufficiently accurate estimate of the solution of a similar problem. The procedures attempt to trace a homotopy path between solutions of the two problems, staying within the local domain of convergence for the series of problems generated. We provide theoretical convergence and tracking results, as well as some numerical results demonstrating the robustness and performance of the methods.


Parametric nonlinear programming Nonlinear programming Nonlinear constraints Sequential quadratic programming SQP methods  Stabilized SQP Regularized methods Model predictive control 

Mathematics Subject Classification

49J20 49J15 49M37 49D37 65F05 65K05 90C30 



We would like to thank Greg Horn for insightful discussions in regards to some of the details of the numerical software implementation of the algorithms, in particular with regards to parallelization. In addition, we’d like to thank the reviewers for their helpful suggestions for improving the presentation of the material in the paper. This research was supported by Research Council KUL: PFV/10/002 Optimization in Engineering Center OPTEC, GOA/10/09 MaNet and GOA/10/11 Global real-time optimal control of autonomous robots and mechatronic systems. Flemish Government: IOF/KP/SCORES4CHEM, FWO: PhD/postdoc Grants and projects: G.0320.08 (convex MPC), G.0377.09 (Mechatronics MPC); IWT: PhD Grants, projects: SBO LeCoPro; Belgian Federal Science Policy Office: IUAP P7 (DYSCO, Dynamical systems, control and optimization, 2012-2017); EU: FP7-EMBOCON (ICT- 248940), FP7-SADCO ( MC ITN-264735), FP7-TEMPO, ERC ST HIGHWIND (259 166), Eurostars SMART, ACCM.

It was also supported by the European social fund within the framework of realizing the project “Support of inter-sectoral mobility and quality enhancement of research teams at the Czech Technical University in Prague”, CZ.1.07/2.3.00/30.0034.


  1. 1.
    Allgower, E.L., Georg, K.: Introduction to Numerical Continuation Methods. Colorado State University Press, Fort Collins (1990)CrossRefGoogle Scholar
  2. 2.
    Bonnans, J.: Local analysis of Newton-type methods for variational inequalities and nonlinear programming. Appl. Math. Optim. 29, 161–186 (1994)CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Bonnans, J.F., Shapiro, A.: Optimization problems with perturbations: a guided tour. SIAM Rev. 40, 228–264 (1998)CrossRefMATHMathSciNetGoogle Scholar
  4. 4.
    Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, Berlin (2000)CrossRefMATHGoogle Scholar
  5. 5.
    Diehl, M.: Real-time optimization for large scale nonlinear processes. PhD thesis, Universität Heidelberg (2001)Google Scholar
  6. 6.
    Facchinei, F., Fischer, A., Herrich, M.: A family of Newton methods for nonsmooth constrained systems with nonisolated solutions. Math. Methods Oper. Res. 1–11 (2011)Google Scholar
  7. 7.
    Facchinei, F., Fischer, A., Kanzow, C.: On the accurate identification of active constraints. SIAM J. Optim. 9, 14–32 (1998)CrossRefMATHMathSciNetGoogle Scholar
  8. 8.
    Fernández, D., Solodov, M.: Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems. Math. Program. 125, 47–73 (2010)CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    Ferreau, H.: An online active set strategy for fast solution of parametric quadratic programs with applications to predictive engine control. Master’s thesis, University of Heidelberg (2006)Google Scholar
  10. 10.
    Ferreau, H.: qpOASES - An open-source implementation of the online active set strategy for fast model predictive control. In: Proceedings of the Workshop on Nonlinear Model Based Control: Software and Applications, Loughborough, pp. 29–30 (2007)Google Scholar
  11. 11.
    Ferreau, H., Kirches, C., Potschka, A., Bock, H., Diehl, M.: qpOASES: A parametric active-set algorithm for quadratic programming. Math. Program. Comput. 1–37 (2013)Google Scholar
  12. 12.
    Gal, T.: A historical sketch on sensitivity analysis and parametric programming. In: Gal, T., Greenberg, H. (eds.) Advances in Sensitivity Analysis and Parametric Programming. International series in operations research and management science, vol. 6, pp. 1–10. Springer, New York (1997)CrossRefGoogle Scholar
  13. 13.
    Gfrerer, H., Guddat, J., Wacker, H.: A globally convergent algorithm based on imbedding and parametric optimization. Computing 30(3), 225–252 (1983)CrossRefMATHMathSciNetGoogle Scholar
  14. 14.
    Gill, P., Kungurtsev, V., Robinson, D.: A regularized SQP method convergent to second order optimal points. Technical Report 13–04, UCSD CCoM (2013)Google Scholar
  15. 15.
    Gould, N., Orban, D., Toint, P.L.: CUTEst: a constrained and unconstrained testing environment with safe threads. Cahier du GERAD G 2013, 27 (2013)Google Scholar
  16. 16.
    Guddat, J., Vasquez, F.G., Jongen, H.: Parametric Optimization: Singularities. Pathfollowing and Jumps. Teubner, Stuttgart (1990)MATHGoogle Scholar
  17. 17.
    Guddat, J., Wacker, H., Zulehner, W.: On imbedding and parametric optimizationa concept of a globally convergent algorithm for nonlinear optimization problems. In: Fiacco, A. (ed.) Sensitivity. Stability and parametric analysis, volume 21 of mathematical programming studies, pp. 79–96. Springer, Berlin (1984)Google Scholar
  18. 18.
    Hager, W.W., Gowda, M.S.: Stability in the presence of degeneracy and error estimation. Math. Program. 85(1), 181–192 (1999)CrossRefMATHMathSciNetGoogle Scholar
  19. 19.
    Hock, W., Schittkowski, K.: Lecture Notes in Economics and Mathematical Systems. Test examples for nonlinear programming codes. Springer, Berlin (1981)Google Scholar
  20. 20.
    Izmailov, A.: Solution sensitivity for Karush–Kuhn–Tucker systems with non-unique Lagrange multipliers. Optimization 59(5), 747–775 (2010)CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    Izmailov, A.F., Solodov, M.V.: Stabilized SQP revisited. Math. Program. 133, 93–120 (2012)CrossRefMATHMathSciNetGoogle Scholar
  22. 22.
    Johnson, T.C., Kirches, C., Wächter. A.: An active-set quadratic programming method based on sequential hot-starts. 2013. Available at optimization online.Google Scholar
  23. 23.
    Jongen, H.T., Jonker, P., Twilt, F.: Critical sets in parametric optimization. Math. Program. 34(3), 333–353 (1986)CrossRefMATHMathSciNetGoogle Scholar
  24. 24.
    Jongen, H.T., Weber, G.W.: On parametric nonlinear programming. Ann. Oper. Res. 27, 253–283 (1990)CrossRefMATHMathSciNetGoogle Scholar
  25. 25.
    Klatte, D., Kummer, B.: Stability properties of infima and optimal solutions of parametric optimization problems. In: Demyanov, V., Pallaschke, D. (eds.) Nondifferentiable Optimization: Motivations and Applications. Lecture notes in economics and mathematical systems, vol. 255, pp. 215–229. Springer, Berlin (1985)CrossRefGoogle Scholar
  26. 26.
    Kojima, M.: Strongly stable stationary solutions in nonlinear programs. In: Robinson, S. M. (ed.) Analysis and Computation of Fixed Points, pp. 93–138. Academic Press, New York (1980)Google Scholar
  27. 27.
    Kungurtsev, V.: Second Derivative SQP Methods. PhD thesis, UC-San Diego (2013)Google Scholar
  28. 28.
    Kyparisis, J.: On uniqueness of Kuhn–Tucker multipliers in nonlinear programming. Math. Program. 32(2), 242–246 (1985)CrossRefMATHMathSciNetGoogle Scholar
  29. 29.
    Levy, A.B.: Solution sensitivity from general principles. SIAM J. Control Optim. 40, 1–38 (2001)CrossRefMATHMathSciNetGoogle Scholar
  30. 30.
    Levy, A.B., Rockafellar, R.: Advances in Nonsmooth Optimization. Sensitivity of solutions in nonlinear programs with nonunique multipliers. World Scientific Publishing, Singapore (1995)Google Scholar
  31. 31.
    Lundberg, B.N., Poore, A.B.: Numerical continuation and singularity detection methods for parametric nonlinear programming. SIAM J. Optim. 3, 134–154 (1993)CrossRefMATHMathSciNetGoogle Scholar
  32. 32.
    Mostafa, E.-S.M., Vicente, L.N., Wright, S.J.: Global Optimization and Constraint Satisfaction. Numerical behavior of a stabilized SQP method for degenerate NLP problems, pp. 123–141. Springer, Berlin (2003)CrossRefGoogle Scholar
  33. 33.
    Poore, A., Tiahrt, C.: Bifurcation problems in nonlinear parametric programming. Math. Program. 39, 189–205 (1987)CrossRefMATHMathSciNetGoogle Scholar
  34. 34.
    Ralph, D., Dempe, S.: Directional derivatives of the solution of a parametric nonlinear program. Math. Program. 70, 159–172 (1995)MATHMathSciNetGoogle Scholar
  35. 35.
    Robinson, S.: Perturbed Kuhn–Tucker points and rates of convergence for a class of nonlinear programming algorithms. Math. Program. 7, 1–16 (1974)CrossRefMATHGoogle Scholar
  36. 36.
    Robinson, S.M.: Stability theory for systems of inequalities. Part I: linear systems. SIAM J. Numer. Anal. 12, 754–769 (1975)CrossRefMATHMathSciNetGoogle Scholar
  37. 37.
    Robinson, S.M.: Stability theory for systems of inequalities, Part II: differentiable nonlinear systems. SIAM J. Numer. Anal. 13(4), 497–513 (1976)CrossRefMATHMathSciNetGoogle Scholar
  38. 38.
    Robinson, S.M.: Optimality and Stability in Mathematical Programming. Generalized equations and their solutions, part II: applications to nonlinear programming. Springer, Berlin (1982)Google Scholar
  39. 39.
    Sequeira, S., Graellis, M., Puigjaner, L.: Real-time evolution for on-line optimization of continuous processes. Ind. Eng. Chem. Res. 41, 1815–1825 (2002)CrossRefGoogle Scholar
  40. 40.
    Seydel, R.: Practical Bifurcation and Stability Analysis. Springer, New York (2010)CrossRefMATHGoogle Scholar
  41. 41.
    Tran-Dinh, Q., Savorgnan, C., Diehl, M.: Adjoint-based predictor–corrector sequential convex programming for parametric nonlinear optimization. SIAM J. Optim. 22(4), 12581284 (2012)Google Scholar
  42. 42.
    Wachsmuth, G.: On LICQ and the uniqueness of Lagrange multipliers. Oper. Res. Lett. 41(1), 78–80 (2013)Google Scholar
  43. 43.
    Watson, L.T.: Solving the nonlinear complementarity problem by a homotopy method. SIAM J. Control Optim. 17, 36–46 (1979)CrossRefMATHMathSciNetGoogle Scholar
  44. 44.
    Wright, S.J.: Superlinear convergence of a stabilized SQP method to a degenerate solution. Comput. Optim. Appl. 11, 253–275 (1998)CrossRefMATHMathSciNetGoogle Scholar
  45. 45.
    Wright, S.J.: An algorithm for degenerate nonlinear programming with rapid local convergence. SIAM J. Optim. 15(3), 673–696 (2005)CrossRefMATHMathSciNetGoogle Scholar
  46. 46.
    Zavala, V., Anitescu, M.: Real-time nonlinear optimization as a generalized equation. SIAM J. Control Optim. 48(8), 5444–5467 (2010)CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Department of Computer Science and Optimization in Engineering Center (OPTEC)KU LeuvenLeuven-HeverleeBelgium
  2. 2.Department of Computer Science, Agent Technology Center, Faculty of Electrical EngineeringCzech Technical University in PraguePragueCzech Republic
  3. 3.Department of Microsystems Engineering IMTEKUniversity of FreiburgFreiburgGermany
  4. 4.Electrical Engineering Department (ESAT-STADIUS) and Optimization in Engineering Center (OPTEC)KU LeuvenLeuven-HeverleeBelgium

Personalised recommendations