Switching Stepsize Strategies for Sequential Quadratic Programming

  • George Tzallas-Regas
  • Berç Rustem


A Sequential Quadratic Programming (in short, SQP) algorithm is presented for solving constrained nonlinear programming problems. The algorithm uses three stepsize strategies, in order to achieve global and superlinear convergence. Switching rules are implemented that combine the merits and avoid the drawbacks of the three stepsize strategies. A penalty parameter is determined, using an adaptive strategy that aims to achieve sufficient decrease of the activated merit function. Global convergence is established and it is also shown that, locally, unity step sizes are accepted. Therefore, superlinear convergence is not impeded under standard assumptions. Global convergence and convergence of the stepsizes are displayed on test problems from the Hock and Schittkowski collection.


Nonlinear programming SQP Global convergence Stepsize convergence Merit functions Switching stepsize strategies 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, New York (1987) zbMATHGoogle Scholar
  2. 2.
    Gill, P.E., Murray, W., Wright, M.H.: Practical Optimization. Academic Press, San Diego (1981) zbMATHGoogle Scholar
  3. 3.
    Pardalos, P.M., Resende, M.G.C.: Handbook of Applied Optimization. Oxford University Press, London (2002) zbMATHGoogle Scholar
  4. 4.
    Floudas, C.A., Pardalos, P.M.: Encyclopedia of Optimization, vols. I–VI. Kluwer Academic, Dordrecht (2001) CrossRefzbMATHGoogle Scholar
  5. 5.
    Tzallas-Regas, G.: Switching stepsize strategies for Nonlinear Programming. Ph.D. thesis, University of London, Imperial College London, London (2007) Google Scholar
  6. 6.
    El-Bakry, A.S., Tapia, R.A., Tsuchiya, T., Zhang, Y.: On the formulation and theory of the Newton interior-point method for nonlinear programming. J. Optim. Theory Appl. 89, 507–541 (1996) CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Shanno, D.F., Simantiraki, E.M.: Interior point methods for linear and nonlinear programming. In: The State of the Art in Numerical Analysis, York, 1996. Inst. Math. Appl. Conf. Ser. New Ser., vol. 63, pp. 339–362. Oxford University Press, London (1997) Google Scholar
  8. 8.
    Pietrzykowski, T.: An exact potential method for constrained maxima. SIAM J. Numer. Anal. 6, 299–304 (1969) CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Han, S.P.: A globally convergent method for nonlinear programming. J. Optim. Theory Appl. 22, 297–309 (1977) CrossRefzbMATHGoogle Scholar
  10. 10.
    Powell, M.J.D.: A fast algorithm for nonlinearly constrained optimization calculations. In: Watson, G. (ed.) Lecture Notes in Mathematics, Numerical Analysis, Dundee, 1977. Springer, Berlin (1977) Google Scholar
  11. 11.
    Powell, M.J.D.: Algorithms for nonlinear constraints that use Lagrangian functions. Math. Program. 14, 224–228 (1978) CrossRefzbMATHGoogle Scholar
  12. 12.
    Han, S.P.: Superlinearly convergent variable metric algorithms for general nonlinear programming problems. Math. Program. 11, 263–282 (1976) CrossRefGoogle Scholar
  13. 13.
    Maratos, N.: Exact penalty function algorithms for finite-dimensional and control optimization problems. Ph.D. thesis, University of London (1978) Google Scholar
  14. 14.
    Rustem, B.: Equality and inequality constrained optimization algorithms with convergent stepsizes. J. Optim. Theory Appl. 76, 429–453 (1993) CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    Chamberlain, R.M., Powell, M.J.D., Lemarechal, C., Pedersen, H.C.: The wathdog technique for forcing convergence in algorithms for constrained optimization. Math. Program. Stud. 16, 1–17 (1982) CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    Pshenichnyi, B.N., Danilin, Y.M.: Numerical Methods in Extremal Problems. Nauka, Moscow (1975), English translation (1978) Google Scholar
  17. 17.
    Mayne, D.Q., Polak, E.: A superlinearly convergent algorithm for constrained optimization problems. Math. Program. Stud. 16, 45–61 (1982) CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Tapia, R.A.: Quasi-Newton methods for equality constrained optimization: equivalence of existing methods and a new implementation. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming, vol. 3, pp. 125–164. Academic Press, San Diego (1978) Google Scholar
  19. 19.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999) CrossRefzbMATHGoogle Scholar
  20. 20.
    Gill, P.E., Murray, W., Saunders, M.A., Wright, M.H.: Some theoretical properties of an augmented Lagrangian merit function. Technical Report SOL 86-6R, SOL, Stanford University (1986) Google Scholar
  21. 21.
    Fukushima, M.: A successive quadratic programming algorithm with global and superlinear convergence properties. Math. Program. 35, 253–264 (1986) CrossRefzbMATHMathSciNetGoogle Scholar
  22. 22.
    Panier, E.R., Tits, A.L.: Avoiding the Maratos effect by means of a nonmonotone line search I. general constrained problems. SIAM J. Numer. Anal. 28, 1183–1195 (1991) CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    Powell, M.J.D., Yuan, Y.: A recursive quadratic programming algorithm that uses differentiable penalty functions. Math. Program. 7, 265–278 (1986) CrossRefMathSciNetGoogle Scholar
  24. 24.
    Yamashita, H.: A globally convergent constrained quasi-Newton method with an augmented Lagrangian type penalty function. Math. Program. 23, 75–86 (1982) CrossRefzbMATHGoogle Scholar
  25. 25.
    Byrd, R.H., Nocedal, J., Waltz, R.A.: Steering exact penalty methods. Technical Report OTC 05, Optimization Technology Center, Northwestern University, Evanston Il 60208 (2005) Google Scholar
  26. 26.
    Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. 91, 239–269 (2002) CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Rustem, B.: Convergent stepsizes for constrained optimization algorithms. J. Optim. Theory Appl. 49, 135–160 (1986) CrossRefzbMATHMathSciNetGoogle Scholar
  28. 28.
    Bertsekas, D.P.: Constrained Optimization and Lagrange Multipliers. Academic Press, San Diego (1982) Google Scholar
  29. 29.
    Hock, W., Schittkowski, K.: Test examples for nonlinear programming codes. In: Lecture Notes in Economics and Mathematical Systems, vol. 187. Springer, Berlin (1981) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Department of ComputingImperial College LondonLondonUK

Personalised recommendations