Principles of Sequential Quadratic Programming Methods for Solving Nonlinear Programs

  • J. Stoer
Part of the NATO ASI Series book series (volume 15)


In this paper it is tried to describe to some extent the theoretical background and several practical aspects of sequential quadratic programming (S.QP) methods for solving the following standard problem
$$\matrix{ {\left( {\rm{p}} \right)\min {\rm{f}}\left( {\rm{x}} \right)} \cr {{\rm{x}} \in {{\rm{R}}^{\rm{n}}}:{{\rm{g}}_{\rm{j}}}\left( {\rm{x}} \right) \le {\rm{0, j = 1,2, \ldots ,mi}}} \cr {{\rm{ }}{{\rm{g}}_{\rm{j}}}\left( {\rm{x}} \right) = {\rm{0, j = mi + 1, \ldots ,m,}}} \cr }$$
Where f,gj ∈ C2 (Rn).


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Baptist, P., Stoer, J.: On the relation between quadratic termination and convergence properties of minimization algorithms. Part I, Numer.Math. 28, 343–366 (1977). Part II, Applications: Numer.Math.28, 367-391 (1977).CrossRefMathSciNetGoogle Scholar
  2. Bartho Iomew-Biggs, M.C.: An improved implementation of the recursive equality quadratic programming method for constrained minimization. Techn.Report No.105, Numerical Optimization Centre. The Hatfield Polytechnic, Hatfield, UK (1979).Google Scholar
  3. Bartho Iomew-Biggs, M.C.: Equality and inequality constrained quadratic programming subproblems for constrained optimization. Techn.Report No. 128, Numerical Optimization Centre. The Hatfield Polytechnic, Hatfield, UK (1982).Google Scholar
  4. Bartho Iomew-Biggs, M.C.; A recursive quadratic programming algorithm based on the augmented Lagrangian function. Techn. Report No.139, Numerical Optimization Centre. The Hatfield Polytechnic, Hatfield. UK (1983).Google Scholar
  5. BeaIe, E.M.L.: On quadratic programming. Naval Research Logistics Quarterly 6, 227–243 (1959).Google Scholar
  6. Bertsekas, D.P.: Constrained optimization and Lagrange multipler methods. New York: Academic Press (1982).Google Scholar
  7. Biggs, M.C.: Constrained minimization using recursive equality quadratic programming. In: Numerical Methods for Nonlinear Optimization (F.A. Lootsma,ed.). Academic Press, 1972.Google Scholar
  8. Biggs, M.C.: Constrained minimization using recursive quadratic programming: some alternative subproblem formulations. In: Towards global optimization (L.C.W. Dixon, G.P. Szego, eds.), Nor th-Holland 1975.Google Scholar
  9. Boggs, P.T., TolIe, J.W.: Augmented Lagrangian which are quadratic in the multiplier. JOTA, 31, 17–26 (1980).Google Scholar
  10. Boggs, P.T., ToI Ie, J.W.: Merit functions for nonlinear programming problems. Operations Research and Systems Analysis Report No.81-2, Univ. of North Carolina at Chapel Hill, N.C. (1981).Google Scholar
  11. Boggs, P.T., ToI Ie, J.W., Wang, P.: On the local convergence of quasi Newton methods for constrained optimization. SIAM J. Control and Opt., 20, 161–171 (1982).Google Scholar
  12. Broyden, C.G.: Quasi-Newton methods and their application to function minimization. Math.Comp. 21, 368–381 (1967).Google Scholar
  13. Broyden, C.G., Dennis, J.E.,More,J.J.: On the local and superlinear convergence of quasi-Newton methods. J. Inst.Math.AppI. 12, 223–245 (1973).Google Scholar
  14. Chamberlain, R., M.: Some examples of cycling in variable metric methods for constrained minimization. Math. Programming 16, 378–384 (1979).Google Scholar
  15. Dennis, J.E., More, J.J.: A characterization of superlinear convergence and its applications to quasi-Newton methods. Math.Comput. 28, 549–560 (1974).Google Scholar
  16. Dennis, J.E., More, J.J.: Quasi-Newton methods, motivation and theory. SIAM Rev.19, 46-89 (1977).Google Scholar
  17. Dennis, J.E., Schnabel, R.B.: Least change secant updates for quasi-Newton methods. SIAM Rev.21, 443-459 (1979).Google Scholar
  18. Dennis, J.E., WaIker, H.F.: Convergence theorems for least-change secant update methods. SIAM J.Numer.Ana l.18, 949-987 (1981).Google Scholar
  19. Di Pillo, G. Grippo, L.: A new class of augmented Lagrangians in nonlinear programming. SIAM J. Control Optim.17, 618-628 (1979).Google Scholar
  20. Di Pillo, G., Grippo, L., Lamparielto, F.: A method for solving equality constrained optimization problems by unconstrained minimization. In: Optimization Techniques Part 2, (K.lracki, K.Malanowski, S. Walukiewicz, eds.), Lecture Notes in Control and Information Sciences, Vol.23, Berlin: Springer (1980).Google Scholar
  21. Fiacco, A.V., McCormick, G.P.: Nonlinear Programming. Sequential Unconstrained Minimization Techniques. New York: Wiley 1968.Google Scholar
  22. Fletcher, R.: The calculation of feasible points for linearly constrained optimization problems. UKAEA Research Group Report, AERE R6354 (1970).Google Scholar
  23. Fletcher, R.: A Fortran subroutine for quadratic pr ogr arrmi ng. UKAEA Research Group Report, AERE R6370 (1970).Google Scholar
  24. Fletcher, R.: A general quadratic programming algorithm. J.lnst. of Math, and its Appl. 76-91 (1971).Google Scholar
  25. Fletcher, R.: Practical Methods of Optimization, Vol.1, Unconstrained Optimization. New York-: Wiley (1980).Google Scholar
  26. Fletcher, R.: Practical Methods of Optimization, Vol.2, Constrained Optimization. New York: Wiley (1981).Google Scholar
  27. Ge Ren-Pu, Powell, M.J.D.: The convergence of variable metric matrices in unconstrained optimization. Math.Programming 27, 123–143 (1983).Google Scholar
  28. Gill, P.E., Murray,W.: Numerically stable methods for quadratic programming. Math.Programming 14, 349–372 (1978).Google Scholar
  29. Goldfarb, D.,Idnani,A.: A numerically stable dual method for solving strictly convex quadratic programs. Math.Programming 27, 1–33 (1983).Google Scholar
  30. Goodman, J.: Newton’s method for constrained optimization. Courant Institute of Math.Sciences, New York,N.Y. (1982).Google Scholar
  31. Griewank, A.,Toint, Ph.L.: Local convergence analysis for partitioned quasi-Newton updates. Numer.Math. 39, 429–448 (1982).Google Scholar
  32. Han, S.-P.: Superlinearly convergent variable metric algorithms for general nonlinear programming problems. Math.Progr. 11, 263–282 (1976).Google Scholar
  33. Han, S.-P.: A globally convergent method for nonlinear programming. JOTA 22, 277–309 (1977).Google Scholar
  34. Hestenes, M.R.: Multiplier and gradient methods. JOTA 4, 303–320 (1969).Google Scholar
  35. Hock, W., Schittkowski, K.: Test examples for nonlinear programming. Lecture Notes in Economics and Mathematical Systems,Vol.187, Berlin-Heidelberg-New York: Springer (1981).Google Scholar
  36. Jittorntrum, K.: Solution point differentiability without strict complementarity in nonlinear programming. Math.Programming Study 21, 127–138 (1984).Google Scholar
  37. Lemke, C.E.: A method of solution for quadratic programs. Management Sci.8, 442-453 (1962).Google Scholar
  38. Maratos, N.: Exact penalty function algorithms for finite dimensional and control optimization problems. Ph.D.Thesis, imperial College, London (1978).Google Scholar
  39. Marwil, E.S.: Exploiting sparsity in Newton-like methods. Ph.D.Thesis, Cornell University, Ithaca, NY. (1978).Google Scholar
  40. Murray, W.: An algorithm for constrained minimization. In: Optimization (R. Fletcher, ed.). Academic Press (1969).Google Scholar
  41. Murray, W.,Wright, M.H.: Projected Lagrangian methods based on the trajectories of penalty and barrier functions. Report SOL 78-23, Rept. of Operations Research, Stanford University, Cal.(1978)Google Scholar
  42. Nocedal, J.,Overton, M.: Projected hessian updating algorithms for nonlinearly constrained optimization. Computer Science Department Report No. 95, 1983. Courant Institute, New York Univ.,New York, N.Y.Google Scholar
  43. Ortega, J.M., RheinboIdt, W.C.: Iterative Solution of Non-linear Equations in Several Variables, New York: Academic Press 1970.Google Scholar
  44. Pietrzykowski, T.: An exact potential method for constrained maxima. SIAM J. Numer.Ana l.6, 299-304 (1969).Google Scholar
  45. Pietrzykowski, T.: The potential method for conditional maxima in the locally convex space. Num.Math.14, 325-329 (1970).Google Scholar
  46. Powell, M.J.D.: A method for nonlinear constraints in minimization problems. In: Optimization (R. Fletcher, ed.). London: Academic Press (1969).Google Scholar
  47. Powell, M.J.D.: The convergence of variable metric methods for nonlinearly constrained optimization calculations. 27-63 in: Nonlinear Programming 3, O.L. Mangasarian, R.R. Meyer, S.M. Robinson, eds., Acad.Press (1978a).Google Scholar
  48. Powell, M.J.D.: A fast algorithm for nonlinearly constrained optimization calculation, in: G.A. Watson (ed.): Numerical Analysis. Lecture Notes in Mathematics, Vol.630. Berlin-Heidelberg-New York: Springer (1978b).Google Scholar
  49. Powell, M.J.D.: Algorithms for nonlinear constraints that use Lagrangian functions. Math.Programming 14,224-248 (1978c).Google Scholar
  50. Powell, M.J.D.: On the rate of convergence of variable metric algorithms for unconstrained optimization. Report DAMTP 1983/NA7, Department of Applied Mathematics and Theoretical Physics, Univ. of Cambridge, England (1983a).Google Scholar
  51. Powell, M.J.D.: ZQPCVX a Fortran subroutine for convex quadratic programming. Technical Report DAMTP/1983/NA17, Department of Applied Mathematics and Theoretical Physics, Cambridge University, England (1983b).Google Scholar
  52. Ritter, K.: On the rate of superlinear convergence of a class of variable metric methods. Numer.Math. 35, 293–313 (1980).Google Scholar
  53. Rockafellar, R. T.: A dual approach to solving nonlinear programming problems by unconstrained optimization. Math. Programming 5, 354–373 (1973).Google Scholar
  54. Schittkowski, K.: Nonlinear proramming codes. Information, tests, performance. Lecture Notes in Economics and Mathematical Systems, Vol. 183, Berlin-Heidelberg-New York: Springer (1980).Google Scholar
  55. Schittkowski, K.: The nonlinear programming method of Wilson, Han and Powell with an augmented Lagrangian type line search function. Part 1: Convergence Analysis. Numer.Math. 38, 83–114 (1983a), Part 2: An efficient implementation with linear least squares subproblems. Numer.Math. 38, 115–127 (1983b).Google Scholar
  56. Schittkowski, K.: On the convergence of a sequential quadratic programing method with an augmented Lagrangian line search function. Math.Operationsforsch.u.Statist., Ser. Optimization 14, 197–216 (1983c).Google Scholar
  57. Schuller, G.: On the order of convergence of certain Quasi-Newton methods. Numer. Math. 23, 181–192 (1974).Google Scholar
  58. Stachurski, A.: Superlinear convergence of Broyden’s bounded β-class of methods. Math.Progr.20, 196-212 (1981).Google Scholar
  59. Stoer, J.: On the numerical solution of constrained least squares problems. SIAM J. Numer.Anal. 8, 382–411 (1971).Google Scholar
  60. Stoer, J.: On the convergence rate of imperfect minimization algorithms in Broyden’s β-ctass. Math. Programming 9, 313–335 (1975).Google Scholar
  61. Stoer, J.: The convergence of matrices generated by rank-2 methods from the restricted β-class of Broyden. Numer. Math. 44, 37–52 (1984).Google Scholar
  62. Tanabe, K.I Feasibility-improving gradient-acute-projection methods: a unified approach to nonlinear programming. Lecture Notes in Num.Appl. Anal. 3, 57–76 (1981).Google Scholar
  63. Tapia, R.A.: Diagonalized multiplier methods and quasi-Newton methods for constrained optimization. JOTA, 22, 135–194 (1977).Google Scholar
  64. Toint, Ph.L.: On sparse and symmetric matrix updating subject to a linear equation. Math.Comp. 31, 954–961 (1977).Google Scholar
  65. Toint, Ph.L.: On the superlinear convergence of an algorithm for solving a sparse minimization problem. SIAM Numer. Ana I. 1063-1045 (1979).Google Scholar
  66. Van de Panne, C., Whinston, A.: The simplex and the dual method for quadratic programming. Operations Research Quarterly 15, 355–389 (1964).Google Scholar
  67. Wilson, R.B.: A simplicial algorithm for concave programming. Ph.D.Thesis, Graduate School of Business Administration, Harvard University, Cambridge, Mass. (1963).Google Scholar
  68. Yuan, Y.: On the least Q-order of convergence of variable metric algorithms. Report DAMTP 1983/NA10. Department of Applied Mathematics and Theoretical Physics, Univ. of Cambridge, England.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1985

Authors and Affiliations

  • J. Stoer
    • 1
  1. 1.Institut für Angewandte Mathematik und StatistikUniversität Würzburg Am HublandWürzburgGermany

Personalised recommendations