Advertisement

Equality and inequality constrained optimization algorithms with convergent stepsizes

  • B. Rustem
Contributed Papers

Abstract

In this paper, we consider two algorithms for nonlinear equality and inequality constrained optimization. Both algorithms utilize stepsize strategies based on differentiable penalty functions and quadratic programming subproblems. The essential difference between the algorithms is in the stepsize strategies used. The objective function in the quadratic subproblem includes a linear term that is dependent on the penalty functions. The quadratic objective function utilizes an approximate Hessian of the Lagrangian augmented by the penalty functions. In this approximation, it is possible to ignore the second-derivative terms arising from the constraints in the penalty functions.

The penalty parameter is determined using a strategy, slightly different for each algorithm, that ensures boundedness as well as a descent property. In particular, the boundedness follows as the strategy is always satisfied for finite values of the parameter.

These properties are utilized to establish global convergence and the condition under which unit stepsizes are achieved. There is also a compatibility between the quadratic objective function and the stepsize strategy to ensure the consistency of the properties for unit steps and subsequent convergence rates.

Key Words

Nonlinear programming sequential quadratic programming nonlinear equality and inequality constraints augmented Lagrangians adaptive penalty parameter convergence analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Polak, E., andTits, A. L.,A Globally Convergent, Implementable Multiplier Method with Automatic Penalty Limitation, Applied Mathematics and Optimization, Vol. 6, pp. 335–360, 1980.Google Scholar
  2. 2.
    Tapia, R. A.,On Secant Updates for Use in General Constrained Optimization, Rice University, Technical Report No. 84-3, 1986.Google Scholar
  3. 3.
    Dixon, L. C. W.,The Choice of Steplength, a Crucial Factor in the Performance of Variable-Metric Algorithms, Numerical Methods for Nonlinear Optimization, Edited by F. A. Lootsma, Academic Press, London, England, 1972.Google Scholar
  4. 4.
    Han, S. P.,A Globally Convergent Method for Nonlinear Programming, Journal of Optimization Theory and Applications, Vol. 22, pp. 297–309, 1977.Google Scholar
  5. 5.
    Powell, M. J. D.,A Fast Algorithm for Nonlinearly Constrained Optimization Calculations, Numerical Analysis Proceedings, Dundee 1977, Edited by G. A. Watson, Springer-Verlag, Berlin, Germany, 1978.Google Scholar
  6. 6.
    Powell, M. J. D.,Variable-Metric Methods for Constrained Optimization, Mathematical Programming: The State of the Art, Edited by A. Bachem, M. Grötschel, and B. Korte, Springer-Verlag, Berlin, Germany, 1987.Google Scholar
  7. 7.
    Powell, M. J. D., andYuan, Y.,A Recursive Quadratic Programming Algorithm That Uses Differentiable Exact Penalty Functions, Mathematical Programming, Vol. 35, pp. 265–278, 1986.Google Scholar
  8. 8.
    Rustem, B.,On the Q- and Two-Step Q-Superlinear Convergence of Successive Quadratic Programming Algorithms, Imperial College, PROPE Discussion Paper No. 89, 1990.Google Scholar
  9. 9.
    Tapia, R. A.,Diagonalized Multiplier Methods and Quasi-Newton Methods for Constrained Optimization, Journal of Optimization Theory and Applications, Vol. 22, pp. 135–194, 1977.Google Scholar
  10. 10.
    Fan, Y., Sarkar, S., andLasdon, L.,Experiments with Successive Quadratic Programming Algorithms, Journal of Optimization Theory and Applications, Vol. 56, pp. 359–383, 1988.Google Scholar
  11. 11.
    Wilson, R. B.,A Simplicial Algorithm for Concave Programming, Harvard University, PhD Dissertation, 1963.Google Scholar
  12. 12.
    Bartholomew-Biggs, M. C.,The Development of a Class of Constrained Minimization Algorithms and Their Application to the Problem of Power Scheduling, University of London, PhD Thesis, 1974.Google Scholar
  13. 13.
    Bartholomew-Biggs, M. C.,On the Convergence of Some Constrained Minimization Algorithms Based on Recursive Quadratic Programming, Journal of the Institute of Mathematics and Its Applications, Vol. 21, pp. 67–81, 1978.Google Scholar
  14. 14.
    Bartholomew-Biggs, M. C.,Line Search Procedures for Nonlinear Programming Algorithms with Quadratic Programming Subproblems, Report No. 116, Numerical Optimization Centre, Hatfield Polytechnic, 1981.Google Scholar
  15. 15.
    Bartholomew-Biggs, M. C.,Recursive Quadratic Programming Methods Based on the Augmented Lagrangian, Mathematical Programming Study, Vol. 31, pp. 21–41, 1987.Google Scholar
  16. 16.
    Bartholomew-Biggs, M. C.,A Globally Convergent Version of REQP for Constrained Optimization, IMA Journal of Numerical Analysis, Vol. 8, pp. 253–271, 1988.Google Scholar
  17. 17.
    Rustem, B.,Convergent Stepsizes for Constrained Optimization Algorithms, Journal of Optimization Theory and Applications, Vol. 49, pp. 135–160, 1986.Google Scholar
  18. 18.
    Mayne, D. Q., andPolak, E.,A Superlinearly Convergent Algorithm for Constrained Minimization Problems, Mathematical Programming Study, Vol. 16, pp. 45–61, 1982.Google Scholar
  19. 19.
    Polak, E., andMayne, D. Q.,A Robust Secant Method for Optimization Problems with Inequality Constraints, Journal of Optimization Theory and Applications, Vol. 33, pp. 463–477, 1981.Google Scholar
  20. 20.
    Gabay, D.,Reduced Quasi-Newton Methods with Feasibility Improvement for Nonlinearly Constrained Optimization, Mathematical Programming Study, Vol. 16, pp. 18–44, 1982.Google Scholar
  21. 21.
    Schittkowski, K.,The Nonlinear Programming Method of Wilson, Han, and Powell with an Augmented Lagrangian-Type Line Search Function, Part 1: Convergence Analysis, Numerische Mathematik, Vol. 38, pp. 84–114, 1981.Google Scholar
  22. 22.
    Schittkowski, K.,The Nonlinear Programming Method of Wilson, Han, and Powell with an Augmented Lagrangian-Type Line Search Function, Part 2: An Efficient Implementation with Linear Least-Squares Subproblems, Numerische Mathematik, Vol. 38, pp. 115–127, 1981.Google Scholar
  23. 23.
    Schittkowski, K.,On the Convergence of a Sequential Quadratic Programming Method with an Augmented Lagrangian Line Search Function, Mathematische Operationsforschung und Statistik, Serie Optimization, Vol. 14, pp. 197–216, 1983.Google Scholar
  24. 24.
    Gill, P. E., Murray, W., Saunders, M. A., andWright, M. H.,Some Theoretical Properties of an Augmented Lagrangian Merit Function, Report No. SOL 86-6R, Stanford University, 1986.Google Scholar
  25. 25.
    Burke, J. V., andHan, S. P.,A Robust Sequential Quadratic Programming Method, Mathematical Programming, Vol. 43, pp. 277–303, 1989.Google Scholar
  26. 26.
    Boggs, P. T., Tolle, J. W., andWang, P.,On the Local Convergence of Quasi-Newton Methods for Constrained Optimization, SIAM Journal on Control and Optimization, Vol. 20, pp. 161–171, 1982.Google Scholar
  27. 27.
    Chiang, J., andTapia, R.,Convergence Rates for the Variable, the Multiplier, and the Pair in SQP Methods, Technical Report No. 89-2, Rice University, 1989.Google Scholar
  28. 28.
    Coleman, T. F., andConn, A. R.,On the Local Convergence of a Quasi-Newton Method for the Nonlinear Programming Problem, SIAM Journal on Numerical Analysis, Vol. 21, pp. 755–769, 1984.Google Scholar
  29. 29.
    Fontecilla, R., Steihaug, T., andTapia, R. A.,A Convergence Theory for a Class of Quasi-Newton Methods for Constrained Optimization, SIAM Journal on Numerical Analysis, Vol. 24, pp. 1133–1151, 1987.Google Scholar
  30. 30.
    Han, S. P.,Superlinearly Convergent Variable-Metric Algorithms for General Nonlinear Programming Problems, Mathematical Programming, Vol. 11, pp. 263–282, 1976.Google Scholar
  31. 31.
    Palomares, U. M. G., andMangasarian, O. L.,Superlinearly Convergent Quasi-Newton Algorithms for Nonlinearly Constrained Optimization Problems, Mathematical Programming, Vol. 11, pp. 1–13, 1976.Google Scholar
  32. 32.
    Stoer, J., andTapia, R. A.,On the Characterization of Q-Superlinear Convergence of Quasi-Newton Methods for Constrained Optimization, Technical Report No. 84-2, Rice University, 1986.Google Scholar
  33. 33.
    Chamberlain, R. M.,Some Examples of Cycling in Variable-Metric Methods for Constrained Minimization, Mathematical Programming, Vol. 16, pp. 378–383, 1978.Google Scholar
  34. 34.
    Maratos, N.,Exact Penalty Function Algorithms for Finite-Dimensional and Control Optimization Problems, PhD Thesis, University of London, 1978.Google Scholar
  35. 35.
    Chamberlain, R. M., Powell, M. J. D., Lemarechal, C., andPedersen, H. C.,The Watchdog Technique for Forcing Convergence in Algorithms for Constrained Optimization, Mathematical Programming Study, Vol. 16, pp. 1–17, 1982.Google Scholar
  36. 36.
    Bartholomew-Biggs, M. C., andNguyen, T. T.,Orthogonal and Conjugate Basis Methods for Solving Equality Constrained Minimization Problems, Report No. 222, Numerical Optimization Centre, Hatfield Polytechnic, 1990.Google Scholar
  37. 37.
    Coleman, T. F., andConn, A. R.,Nonlinear Programming via an Exact Penalty Function: Asymptotic Analysis, Mathematical Programming, Vol. 24, pp. 123–136, 1982.Google Scholar
  38. 38.
    Dixon, L. C. W.,Conjugate Bases for Constrained Optimisation, Report No. 181, Numerical Optimization Centre, Hatfield Polytechnic, 1987.Google Scholar
  39. 39.
    Dew, M. C.,An Exact Penalty Function Algorithm for Accurate Optimisation of Industrial Problems, Hatfield Polytechnic, PhD Thesis, 1985.Google Scholar
  40. 40.
    Bertsekas, D. P.,Constrained Optimization and Lagrange Multiplier Methods, Academic Press, New York, New York, 1982.Google Scholar
  41. 41.
    Di Pillo, G., andGrippo, L.,Exact Penalty Functions in Constrained Optimization, SIAM Journal on Control and Optimization, Vol. 27, pp. 1333–1360, 1989.Google Scholar
  42. 42.
    Dixon, L. C. W.,On the Convergence Properties of Variable Metric Recursive Quadratic Programming Algorithms, Report No. 110, Numerical Optimization Centre, Hatfield Polytechnic, 1980.Google Scholar
  43. 43.
    Rustem, B.,A Class of Superlinearly Convergent Projection Algorithms with Relaxed Stepsizes, Applied Mathematics and Optimization, Vol. 12, pp. 29–43, 1984.Google Scholar
  44. 44.
    Hock, W., andSchittkowski, K.,Test Examples for Nonlinear Programming Problems, Springer-Verlag, Berlin, Germany, 1981.Google Scholar

Copyright information

© Plenum Publishing Corporation 1993

Authors and Affiliations

  • B. Rustem
    • 1
  1. 1.Department of ComputingImperial College of Science, Technology, and MedicineLondonUK

Personalised recommendations