Journal of Optimization Theory and Applications

, Volume 36, Issue 2, pp 221–252 | Cite as

Enlarging the region of convergence of Newton's method for constrained optimization

  • D. P. Bertsekas
Contributed Papers

Abstract

In this paper, we consider Newton's method for solving the system of necessary optimality conditions of optimization problems with equality and inequality constraints. The principal drawbacks of the method are the need for a good starting point, the inability to distinguish between local maxima and local minima, and, when inequality constraints are present, the necessity to solve a quadratic programming problem at each iteration. We show that all these drawbacks can be overcome to a great extent without sacrificing the superlinear convergence rate by making use of exact differentiable penalty functions introduced by Di Pillo and Grippo (Ref. 1). We also show that there is a close relationship between the class of penalty functions of Di Pillo and Grippo and the class of Fletcher (Ref. 2), and that the region of convergence of a variation of Newton's method can be enlarged by making use of one of Fletcher's penalty functions.

Key Words

Constrained minimization Newton's method differentiable exact penalty functions superlinear convergence 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Di Pillo, G., andGrippo, L.,A New Class of Augmented Lagrangians in Nonlinear Programming, SIAM Journal on Control and Optimization, Vol. 17, pp. 618–628, 1979.Google Scholar
  2. 2.
    Fletcher, R.,A Class of Methods for Nonlinear Programming with Termination and Convergence Properties, Integer and Nonlinear Programming, Edited by J. Abadie, North-Holland Publishing Company, Amsterdam, Holland, 1970.Google Scholar
  3. 3.
    Mayne, D. Q., andPolak, E.,A Superlinearly Convergent Algorithm for Constrained Optimization Problems, Imperial College, Department of Computing and Control, Research Report No. 78/52, 1978.Google Scholar
  4. 4.
    Powell, M. J. D.,Algorithms for Nonlinear Constraints that Use Lagrangian Functions, Mathematical Programming, Vol. 14, pp. 224–248, 1978.Google Scholar
  5. 5.
    Powell, M. J. D.,The Convergence of Variable Metric Methods for Nonlinearly Constrained Optimization Calculations, Nonlinear Programming 3, Edited by O. L. Mangasarian, R. Meyer, and S. Robinson, Academic Press, New York, New York, 1978.Google Scholar
  6. 6.
    Pshenichnyi, B. N.,Algorithm for the General Problem of Mathematical Programming, Kibernetika, No. 5, pp. 120–125, 1970 (in Russian).Google Scholar
  7. 7.
    Pshenichnyi, B. N., andDanilin, Y. M.,Numerical Methods in Extremal Problems, Mir Publishers, Moscow, USSR, 1978.Google Scholar
  8. 8.
    Han, S. P.,A Globally Convergent Method for Nonlinear Programming, Journal of Optimization Theory and Applications, Vol. 22, pp. 297–309, 1977.Google Scholar
  9. 9.
    Mayne, D. Q., andMaratos, N.,A First-Order Exact Penalty Function Algorithm for Equality Constrained Optimization Problems, Mathematical Programming, Vol. 16, pp. 303–324, 1979.Google Scholar
  10. 10.
    Wilson, R. B.,A Simplicial Algorithm for Concave Programming, Harvard University, Graduate School of Business Administration, PhD Dissertation, 1963.Google Scholar
  11. 11.
    Ortega, J. M., andRheinboldt, W. C.,Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, New York, 1970.Google Scholar
  12. 12.
    Tapia, R. A.,Diagonalized Multiplier Methods and Quasi-Newton Methods for Constrained Optimization, Journal of Optimization Theory and Applications, Vol. 22, pp. 135–194, 1977.Google Scholar
  13. 13.
    Di Pillo, G., Grippo, L., andLampariello, F.,A Method for Solving Equality Constrained Optimization Problems by Unconstrained Minimization, Proceedings of the 9th IFIP Conference on Optimization Techniques, Warsaw, Poland, 1970.Google Scholar
  14. 14.
    Fletcher, R., andLill, S.,A Class of Methods for Nonlinear Programming: II Computational Experience, Nonlinear Programming, Edited by J. B. Rosen, O. L. Mangasarian, and K. Ritter, Academic Press, New York, New York, 1971.Google Scholar
  15. 15.
    Fletcher, R.,A Class of Methods for Nonlinear Programming, III, Rates of Convergence, Numerical Methods for Nonlinear Optimization, Edited by F. A. Lootsma, Academic Press, New York, New York, 1973.Google Scholar
  16. 16.
    Mukai, H., andPolak, E.,A Quadratically Convergent Primal-Dual Method with Global Convergence Properties for Solving Optimization Problems with Equality Constraints, Mathematical Programming, Vol. 9, pp. 336–349, 1975.Google Scholar
  17. 17.
    Glad, T., andPolak, E.,A Multiplier Method with Automatic Limitation of Penalty Growth, Mathematical Programming, Vol. 17, pp. 140–155, 1979.Google Scholar
  18. 18.
    Maratos, N.,Exact Penalty Function Algorithms for Finite-Dimensional and Control Problems, Imperial College, PhD Thesis, 1978.Google Scholar
  19. 19.
    Chamberlain, R. M., Lemarechal, C., Pedersen, H. C., andPowell, M. J. D.,The Watchdog Technique for Forcing Convergence in Algorithms for Constrained Optimization, Tenth International Symposium on Mathematical Programming, Montreal, Canada, 1979.Google Scholar
  20. 20.
    Garcia Palomares, U. M., andMangasarian, O. L.,Superlinearly Convergent Quasi-Newton Algorithms for Nonlinearly Constrained Optimization Problems, Mathematical Programming, Vol. 11, pp. 1–13, 1976.Google Scholar
  21. 21.
    Glad, T.,Properties of Updating Methods for the Multipliers in Augmented Lagrangians, Journal of Optimization Theory and Applications, Vol. 28, pp. 135–156, 1979.Google Scholar
  22. 22.
    Bertsekas, D. P.,Constrained Optimization and Lagrange Multiplier Methods, Academic Press, New York, New York, 1982 (to appear).Google Scholar
  23. 23.
    Di Pillo, G., andGrippo, L.,An Augmented Lagrangian for Inequality Constraints in Nonlinear Programming Problems, Università di Roma, Instituto di Automatica, Report No. 79–22, 1979.Google Scholar
  24. 24.
    Poljak, B. T.,Iterative Methods Using Lagrange Multipliers for solving Extremal Problems with Constraints of the Equation Type, Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki, Vol. 10, pp. 1098–1106, 1970 (in Russian).Google Scholar
  25. 25.
    Dennis, J. E., andMore, J. J.,A Characterization of Superlinear Convergence and Its Application to Quasi-Newton Methods, Mathematics of Computation, Vol. 28, pp. 549–560, 1974.Google Scholar
  26. 26.
    Robinson, S. M.,Perturbed Kuhn-Tucker Points and Rates of Convergence for a Class of Nonlinear Programming Algorithms, Mathematical Programming, Vol. 7, pp. 1–16, 1974.Google Scholar
  27. 27.
    Tapia, R. A.,Quasi-Newton Methods for Equality Constrained Optimization: Equivalence of Existing Methods and a New Implementation, Nonlinear Programming 3, Edited by O. L. Managasarian, R. Meyer, and S. Robinson, Academic Press, New York, New York, 1978.Google Scholar
  28. 28.
    Bertsekas, D. P.,Variable Metric Methods for Constrained Optimization Based on Exact Differentiable Penalty Functions, Proceedings of the 18th Allerton Conference on Communication, Control, and Computing, University of Illinois, Urbana, Illinois, pp. 584–593, 1980.Google Scholar

Copyright information

© Plenum Publishing Corporation 1982

Authors and Affiliations

  • D. P. Bertsekas
    • 1
  1. 1.Laboratory for Information and Decision SystemsMassachusetts Institute of TechnologyCambridge

Personalised recommendations