Skip to main content
SpringerLink
Log in

Enlarging the region of convergence of Newton's method for constrained optimization

  • Contributed Papers
  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this paper, we consider Newton's method for solving the system of necessary optimality conditions of optimization problems with equality and inequality constraints. The principal drawbacks of the method are the need for a good starting point, the inability to distinguish between local maxima and local minima, and, when inequality constraints are present, the necessity to solve a quadratic programming problem at each iteration. We show that all these drawbacks can be overcome to a great extent without sacrificing the superlinear convergence rate by making use of exact differentiable penalty functions introduced by Di Pillo and Grippo (Ref. 1). We also show that there is a close relationship between the class of penalty functions of Di Pillo and Grippo and the class of Fletcher (Ref. 2), and that the region of convergence of a variation of Newton's method can be enlarged by making use of one of Fletcher's penalty functions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Di Pillo, G., andGrippo, L.,A New Class of Augmented Lagrangians in Nonlinear Programming, SIAM Journal on Control and Optimization, Vol. 17, pp. 618–628, 1979.

    Google Scholar 

  2. Fletcher, R.,A Class of Methods for Nonlinear Programming with Termination and Convergence Properties, Integer and Nonlinear Programming, Edited by J. Abadie, North-Holland Publishing Company, Amsterdam, Holland, 1970.

    Google Scholar 

  3. Mayne, D. Q., andPolak, E.,A Superlinearly Convergent Algorithm for Constrained Optimization Problems, Imperial College, Department of Computing and Control, Research Report No. 78/52, 1978.

  4. Powell, M. J. D.,Algorithms for Nonlinear Constraints that Use Lagrangian Functions, Mathematical Programming, Vol. 14, pp. 224–248, 1978.

    Google Scholar 

  5. Powell, M. J. D.,The Convergence of Variable Metric Methods for Nonlinearly Constrained Optimization Calculations, Nonlinear Programming 3, Edited by O. L. Mangasarian, R. Meyer, and S. Robinson, Academic Press, New York, New York, 1978.

    Google Scholar 

  6. Pshenichnyi, B. N.,Algorithm for the General Problem of Mathematical Programming, Kibernetika, No. 5, pp. 120–125, 1970 (in Russian).

  7. Pshenichnyi, B. N., andDanilin, Y. M.,Numerical Methods in Extremal Problems, Mir Publishers, Moscow, USSR, 1978.

    Google Scholar 

  8. Han, S. P.,A Globally Convergent Method for Nonlinear Programming, Journal of Optimization Theory and Applications, Vol. 22, pp. 297–309, 1977.

    Google Scholar 

  9. Mayne, D. Q., andMaratos, N.,A First-Order Exact Penalty Function Algorithm for Equality Constrained Optimization Problems, Mathematical Programming, Vol. 16, pp. 303–324, 1979.

    Google Scholar 

  10. Wilson, R. B.,A Simplicial Algorithm for Concave Programming, Harvard University, Graduate School of Business Administration, PhD Dissertation, 1963.

  11. Ortega, J. M., andRheinboldt, W. C.,Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, New York, 1970.

    Google Scholar 

  12. Tapia, R. A.,Diagonalized Multiplier Methods and Quasi-Newton Methods for Constrained Optimization, Journal of Optimization Theory and Applications, Vol. 22, pp. 135–194, 1977.

    Google Scholar 

  13. Di Pillo, G., Grippo, L., andLampariello, F.,A Method for Solving Equality Constrained Optimization Problems by Unconstrained Minimization, Proceedings of the 9th IFIP Conference on Optimization Techniques, Warsaw, Poland, 1970.

  14. Fletcher, R., andLill, S.,A Class of Methods for Nonlinear Programming: II Computational Experience, Nonlinear Programming, Edited by J. B. Rosen, O. L. Mangasarian, and K. Ritter, Academic Press, New York, New York, 1971.

    Google Scholar 

  15. Fletcher, R.,A Class of Methods for Nonlinear Programming, III, Rates of Convergence, Numerical Methods for Nonlinear Optimization, Edited by F. A. Lootsma, Academic Press, New York, New York, 1973.

    Google Scholar 

  16. Mukai, H., andPolak, E.,A Quadratically Convergent Primal-Dual Method with Global Convergence Properties for Solving Optimization Problems with Equality Constraints, Mathematical Programming, Vol. 9, pp. 336–349, 1975.

    Google Scholar 

  17. Glad, T., andPolak, E.,A Multiplier Method with Automatic Limitation of Penalty Growth, Mathematical Programming, Vol. 17, pp. 140–155, 1979.

    Google Scholar 

  18. Maratos, N.,Exact Penalty Function Algorithms for Finite-Dimensional and Control Problems, Imperial College, PhD Thesis, 1978.

  19. Chamberlain, R. M., Lemarechal, C., Pedersen, H. C., andPowell, M. J. D.,The Watchdog Technique for Forcing Convergence in Algorithms for Constrained Optimization, Tenth International Symposium on Mathematical Programming, Montreal, Canada, 1979.

  20. Garcia Palomares, U. M., andMangasarian, O. L.,Superlinearly Convergent Quasi-Newton Algorithms for Nonlinearly Constrained Optimization Problems, Mathematical Programming, Vol. 11, pp. 1–13, 1976.

    Google Scholar 

  21. Glad, T.,Properties of Updating Methods for the Multipliers in Augmented Lagrangians, Journal of Optimization Theory and Applications, Vol. 28, pp. 135–156, 1979.

    Google Scholar 

  22. Bertsekas, D. P.,Constrained Optimization and Lagrange Multiplier Methods, Academic Press, New York, New York, 1982 (to appear).

    Google Scholar 

  23. Di Pillo, G., andGrippo, L.,An Augmented Lagrangian for Inequality Constraints in Nonlinear Programming Problems, Università di Roma, Instituto di Automatica, Report No. 79–22, 1979.

  24. Poljak, B. T.,Iterative Methods Using Lagrange Multipliers for solving Extremal Problems with Constraints of the Equation Type, Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki, Vol. 10, pp. 1098–1106, 1970 (in Russian).

    Google Scholar 

  25. Dennis, J. E., andMore, J. J.,A Characterization of Superlinear Convergence and Its Application to Quasi-Newton Methods, Mathematics of Computation, Vol. 28, pp. 549–560, 1974.

    Google Scholar 

  26. Robinson, S. M.,Perturbed Kuhn-Tucker Points and Rates of Convergence for a Class of Nonlinear Programming Algorithms, Mathematical Programming, Vol. 7, pp. 1–16, 1974.

    Google Scholar 

  27. Tapia, R. A.,Quasi-Newton Methods for Equality Constrained Optimization: Equivalence of Existing Methods and a New Implementation, Nonlinear Programming 3, Edited by O. L. Managasarian, R. Meyer, and S. Robinson, Academic Press, New York, New York, 1978.

    Google Scholar 

  28. Bertsekas, D. P.,Variable Metric Methods for Constrained Optimization Based on Exact Differentiable Penalty Functions, Proceedings of the 18th Allerton Conference on Communication, Control, and Computing, University of Illinois, Urbana, Illinois, pp. 584–593, 1980.

Download references

Author information

Authors and Affiliations

  1. Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, Massachusetts

    D. P. Bertsekas (Associate Professor)

Authors
  1. D. P. Bertsekas
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Communicated by D. Q. Mayne

This work was supported by the National Science Foundation, Grant No. ENG-79-06332.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bertsekas, D.P. Enlarging the region of convergence of Newton's method for constrained optimization. J Optim Theory Appl 36, 221–252 (1982). https://doi.org/10.1007/BF00933831

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00933831

Key Words

Search

Navigation