Abstract
It is known that augmented Lagrangian or multiplier methods for solving constrained optimization problems can be interpreted as techniques for maximizing an augmented dual functionD c(λ). For a constantc sufficiently large, by considering maximizing the augmented dual functionD c(λ) with respect toλ, it is shown that the Newton iteration forλ based on maximizingD c(λ) can be decomposed into taking a Powell/Hestenes iteration followed by a Newton-like correction. Superimposed on the original Powell/Hestenes method, a simple acceleration technique is devised to make use of information from the previous iteration. For problems with only one constraint, the acceleration technique is equivalent to replacing the second (Newton-like) part of the decomposition by a finite difference approximation. Numerical results are presented.
Similar content being viewed by others
References
D.P. Bertsekas, “Combined primal-dual and penalty methods for constrained Minimization”,SIAM Journal on Control 13 (1975) 521–544.
D.P. Bertsekas, “On penalty and multiplier methofs for constrained minimization”,SIAM Journal on Control and Optimization 14 (1976) 217–235.
J.D. Buys, “Dual algorithms for constrained optimization problems”, Ph.D. Thesis, University of Leiden (Leiden, 1972).
A.R. Colville, “A comparative study on nonlinear programming codes”, IBM New York Scientific Centre, Technical Report No. 320-2949 (1968).
R. Fletcher, “An ideal penalty function for constrained optimization”,Journal of the Institute of Mathematics and its Applications 15 (1975) 319–342.
P.C. Haarhoff and J.D. Buys, “A new method for the optimization of a nonlinear function subject to nonlinear constraints”,Computer Journal 13 (1970) 178–184.
S.P. Han, “Dual variable-metric algorithms for constrained optimization”,SIAM Journal on Control 15 (1977) 546–565.
M.R. Hestenes, “Multiplier and gradient methods”,Journal of Optimization Theory and Applications 4 (1969) 303–320.
B.W. Kort and D.P. Bertsekas, “Combined primal—dual and penalty methods for convex programming”,SIAM Journal on Control and Optimization 14 (1976) 268–294.
A. Miele, P.E. Moseley, A.V. Levy and G.M. Coggins, “On the method of multiplier for mathematical programming problems”,Journal of Optimization Theory and Applications 10 (1972) 1–33.
M.J.D. Powell, “A method for nonlinear constraints in minimization problems”. In: R. Fletcher, ed.,Optimization (Academic Press, London, 1969).
M.J.D. Powell, “A fast algorithm for nonlinearly constrained optimization calculation”. In: G.A. Watson, ed.,Proceeding of the 1977 Dundee conference on numerical analysis (Springer-Verlag, Berlin, 1977).
R.T. Rockafellar, “Augmented Lagrange multiplier functions and duality in nonconvex programming”,SIAM Journal on Control 12 (1974) 269–285.
J.B. Rosen and S. Suzuki, “Construction of nonlinear programming test problems”,Communications of the ACM 8 (1965) 113.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Jittorntrum, K. Accelerated convergence for the Powell/Hestenes multiplier method. Mathematical Programming 18, 197–214 (1980). https://doi.org/10.1007/BF01588314
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01588314