Abstract
Augmented Lagrangian algorithms are very popular tools for solving nonlinear programming problems. At each outer iteration of these methods a simpler optimization problem is solved, for which efficient algorithms can be used, especially when the problems are large. The most famous Augmented Lagrangian algorithm for minimization with inequality constraints is known as Powell-Hestenes-Rockafellar (PHR) method. The main drawback of PHR is that the objective function of the subproblems is not twice continuously differentiable. This is the main motivation for the introduction of many alternative Augmented Lagrangian methods. Most of them have interesting interpretations as proximal point methods for solving the dual problem, when the original nonlinear programming problem is convex. In this paper a numerical comparison between many of these methods is performed using all the suitable problems of the CUTE collection.
Similar content being viewed by others
References
R.R. Allran and S.E.J. Johnsen, “An algorithm for solving nonlinear programming problems subject to nonlinear inequality constraints,” The Computer Journal, vol. 13, pp. 171–177, 1970.
A. Auslender, M. Teboulle, and S. Ben-Tiba, “Interior proximal and multiplier methods based on second order homogeneous kernels,” Mathematics of Operations Research, vol. 24, pp. 645–668, 1999.
A. Ben Tal, I. Yuzefovich, and M. Zibulevsky, “Penalty/barrier multiplier methods for minimax and constrained smooth convex programs,” Research Report 9/92, Optimization Laboratory, Faculty of Industrial Engineering Management, Technion, Haifa, Israel, 1992.
A. Ben-Tal and M. Zibulevsky, “Penalty/barrier multiplier methods for convex programming problems,” SIAM Journal on Optimization, vol. 7, pp. 347–366, 1997.
D.P. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods, Academic Press: New York, 1982.
D.P. Bertsekas, Nonlinear Programming, 2nd edition, Athena Scientific: Belmont, MA, 1999.
E.G. Birgin and J.M. Martínez, “Large-scale active-set box-constrained optimization method with spectral projected gradients,” Computational Optimization and Applications, vol. 23, pp. 101–125, 2002.
E.G. Birgin, J.M. Martínez, and M. Raydan, “Nonmonotone spectral projected gradient methods on convex sets,” SIAM Journal on Optimization, vol. 10, pp. 1196–1211, 2000.
E.G. Birgin, J.M. Martínez, and M. Raydan, “Algorithm 813: SPG—Software for convex-constrained optimization,” ACM Transactions on Mathematical Software, vol. 27, pp. 340–349, 2001.
R.A. Castillo, “Métodos de Lagrangiano Aumentado usando penalidades general-izadas para programação não linear,” Tese de Doutorado, COPPE, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brasil, 1998.
A.R. Conn, N.I.M. Gould, and Ph.L. Toint, “A globally convergent Augmented Lagrangian algorithm for optimization with general constraints and simple bounds,” SIAM Journal on Numerical Analysis, vol. 28, pp. 545–572, 1991.
E.D. Dolan and J.J. Moré, “Benchmarking optimization software with performance profiles,” Mathematical Programming, vol. 91, pp. 201–213, 2002.
Z. Dostál, A. Friedlander, and S.A. Santos, “Augmented Lagrangians with adaptive precision control for quadratic programming with equality constraints,” Computational Optimization and Applications, vol. 14, pp. 37–53, 1999.
Z. Dostál, A. Friedlander, and S.A. Santos, “Augmented Lagrangians with adaptive precision control for quadratic programming with simple bounds and equality constraints,” SIAM Journal on Optimization, vol. 13, pp. 1120–1140, 2003.
Z. Dostál, F.A.M. Gomes, and S.A. Santos, “Solution of contact problems by FETI domain decomposition with natural coarse space projection,” Computational Method in Applied Mechanics and Engineering, vol. 190, pp. 1611–1627, 2002.
R. Fletcher, Practical Methods for Optimization, John Wiley and Sons: Chichester, 1987.
C.C. Gonzaga and R.A. Castillo, “A nonlinear programming algorithm based on non-coercive penalty functions,” Mathematical Programming, vol. 96, pp. 87–101, 2003.
W.W. Hager, “Dual techniques for constrained optimization,” Journal of Optimization Theory and Applications, vol. 55, pp. 33–72, 1987.
W.W. Hager, “Analysis and implementation of a dual algorithm for constrained optimization,” Journal of Optimization Theory and Applications, vol. 79, pp. 427–462, 1993.
M.R. Hestenes, “Multiplier and gradient methods,” Journal of Optimization Theory and Applications, vol 4, pp. 303–320, 1969.
C. Humes and P.S. Silva, “Strict convex regularizations, proximal points and Augmented Lagrangians,” RAIRO Operations Research, vol. 34, pp. 283–303, 2000.
A.N. Iusem, “Augmented Lagrangian methods and proximal point methods for convex optimization,” Investigacion Operativa, vol. 8, pp. 11–50, 1999.
K.C. Kiwiel, “Proximal minimization methods with generalized Bregman functions,” SIAM Journal on Control and Optimization, vol. 35, pp. 1142–1168, 1997.
B.W. Kort and D.P. Bertsekas, “Multiplier methods for convex programming,” in Proceeding of the IEEE Decision and Control Conference, San Diego, CA, 1973, pp. 260–264.
B.W. Kort and D.P. Bertsekas, “Combined primal-dual and penalty methods for convex programming,” SIAM Journal on Control and Optimization, vol. 14, pp. 268–294, 1976.
O.L. Mangasarian and S. Fromovitz, “The Fritz-John necessary optimality conditons in presence of equality and inequality constraints,” Journal of Mathematical Analysis and Applications, vol. 17, pp. 37–47, 1967.
L.C. Matioli, “Uma nova metodologia para construção de funções de penalização para algoritoms de Lagrangiano Aumentado,” Tese de Doutorado, Universidade Federal de Santa Catarina, Florianópolis, Brasil, 2001.
F.H. Murphy, “A class of exponential penalty functions,” SIAM Journal on Control, vol. 12, pp. 679–687, 1974.
H. Nakayama, H. Samaya, and Y. Sawaragi, “A generalized Lagrangian function and multiplier method,” Journal of Optimization Theory and Applications, vol. 17, pp. 211–227, 1975.
J. Nocedal and S.J. Wright, Numerical Optimization, Springer Verlag: New York, 1999.
R.A. Polyak, “Log-sigmoid multiplier method in constrained optimization,” Annals of Operation Research, vol. 101, pp. 427–460, 2001.
R. Polyak and M. Teboulle, “Nonliner rescaling and proximal-like methods in convex optimization,” Mathematical Programming, vol. 76, pp. 265–284, 1997.
M.J.D. Powell, “A method for nonlinear constraints in minimization problems,” in Optimization, R. Fletcher (Ed.), Academic Press, New York, NY, 1969, pp. 283–298.
R.T. Rockafellar, “The multiplier method of Hestenes and Powell applied to convex programming,” Journal of Optimization Theory and Applications, vol. 12, pp. 555–562, 1973.
P. Tseng and D. Bertseaks, “On the convergence of the exponential multiplier method for convex programming,” Mathematical Programming, vol. 17, pp. 670–690, 1993.
A.E. Xavier, “Penalização hiperbolica,” Tese de Doutorado, COPPE, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brasil, 1992.
Author information
Authors and Affiliations
Corresponding author
Additional information
This author was supported by ProNEx MCT/CNPq/FAPERJ 171.164/2003, FAPESP (Grants 2001/04597-4 and 2002/00094-0 and 2003/09169-6) and CNPq (Grant 302266/2002-0).
This author was partially supported by CNPq-Brasil and CDCHT-Venezuela.
This author was supported by ProNEx MCT/CNPq/FAPERJ 171.164/2003, FAPESP (Grant 2001/04597-4) and CNPq.
Rights and permissions
About this article
Cite this article
Birgin, E.G., Castillo, R.A. & MartÍnez, J.M. Numerical Comparison of Augmented Lagrangian Algorithms for Nonconvex Problems. Comput Optim Applic 31, 31–55 (2005). https://doi.org/10.1007/s10589-005-1066-7
Received:
Revised:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s10589-005-1066-7