Computational Optimization and Applications

, Volume 69, Issue 1, pp 51–75 | Cite as

Augmented Lagrangians with constrained subproblems and convergence to second-order stationary points

  • E. G. Birgin
  • G. Haeser
  • A. Ramos


Augmented Lagrangian methods with convergence to second-order stationary points in which any constraint can be penalized or carried out to the subproblems are considered in this work. The resolution of each subproblem can be done by any numerical algorithm able to return approximate second-order stationary points. The developed global convergence theory is stronger than the ones known for current algorithms with convergence to second-order points in the sense that, besides the flexibility introduced by the general lower-level approach, it includes a loose requirement for the resolution of subproblems. The proposed approach relies on a weak constraint qualification, that allows Lagrange multipliers to be unbounded at the solution. It is also shown that second-order resolution of subproblems increases the chances of finding a feasible point, in the sense that limit points are second-order stationary for the problem of minimizing the squared infeasibility. The applicability of the proposed method is illustrated in numerical examples with ball-constrained subproblems.


Augmented Lagrangian methods Nonlinear programming Second-order stationary points Algorithms 


  1. 1.
    Agarwal, N., Allen-Zhu, Z., Bullins, B., Hazan, E., Ma T.: Finding local minima for nonconvex optimization in linear time. arXiv:1611.01146
  2. 2.
    Anandkumar, A., Ge, R.: Efficient approaches for escaping higher order saddle points in non-convex optimization. arXiv:1602.05908v1
  3. 3.
    Andreani, R., Behling, R., Haeser, G., Silva, P.J.S.: On second order optimality conditions for nonlinear optimization. Optim. Methods Softw. 32, 22–38 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: Augmented Lagrangian methods under the constant positive linear dependence constraint qualification. Math. Program. 112, 5–32 (2008)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: On Augmented Lagrangian methods with general lower-level constraints. SIAM J. Optim. 18, 1286–1309 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: Second-order negative-curvature methods for box-constrained and general constrained optimization. Comput. Optim. Appl. 45, 209–236 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Andreani, R., Haeser, G., Martínez, J.M.: On sequential optimality conditions for smooth constrained optimization. Optimization 60, 627–641 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Andreani, R., Haeser, G., Ramos, A., Silva, P.J.S.: A second-order sequential optimality condition associated to the convergence of optimization algorithms. IMA J. Numer. Anal. (2017). doi: 10.1093/imanum/drw064 MathSciNetGoogle Scholar
  9. 9.
    Andreani, R., Haeser, G., Schuverdt, M.L., Silva, P.J.S.: A relaxed constant positive linear dependence constraint qualification and applications. Math. Program. 135, 255–273 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Andreani, R., Haeser, G., Schuverdt, M.L., Silva, P.J.S.: Two new weak constraint qualifications and applications. SIAM J. Optim. 22, 1109–1135 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Andreani, R., Martínez, J.M., Ramos, A., Silva, P.J.S.: A cone-continuity constraint qualification and algorithmic consequences. SIAM J. Optim. 26, 96–110 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Andreani, R., Martínez, J.M., Schuverdt, M.L.: On second-order optimality conditions for nonlinear programming. Optimization 56, 529–542 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Andreani, R., Martínez, J.M., Svaiter, B.F.: A new sequential optimality condition for constrained optimization and algorithmic consequences. SIAM J. Optim. 20, 3533–3554 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Avelino, C.P., Moguerza, J.M., Olivares, A., Prieto, F.J.: Combining and scaling descent and negative curvature directions. Math. Program. 128, 285–319 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Baccari, A., Trad, A.: On the classical necessary second-order optimality conditions in the presence of equality and inequality constraints. SIAM J. Optim. 15, 394–408 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Behling, R., Haeser, G., Ramos, A., Viana, D.S.: On a conjecture in second-order optimality conditions. Available at Optimization OnlineGoogle Scholar
  17. 17.
    Bertsekas, D.P.: Constrained Optimization and Lagrange Multiplier Methods. Athena Scientific, Belmont (1996)zbMATHGoogle Scholar
  18. 18.
    Bertsekas, D.P.: Nonlinear programming, 2d edn. Athenas Scientific, Belmont (1999)zbMATHGoogle Scholar
  19. 19.
    Birgin, E.G., Bueno, L.F., Martínez, J.M.: Sequential equality-constrained optimization for nonlinear programming. Comput. Optim. Appl. 65, 699–721 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Birgin, E.G., Castelani, E.V., Martinez, A.L.M., Martínez, J.M.: Outer trust-region method for constrained optimization. J. Optim. Theory Appl. 150, 142–155 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Birgin, E.G., Fernandez, D., Martínez, J.M.: On the boundedness of penalty parameters in an Augmented Lagrangian method with lower level constraints. Optim. Methods Softw. 27, 1001–1024 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Birgin, E.G., Floudas, C.A., Martínez, J.M.: Global minimization using an Augmented Lagrangian method with variable lower-level constraints. Math. Program. 125, 139–162 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Birgin, E.G., Martínez, J.M.: The use of quadratic regularization with a cubic descent condition for unconstrained optimization. SIAM J. Optim. 27, 1049–1074 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Birgin, E.G., Martínez, J.M.: Practical Augmented Lagrangian Methods for Constrained Optimization, Fundamentals of Algorithms, vol. 10. Society for Industrial and Applied Mathematics, Philadelphia (2014). doi: 10.1137/1.9781611973365
  25. 25.
    Birgin, E.G., Martínez, J.M., Prudente, L.F.: Augmented Lagrangians with possible infeasibility and finite termination for global nonlinear programming. J. Glob. Optim. 58, 207–242 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Birgin, E.G., Martínez, J.M., Prudente, L.F.: Optimality properties of an Augmented Lagrangian method on infeasible problems. Comput. Optim. Appl. 60, 609–631 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Birgin, E.G., Martínez, J.M., Raydan, M.: Algorithm 813: SPG–software for convex-constrained optimization. ACM Trans. Math. Softw. 27, 340–349 (2001)CrossRefzbMATHGoogle Scholar
  28. 28.
    Birgin, E.G., Martínez, J.M., Ronconi, D.P.: Optimizing the packing of cylinders into a rectangular container: a nonlinear approach. Eur. J. Oper. Res. 160, 19–33 (2005)CrossRefzbMATHGoogle Scholar
  29. 29.
    Bonnans, J.F., Shapiro, A.: Pertubation Analysis of Optimization Problems. Springer, New York (2000)CrossRefzbMATHGoogle Scholar
  30. 30.
    Carmon, Y., Duchi, J.C., Hinder, O., Sidford, A.: Accelerated methods for nonconvex optimization. ArXiv:1611.00756
  31. 31.
    Castelani, E.V., Martinez, A.L., Martínez, J.M., Svaiter, B.F.: Addressing the greediness phenomenon in nonlinear programming by means of proximal Augmented Lagrangians. Comput. Optim. Appl. 46, 229–245 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Conn, A.R., Gould, N.I.M.: Toint, PhL: A globally convergent Augmented Lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J. Numer. Anal. 28, 545–572 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Conn, A.R., Gould, N.I.M.: Toint, PhL: Lancelot: A Fortran Package for Large-Scale Nonlinear Optimization (Release A). Springer, Berlin (1992)zbMATHGoogle Scholar
  34. 34.
    Debreu, G.: Definite and semidefinite quadratic forms. Econometrica 20, 295–300 (1952)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Dempe, S.: Foundations of Bilevel Programming. Kluwer Academic Publishers, Dordrecht (2002)zbMATHGoogle Scholar
  36. 36.
    Eckstein, J., Silva, P.J.S.: A practical relative error criterion for Augmented Lagrangians. Math. Program. 141, 319–348 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20, 303–353 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Facchinei, F., Lucidi, S.: Convergence to second order stationary points in inequality constrained optimization. Math. Oper. Res. 23, 746–766 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Gill, P.E., Kungurtsev, V., Robinson, D.P.: A stabilized SQP method: global convergence. IMA J. Numer. Anal. 37(1), 407–443 (2017). doi: 10.1093/imanum/drw004 MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Gould, N.I.M., Conn, A.R.: Toint, PhL: A note on the convergence of barrier algorithms for second-order necessary points. Math. Program. 85, 433–438 (1998)CrossRefGoogle Scholar
  41. 41.
    Gould, N.I.M., Orban, D.: Toint, PhL: CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60, 545–557 (2014)CrossRefzbMATHGoogle Scholar
  42. 42.
    Haeser, G.: A second-order optimality condition with first- and second-order complementarity associated with global convergence of algorithms. Available at Optimization OnlineGoogle Scholar
  43. 43.
    Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Janin, R.: Direction derivative of the marginal function in nonlinear programming. Math. Program. Stud. 21, 127–138 (1984)CrossRefzbMATHGoogle Scholar
  45. 45.
    Kanzow, C., Steck, D.: An Example Comparing the Standard and Modified Augmented Lagrangian Methods, Preprint, Institute of Mathematics, University of Würzburg, Würzburg, Feb 2017.
  46. 46.
    Karas, E.W., Santos, S.A., Svaiter, B.F.: Algebraic rules for quadratic regularization of Newton’s method. Comput. Optim. Appl. 60, 343–376 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Lehoucq, R.B., Sorensen, D.C., Yang, C.: ARPACK Users’ Guide: Solution of Large-Scale Eigenvalue Problems with Implicitly Restarted Arnoldi Methods, Software, Environments and Tools, vol. 6. Society for Industrial and Applied Mathematics, Philadelphia (1998)CrossRefzbMATHGoogle Scholar
  48. 48.
    Liu, H., Yao, T., Li, R., Ye, Y.: Folded concave penalized sparse linear regression: sparsity, statistical performance and algorithmic theory for local solutions. Math. Program. (2017). doi: 10.1007/s10107-017-1114-y MathSciNetzbMATHGoogle Scholar
  49. 49.
    Mangasarian, O.L., Fromovitz, S.: The Fritz-John necessary conditions in presence of equality and inequality constraints. J. Math. Anal. Appl. 17, 37–47 (1967)MathSciNetCrossRefzbMATHGoogle Scholar
  50. 50.
    Martínez, J.M., Santos, S.A.: A trust-region strategy for minimization on arbitrary domains. Math. Program. 68, 267–301 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  51. 51.
    Martínez, J.M., Santos, S.A.: New convergence results on an algorithm for norm constrained regularization and related problems. RAIRO-Oper. Res. 31, 269–294 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  52. 52.
    Martínez, L., Andrade, R., Birgin, E.G., Martínez, J.M.: Packmol: a package for building initial configurations for molecular dynamics simulations. J. Comput. Chem. 30, 2157–2164 (2009)CrossRefGoogle Scholar
  53. 53.
    Martínez, L., Martínez, J.M.: Packing optimization for automated generation of complex system’s initial configurations for molecular dynamics and docking. J. Comput. Chem. 24, 819–825 (2003)CrossRefGoogle Scholar
  54. 54.
    Minchenko, L., Stakhovski, S.: On relaxed constant rank regularity condition in mathematical programming. Optimization 60, 429–440 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  55. 55.
    Moguerza, J.M., Prieto, F.J.: An Augmented Lagrangian interior-point method using directions of negative curvature. Math. Program. 95, 573–616 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  56. 56.
    Murty, K.G., Kabadi, S.N.: Some NP-complete problems in quadratic and nonlinear programming. Math. Program. 39, 117–129 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  57. 57.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (2006)zbMATHGoogle Scholar
  58. 58.
    Nurmela, K.J., Östergåd, P.R.J.: Packing up to 50 equal circles in a square. Discrete Comput. Geom. 18, 111–120 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  59. 59.
    Powell, M.J.D.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, New York (1969)Google Scholar
  60. 60.
    Qi, L., Wei, Z.: On the constant positive linear dependence conditions and its application to SQP methods. SIAM J. Optim. 10, 963–981 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  61. 61.
    Rockafellar, R.T.: Augmented Lagrange multiplier functions and duality in nonconvex programming. SIAM J. Control Optim. 12, 268–285 (1974)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Department of Computer Science, Institute of Mathematics and StatisticsUniversity of São PauloSão PauloBrazil
  2. 2.Department of Applied Mathematics, Institute of Mathematics and StatisticsUniversity of São PauloSão PauloBrazil
  3. 3.Department of Management Science and EngineeringStanford UniversityStanfordUSA
  4. 4.Department of MathematicsFederal University of ParanáCuritibaBrazil

Personalised recommendations