Skip to main content
SpringerLink
Log in

Augmented Lagrangians with constrained subproblems and convergence to second-order stationary points

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

Augmented Lagrangian methods with convergence to second-order stationary points in which any constraint can be penalized or carried out to the subproblems are considered in this work. The resolution of each subproblem can be done by any numerical algorithm able to return approximate second-order stationary points. The developed global convergence theory is stronger than the ones known for current algorithms with convergence to second-order points in the sense that, besides the flexibility introduced by the general lower-level approach, it includes a loose requirement for the resolution of subproblems. The proposed approach relies on a weak constraint qualification, that allows Lagrange multipliers to be unbounded at the solution. It is also shown that second-order resolution of subproblems increases the chances of finding a feasible point, in the sense that limit points are second-order stationary for the problem of minimizing the squared infeasibility. The applicability of the proposed method is illustrated in numerical examples with ball-constrained subproblems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Agarwal, N., Allen-Zhu, Z., Bullins, B., Hazan, E., Ma T.: Finding local minima for nonconvex optimization in linear time. arXiv:1611.01146

  2. Anandkumar, A., Ge, R.: Efficient approaches for escaping higher order saddle points in non-convex optimization. arXiv:1602.05908v1

  3. Andreani, R., Behling, R., Haeser, G., Silva, P.J.S.: On second order optimality conditions for nonlinear optimization. Optim. Methods Softw. 32, 22–38 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  4. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: Augmented Lagrangian methods under the constant positive linear dependence constraint qualification. Math. Program. 112, 5–32 (2008)

    MathSciNet  MATH  Google Scholar 

  5. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: On Augmented Lagrangian methods with general lower-level constraints. SIAM J. Optim. 18, 1286–1309 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  6. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: Second-order negative-curvature methods for box-constrained and general constrained optimization. Comput. Optim. Appl. 45, 209–236 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  7. Andreani, R., Haeser, G., Martínez, J.M.: On sequential optimality conditions for smooth constrained optimization. Optimization 60, 627–641 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  8. Andreani, R., Haeser, G., Ramos, A., Silva, P.J.S.: A second-order sequential optimality condition associated to the convergence of optimization algorithms. IMA J. Numer. Anal. (2017). doi:10.1093/imanum/drw064

    MathSciNet  Google Scholar 

  9. Andreani, R., Haeser, G., Schuverdt, M.L., Silva, P.J.S.: A relaxed constant positive linear dependence constraint qualification and applications. Math. Program. 135, 255–273 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  10. Andreani, R., Haeser, G., Schuverdt, M.L., Silva, P.J.S.: Two new weak constraint qualifications and applications. SIAM J. Optim. 22, 1109–1135 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  11. Andreani, R., Martínez, J.M., Ramos, A., Silva, P.J.S.: A cone-continuity constraint qualification and algorithmic consequences. SIAM J. Optim. 26, 96–110 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  12. Andreani, R., Martínez, J.M., Schuverdt, M.L.: On second-order optimality conditions for nonlinear programming. Optimization 56, 529–542 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  13. Andreani, R., Martínez, J.M., Svaiter, B.F.: A new sequential optimality condition for constrained optimization and algorithmic consequences. SIAM J. Optim. 20, 3533–3554 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Avelino, C.P., Moguerza, J.M., Olivares, A., Prieto, F.J.: Combining and scaling descent and negative curvature directions. Math. Program. 128, 285–319 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  15. Baccari, A., Trad, A.: On the classical necessary second-order optimality conditions in the presence of equality and inequality constraints. SIAM J. Optim. 15, 394–408 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  16. Behling, R., Haeser, G., Ramos, A., Viana, D.S.: On a conjecture in second-order optimality conditions. Available at Optimization Online

  17. Bertsekas, D.P.: Constrained Optimization and Lagrange Multiplier Methods. Athena Scientific, Belmont (1996)

    MATH  Google Scholar 

  18. Bertsekas, D.P.: Nonlinear programming, 2d edn. Athenas Scientific, Belmont (1999)

    MATH  Google Scholar 

  19. Birgin, E.G., Bueno, L.F., Martínez, J.M.: Sequential equality-constrained optimization for nonlinear programming. Comput. Optim. Appl. 65, 699–721 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  20. Birgin, E.G., Castelani, E.V., Martinez, A.L.M., Martínez, J.M.: Outer trust-region method for constrained optimization. J. Optim. Theory Appl. 150, 142–155 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  21. Birgin, E.G., Fernandez, D., Martínez, J.M.: On the boundedness of penalty parameters in an Augmented Lagrangian method with lower level constraints. Optim. Methods Softw. 27, 1001–1024 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  22. Birgin, E.G., Floudas, C.A., Martínez, J.M.: Global minimization using an Augmented Lagrangian method with variable lower-level constraints. Math. Program. 125, 139–162 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  23. Birgin, E.G., Martínez, J.M.: The use of quadratic regularization with a cubic descent condition for unconstrained optimization. SIAM J. Optim. 27, 1049–1074 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  24. Birgin, E.G., Martínez, J.M.: Practical Augmented Lagrangian Methods for Constrained Optimization, Fundamentals of Algorithms, vol. 10. Society for Industrial and Applied Mathematics, Philadelphia (2014). doi:10.1137/1.9781611973365

  25. Birgin, E.G., Martínez, J.M., Prudente, L.F.: Augmented Lagrangians with possible infeasibility and finite termination for global nonlinear programming. J. Glob. Optim. 58, 207–242 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  26. Birgin, E.G., Martínez, J.M., Prudente, L.F.: Optimality properties of an Augmented Lagrangian method on infeasible problems. Comput. Optim. Appl. 60, 609–631 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  27. Birgin, E.G., Martínez, J.M., Raydan, M.: Algorithm 813: SPG–software for convex-constrained optimization. ACM Trans. Math. Softw. 27, 340–349 (2001)

    Article  MATH  Google Scholar 

  28. Birgin, E.G., Martínez, J.M., Ronconi, D.P.: Optimizing the packing of cylinders into a rectangular container: a nonlinear approach. Eur. J. Oper. Res. 160, 19–33 (2005)

    Article  MATH  Google Scholar 

  29. Bonnans, J.F., Shapiro, A.: Pertubation Analysis of Optimization Problems. Springer, New York (2000)

    Book  MATH  Google Scholar 

  30. Carmon, Y., Duchi, J.C., Hinder, O., Sidford, A.: Accelerated methods for nonconvex optimization. ArXiv:1611.00756

  31. Castelani, E.V., Martinez, A.L., Martínez, J.M., Svaiter, B.F.: Addressing the greediness phenomenon in nonlinear programming by means of proximal Augmented Lagrangians. Comput. Optim. Appl. 46, 229–245 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  32. Conn, A.R., Gould, N.I.M.: Toint, PhL: A globally convergent Augmented Lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J. Numer. Anal. 28, 545–572 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  33. Conn, A.R., Gould, N.I.M.: Toint, PhL: Lancelot: A Fortran Package for Large-Scale Nonlinear Optimization (Release A). Springer, Berlin (1992)

    MATH  Google Scholar 

  34. Debreu, G.: Definite and semidefinite quadratic forms. Econometrica 20, 295–300 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  35. Dempe, S.: Foundations of Bilevel Programming. Kluwer Academic Publishers, Dordrecht (2002)

    MATH  Google Scholar 

  36. Eckstein, J., Silva, P.J.S.: A practical relative error criterion for Augmented Lagrangians. Math. Program. 141, 319–348 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  37. Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20, 303–353 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  38. Facchinei, F., Lucidi, S.: Convergence to second order stationary points in inequality constrained optimization. Math. Oper. Res. 23, 746–766 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  39. Gill, P.E., Kungurtsev, V., Robinson, D.P.: A stabilized SQP method: global convergence. IMA J. Numer. Anal. 37(1), 407–443 (2017). doi:10.1093/imanum/drw004

    Article  MathSciNet  MATH  Google Scholar 

  40. Gould, N.I.M., Conn, A.R.: Toint, PhL: A note on the convergence of barrier algorithms for second-order necessary points. Math. Program. 85, 433–438 (1998)

    Article  Google Scholar 

  41. Gould, N.I.M., Orban, D.: Toint, PhL: CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60, 545–557 (2014)

    Article  MATH  Google Scholar 

  42. Haeser, G.: A second-order optimality condition with first- and second-order complementarity associated with global convergence of algorithms. Available at Optimization Online

  43. Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  44. Janin, R.: Direction derivative of the marginal function in nonlinear programming. Math. Program. Stud. 21, 127–138 (1984)

    Article  MATH  Google Scholar 

  45. Kanzow, C., Steck, D.: An Example Comparing the Standard and Modified Augmented Lagrangian Methods, Preprint, Institute of Mathematics, University of Würzburg, Würzburg, Feb 2017. http://www.mathematik.uni-wuerzburg.de/~kanzow/paper/ExampleALM.pdf

  46. Karas, E.W., Santos, S.A., Svaiter, B.F.: Algebraic rules for quadratic regularization of Newton’s method. Comput. Optim. Appl. 60, 343–376 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  47. Lehoucq, R.B., Sorensen, D.C., Yang, C.: ARPACK Users’ Guide: Solution of Large-Scale Eigenvalue Problems with Implicitly Restarted Arnoldi Methods, Software, Environments and Tools, vol. 6. Society for Industrial and Applied Mathematics, Philadelphia (1998)

    Book  MATH  Google Scholar 

  48. Liu, H., Yao, T., Li, R., Ye, Y.: Folded concave penalized sparse linear regression: sparsity, statistical performance and algorithmic theory for local solutions. Math. Program. (2017). doi:10.1007/s10107-017-1114-y

    MathSciNet  MATH  Google Scholar 

  49. Mangasarian, O.L., Fromovitz, S.: The Fritz-John necessary conditions in presence of equality and inequality constraints. J. Math. Anal. Appl. 17, 37–47 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  50. Martínez, J.M., Santos, S.A.: A trust-region strategy for minimization on arbitrary domains. Math. Program. 68, 267–301 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  51. Martínez, J.M., Santos, S.A.: New convergence results on an algorithm for norm constrained regularization and related problems. RAIRO-Oper. Res. 31, 269–294 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  52. Martínez, L., Andrade, R., Birgin, E.G., Martínez, J.M.: Packmol: a package for building initial configurations for molecular dynamics simulations. J. Comput. Chem. 30, 2157–2164 (2009)

    Article  Google Scholar 

  53. Martínez, L., Martínez, J.M.: Packing optimization for automated generation of complex system’s initial configurations for molecular dynamics and docking. J. Comput. Chem. 24, 819–825 (2003)

    Article  Google Scholar 

  54. Minchenko, L., Stakhovski, S.: On relaxed constant rank regularity condition in mathematical programming. Optimization 60, 429–440 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  55. Moguerza, J.M., Prieto, F.J.: An Augmented Lagrangian interior-point method using directions of negative curvature. Math. Program. 95, 573–616 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  56. Murty, K.G., Kabadi, S.N.: Some NP-complete problems in quadratic and nonlinear programming. Math. Program. 39, 117–129 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  57. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (2006)

    MATH  Google Scholar 

  58. Nurmela, K.J., Östergåd, P.R.J.: Packing up to 50 equal circles in a square. Discrete Comput. Geom. 18, 111–120 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  59. Powell, M.J.D.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, New York (1969)

    Google Scholar 

  60. Qi, L., Wei, Z.: On the constant positive linear dependence conditions and its application to SQP methods. SIAM J. Optim. 10, 963–981 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  61. Rockafellar, R.T.: Augmented Lagrange multiplier functions and duality in nonconvex programming. SIAM J. Control Optim. 12, 268–285 (1974)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Institute of Mathematics and Statistics, University of São Paulo, Rua do Matão, 1010, Cidade Universitária, São Paulo, SP, 05508-090, Brazil

    E. G. Birgin

  2. Department of Applied Mathematics, Institute of Mathematics and Statistics, University of São Paulo, Rua do Matão, 1010, Cidade Universitária, São Paulo, SP, 05508-090, Brazil

    G. Haeser

  3. Department of Management Science and Engineering, Stanford University, Stanford, CA, USA

    G. Haeser

  4. Department of Mathematics, Federal University of Paraná, Curitiba, PR, Brazil

    A. Ramos

Authors
  1. E. G. Birgin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. Haeser
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Ramos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. G. Birgin.

Additional information

This work has been partially supported by FAPESP (Grants 2013/03447-6, 2013/05475-7, 2013/07375-0, 2016/01860-1, and 2016/02092-8) and CNPq (Grants 481992/2013-8, 309517/2014-1, and 303264/2015-2).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Birgin, E.G., Haeser, G. & Ramos, A. Augmented Lagrangians with constrained subproblems and convergence to second-order stationary points. Comput Optim Appl 69, 51–75 (2018). https://doi.org/10.1007/s10589-017-9937-2

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-017-9937-2

Keywords

Search

Navigation