Augmented Lagrangian Method with Alternating Constraints for Nonlinear Optimization Problems

Abstract

The augmented Lagrangian method is a classical solution method for nonlinear optimization problems. At each iteration, it minimizes an augmented Lagrangian function that consists of the constraint functions and the corresponding Lagrange multipliers. If the Lagrange multipliers in the augmented Lagrangian function are close to the exact Lagrange multipliers at an optimal solution, the method converges steadily. Since the conventional augmented Lagrangian method uses inaccurate estimated Lagrange multipliers, it sometimes converges slowly. In this paper, we propose a novel augmented Lagrangian method that allows the augmented Lagrangian function and its minimization problem to have variable constraints at each iteration. This allowance enables the new method to get more accurate estimated Lagrange multipliers by exploiting Karush–Kuhn–Tucker points of the subproblems and consequently to converge more efficiently and steadily.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    Beck, A., Teboulle, M.: Mirror descent and nonlinear projected subgradient methods for convex optimization. Oper. Res. Lett. 31, 167–175 (2003)

    MathSciNet  Article  MATH  Google Scholar 

  2. 2.

    Tseng, P.: Approximation accuracy gradient methods, and error bound for structured convex optimization. Math. Program. 125, 263–295 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  3. 3.

    Bartlett, P., Collins, M., Taskar, B., McAllester, D.: Exponentiated gradient algorithms for large-margin structured classification. In: Advances in Neural Information Processing Systems, vol. 17, pp. 113–120 (2004)

  4. 4.

    Beck, A., Teboulle, M.: A fast iterative shrinkage–thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2, 183–202 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  5. 5.

    Bertsekas, D.: Nonlinear Programming. Athena Scientific, Athena (1999)

    Google Scholar 

  6. 6.

    Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    MathSciNet  Article  MATH  Google Scholar 

  7. 7.

    Powell, M.J.D.: A Method for Nonlinear Constraints in Minimization Problems. Academic, New York (1969)

    Google Scholar 

  8. 8.

    Liuzzi, G., Lucidi, S., Piccialli, V.: Exploiting derivative-free local searches in DIRECT-type algorithms for global optimization. Comput. Optim. Appl. 65, 449–475 (2016)

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Knowl. Discov. Data Min. 2, 121–167 (1998)

    Article  Google Scholar 

  10. 10.

    Cristianini, N., Shawe-Taylor, J.: Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge (2000)

    Google Scholar 

  11. 11.

    Sun, M., Aronson, J., Mckeown, P., Drinka, M.: A tabu search heuristic procedure for the fixed charge transportation problem. Eur. J. Oper. Res. 106, 441–456 (1998)

    Article  MATH  Google Scholar 

  12. 12.

    Conn, A.R., Gould, N., Sartenaer, A., Toint, P.H.L.: Convergence properties of an augmented Lagrangian algorithm for optimization with a combination of general equality and linear constraints. SIAM J. Optim. 6, 674–703 (1996)

    MathSciNet  Article  MATH  Google Scholar 

  13. 13.

    Ben-Tal, A., Zibulevski, M.: Penalty/Barrier multiplier methods for convex programming problems. J. Optim. 7, 347–366 (1997)

    MathSciNet  MATH  Google Scholar 

  14. 14.

    Ben-Tal, A., Margalit, T., Nemirovski, A.: The ordered subsets mirror descent optimization method with applications to tomography. J. Optim. 12, 79–108 (2001)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Siti Nor Habibah Binti Hassan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hassan, S.N.H.B., Niimi, T. & Yamashita, N. Augmented Lagrangian Method with Alternating Constraints for Nonlinear Optimization Problems. J Optim Theory Appl 181, 883–904 (2019). https://doi.org/10.1007/s10957-019-01488-w

Download citation

Keywords

  • Augmented Lagrangian functions
  • Gradient descent method
  • Large-scale problem
  • Nonlinear optimization

Mathematics Subject Classification

  • 26A16
  • 41A25
  • 47B36