Skip to main content
SpringerLink
Log in

An augmented Lagrangian method for nonconvex composite optimization problems with nonlinear constraints

  • Research Article
  • Published:
Optimization and Engineering Aims and scope Submit manuscript

Abstract

In this paper, we propose an augmented Lagrangian method with Backtracking Line Search for solving nonconvex composite optimization problems including both nonlinear equality and inequality constraints. In case the variable spaces are homogeneous, our setting yields a generic nonlinear mathematical programming model. When some variables belong to the real Hilbert space and others to the integer space, one obtains a nonconvex mixed-integer/-binary nonlinear programming model for which the nonconvexity is not limited to the integrality constraints. Together with the formal proof of its iteration complexity, the proposed algorithm is then numerically evaluated to solve a multi-constrained network design problem. Extensive numerical executions on a set of instances extracted from the SNDlib repository are then performed to study its behavior and performance as well as identify potential improvement of this method. Finally, analysis of the results and their comparison against those obtained when solving its convex relaxation using mixed-integer programming solvers are reported.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1

Similar content being viewed by others

Data Availability

The network topology datasets generated during and/or analysed during the current study are available in the SNDlib repository (Orlowski et al. 2010), http://sndlib.zib.de/problems.overview.action. The demand datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Notes

  1. Remember that, following the right scalar multiplication rule with \(\gamma > 0\), the proximal operator associated with the function \(\gamma f(\gamma ^{-1} x)\) is \(\gamma {\text {prox}}_{\gamma ^{-1} f}(\gamma ^{-1} x)\).

  2. \({\mathcal {N}}_C\) denotes the limiting (or Mordukhovich) normal cone; this cone is generally nonconvex; hence, it cannot be polar to any tangential approximation of C.

  3. The set of KKT multipliers (\(\lambda _1,\lambda _2\)) at \(u^{\dagger }\) is nonempty.

  4. If c(u) is a vector function, \(c(u)_+ {:}{=}\max [c(u), 0]\), where the maximum is taken componentwise.

References

  • Andreani R, Birgin EG, Martínez JM, Schuverdt ML (2008) Augmented Lagrangian methods under the constant positive linear dependence constraint qualification. Math Program Ser B 111:5–32

    Article  MathSciNet  MATH  Google Scholar 

  • Attouch H, Bolte J, Redont P, Soubeyran A (2010) Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka–Lojasiewicz inequality. Math Oper Res 35(2):438–457

    Article  MathSciNet  MATH  Google Scholar 

  • Attouch H, Bolte J, Svaiter BF (2013) Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss–Seidel methods. Math Program Ser A 137:91–129

    Article  MathSciNet  MATH  Google Scholar 

  • Bauschke HH, Combettes PL (2017) Convex analysis and monotone operator theory in Hilbert spaces, 2nd edn. Springer, New York

    Book  MATH  Google Scholar 

  • Beck A, Teboulle M (2012) Smoothing and first order methods: a unified framework. SIAM J Optim 22(2):557–580

    Article  MathSciNet  MATH  Google Scholar 

  • Becker S, Bobin J, Candès EJ (2011) NESTA: a fast and accurate first-order method for sparse recovery. SIAM J Imaging Sci 4(1):1–39

    Article  MathSciNet  MATH  Google Scholar 

  • Ben-Ameur W, Ouorou A (2006) Mathematical models of the delay constrained routing problem. Algorithmic Oper Res 1(2):94–103

    MathSciNet  MATH  Google Scholar 

  • Bolte J, Daniilidis A, Lewis AS (2007) The Lojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J Optim 17(4):1205–1223

    Article  MATH  Google Scholar 

  • Bolte J, Sabach S, Teboulle M (2018) Nonconvex Lagrangian-based optimization: monitoring schemes and global convergence. Math Oper Res 43(4):1210–1232

    Article  MathSciNet  MATH  Google Scholar 

  • Burke JV, Engle A (2018) Line search and trust-region methods for convex-composite optimization. arXiv:1806.05218

  • Fischer A (1992) A special Newton-type optimization method. Optimization 24(3–4):269–284

    Article  MathSciNet  MATH  Google Scholar 

  • Gabay D (1983) Applications of the method of multipliers to variational inequalities. In: Fortinand M, Glowinski R (eds) Augmented Lagrangian methods: applications to the solution of boundary-value problems, North-Holland, Amsterdam

  • Garrigos PG, Garrigos G, Peypouquet J (2015) Splitting methods with variable metric for Kurdyka–Łojasiewicz functions and general convergence rates. J Optim Theory Appl 165:874–900

    Article  MathSciNet  MATH  Google Scholar 

  • Giorgi G (2018) A guided tour in constraint qualifications for nonlinear programming under differentiability assumptions. In: DEM working paper series. ISSN: 2281-1346

  • Grapiglia GN, Yuan Y (2021) On the complexity of an augmented Lagrangian method for nonconvex optimization. IMA J Numer Anal 41(2):1546–1568

    Article  MathSciNet  MATH  Google Scholar 

  • Guo L, Ye JJ (2018) Necessary optimality conditions and exact penalization for non-Lipschitz nonlinear programs. Math Program Ser B 168:571–598

    Article  MathSciNet  MATH  Google Scholar 

  • Hestenes MR (1969) Multiplier and gradient methods. J Optim Theory Appl 4(5):303–320

    Article  MathSciNet  MATH  Google Scholar 

  • Hijazi HL, Bonami P, Ouorou A (2013) Robust delay-constrained routing in telecommunications. Ann Oper Res 206:163–181

    Article  MathSciNet  MATH  Google Scholar 

  • Khintchine AY (1932) Mathematical theory of a stationary queue. Mat Sb 39(4):73–84

    Google Scholar 

  • Li Z et al (2021) Rate-improved inexact augmented Lagrangian method for constrained nonconvex optimization. In: Proceedings of the 24th international conference on artificial intelligence and statistics (AISTATS) 2021, San Diego, California, USA. PMLR: Volume 130

  • Luenberger DG, Yase Y et al (2007) Linear and nonlinear programming, vol 2, 3rd edn. Springer, Berlin

    Google Scholar 

  • McCormick GP (1976) Computability of global solutions to factorable nonconvex programs: part I—convex underestimating problems. Math Program 10:147–175

    Article  MATH  Google Scholar 

  • Murray W, Ng K (2010) An algorithm for nonlinear optimization problems with binary variables. Comput Optim Appl 47(2):257–288

    Article  MathSciNet  MATH  Google Scholar 

  • Nesterov Y (2005) Smooth minimization of non-smooth functions. Math Program Ser A 103:127–152

    Article  MathSciNet  MATH  Google Scholar 

  • Orlowski, S Pioro M, Tomaszewski A, Wessaly R (2007) SNDlib 1.0—survivable network design library. In: Proceedings of the 3rd international network optimization conference (INOC 2007), Spa, Belgium. http://sndlib.zib.de. Extended version accepted in Networks, 2009

  • Orlowski S, Wessäly R, Pióro M, Tomaszewski A (2010) SNDlib 1.0–survivable network design library. Networks 55:276–286

    Article  Google Scholar 

  • Papadimitriou D, Fortz B, Gorgone E (2015) Lagrangian relaxation for the time-dependent combined network design and routing problem. In: Proceedings of 2015 IEEE international conference on communications (ICC), pp 6030–6036

  • Pollaczek F (1930) Uber eine Aufgabe der Wahrscheinlichkeitstheorie. Math Z 32:64–100

    Article  MathSciNet  MATH  Google Scholar 

  • Poss M, Raack C (2013) Affine recourse for the robust network design problem: between static and dynamic routing. Networks 61(2):180–198

    Article  MathSciNet  MATH  Google Scholar 

  • Powell MJD (1969) A method for non-linear constraints in minimization problems. In: Fletcher R (ed) Optimization. Academic Press, New York, pp 283–298

  • Qi L, Wei Z (2000) On the constant positive linear dependence condition and its application to SQP methods. SIAM J Optim 10:963–981

    Article  MathSciNet  MATH  Google Scholar 

  • Sahin FM, Eftekhari A, Alacaoglu A, Latorre F, Cevher V (2019) An inexact augmented Lagrangian framework for nonconvex optimization with nonlinear constraints. In: Proceedings of advances in neural information processing systems 32 (NeurIPS 2019), pp 13943–13955

  • Scholtes S (2001) Convergence properties of a regularization scheme for mathematical programs with complementarity constraints. SIAM J Optim 11(4):918–936

    Article  MathSciNet  MATH  Google Scholar 

  • Stein O (2004) On constraint qualifications in nonsmooth optimization. J Optim Theory Appl 121(3):647–671

    Article  MathSciNet  MATH  Google Scholar 

  • Valkonen T (2014) A primal-dual hybrid gradient method for nonlinear operators with applications to MRI. Inverse Prob 30(5):055012

    Article  MathSciNet  MATH  Google Scholar 

  • Xie Y, Wright SJ (2021) Complexity of proximal augmented Lagrangian for Nonconvex optimization with nonlinear equality constraints. J Sci Comput 86(38):1–30

    MathSciNet  MATH  Google Scholar 

  • Yuan G, Ghanem B (2017) An exact penalty method for binary optimization based on MPEC formulation. In: Proceedings of the thirty-first AAAI conference on artificial intelligence (AAAI-17), pp 2867–2875

Download references

Author information

Authors and Affiliations

  1. 3nLab & N &O Lab, Belgium Research Center (BeRC), Huawei, 3001, Leuven, Belgium

    Dimitri Papadimitriou

  2. N &O Lab, Belgium Research Center (BeRC), Huawei, 3001, Leuven, Belgium

    Bằng Công Vũ

Authors
  1. Dimitri Papadimitriou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bằng Công Vũ
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dimitri Papadimitriou.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Papadimitriou, D., Vũ, B.C. An augmented Lagrangian method for nonconvex composite optimization problems with nonlinear constraints. Optim Eng (2023). https://doi.org/10.1007/s11081-023-09867-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11081-023-09867-z

Keywords

Search

Navigation