Abstract
In this article we present a general theory of augmented Lagrangian functions for cone constrained optimization problems that allows one to study almost all known augmented Lagrangians for these problems within a unified framework. We develop a new general method for proving the existence of global saddle points of augmented Lagrangian functions, called the localization principle. The localization principle unifies, generalizes and sharpens most of the known results on the existence of global saddle points, and, in essence, reduces the problem of the existence of global saddle points to a local analysis of optimality conditions. With the use of the localization principle we obtain first necessary and sufficient conditions for the existence of a global saddle point of an augmented Lagrangian for cone constrained minimax problems via both second and first order optimality conditions. In the second part of the paper, we present a general approach to the construction of globally exact augmented Lagrangian functions. The general approach developed in this paper allowed us not only to sharpen most of the existing results on globally exact augmented Lagrangians, but also to construct first globally exact augmented Lagrangian functions for equality constrained optimization problems, for nonlinear second order cone programs and for nonlinear semidefinite programs. These globally exact augmented Lagrangians can be utilized in order to design new superlinearly (or even quadratically) convergent optimization methods for cone constrained optimization problems.
Similar content being viewed by others
References
Rubinov, A.M., Yang, X.Q.: Lagrange-Type Functions in Constrained Non-convex Optimization. Kluwer Academic Publishers, Dordrecht (2003)
Wang, C.Y., Yang, X.Q., Yang, X.M.: Unified nonlinear Lagrangian approach to duality and optimal paths. J. Optim. Theory Appl. 135, 85–100 (2007)
Giannessi, F.: On the theory of Lagrangian duality. Optim. Lett. 1, 9–20 (2007)
Li, J., Feng, S.Q., Zhang, Z.: A unified approach for constrained extremum problems: image space analysis. J. Optim. Theory Appl. 159, 69–92 (2013)
Zhu, S.K., Li, S.J.: Unified duality theory for constrained extremum problems. Part I: image space analysis. J. Optim. Theory Appl. 161, 738–762 (2014)
Zhu, S.K., Li, S.J.: Unified duality theory for constrained extremum problems. Part II: special duality schemes. J. Optim. Theory Appl. 161, 763–782 (2014)
Burachik, R.S., Iusem, A.N., Melo, J.G.: Duality and exact penalization for general augmented Lagrangians. J. Optim. Theory Appl. 147, 125–140 (2010)
Wang, C.Y., Yang, X.Q., Yang, X.M.: Nonlinear augmented Lagrangian and duality theory. Math. Oper. Res. 38, 740–760 (2012)
Wang, C., Liu, Q., Qu, B.: Global saddle points of nonlinear augmented Lagrangian functions. J. Glob. Optim. 68, 125–146 (2017)
Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer-Verlag, Berlin (1998)
Wang, C.Y., Li, D.: Unified theory of augmented Lagrangian methods for constrained global optimization. J. Glob. Optim. 44, 433–458 (2009)
Liu, Q., Yang, X.: Zero duality and saddle points of a class of augmented Lagrangian functions in constrained non-convex optimization. Optimization 57, 655–667 (2008)
Rockafellar, R.T.: Augmented Lagrange multiplier functions and duality in nonconvex programming. SIAM J. Control Optim. 12, 268–285 (1974)
Rockafellar, R.T.: Lagrange multipliers and optimality. SIAM Rev. 35, 183–238 (1993)
Birgin, E.G., Martinez, J.M.: Practical Augmented Lagrangian Methods for Constrained Optimization. SIAM, Philadelphia (2014)
Kiwiel, K.C.: On the twice differentiable cubic augmented Lagrangian. J. Optim. Theory Appl. 88, 233–236 (1996)
Mangasarian, O.L.: Unconstrained Lagrangians in nonlinear programming. SIAM J. Control 12, 772–791 (1975)
Wu, H.X., Luo, H.Z.: Saddle points of general augmented Lagrangians for constrained nonconvex optimization. J. Glob. Optim. 53, 683–697 (2012)
Bertsekas, D.P.: Constrained Optimization and Lagrange Multiplier Methods. Academic Press, New York (1982)
Tseng, P., Bertsekas, D.P.: On the convergence of the exponential multiplier method for convex programming. Math. Program. 60, 1–19 (1993)
Sun, X.L., Li, D., McKinnon, K.I.M.: On saddle points of augmented Lagrangians for constrained nonconvex optimization. SIAM J. Optim. 15, 1128–1146 (2005)
Polyak, R.A.: Log-Sigmoid multipliers method in constrained optimization. Ann. Oper. Res. 101, 427–460 (2001)
Polyak, R.A.: Nonlinear rescaling vs. smoothing technique in convex optimization. Math. Program. 92, 197–235 (2002)
Polyak, R.: Modified barrier functions: theory and methods. Math. Program. 54, 177–222 (1992)
Li, D., Sun, X.L.: Convexification and existence of saddle point in a pth-power reformulation for nonconvex constrained optimization. Nonlinear Anal. 47, 5611–5622 (2001)
Li, D.: Zero duality gap for a class of nonconvex optimization problems. J. Optim. Theory Appl. 85, 309–324 (1995)
Li, D.: Saddle-point generation in nonlinear nonconvex optimization. Nonlinear Anal. 30, 4339–4344 (1997)
Xu, Z.K.: Local saddle points and convexification for nonconvex optimization problems. J. Optim. Theory Appl. 94, 739–746 (1997)
Li, D., Sun, X.L.: Existence of a saddle point in nonconvex constrained optimization. J. Glob. Optim. 21, 39–50 (2001)
Wu, H.X., Luo, H.Z.: A note on the existence of saddle points of p-th power Lagrangian for constrained nonconvex optimization. Optimization 61, 1231–1245 (2012)
He, S., Wu, L., Meng, H.: A nonlinear Lagrangian for constrained optimization problems. J. Appl. Math. Comput. 38, 669–685 (2012)
Liu, Y.J., Zhang, L.W.: Convergence analysis of the augmented Lagrangian method for nonlinear second-order cone optimization problems. Nonlinear Anal. Theory Methods Appl. 67, 1359–1373 (2007)
Liu, Y.J., Zhang, L.W.: Convergence of the augmented Lagrangian method for nonlinear optimization problems over second-order cones. J. Optim. Theory Appl. 139, 557–575 (2008)
Zhou, J., Chen, J.S.: On the existence of saddle points for nonlinear second-order cone programming problems. J. Glob. Optim. 62, 459–480 (2015)
Sun, J., Zhang, L.W., Wu, Y.: Properties of the augmented Lagrangian in nonlinear semidefinite optimization. J. Optim. Theory Appl. 129, 437–456 (2006)
Sun, D., Sun, J., Zhang, L.: The rate of convergence of the augmented Lagrangian method for nonlinear semidefinite programming. Math. Program. 114, 349–391 (2008)
Zhao, X.Y., Sun, D., Toh, K.-C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737–1765 (2010)
Wen, Z., Goldfarb, D., Yin, W.: Alternating direction augmented Lagrangian methods for semidefinite programming. Math. Program. Comput. 2, 203–230 (2010)
Sun, J.: On methods for solving nonlinear semidefinite optimization problems. Numer. Algebra Control Optim. 1, 1–14 (2011)
Luo, H.Z., Wu, H.X., Chen, G.T.: On the convergence of augmented Lagrangian methods for nonlinear semidefinite programming. J. Glob. Optim. 54, 599–618 (2012)
Wu, H., Luo, H., Ding, X., Chen, G.: Global convergence of modified augmented Lagrangian methods for nonlinear semidefintie programming. Comput. Optim. Appl. 56, 531–558 (2013)
Wu, H.X., Luo, H.Z., Yang, J.F.: Nonlinear separation approach for the augmented Lagrangian in nonlinear semidefinite programming. J. Glob. Optim. 59, 695–727 (2014)
Yamashita, H., Yabe, H.: A survey of numerical methods for nonlinear semidefinite programming. J. Oper. Res. Soc. Jpn. 58, 24–60 (2015)
Rückmann, J.-J., Shapiro, A.: Augmented Lagrangians in semi-infinite programming. Math. Program. Ser. B. 116, 499–512 (2009)
Huy, N.Q., Kim, D.S.: Stability and augmented Lagrangian duality in nonconvex semi-infinite programming. Nonlinear Anal. 75, 163–176 (2012)
Son, T.Q., Kim, D.S., Tam, N.N.: Weak stability and strong duality of a class of nonconvex infinite programs via augmented Lagrangian. J. Glob. Optim. 53, 165–184 (2012)
Burachik, R.S., Yang, X.Q., Zhou, Y.Y.: Existence of augmented Lagrange multipliers for semi-infinite programming problems. J. Optim. Theory Appl. 173, 471–503 (2017)
Zhang, L., Gu, J., Xiao, X.: A class of nonlinear Lagrangians for nonconvex second order cone programming. Comput. Optim. Appl. 49, 61–99 (2011)
Stingl, M.: On the solution of nonlinear semidefinite programs by augmented Lagrangian methods. Ph.D. thesis, Institute of Applied Mathematics II, Friedrech-Alexander University of Erlangen-Nuremberg, Erlangen, Germany (2006)
Noll, D.: Local convergence of an augmented Lagrangian method for matrix inequality constrained programming. Optim. Methods Softw. 22, 777–802 (2007)
Li, Y., Zhang, L.: A new nonlinear Lagrangian method for nonconvex semidefinite programming. J. Appl. Anal. 15, 149–172 (2009)
Zhang, L., Li, Y., Wu, J.: Nonlinear rescaling Lagrangians for nonconvex semidefinite programming. Optim. 63, 899–920 (2014)
Luo, H., Wu, H., Liu, J.: On saddle points in semidefinite optimization via separation scheme. J. Optim. Theory Appl. 165, 113–150 (2015)
Shapiro, A., Sun, J.: Some properties of the augmented Lagrangian in cone constrained optimization. Math. Oper. Res. 29, 479–491 (2004)
Zhou, Y.Y., Zhou, J.C., Yang, X.Q.: Existence of augmented Lagrange multipliers for cone constrained optimization problems. J. Glob. Optim. 58, 243–260 (2014)
Liu, Q., Tang, W.M., Yang, X.M.: Properties of saddle points for generalized augmented Lagrangian. Math. Meth. Oper. Res. 69, 111–124 (2009)
Wang, C., Zhou, J., Xu, X.: Saddle points theory of two classes of augmented Lagrangians and its applications to generalized semi-infinite programming. Appl. Math. Optim. 59, 413–434 (2009)
Luo, H.Z., Mastroeni, G., Wu, H.X.: Separation approach for augmented Lagrangians in constrained nonconvex optimization. J. Optim. Theory Appl. 144, 275–290 (2010)
Zhou, J., Xiu, N., Wang, C.: Saddle point and exact penalty representation for generalized proximal Lagrangians. J. Glob. Optim. 56, 669–687 (2012)
Dolgopolik, M.V.: A unifying theory of exactness of linear penalty functions. Optimization 65, 1167–1202 (2016)
Dolgopolik, M.V.: A unifying theory of exactness of linear penalty functions II: parametric penalty functions. Optimization 66, 1577–1622 (2017)
Dolgopolik, M.V.: Existence of augmented Lagrange multipliers: reduction to exact penalty functions and localization principle. Math. Program. 166, 297–326 (2017)
Malozemov, V.N., Pevnyi, A.B.: Alternation properties of solutions of nonlinear minimax problems. Soviet Math. Dokl. 14(5), 1303–1306 (1973)
Daugavet, V.A., Malozemov, V.N.: Alternance properties of solutions of nonlinear minimax problems with nonconvex constraints. Soviet Math. Dokl. 16(6), 1474–1476 (1975)
Daugavet, V.A.: Alternance properties of the solutions of non-linear minimax problems with non-linear constraints. USSR Comput. Math. Math. Phys. 16(3), 236–241 (1976)
Daugavet, V.A., Malozemov, V.N.: Quadratic rate of convergence of a linearization method for solving discrete minimax problems. USSR Comput. Math. Math. Phys. 21(4), 19–28 (1981)
Demyanov, V.F., Malozemov, V.N.: Optimality conditions in terms of alternance: two approaches. J. Optim. Theory Appl. 162, 805–820 (2014)
Demyanov, V.F., Malozemov, V.N.: Alternance form of optimality conditions in the finite-dimensional space. In: Demyanov, V.F., Pardalos, P.M., Batsyn, M. (eds.) Constructive Nonsmooth Analysis and Related Topics, pp. 185–205. Springer, New York (2014)
Di Pillo, G., Grippo, L.: A new class of augmented Lagrangians in nonlinear programming. SIAM J. Control Optim. 17, 618–628 (1979)
Di Pillo, G., Grippo, L., Lampariello, F.: A method for solving equality constrained optimization problems by unconstrained minimization. In: Iracki, K., Malanowski, K., Walukiewicz, S. (eds.) Optimization techniques: proceedings of the 9th IFIP Conference on Optimization Techniques, pp. 96–105. Springer-Verlag, Berlin, Heidelberg (1980)
Di Pillo, G., Grippo, L.: A new augmented Lagrangian function for inequality constraints in nonlinear programming problems. J. Optim. Theory Appl. 36, 495–519 (1982)
Lucidi, S.: New results on a class of exact augmented Lagrangians. J. Optim. Theory Appl. 58, 259–282 (1988)
Di Pillo, G., Lucidi, S.: On exact augmented Lagrangian functions in nonlinear programming. In: Di Pillo, G., Giannessi, F. (eds.) Nonlinear Optimization and Applications, pp. 85–100. Plenum Press, New York (1996)
Di Pillo, G., Lucidi, S.: An augmented Lagrangian function with improved exactness properties. SIAM J. Optim. 12, 376–406 (2001)
Di Pillo, G., Liuzzi, G., Lucidi, S., Palagi, L.: Fruitful uses of smooth exact merit functions in constrained optimization. In: Di Pillo, G., Murli, A. (eds.) High Performance Algorithms and Software for Nonlinear Optimization, pp. 201–225. Kluwer Academic Publishers, Dordrecht (2003)
Di Pillo, G., Liuzzi, G., Lucidi, S., Palagi, L.: An exact augmented Lagrangian function for nonlinear programming with two-sided constraints. Comput. Optim. Appl. 25, 57–83 (2003)
Du, X., Zhang, L., Gao, Y.: A class of augmented Lagrangians for equality constraints in nonlinear programming problems. Appl. Math. Comput. 172, 644–663 (2006)
Du, X., Liang, Y., Zhang, L.: Further study on a class of augmented Lagrangians of Di Pillo and Grippo in nonlinear programming. J. Shanghai Univ. 10, 293–298 (2006)
Luo, H., Wu, H., Liu, J.: Some results on augmented Lagrangians in constrained global optimization via image space analysis. J. Optim. Theory Appl. 159, 360–385 (2013)
Di Pillo, G., Lucidi, S., Palagi, L.: An exact penalty-Lagrangian approach for a class of constrained optimization problems with bounded variables. Optimization 28, 129–148 (1993)
Di Pillo, G., Girppo, L., Lucidi, S.: A smooth method for the finite minimax problem. Math. Program. 60, 187–214 (1993)
Di Pillo, G., Lucidi, S., Palagi, L.: A truncated Newton method for constrained optimization. In: Di Pillo, G., Giannessi, F. (eds.) Nonlinear Optimization and Related Topics, pp. 79–103. Kluwer Academic Publishers, Dordrecht (2000)
Di Pillo, G., Lucidi, S., Palagi, L.: Convergence to second-order stationary points of a primal-dual algorithm model for nonlinear programming. Math. Oper. Res. 30, 897–915 (2005)
Di Pillo, G., Liuzzi, G., Lucidi, S., Palagi, L.: A truncated Newton method in an augmented Lagrangian framework for nonlinear programming. Comput. Optim. Appl. 45, 311–352 (2010)
Di Pillo, G., Luizzi, G., Lucidi, S.: An exact penalty-Lagrangian approach for large-scale nonlinear programming. Optimization 60, 223–252 (2011)
Fukuda, E.H., Lourenco, B.F.: Exact augmented Lagrangian functions for nonlinear semidefinite programming. arXiv: 1705.06551 (2017)
Huang, X.X., Yang, X.Q.: A unified augmented Lagrangian approach to duality and exact penalization. Math. Oper. Res. 28, 533–552 (2003)
Huang, X.X., Yang, X.Q.: Further study on augmented Lagrangian duality theory. J. Glob. Optim. 31, 193–210 (2005)
Chen, J.-X., Chen, X., Tseng, P.: Analysis of nonsmooth vector-valued functions associated with second-order cones. Math. Program. Ser. B. 101, 95–117 (2004)
Sun, D., Sun, J.: Löwner’s operator and spectral functions in Euclidean Jordan algebras. Math. Oper. Res. 33, 421–445 (2008)
Shapiro, A.: On differentiability of symmetric matrix valued functions. Technical report. School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia, USA (2002)
Chen, X., Qi, H., Tseng, P.: Analysis of nonsmooth symmetric-matrix-valued functions with applications to semidefinite complementarity problems. SIAM J. Optim. 13, 960–985 (2003)
Horn, R.A., Johnson, C.R.: Topics in Matrix Analysis. Cambridge University Press, Cambridge (1991)
Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer Science+Business Media, New York (2000)
Zhao, W., Zhang, J., Zhou, J.: Existence of local saddle points for a new augmented Lagrangian function. Math. Probl. Eng. 2010, 1–13 (2010)
Polak, E., Royset, J.O.: On the use of augmented Lagrangians in the solution of generalized semi-infinite min-max problems. Comput. Optim. Appl. 31, 173–192 (2005)
Dolgopolik, M.V.: A unified approach to the global exactness of penalty and augmented Lagrangian functions II: extended exactness. arXiv: 1710.01961 (2017)
Fukuda, E.H., Silva, P.J.S., Fukushima, M.: Differentiable exact penalty functions for nonlinear second-order cone programs. SIAM J. Optim. 22, 1607–1633 (2012)
Bonnans, J.F., Ramírez, C.H.: Perturbation analysis of second-order cone programming problems. Math. Program. 104, 205–227 (2005)
Pedregal, P.: Direct numerical algorithm for constrained variational problems. Numer. Funct. Anal. Optim. 38, 486–506 (2017)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dolgopolik, M.V. Augmented Lagrangian functions for cone constrained optimization: the existence of global saddle points and exact penalty property. J Glob Optim 71, 237–296 (2018). https://doi.org/10.1007/s10898-017-0603-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-017-0603-0
Keywords
- Augmented Lagrangian
- Cone constrained optimization
- Localization principle
- Saddle point
- Second order cone
- Semidefinite programming
- Semi-infinite programming