Abstract
In this paper, we study nonlinear optimization problems involving eigenvalues of symmetric matrices. One of the difficulties in solving these problems is that the eigenvalue functions are not differentiable when the multiplicity of the function is not one. We apply the \({\mathcal {U}}\)-Lagrangian theory to analyze the largest eigenvalue function of a convex matrix-valued mapping which extends the corresponding results for linear mapping in the literature. We also provides the formula of first-and second-order derivatives of the \({\mathcal {U}}\)-Lagrangian under mild assumptions. These theoretical results provide us new second-order information about the largest eigenvalue function along a suitable smooth manifold, and leads to a new algorithmic framework for analyzing the underlying optimization problem.
Similar content being viewed by others
Change history
08 February 2018
We apologize for the inconvenience caused.
References
Alizadeh F, Haeberly J-PA, Overton ML (1998) Primal-dual interior-point methods for semidefinite programming: convergence rates, stability and numerical results. SIAM J Optim 8:746–768
Abraham R, Marsden JE, Ratiu T (1988) Manifolds, tensor analysis, and applications, applied mathematical sciences, vol 75. Springer, Berlin
Apkarian P, Noll D, Thevenet J-B, Tuan HD (2004) A spectral quadratic-SDP method with applications to fixed-order \(H^{}_2\) and \(H^{}_{\infty }\) synthesis. Eur J Control 10:527–538
Bellman R, Fan K (1963) On systems of linear inequalities in Hermitian matrix variables. In: Klee VL (ed.) Convexity volume 7 of proceedings of symposia in pure mathematics, American Mathematical Society, pp 1-11
Bertsekas DP, Nedic A, Ozdaglar AE (2003) Convex analysis and optimization. Athena Scientific and Tsinghua University Press, Belmont
Bonnans JF, Shapiro A (2000) Perturbation analysis of optimization problems. Springer, New York
Cullum J, Donath WE, Wolfe P (1975) The minimization of certain nondifferentiable sums of eigenvalues of symmetric matrices. In: Nondifferentiable optimization, mathmatical programming study, vol. 3, pp 35–55
Cox S, Lipton R (1996) Extremal eigenvalue problems for two-phase conductors. Arch Rational Mech Anal 136:101–117
Díaz AR, Kikuchi N (1992) Solution to shape and topology eigenvalue optimization problems using a homogenization method. Int J Numer Methods Eng 35:1487–1502
Fukuda M, Kojima M (2001) Branch-and-cut algorithms for the bilinear matrix inequality eigenvalue problem. Comput Optim Appl 19:79–105
Fletcher R (1985) Semi-definite matrix constraints in optimization. SIAM J Control Optim 23:493–513
Goemans MX (1997) Semidefinite programming in combinatorial optimization. Math Program 79:143–161
Grötschel M, Lovász L, Schrijver A (1983) Geometric algorithms and combinatorial optimization. Springer, Berlin
Helmberg C, Kiwiel KC (2002) A spectral bundle method with bounds. Math Program 93:173–194
Helmberg C, Rendl F (2000) A spectral bundle method for semidefinite programming. SIAM J Optim 10:673–696
Helmberg C, Rendl F, Weismantel R (2000) A semidefinite programming approach to the quadratic knapsack problem. J Comb Optim 4:197–215
Huang M, Pang LP, Xia ZQ (2014) The space decomposition theory for a class of eigenvalue optimizations. Comput Optim Appl 58:423–454
Hiriart-Urruty J-B, Lemaréchal C (1993) Convex analysis and minimization algorithms I-II. In: Grundlehren der mathematischen Wissenschaften, vol. 305–306. Springer-Verlag, Berlin
Hiriart-Urruty J-B, Ye D (1995) Sensitivity analysis of all eigenvalues of a symmetric matrix. Numer Math 70:45–72
Kojima M, Shida M, Shindoh S (1998) Local convergence of predictor-corrector infeasible-interior point algorithms for sdps and sdlcps. Math Program 80:129–161
Lemaréchal C, Oustry F, Sagastizábal C (2000) The \({\cal{U}}\)-Lagrangian of a convex function. Trans Am. Math. Soc. 352:711–729
Lewis AS, Overton ML (1996) Eigenvalue optimization. Acta Numer 5:149–190
Lewis AS (1996) Derivatives of spectral functions. Math Oper Res 21:576–588
Lewis AS (2003) Active sets, nonsmoothness and sensitivity. SIAM J Optim 13:702–725
Mangasarian OL (1985) Sufficiency of exact penalty minimization. SIAM J Control Optim 23:30–37
Mifflin R, Sagastizábal C (2005) A \({\cal{VU}}\)-algorithm for convex minimization. Math Program Ser B 104:583–608
Noll D, Apkarian P (2005) Spectral bundle method for nonconvex maximum eigenvalue functions: second-order methods. Math Program Ser B 104:729–747
Nesterov Y (1997) Interior-point methods: an old and new approach to nonlinear programming. Math Program 79:285–297
Nesterov Y, Nemirovskii A (1988) A general approach to polynomial-time algorithms design for convex programming. Technical report. Centr. Econ. & Math. Inst., USSR Academy of Sciences, Moscow, USSR
Noll D, Torki M, Apkarian P (2004) Partially augmented Lagrangian method for matrix inequality constraints. SIAM J Optim 15:161–184
Oustry F (1999) The \({\cal{U}}\)-Lagrangian of the maximum eigenvalue function. SIAM J Optim 9:526–549
Oustry F (2000) A second-order bundle method to minimize the maximum eigenvalue function. Math Program Ser A 89:1–33
Overton ML (1992) Large-scale optimization of eigenvalues. SIAM J Optim 2:88–120
Overton ML, Womersley RS (1993) Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices. Math Program 62:321–357
Overton ML, Womersley RS (1995) Second derivatives for optimizing eigenvalues of symmetric matrices. SIAM J Matrix Anal Appl 16:697–718
Overton ML, Ye X (1994) Towards second-order methods for structured nonsmooth optimization. In: Gomez S, Hennart J-P (eds) Advances in optimization and numerical analysis, vol 275., Series mathematics and its applications, Kluwer Academic Publishers, Norwell, MA, pp 97–109
Pataki G (1998) On the rank of extreme matrices in semidefinite programming and the multiplicity of optimal eigenvalues. Math Oper Res 23:339–358
Rockafellar RT (1970) Convex analysis. Princeton University Press, New Jersey
Shapiro A, Fan MKH (1995) On eigenvalue optimization. SIAM J Optim 5:552–569
Shapiro A (1997) First and second order analysis of nonlinear semidefinite programs. Math Program Ser B 77:301–320
Thevenet JB, Noll D, Apkarian P (2006) Nonlinear spectral SDP method for BMI-constrained problems: applications to control design. In: Braz J et al (eds) Informatics in control, automation and robotics. Springer, Berlin, pp 61–72
Vandenberghe L, Boyd S (1996) Semidefinite programming. SIAM Rev 38:49–95
Zhao Z, Braams B, Fukuda M, Overton M, Percus J (2004) The reduced density matrix method for electronic structure calculations and the role of three-index representability conditions. J Chem Phys 120:2095–2104
Acknowledgements
The authors would thank the associate editor and the anonymous referees for their valuable and helpful comments and good advice for improving the presentation of this paper. This article is supported by the National Natural Science Foundation of China under Grant Nos. 11301347, 11626053 and 11601389, the Project funded by China Postdoctoral Science Foundation Under No. 2016M601296 and the Fundamental Research Funds for the Central Universities under Project No. 3132016108, the Scientific Research Foundation Funds of DLMU under Project No. 02501102, and Doctoral Foundation of Tianjin Normal University under Project No. 52XB1513.
Author information
Authors and Affiliations
Corresponding author
Additional information
A correction to this article is available online at https://doi.org/10.1007/s00186-017-0622-0.
Rights and permissions
About this article
Cite this article
Huang, M., Lu, Y., Pang, L.P. et al. A space decomposition scheme for maximum eigenvalue functions and its applications. Math Meth Oper Res 85, 453–490 (2017). https://doi.org/10.1007/s00186-017-0579-z
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00186-017-0579-z
Keywords
- Nonsmooth optimization
- Eigenvalue optimization
- Matrix-convex
- Semidefinite programming
- \({\mathcal {VU}}\)-decomposition
- \({\mathcal {U}}\)-Lagrangian
- Smooth manifold
- Second-order derivative
- Bilinear matrix inequality