Computational Optimization and Applications

, Volume 58, Issue 2, pp 423–454 | Cite as

The space decomposition theory for a class of eigenvalue optimizations

  • Ming Huang
  • Li-Ping PangEmail author
  • Zun-Quan Xia


In this paper we study optimization problems involving eigenvalues of symmetric matrices. One of the difficulties with numerical analysis of such problems is that the eigenvalues, considered as functions of a symmetric matrix, are not differentiable at those points where they coalesce. Here we apply the \(\mathcal{U}\)-Lagrangian theory to a class of D.C. functions (the difference of two convex functions): the arbitrary eigenvalue function λ i , with affine matrix-valued mappings, where λ i is a D.C. function. We give the first-and second-order derivatives of \({\mathcal{U}}\)-Lagrangian in the space of decision variables R m when transversality condition holds. Moreover, an algorithm framework with quadratic convergence is presented. Finally, we present an application: low rank matrix optimization; meanwhile, list its \(\mathcal{VU}\) decomposition results.


Nonsmooth optimization Eigenvalue optimization \(\mathcal{VU}\)-Decomposition \({\mathcal{U}}\)-Lagrangian D.C. function Second-order derivative 



We would like to thank Xi-jun Liang and Yue Lu from Dalian University of Technology and Yuan Lu from Shenyang University for numerous fruitful discussions. The authors also thank two anonymous referees for a number of valuable and constructive suggestions that helped to improved the presentation. This research was supported in part by the Natural Science Foundation of China, Grant 11171049, 11226230 and 11301347, and General Project of the Education Department of Liaoning Province L2012427.


  1. 1.
    Apkarian, P., Noll, D., Thevenet, J.-B., Tuan, H.D.: A spectral quadratic-SDP method with applications to fixed-order H 2 and H synthesis. Eur. J. Control 10, 527–538 (2004) CrossRefMathSciNetGoogle Scholar
  2. 2.
    Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2000) CrossRefzbMATHGoogle Scholar
  3. 3.
    Clarke, F.H., Ledyaev, Y.S., Stern, R.J., Wolenski, P.R.: Nonsmooth Analysis and Control Theory. Springer, New York (1998) zbMATHGoogle Scholar
  4. 4.
    Cox, S., Lipton, R.: Extremal eigenvalue problems for two-phase conductors. Arch. Ration. Mech. Anal. 136, 101–117 (1996) CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9, 717–772 (2009) CrossRefzbMATHMathSciNetGoogle Scholar
  6. 6.
    Candès, E.J., Tao, T.: The power of convex relaxation: near-optimal matrix completion. IEEE Trans. Inf. Theory 56(5), 2053–2080 (2010) CrossRefGoogle Scholar
  7. 7.
    Diaaz, A.R., Kikuchi, N.: Solution to shape and topology eigenvalue optimization problems using a homogenization method. J. Numer. Methods Eng. 35(7), 1487–1502 (2005) CrossRefGoogle Scholar
  8. 8.
    Demyanov, V.F., Rubinov, A.M.: Constructive Nonsmooth Analysis, Approximation & Optimization No. 7. Peter Lang, Frankfurt (1995) Google Scholar
  9. 9.
    Dinh, T.P., Le Thi, H.A.: A DC optimization algorithm for solving the trust region subproblem. SIAM J. Optim. 8(2), 476–505 (1998) CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Dinh, T.P., Le Thi, H.A.: The DC programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann. Oper. Res. 133, 23–46 (2005) CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Gross, D.: Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inf. Theory 57(3), 1548–1566 (2011) CrossRefGoogle Scholar
  12. 12.
    Gross, D., Liu, Y.-K., Flammia, S.T., Becker, S., Eisert, J.: Quantum state tomography via compressed sensing. Phys. Rev. Lett. 105, 150401 (2010) CrossRefGoogle Scholar
  13. 13.
    Helmberg, C., Rendl, F., Weismantel, R.: A semidefinite programming approach to the quadratic knapsack problem. J. Comb. Optim. 4, 197–215 (2000) CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Hiriart-Urruty, J.-B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. Springer, Berlin (1993) Google Scholar
  15. 15.
    Hiriart-Urruty, J.-B., Ye, D.: Sensitivity analysis of all eigenvalues of a symmetric matrix. Numer. Math. 70, 45–72 (1995) CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    Lu, Y., Pang, L.P., Xia, Z.Q.: A \(\mathcal{VU}\)-decomposition method for a second-order cone programming problem. Appl. Math. Mech. 31(2), 263–270 (2010) CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Lu, Y., Pang, L.P., Guo, F.F., Xia, Z.Q.: A superlinear space decomposition algorithm for constrained nonsmooth convex program. J. Comput. Appl. Math. 234(1), 224–232 (2010) CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Lemaréchal, C., Oustry, F., Sagastizábal, C.: The \({\mathcal{U}}\)-Lagrangian of a convex function. Trans. Am. Math. Soc. 352(2), 711–729 (2000) CrossRefzbMATHGoogle Scholar
  19. 19.
    Lemaréchal, C., Sagastizábal, C.: More than first-order developments of convex function: primal-dual relations. J. Convex Anal. 3(2), 255–268 (1996) zbMATHMathSciNetGoogle Scholar
  20. 20.
    Lewis, A.S., Overton, M.L.: Eigenvalue optimization. Acta Numer. 5, 149–190 (1996) CrossRefMathSciNetGoogle Scholar
  21. 21.
    Mohan, K., Fazel, M.: Reweighted nuclear norm minimization with application to system identification. In: Proceedings of the American Control Conference, pp. 2953–2959. IEEE Press, Piscataway (2010) Google Scholar
  22. 22.
    Mifflin, R., Sagastizábal, C.: On \(\mathcal{VU}\)-theory for functions with primal-dual gradient structure. SIAM J. Optim. 11(2), 547–571 (2000) CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    Mifflin, R., Sagastizábal, C.: \(\mathcal{VU}\)-smoothness and proximal point results for some nonconvex functions. Optim. Methods Softw. 19(5), 463–478 (2004) CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    Mifflin, R., Sagastizábal, C.: A \(\mathcal{VU}\)-algorithm for convex minimization. Math. Program., Ser. B 104, 583–608 (2005) CrossRefzbMATHGoogle Scholar
  25. 25.
    Noll, D., Apkarian, P.: Spectral bundle method for non-convex maximum eigenvalue functions: first-order methods. Math. Program., Ser. B 104, 701–727 (2005) CrossRefzbMATHMathSciNetGoogle Scholar
  26. 26.
    Noll, D., Apkarian, P.: Spectral bundle method for non-convex maximum eigenvalue functions: second-order methods. Math. Program., Ser. B 104, 729–747 (2005) CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Niu, Y.S., Dinh, T.P., Le Thi, H.A., Judice, J.J.: Efficient DC programming approaches for the asymmetric eigenvalue complementarity problem. Optim. Methods Softw. 28(4), 812–829 (2013) CrossRefzbMATHMathSciNetGoogle Scholar
  28. 28.
    Noll, D., Torki, M., Apkarian, P.: Partially augmented Lagrangian method for matrix inequality constraints. SIAM J. Optim. 15(1), 161–184 (2004) CrossRefzbMATHMathSciNetGoogle Scholar
  29. 29.
    Oustry, F.: The \({\mathcal{U}}\)-Lagrangian of the maximum eigenvalue functions. SIAM J. Optim. 9(2), 526–549 (1999) CrossRefzbMATHMathSciNetGoogle Scholar
  30. 30.
    Oustry, F.: A second-order bundle method to minimize the maximum eigenvalue function. Math. Program., Ser. A 89, 1–33 (2000) CrossRefzbMATHMathSciNetGoogle Scholar
  31. 31.
    Overton, M.L.: Large-scale optimization of eigenvalues. SIAM J. Optim. 2(1), 88–120 (1992) CrossRefzbMATHMathSciNetGoogle Scholar
  32. 32.
    Overton, M.L., Womersley, R.S.: Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices. Math. Program. 62, 321–357 (1993) CrossRefzbMATHMathSciNetGoogle Scholar
  33. 33.
    Overton, M.L., Womersley, R.S.: Second derivatives for optimizing eigenvalues of symmetric matrices. SIAM J. Matrix Anal. Appl. 16(3), 697–718 (1995) CrossRefzbMATHMathSciNetGoogle Scholar
  34. 34.
    Recht, B.: A simpler approach to matrix completion. J. Mach. Learn. Res. 12, 3413–3430 (2011) zbMATHMathSciNetGoogle Scholar
  35. 35.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, New Jersey (1970) zbMATHGoogle Scholar
  36. 36.
    Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (1998) CrossRefzbMATHGoogle Scholar
  37. 37.
    Shapiro, A., Fan, M.K.H.: On eigenvalue optimization. SIAM J. Optim. 5(3), 552–556 (1995) CrossRefzbMATHMathSciNetGoogle Scholar
  38. 38.
    Shapiro, A.: First and second order analysis of nonlinear semidefinite programs. Math. Program., Ser. B 77, 301–320 (1997) zbMATHGoogle Scholar
  39. 39.
    Thiao, M., Dinh, T.P., Le Thi, H.A.: A DC programming approach for sparse eigenvalue problem. In: Proceedings of the 27-th International Conference on Machine Learning, Haifa, Israel (2010) Google Scholar
  40. 40.
    Vandenberghe, L., Boyd, S.: Semidefinite programming. SIAM Rev. 38(1), 49–95 (1996) CrossRefzbMATHMathSciNetGoogle Scholar
  41. 41.
    Zhao, Z., Braams, B.Z., Fukuda, M., Overton, M.L., Percus, J.K.: The reduced density matrix method for electronic structure calculations and the role of three-index representability conditions. J. Chem. Phys. 120(5), 2095–2104 (2004) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.CORA, School of Mathematical SciencesDalian University of TechnologyDalianChina

Personalised recommendations