Advertisement

Mathematical Programming

, Volume 144, Issue 1–2, pp 141–179 | Cite as

An introduction to a class of matrix cone programming

  • Chao Ding
  • Defeng SunEmail author
  • Kim-Chuan Toh
Full Length Paper Series A

Abstract

In this paper, we define a class of linear conic programming (which we call matrix cone programming or MCP) involving the epigraphs of five commonly used matrix norms and the well studied symmetric cone. MCP has recently been found to have many important applications, for example, in nuclear norm relaxations of affine rank minimization problems. In order to make the defined MCP tractable and meaningful, we must first understand the structure of these epigraphs. So far, only the epigraph of the Frobenius matrix norm, which can be regarded as a second order cone, has been well studied. Here, we take an initial step to study several important properties, including its closed form solution, calm Bouligand-differentiability and strong semismoothness, of the metric projection operator over the epigraph of the \(l_1,\,l_\infty \), spectral or operator, and nuclear matrix norm, respectively. These properties make it possible to apply augmented Lagrangian methods, which have recently received a great deal of interests due to their high efficiency in solving large scale semidefinite programming, to this class of MCP problems. The work done in this paper is far from comprehensive. Rather it is intended as a starting point to call for more insightful research on MCP so that it can serve as a basic tool to solve more challenging convex matrix optimization problems in years to come.

Keywords

Matrix cones Metric projectors Conic optimization 

Mathematics Subject Classification (2000)

65K05 90C25 90C30 

Notes

Acknowledgments

We wish to thank the anonymous referees and the Associate Editor for helpful comments that led to an improved version of the original submission.

References

  1. 1.
    Bhatia, R.: Matrix Analysis. Springer, New York (1997)CrossRefGoogle Scholar
  2. 2.
    Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9, 717–772 (2008)CrossRefGoogle Scholar
  3. 3.
    Candès, E.J., Tao, T.: The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inf. Theory 56, 2053–2080 (2009)CrossRefGoogle Scholar
  4. 4.
    Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM. 58, article No. 11 (2011)Google Scholar
  5. 5.
    Chandrasekaran, V., Sanghavi, S., Parrilo, P.A., Willsky, A.: Rank-sparsity incoherence for matrix decomposition. SIAM J. Optim. 21, 572–596 (2011)Google Scholar
  6. 6.
    Chen, X., Tseng, P.: Non-interior continuation methods for solving semidefinite complementarity problems. Math. Program. 95, 431–474 (2003)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Chen, X., Qi, H.D., Tseng, P.: Analysis of nonsmooth symmetric-matrix-valued functions with applications to semidefinite complement problems. SIAM J. Optim. 13, 960–985 (2003)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Chen, X.D., Sun, D.F., Sun, J.: Complementarity functions and numerical experiments for second-order-cone complementarity problems. Comput. Optim. Appl. 25, 39–56 (2003)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Chu, M., Funderlic, R., Plemmons, R.: Structured low rank approximation. Linear Algebra Appl. 366, 157–172 (2003)CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)zbMATHGoogle Scholar
  11. 11.
    Donoghue, W.F.: Monotone Matrix Functions and Analytic Continuation. Springer, New York (1974)CrossRefzbMATHGoogle Scholar
  12. 12.
    Fan, K.: On a theorem of Weyl concerning eigenvalues of linear transformations. Proc. Natl. Acad. Sci. USA 35, 652–655 (1949)CrossRefGoogle Scholar
  13. 13.
    Faraut, J., Korányi, A.: Analysis on Symmetric Cones. Clarendon Press, Oxford (1994)zbMATHGoogle Scholar
  14. 14.
    Gao, Y., Sun, D.F.: A majorized penalty approach for calibrating rank constrained correlation matrix problems. Preprint available at http://www.math.nus.edu.sg/~matsundf/MajorPen.pdf
  15. 15.
    Greenbaum, A., Trefethen, L.N.: GMRES/CR and Arnoldi/Lanczos as matrix approximation problems. SIAM J. Sci. Comput. 15, 359–368 (1994)CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    Gross, D.: Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inf. Theory 57, 1548–1566 (2011)Google Scholar
  17. 17.
    Güler, O.: Hyperbolic polynomials and interior point methods for convex programming. Math. Oper. Res. 22, 350–377 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Haraux, A.: How to differentiate the projection on a convex set in Hilbert space. Some applications to variational inequalities. J. Math. Soc. Jpn 29, 615–631 (1977)CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Hardy, G.H., Littlewood, J.E., Pólya, G.: Inequalities, 2nd edn. University Press, Cambridge (1952)zbMATHGoogle Scholar
  20. 20.
    Higham, N.J.: Computing a nearest symmetric positive semidefinite matrix. Linear Algebra Appl. 103, 103–118 (1988)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1985)CrossRefzbMATHGoogle Scholar
  22. 22.
    Horn, R.A., Johnson, C.R.: Topics in Matrix Analysis. Cambridge University Press, Cambridge (1991)CrossRefzbMATHGoogle Scholar
  23. 23.
    Keshavan, R.H., Montanari, A., Oh, S.: Matrix completion from a few entries. IEEE Trans. Inf. Theory 56, 2980–2998 (2010)Google Scholar
  24. 24.
    Korányi, A.: Monotone functions on formally real Jordan algebras. Mathematische Annalen 269, 73–76 (1984)CrossRefzbMATHMathSciNetGoogle Scholar
  25. 25.
    Lancaster, P.: On eigenvalues of matrices dependent on a parameter. Numerische Mathematik 6, 377–387 (1964)CrossRefzbMATHMathSciNetGoogle Scholar
  26. 26.
    Lewis, A.S., Sendov, H.S.: Twice differentiable spectral functions. SIAM J. Matrix Anal. Appl. 23, 368–386 (2001)CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Lewis, A.S., Sendov, H.S.: Nonsmooth analysis of singular values. Part II: Applications. Set Valued Anal. 13, 243–264 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  28. 28.
    Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: Proceedings of the 26th International Conference on Machine Learning. Haifa, Israel (2010)Google Scholar
  29. 29.
    Löwner, K.: Über monotone matrixfunktionen. Mathematische Zeitschrift 38, 177–216 (1934)CrossRefMathSciNetGoogle Scholar
  30. 30.
    Malick, J., Povh, J., Rendl, F., Wiegele, A.: Regularization methods for semidefinite programming. SIAM J. Optim. 20, 336–356 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  31. 31.
    Moreau, J.J.: Décomposition orthogonale d’un espace hilbertien selon deux cones mutuellement polaires. Comptes Rendus de l’Académie des Sciences 255, 238–240 (1962)zbMATHMathSciNetGoogle Scholar
  32. 32.
    von Neumann, J.: Some matrix-inequalities and metrization of matric-space. Tomsk University Review 1, 286–300 (1937). In: Collected Works. Pergamon, Oxford, 1962, Volume IV, 205–218Google Scholar
  33. 33.
    Pang, J.S.: Newton’s method for B-differentiable equations. Math. Oper. Res. 15, 149–160 (1990)CrossRefGoogle Scholar
  34. 34.
    Povh, J., Rendl, F., Wiegele, A.: A boundary point method to solve semidefinite programs. Computing 78, 277–286 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  35. 35.
    Qi, L.: Convergence analysis of some algorithms for solving nonsmooth equations. Math. Oper. Res. 18, 227–244 (1993)CrossRefzbMATHMathSciNetGoogle Scholar
  36. 36.
    Recht, B.: A Simpler approach to matrix completion. J. Mach. Learn. Res. 12, 3413–3430 (2011)Google Scholar
  37. 37.
    Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum rank solutions to linear matrix equations via nuclear norm minimization. SIAM Rev. 52, 471–501 (2010)Google Scholar
  38. 38.
    Rockafellar, R.T.: Convex Analyis. Princeton University Press, Princeton (1970)Google Scholar
  39. 39.
    Rockafellar, R.T.: Conjugate Duality and Optimization. SIAM, Philadelphia (1974)CrossRefzbMATHGoogle Scholar
  40. 40.
    Scholtes, S.: Introduction to Piecewise Differentiable Equations. Habilitation Thesis, Institut für Statistik und Mathematische Wirtschaftstheorie, Universit”at Karlsruhe (1994)Google Scholar
  41. 41.
    Schwertman, N.C., Allen, D.M.: Smoothing an indefinite variance-covariance matrix. J. Stat. Comput. Simul. 9, 183–194 (1979)CrossRefGoogle Scholar
  42. 42.
    Shapiro, A.: On differentiability of symmetric matrix valued functions. Optimization, Online (2002)Google Scholar
  43. 43.
    Stewart, G.W., Sun, J.G.: Matrix Perturbation Theory. Academic Press, New York (1990)zbMATHGoogle Scholar
  44. 44.
    Sun, D.F.: Algorithms and Convergence Analysis for Nonsmooth Optimization and Nonsmooth Equations. PhD Thesis, Institute of Applied Mathematics, Chinese Academy of Sciences, China (1994)Google Scholar
  45. 45.
    Sun, D.F., Sun, J.: Semismooth matrix-valued functions. Math. Oper. Res. 27, 150–169 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  46. 46.
    Sun, D.F., Sun, J.: Strong semismoothness of eigenvalues of symmetric matrices and its applications in inverse eigenvalue problems. SIAM J. Numer. Anal. 40, 2352–2367 (2003)CrossRefzbMATHGoogle Scholar
  47. 47.
    Sun, D.F., Sun, J.: Löwner’s operator and spectral functions in Euclidean Jordan algebras. Math. Oper. Res. 33, 421–445 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  48. 48.
    Toh, K.C.: GMRES vs. ideal GMRES. SIAM J. Matrix Anal. Appl. 18, 30–36 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  49. 49.
    Toh, K.C., Trefethen, L.N.: The Chebyshev polynomials of a matrix. SIAM J. Matrix Anal. Appl. 20, 400–419 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  50. 50.
    Torki, M.: Second-order directional derivatives of all eigenvalues of a symmetric matrix. Nonlinear Anal. Ser. A Theory Methods 46, 1133–1150 (2001)CrossRefzbMATHMathSciNetGoogle Scholar
  51. 51.
    Tseng, P.: Merit functions for semi-definite complementarity problems. Math. Program. 83, 159–185 (1998)zbMATHGoogle Scholar
  52. 52.
    Tsing, N.K., Fan, M.K.H., Verriest, E.I.: On analyticity of functions involving eigenvalues. Linear Algebra Appl. 207, 159–180 (1994)CrossRefzbMATHMathSciNetGoogle Scholar
  53. 53.
    Vandenberghe, L., Boyd, S.: Semidefinite programming. SIAM Rev. 38, 49–95 (1996)CrossRefzbMATHMathSciNetGoogle Scholar
  54. 54.
    Wen, Z., Goldfarb, D., Yin, W.: Alternating Direction Augmented Lagrangian Methods for Semidefinite Programming. Rice University CAAM Technical, Report TR09-42, 2009 (1996)Google Scholar
  55. 55.
    Wright, J., Ma, Y., Ganesh, A., Rao, S.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: Bengio, Y., Schuurmans, D., Lafferty, J., Williams, C. (eds.) Advances in Neural Information Processing Systems, vol. 22 (2009)Google Scholar
  56. 56.
    Zarantonello, E.H.: Projections on convex sets in Hilbert space and spectral theory I and II. In: Zarantonello, E.H. (ed.) Contributions to Nonlinear Functional Analysis, pp. 237–424. Academic Press, New York (1971)Google Scholar
  57. 57.
    Zhao, X.Y.: A Semismooth Newton-CG Augmented Lagrangian Method for Large Scale Linear and Convex Quadratic SDPs. PhD thesis, Department of Mathematics, National University of Singapore (2009)Google Scholar
  58. 58.
    Zhao, X.Y., Sun, D.F., Toh, K.C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737–1765 (2010)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2012

Authors and Affiliations

  1. 1.National Center for Mathematics and Interdisciplinary SciencesChinese Academy of SciencesBeijingP. R. China
  2. 2.Department of Mathematics and Risk Management InstituteNational University of SingaporeSingaporeRepublic of Singapore
  3. 3.Department of MathematicsNational University of SingaporeSingaporeRepublic of Singapore

Personalised recommendations