An introduction to a class of matrix cone programming

Abstract

In this paper, we define a class of linear conic programming (which we call matrix cone programming or MCP) involving the epigraphs of five commonly used matrix norms and the well studied symmetric cone. MCP has recently been found to have many important applications, for example, in nuclear norm relaxations of affine rank minimization problems. In order to make the defined MCP tractable and meaningful, we must first understand the structure of these epigraphs. So far, only the epigraph of the Frobenius matrix norm, which can be regarded as a second order cone, has been well studied. Here, we take an initial step to study several important properties, including its closed form solution, calm Bouligand-differentiability and strong semismoothness, of the metric projection operator over the epigraph of the \(l_1,\,l_\infty \), spectral or operator, and nuclear matrix norm, respectively. These properties make it possible to apply augmented Lagrangian methods, which have recently received a great deal of interests due to their high efficiency in solving large scale semidefinite programming, to this class of MCP problems. The work done in this paper is far from comprehensive. Rather it is intended as a starting point to call for more insightful research on MCP so that it can serve as a basic tool to solve more challenging convex matrix optimization problems in years to come.

This is a preview of subscription content, log in to check access.

References

  1. 1.

    Bhatia, R.: Matrix Analysis. Springer, New York (1997)

    Google Scholar 

  2. 2.

    Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9, 717–772 (2008)

    Article  Google Scholar 

  3. 3.

    Candès, E.J., Tao, T.: The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inf. Theory 56, 2053–2080 (2009)

    Article  Google Scholar 

  4. 4.

    Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM. 58, article No. 11 (2011)

  5. 5.

    Chandrasekaran, V., Sanghavi, S., Parrilo, P.A., Willsky, A.: Rank-sparsity incoherence for matrix decomposition. SIAM J. Optim. 21, 572–596 (2011)

    Google Scholar 

  6. 6.

    Chen, X., Tseng, P.: Non-interior continuation methods for solving semidefinite complementarity problems. Math. Program. 95, 431–474 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  7. 7.

    Chen, X., Qi, H.D., Tseng, P.: Analysis of nonsmooth symmetric-matrix-valued functions with applications to semidefinite complement problems. SIAM J. Optim. 13, 960–985 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  8. 8.

    Chen, X.D., Sun, D.F., Sun, J.: Complementarity functions and numerical experiments for second-order-cone complementarity problems. Comput. Optim. Appl. 25, 39–56 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  9. 9.

    Chu, M., Funderlic, R., Plemmons, R.: Structured low rank approximation. Linear Algebra Appl. 366, 157–172 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  10. 10.

    Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)

    Google Scholar 

  11. 11.

    Donoghue, W.F.: Monotone Matrix Functions and Analytic Continuation. Springer, New York (1974)

    Google Scholar 

  12. 12.

    Fan, K.: On a theorem of Weyl concerning eigenvalues of linear transformations. Proc. Natl. Acad. Sci. USA 35, 652–655 (1949)

    Article  Google Scholar 

  13. 13.

    Faraut, J., Korányi, A.: Analysis on Symmetric Cones. Clarendon Press, Oxford (1994)

    Google Scholar 

  14. 14.

    Gao, Y., Sun, D.F.: A majorized penalty approach for calibrating rank constrained correlation matrix problems. Preprint available at http://www.math.nus.edu.sg/~matsundf/MajorPen.pdf

  15. 15.

    Greenbaum, A., Trefethen, L.N.: GMRES/CR and Arnoldi/Lanczos as matrix approximation problems. SIAM J. Sci. Comput. 15, 359–368 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  16. 16.

    Gross, D.: Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inf. Theory 57, 1548–1566 (2011)

    Google Scholar 

  17. 17.

    Güler, O.: Hyperbolic polynomials and interior point methods for convex programming. Math. Oper. Res. 22, 350–377 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  18. 18.

    Haraux, A.: How to differentiate the projection on a convex set in Hilbert space. Some applications to variational inequalities. J. Math. Soc. Jpn 29, 615–631 (1977)

    Article  MATH  MathSciNet  Google Scholar 

  19. 19.

    Hardy, G.H., Littlewood, J.E., Pólya, G.: Inequalities, 2nd edn. University Press, Cambridge (1952)

    Google Scholar 

  20. 20.

    Higham, N.J.: Computing a nearest symmetric positive semidefinite matrix. Linear Algebra Appl. 103, 103–118 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  21. 21.

    Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1985)

    Google Scholar 

  22. 22.

    Horn, R.A., Johnson, C.R.: Topics in Matrix Analysis. Cambridge University Press, Cambridge (1991)

    Google Scholar 

  23. 23.

    Keshavan, R.H., Montanari, A., Oh, S.: Matrix completion from a few entries. IEEE Trans. Inf. Theory 56, 2980–2998 (2010)

    Google Scholar 

  24. 24.

    Korányi, A.: Monotone functions on formally real Jordan algebras. Mathematische Annalen 269, 73–76 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  25. 25.

    Lancaster, P.: On eigenvalues of matrices dependent on a parameter. Numerische Mathematik 6, 377–387 (1964)

    Article  MATH  MathSciNet  Google Scholar 

  26. 26.

    Lewis, A.S., Sendov, H.S.: Twice differentiable spectral functions. SIAM J. Matrix Anal. Appl. 23, 368–386 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  27. 27.

    Lewis, A.S., Sendov, H.S.: Nonsmooth analysis of singular values. Part II: Applications. Set Valued Anal. 13, 243–264 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  28. 28.

    Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: Proceedings of the 26th International Conference on Machine Learning. Haifa, Israel (2010)

  29. 29.

    Löwner, K.: Über monotone matrixfunktionen. Mathematische Zeitschrift 38, 177–216 (1934)

    Article  MathSciNet  Google Scholar 

  30. 30.

    Malick, J., Povh, J., Rendl, F., Wiegele, A.: Regularization methods for semidefinite programming. SIAM J. Optim. 20, 336–356 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  31. 31.

    Moreau, J.J.: Décomposition orthogonale d’un espace hilbertien selon deux cones mutuellement polaires. Comptes Rendus de l’Académie des Sciences 255, 238–240 (1962)

    MATH  MathSciNet  Google Scholar 

  32. 32.

    von Neumann, J.: Some matrix-inequalities and metrization of matric-space. Tomsk University Review 1, 286–300 (1937). In: Collected Works. Pergamon, Oxford, 1962, Volume IV, 205–218

  33. 33.

    Pang, J.S.: Newton’s method for B-differentiable equations. Math. Oper. Res. 15, 149–160 (1990)

    Article  Google Scholar 

  34. 34.

    Povh, J., Rendl, F., Wiegele, A.: A boundary point method to solve semidefinite programs. Computing 78, 277–286 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  35. 35.

    Qi, L.: Convergence analysis of some algorithms for solving nonsmooth equations. Math. Oper. Res. 18, 227–244 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  36. 36.

    Recht, B.: A Simpler approach to matrix completion. J. Mach. Learn. Res. 12, 3413–3430 (2011)

    Google Scholar 

  37. 37.

    Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum rank solutions to linear matrix equations via nuclear norm minimization. SIAM Rev. 52, 471–501 (2010)

    Google Scholar 

  38. 38.

    Rockafellar, R.T.: Convex Analyis. Princeton University Press, Princeton (1970)

    Google Scholar 

  39. 39.

    Rockafellar, R.T.: Conjugate Duality and Optimization. SIAM, Philadelphia (1974)

    Google Scholar 

  40. 40.

    Scholtes, S.: Introduction to Piecewise Differentiable Equations. Habilitation Thesis, Institut für Statistik und Mathematische Wirtschaftstheorie, Universit”at Karlsruhe (1994)

  41. 41.

    Schwertman, N.C., Allen, D.M.: Smoothing an indefinite variance-covariance matrix. J. Stat. Comput. Simul. 9, 183–194 (1979)

    Article  Google Scholar 

  42. 42.

    Shapiro, A.: On differentiability of symmetric matrix valued functions. Optimization, Online (2002)

  43. 43.

    Stewart, G.W., Sun, J.G.: Matrix Perturbation Theory. Academic Press, New York (1990)

    Google Scholar 

  44. 44.

    Sun, D.F.: Algorithms and Convergence Analysis for Nonsmooth Optimization and Nonsmooth Equations. PhD Thesis, Institute of Applied Mathematics, Chinese Academy of Sciences, China (1994)

  45. 45.

    Sun, D.F., Sun, J.: Semismooth matrix-valued functions. Math. Oper. Res. 27, 150–169 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  46. 46.

    Sun, D.F., Sun, J.: Strong semismoothness of eigenvalues of symmetric matrices and its applications in inverse eigenvalue problems. SIAM J. Numer. Anal. 40, 2352–2367 (2003)

    Article  MATH  Google Scholar 

  47. 47.

    Sun, D.F., Sun, J.: Löwner’s operator and spectral functions in Euclidean Jordan algebras. Math. Oper. Res. 33, 421–445 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  48. 48.

    Toh, K.C.: GMRES vs. ideal GMRES. SIAM J. Matrix Anal. Appl. 18, 30–36 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  49. 49.

    Toh, K.C., Trefethen, L.N.: The Chebyshev polynomials of a matrix. SIAM J. Matrix Anal. Appl. 20, 400–419 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  50. 50.

    Torki, M.: Second-order directional derivatives of all eigenvalues of a symmetric matrix. Nonlinear Anal. Ser. A Theory Methods 46, 1133–1150 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  51. 51.

    Tseng, P.: Merit functions for semi-definite complementarity problems. Math. Program. 83, 159–185 (1998)

    MATH  Google Scholar 

  52. 52.

    Tsing, N.K., Fan, M.K.H., Verriest, E.I.: On analyticity of functions involving eigenvalues. Linear Algebra Appl. 207, 159–180 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  53. 53.

    Vandenberghe, L., Boyd, S.: Semidefinite programming. SIAM Rev. 38, 49–95 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  54. 54.

    Wen, Z., Goldfarb, D., Yin, W.: Alternating Direction Augmented Lagrangian Methods for Semidefinite Programming. Rice University CAAM Technical, Report TR09-42, 2009 (1996)

  55. 55.

    Wright, J., Ma, Y., Ganesh, A., Rao, S.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: Bengio, Y., Schuurmans, D., Lafferty, J., Williams, C. (eds.) Advances in Neural Information Processing Systems, vol. 22 (2009)

  56. 56.

    Zarantonello, E.H.: Projections on convex sets in Hilbert space and spectral theory I and II. In: Zarantonello, E.H. (ed.) Contributions to Nonlinear Functional Analysis, pp. 237–424. Academic Press, New York (1971)

  57. 57.

    Zhao, X.Y.: A Semismooth Newton-CG Augmented Lagrangian Method for Large Scale Linear and Convex Quadratic SDPs. PhD thesis, Department of Mathematics, National University of Singapore (2009)

  58. 58.

    Zhao, X.Y., Sun, D.F., Toh, K.C.: A Newton-CG augmented Lagrangian method for semidefinite programming. SIAM J. Optim. 20, 1737–1765 (2010)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

We wish to thank the anonymous referees and the Associate Editor for helpful comments that led to an improved version of the original submission.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Defeng Sun.

Additional information

This work was done while C. Ding was with Department of Mathematics, National University of Singapore, Republic of Singapore.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ding, C., Sun, D. & Toh, K. An introduction to a class of matrix cone programming. Math. Program. 144, 141–179 (2014). https://doi.org/10.1007/s10107-012-0619-7

Download citation

Keywords

  • Matrix cones
  • Metric projectors
  • Conic optimization

Mathematics Subject Classification (2000)

  • 65K05
  • 90C25
  • 90C30