Skip to main content
SpringerLink
Log in

A penalty method for rank minimization problems in symmetric matrices

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

The problem of minimizing the rank of a symmetric positive semidefinite matrix subject to constraints can be cast equivalently as a semidefinite program with complementarity constraints (SDCMPCC). The formulation requires two positive semidefinite matrices to be complementary. This is a continuous and nonconvex reformulation of the rank minimization problem. We investigate calmness of locally optimal solutions to the SDCMPCC formulation and hence show that any locally optimal solution is a KKT point. We develop a penalty formulation of the problem. We present calmness results for locally optimal solutions to the penalty formulation. We also develop a proximal alternating linearized minimization (PALM) scheme for the penalty formulation, and investigate the incorporation of a momentum term into the algorithm. Computational results are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Alfakih, A.Y., Anjos, M.F., Piccialli, V., Wolkowicz, H.: Euclidean distance matrices, semidefinite programming, and sensor network localization. Port. Math. 68, 53–102 (2011)

    Article  MathSciNet  Google Scholar 

  2. Bai, L., Mitchell, J.E., Pang, J.: On conic QPCCs, conic QCQPs and completely positive programs. Math. Program. 159, 109–136 (2016)

    Article  MathSciNet  Google Scholar 

  3. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014)

    Article  MathSciNet  Google Scholar 

  4. Burdakov, O., Kanzow, C., Schwartz, A.: Mathematical programs with cardinality constraints: reformulation by complementarity-type conditions and a regularization method. SIAM J. Optim. 26, 397–425 (2016)

    Article  MathSciNet  Google Scholar 

  5. Cai, J.-F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20, 1956–1982 (2010)

    Article  MathSciNet  Google Scholar 

  6. Cambier, L., Absil, P.-A.: Robust low-rank matrix completion by Riemannian optimization. SIAM J. Sci. Comput. 38, S440–S460 (2016)

    Article  MathSciNet  Google Scholar 

  7. Clarke, F.H.: Optimization and Nonsmooth Analysis. SIAM, Philadelphia (1990)

    Book  Google Scholar 

  8. Ding, C., Sun, D., Ye, J.: First order optimality conditions for mathematical programs with semidefinite cone complementarity constraints. Math. Program. 147, 539–579 (2014)

    Article  MathSciNet  Google Scholar 

  9. Ding, Y., Krislock, N., Qian, J., Wolkowicz, H.: Sensor network localization, euclidean distance matrix completions, and graph realization. Optim. Eng. 11, 45–66 (2008)

    Article  MathSciNet  Google Scholar 

  10. Fazel, M., Hindi, H., Boyd, S.P.: A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the 2001 American Control Conference, June 2001, IEEE, pp. 4734–4739 (2001). https://faculty.washington.edu/mfazel/nucnorm.html

  11. Feng, M., Mitchell, J.E., Pang, J., Shen, X., Wächter, A.: Complementarity formulations of \(\ell \)0-norm opti-mization problems. Tech. Report, Department of Mathematical Sciences, Rensselaer Polytechnic Institute, Troy, NY (2013). Revised: May 2016

  12. Ghadimi, S., Lan, G.: Accelerated gradient methods for nonconvex nonlinear and stochastic programming. Math. Program. 156, 59–99 (2016)

    Article  MathSciNet  Google Scholar 

  13. Grant, M., Boyd, S., Ye, Y.: Disciplined convex programming, in global optimization: from theory to implementation. In: Liberti, L., Maculan, N. (eds.) Nonconvex Optimization and its Applications, pp. 155–210. Springer, Berlin (2006)

    Google Scholar 

  14. Hestenes, M.R., Stiefel, E.: Methods of Conjugate Gradients for Solving Linear Systems. NBS, Bikaner (1952)

    MATH  Google Scholar 

  15. Hsieh, C.-J., Olsen, P.: Nuclear norm minimization via active subspace selection. In: International Conference Machine Learning Proceedings, Beijing, China, pp. 575–583 (2014)

  16. Huang, X.X., Teo, K.L., Yang, X.Q.: Calmness and exact penalization in vector optimization with cone constraints. Comput. Optim. Appl. 35, 47–67 (2006)

    Article  MathSciNet  Google Scholar 

  17. Keshavan, R.H., Montanari, A., Oh, S.: Matrix completion from a few entries. IEEE Trans. Inf. Theory 56, 2980–2998 (2010)

    Article  MathSciNet  Google Scholar 

  18. Kocvara, M., Stingl, M.: PENNON—a code for convex nonlinear and semidefinite programming. Optim. Methods Softw. 18, 317–333 (2003)

    Article  MathSciNet  Google Scholar 

  19. Krislock, N., Wolkowicz, H.: Euclidean distance matrices and applications. In: Anjos, M., Lasserre, J. (eds.) Handbook on Semidefinite, Conic and Polynomial Optimization, International Series in Operational Research and Management Science. Springer, pp. 879–914 (2012)

  20. Laurent, M.: A tour d’horizon on positive semidefinite and Euclidean distance matrix completion problems. In: Topics in Semidefinite and Interior Point Methods, vol. 8 of The Fields Institute for Research in Mathematical Sciences, Communications Series, AMS, Providence, RI (1998)

  21. Li, H., Lin, Z.: Accelerated proximal gradient methods for nonconvex programming. In: Advances in Neural Information Processing Systems, Montreal, Canada, pp. 379–387 (2015)

  22. Li, Q., Qi, H.-D.: A sequential semismooth Newton method for the nearest low-rank correlation matrix problem. SIAM J. Optim. 21, 1641–1666 (2011)

    Article  MathSciNet  Google Scholar 

  23. Li, X., Ling, S., Strohmer, T., Wei, K.: Rapid, robust, and reliable blind deconvolution via nonconvex optimization. Tech. Report, Department of Mathematics, University of California Davis, Davis, CA 95616 (2016)

  24. Lin, Q., Lu, Z., Xiao, L.: An accelerated randomized proximal coordinate gradient method and its application to regularized empirical risk minimization. SIAM J. Optim. 25, 2244–2273 (2015)

    Article  MathSciNet  Google Scholar 

  25. Lin, Z., Chen, M., Ma, Y.: The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. Tech. Report, Perception and Decision Lab, University of Illinois, Urbana-Champaign, IL (2010)

  26. Liu, Y.-J., Sun, D., Toh, K.-C.: An implementable proximal point algorithmic framework for nuclear norm minimization. Math. Program. 133, 399–436 (2012)

    Article  MathSciNet  Google Scholar 

  27. Liu, Z., Vandenberghe, L.: Interior-point method for nuclear norm approximation with application to system identification. SIAM J. Matrix Anal. Appl. 31, 1235–1256 (2009)

    Article  MathSciNet  Google Scholar 

  28. Lu, S.: Relation between the constant rank and the relaxed constant rank constraint qualifications. Optimization 61, 555–566 (2012)

    Article  MathSciNet  Google Scholar 

  29. Luenberger, D.G.: Linear and Nonlinear Programming, 2nd edn. Addison-Wesley, Menlo Park (1984)

    MATH  Google Scholar 

  30. Ma, S., Goldfarb, D., Chen, L.: Fixed point and Bregman iterative methods for matrix rank minimization. Math. Program. 128, 321–353 (2011)

    Article  MathSciNet  Google Scholar 

  31. Mohan, K., Fazel, M.: Reweighted nuclear norm minimization with application to system identification. In: Proceedings of the American Control Conference, Baltimore, MD. IEEE, pp. 2953–2959 (2010)

  32. Mohan, K., Fazel, M.: Iterative reweighted algorithms for matrix rank minimization. Journal of Machine Learning Research 13, 3441–3473 (2012)

    MathSciNet  MATH  Google Scholar 

  33. Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Springer, New York (2013)

    MATH  Google Scholar 

  34. Nesterov, Y.E.: A method for unconstrained convex minimization problem with the rate of convergence \(o(1/k^2)\). In: Doklady AN SSSR, vol 269, pp. 543–547 (1983). translated as Soviet Mathematics Doklady

  35. Nesterov, Y.E.: How to advance in structural convex optimization. Optima 78, 2–5 (2008)

    Google Scholar 

  36. Pong, T.K., Tseng, P.: (Robust) edge-based semidefinite programming relaxation of sensor network localization. Math. Program. 130, 321–358 (2011)

    Article  MathSciNet  Google Scholar 

  37. Qi, H., Sun, D.: A quadratically convergent Newton method for computing the nearest correlation matrix. SIAM J. Matrix Anal. Appl. 28, 360–385 (2006)

    Article  MathSciNet  Google Scholar 

  38. Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum-rank solutions of linear matrix inequalities via nuclear norm minimization. SIAM Rev. 52, 471–501 (2010)

    Article  MathSciNet  Google Scholar 

  39. Schmidt, M., Roux, N.L., Bach, F.R.: Convergence rates of inexact proximal-gradient methods for convex optimization. In: Advances in Neural Information Processing Systems, Granada, Spain, pp. 1458–1466 (2011)

  40. So, A.M.-C., Ye, Y.: Theory of semidefinite programming for sensor network localization. Math. Program. 109, 367–384 (2007)

    Article  MathSciNet  Google Scholar 

  41. Srebro, N., Jaakkola, T.: Weighted low-rank approximations. In: Proceedings of the International Conference on Machine Learning, Atlanta, GA, pp. 720–727 (2003)

  42. Sun, R., Luo, Z.-Q., Guaranteed matrix completion via nonconvex factorization. In: 2015 IEEE 56th Annual Symposium on Foundations of Computer Science, pp. 270–289 (2015)

  43. Tanner, J., Wei, K.: Low rank matrix completion by alternating steepest descent methods. Appl. Comput. Harmonic Anal. 40, 417–429 (2016)

    Article  MathSciNet  Google Scholar 

  44. Toh, K.C., Yun, S.: An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pac. J. Optim. 6, 615–640 (2010)

    MathSciNet  MATH  Google Scholar 

  45. Vandereycken, B.: Low-rank matrix completion by Riemannian optimization. SIAM J. Optim. 23, 1214–1236 (2013)

    Article  MathSciNet  Google Scholar 

  46. Wei, K., Cai, J.-F., Chan, T.F., Leung, S.: Guarantees of Reimannian optimization for low rank matrix completion. Tech. Report, Department of Mathematics, University of California at Davis, CA, USA (2016)

  47. Wu, J., Zhang, L.: On properties of the bilinear penalty function method for mathematical programs with semidefinite cone complementarity constraints. Set Valued Var. Anal. 23, 277–294 (2015)

    Article  MathSciNet  Google Scholar 

  48. Yamashita, H., Yabe, H.: A survey of numerical methods for nonlinear semidefinite programming. J. Oper. Res. Soc. Jpn. 58, 24–60 (2015)

    Article  MathSciNet  Google Scholar 

  49. Ye, J.J., Zhou, J.: First-order optimality conditions for mathematical programs with second-order cone complementarity constraints. Tech. Report, Department of Mathematics and Statistics, University of Victoria, Victoria, BC, Canada (2015)

  50. Zhai, J., Huang, X.X.: Calmness and exact penalization in vector optimization under nonlinear perturbations. J. Optim. Theory Appl. 162, 856–872 (2014)

    Article  MathSciNet  Google Scholar 

  51. Zhang, H., Lin, Z., Zhang, C.: A Counterexample for the Validity of Using Nuclear Norm as a Convex Surrogate of Rank, in Machine Learning and Knowledge Discovery in Databases, pp. 226–241. Springer, Berlin (2013)

    Google Scholar 

  52. Zhang, Y., Zhang, L., Wu, J.: Convergence properties of a smoothing approach for mathematical programs with second-order cone complementarity constraints. Set Valued Var. Anal. 19, 609–646 (2011)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, Rensselaer Polytechnic Institute, Troy, NY, 12180, USA

    Xin Shen & John E. Mitchell

Authors
  1. Xin Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John E. Mitchell
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John E. Mitchell.

Additional information

This work was supported in part by the Air Force Office of Sponsored Research under Grants FA9550-08-1-0081 and FA9550-11-1-0260 and by the National Science Foundation under Grants Numbers CMMI-1334327 and DMS-1736326.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shen, X., Mitchell, J.E. A penalty method for rank minimization problems in symmetric matrices. Comput Optim Appl 71, 353–380 (2018). https://doi.org/10.1007/s10589-018-0010-6

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-018-0010-6

Keywords

Mathematics Subject Classification

Search

Navigation