Skip to main content
SpringerLink
Log in

Two Relaxation Methods for Rank Minimization Problems

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

The problem of minimizing the rank of a symmetric positive semidefinite matrix subject to constraints can be lifted to give an equivalent semidefinite program with complementarity constraints. The formulation requires two positive semidefinite matrices to be complementary. This is a continuous and nonconvex reformulation of the rank minimization problem. We develop two relaxations and show that constraint qualification holds at any stationary point of either relaxation of the rank minimization problem, and we explore the structure of the local minimizers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Fazel, M., Hindi, H., Boyd, S.P.: A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the 2001 American Control Conference, June 2001. IEEE, pp. 4734–4739 (2001). https://faculty.washington.edu/mfazel/nucnorm.html

  2. Srebro, N., Jaakkola, T.: Weighted low-rank approximations. In: International Conference on Machine Learning. Atlanta, GA, pp. 720–727 (2003)

  3. Liu, Z., Vandenberghe, L.: Interior-point method for nuclear norm approximation with application to system identification. SIAM J. Matrix Anal. Appl. 31(3), 1235–1256 (2009)

    Article  MathSciNet  Google Scholar 

  4. Ma, S., Goldfarb, D., Chen, L.: Fixed point and Bregman iterative methods for matrix rank minimization. Math. Program. 128(1–2), 321–353 (2011)

    Article  MathSciNet  Google Scholar 

  5. Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)

    Article  MathSciNet  Google Scholar 

  6. Feng, M., Mitchell, J.E., Pang, J., Shen, X., Wächter, A.: Complementarity formulations of \(\ell _0\)-norm optimization problems. Pac. J. Optim. 14(2), 273–305 (2018)

    MathSciNet  Google Scholar 

  7. Shen, X., Mitchell, J.E.: A penalty method for rank minimization in symmetric matrices. Comput. Optim. Appl. 71(2), 353–380 (2018). https://doi.org/10.1007/s10589-018-0010-6

    Article  MathSciNet  MATH  Google Scholar 

  8. Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)

    Article  MathSciNet  Google Scholar 

  9. Gao, P., Wang, M., Ghiocel, S.G., Chow, J.H., Fardanesh, B., Stephopoulos, G.: Missing data recovery by exploiting low-dimensionality in power systems synchrophasor measurements. IEEE Trans. Power Syst. 31(2), 1006–1013 (2016)

    Article  Google Scholar 

  10. Gao, P., Wang, M., Ghiocel, S.G., Chow, J.H., Fardanesh, B., Stephopoulos, G., Razanousky, M.P.: Identification of successive “unobservable” cyber data attacks in power systems. IEEE Trans. Signal Process. 64(21), 5557–5570 (2016)

    Article  MathSciNet  Google Scholar 

  11. Jain, P., Netrapalli, P.: Fast exact matrix completion with finite samples. Technical report, Microsoft Research, India (2014)

  12. Cherapanamjeri, Y., Gupta, K., Jain, P.: Nearly-optimal robust matrix completion. Technical report, Microsoft Research, India (2016)

  13. Alfakih, A.Y., Anjos, M.F., Piccialli, V., Wolkowicz, H.: Euclidean distance matrices, semidefinite programming, and sensor network localization. Port. Math. 68(1), 53–102 (2011)

    Article  MathSciNet  Google Scholar 

  14. Miao, W., Pan, S., Sun, D.: A rank-corrected procedure for matrix completion with fixed basis coefficients. Math. Program. 159, 289–338 (2016)

    Article  MathSciNet  Google Scholar 

  15. Zhao, Y.B.: An approximation theory of matrix rank minimization and its application to quadratic equations. Linear Algebra Appl. 437, 77–93 (2013)

    Article  MathSciNet  Google Scholar 

  16. Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum-rank solutions of linear matrix inequalities via nuclear norm minimization. SIAM Rev. 52(3), 471–501 (2010)

    Article  MathSciNet  Google Scholar 

  17. Fazel, M., Hindi, H., Boyd, S.: Rank minimization and applications in system theory. In: Proceedings of the 2004 American Control Conference, Boston, Massachusetts, June 30–Jul 2, 2004, pp. 3273–3278 (2004)

  18. Lin, Z., Chen, M., Ma, Y.: The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. Technical report, Perception and Decision Lab, University of Illinois, Urbana-Champaign, IL (2010)

  19. Liu, Y.J., Sun, D., Toh, K.C.: An implementable proximal point algorithmic framework for nuclear norm minimization. Math. Program. 133(1–2), 399–436 (2012)

    Article  MathSciNet  Google Scholar 

  20. Hsieh, C.J., Olsen, P.: Nuclear norm minimization via active subspace selection. In: International Conference on Machine Learning. Beijing, China, pp. 575–583 (2014)

  21. Candès, E.J., Tao, T.: The power of convex relaxation: near-optimal matrix completion. IEEE Trans. Inf. Theory 56(5), 2053–2080 (2009)

    Article  MathSciNet  Google Scholar 

  22. Zhang, H., Lin, Z., Zhang, C.: A counterexample for the validity of using nuclear norm as a convex surrogate of rank. In: Machine Learning and Knowledge Discovery in Databases. Springer, Berlin, Germany, pp. 226–241 (2013)

  23. Goldfarb, D., Ma, S., Wen, Z.: Solving low-rank matrix completion problems efficiently. In: Forty-Seventh Annual Allerton Conference, Allerton House, UIUC, Illinois, USA, September 30–October 2, pp. 1013–1020 (2009)

  24. Luss, R., Teboulle, M.: Conditional gradient algorithms for rank-one matrix approximations with a sparsity constraint. SIAM Rev. 55(1), 65–98 (2013)

    Article  MathSciNet  Google Scholar 

  25. Fawzi, H., Parrilo, P.A.: Lower bounds on nonnegative rank via nonnegative nuclear norms. Math. Program. 153(1), 41–66 (2015)

    Article  MathSciNet  Google Scholar 

  26. Fazel, M., Hindi, H., Boyd, S.: Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices. In: Proceedings of the 2003 American Control Conference, pp. 2156–2162 (2003)

  27. Zhang, D., Hu, Y., Ye, J., Li, X., He, X.: Matrix completion by truncated nuclear norm regularization. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2192–2199 (2012). https://doi.org/10.1109/CVPR.2012.6247927

  28. Mohan, K., Fazel, M.: Reweighted nuclear norm minimization with application to system identification. In: Proceedings of the American Control Conference. IEEE, Baltimore, MD, pp. 2953–2959 (2010)

  29. Wang, S., Liu, D., Zhang, Z.: Nonconvex relaxation approaches to robust matrix recovery. In: Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI’13. AAAI Press, pp. 1764–1770 (2013). http://dl.acm.org/citation.cfm?id=2540128.2540381

  30. Keshavan, R.H., Montanari, A., Oh, S.: Matrix completion from a few entries. IEEE Trans. Inf. Theory 56(6), 2980–2998 (2010)

    Article  MathSciNet  Google Scholar 

  31. Li, X., Ling, S., Strohmer, T., Wei, K.: Rapid, robust, and reliable blind deconvolution via nonconvex optimization. Appl. Comput. Harmonic Anal. 47(3), 893–934 (2019)

  32. Sun, R., Luo, Z.Q.: Guaranteed matrix completion via nonconvex factorization. In: 2015 IEEE 56th Annual Symposium on Foundations of Computer Science, pp. 270–289 (2015)

  33. Tanner, J., Wei, K.: Low rank matrix completion by alternating steepest descent methods. Appl. Comput. Harmonic Anal. 40, 417–429 (2016)

    Article  MathSciNet  Google Scholar 

  34. Wei, K., Cai, J.F., Chan, T.F., Leung, S.: Guarantees of Reimannian optimization for low rank matrix recovery. SIAM J. Matrix Anal. Appl. 37(3), 1198–1222 (2016)

    Article  MathSciNet  Google Scholar 

  35. Yamashita, H., Yabe, H.: A survey of numerical methods for nonlinear semidefinite programming. J. Oper. Res. Soc. Jpn. 58(1), 24–60 (2015)

    Article  MathSciNet  Google Scholar 

  36. Ding, C., Sun, D., Ye, J.: First order optimality conditions for mathematical programs with semidefinite cone complementarity constraints. Math. Program. 147(1–2), 539–579 (2014)

    Article  MathSciNet  Google Scholar 

  37. Zhang, Y., Wu, J., Zhang, L.: First order necessary optimality conditions for mathematical programs with second-order cone complementarity constraints. J. Glob. Optim. 63(2), 253–279 (2015)

    Article  MathSciNet  Google Scholar 

  38. Li, Q., Qi, H.D.: A sequential semismooth Newton method for the nearest low-rank correlation matrix problem. SIAM J. Optim. 21, 1641–1666 (2011)

    Article  MathSciNet  Google Scholar 

  39. Liu, Y., Bi, S., Pan, S.: Equivalent Lipschitz surrogates for zero-norm and rank optimization problems. Technical report, School of Mathematics, GuangDong University of Technology, Guangzhou, China (2018)

  40. Yuan, G., Ghanem, B.: A proximal alternating direction method for semi-definite rank minimization. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, vol. AAAI-16, pp. 2300–2308 (2016)

  41. Robinson, S.M.: Stability theory for systems of inequalities. Part II: differentiable nonlinear systems. SIAM J. Numer. Anal. 13(4), 497–513 (1976)

    Article  MathSciNet  Google Scholar 

  42. Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2000)

    Book  Google Scholar 

  43. Sagan, A., Mitchell, J.E.: Alternating minimization for generalized nonconvex relaxations to rank minmization. Technical report, Mathematical Sciences, Rensselaer Polytechnic Institute, Troy, NY 12180 (2020)

Download references

Acknowledgements

This work was supported in part by the Air Force Office of Sponsored Research under Grants FA9550-08-1-0081 and FA9550-11-1-0260 and by the National Science Foundation under Grant Numbers CMMI-1334327 and DMS-1736326.

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, Rensselaer Polytechnic Institute, Troy, NY, 12180, USA

    April Sagan & John E. Mitchell

  2. Yelp Inc., San Francisco, CA, 94105, USA

    Xin Shen

Authors
  1. April Sagan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. John E. Mitchell
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John E. Mitchell.

Additional information

Communicated by Liqun Qi.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sagan, A., Shen, X. & Mitchell, J.E. Two Relaxation Methods for Rank Minimization Problems. J Optim Theory Appl 186, 806–825 (2020). https://doi.org/10.1007/s10957-020-01731-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-020-01731-9

Keywords

Mathematics Subject Classification

Search

Navigation