Advertisement

Knowledge and Information Systems

, Volume 42, Issue 3, pp 525–544 | Cite as

Joint Schatten \(p\)-norm and \(\ell _p\)-norm robust matrix completion for missing value recovery

  • Feiping Nie
  • Hua Wang
  • Heng Huang
  • Chris Ding
Regular Paper

Abstract

The low-rank matrix completion problem is a fundamental machine learning and data mining problem with many important applications. The standard low-rank matrix completion methods relax the rank minimization problem by the trace norm minimization. However, this relaxation may make the solution seriously deviate from the original solution. Meanwhile, most completion methods minimize the squared prediction errors on the observed entries, which is sensitive to outliers. In this paper, we propose a new robust matrix completion method to address these two problems. The joint Schatten \(p\)-norm and \(\ell _p\)-norm are used to better approximate the rank minimization problem and enhance the robustness to outliers. The extensive experiments are performed on both synthetic data and real-world applications in collaborative filtering prediction and social network link recovery. All empirical results show that our new method outperforms the standard matrix completion methods.

Keywords

Matrix completion Schatten \(p\)-norm \(\ell _p\)-norm Recommendation system Social network 

Notes

Acknowledgments

This research was partially supported by NSF DMS-0915228, IIS-1117965, IIS-1302675, and IIS-1344152.

References

  1. 1.
    Srebro N, Rennie J, Jaakkola T (2004) Maximum margin matrix factorization. Conf Neural Inf Process Syst (NIPS) 17:1329–1336Google Scholar
  2. 2.
    Rennie J, Srebro N (2005) Fast maximum margin matrix factorization for collaborative prediction. In: International conference on machine learning (ICML)Google Scholar
  3. 3.
    Abernethy J, Bach F, Evgeniou T, Vert JP (2009) A new approach to collaborative filtering: operator estimation with spectral regularization. J Mach Learn Res (JMLR) 10:803–826zbMATHGoogle Scholar
  4. 4.
    Candès E, Recht B (2009) Exact matrix completion via convex optimization. Found Comput Math 9(6):717–772CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Candes EJ, Tao T (2009) The power of convex relaxation: near-optimal matrix completion. IEEE Trans Inform Theory 56(5):2053–2080CrossRefMathSciNetGoogle Scholar
  6. 6.
    Recht B, Fazel M, Parrilo PA (2010) Guaranteed minimum rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev 52(3):471–501CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Mazumder R, Hastie T, Tibshirani R (2010) Spectral regularization algorithms for learning large incomplete matrices. J Mach Learn Res (JMLR) 11:2287–2322Google Scholar
  8. 8.
    Cai J-F, Candes EJ, Shen Z (2008) A singular value thresholding algorithm for matrix completion. SIAM J Opt 20(4):1956–1982CrossRefMathSciNetGoogle Scholar
  9. 9.
    Toh K, Yun S (2010) An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pac J Opt 6:615–640zbMATHMathSciNetGoogle Scholar
  10. 10.
    Ji S, Ye Y (2009) An accelerated gradient method for trace norm minimization. In: International conference on machine learning (ICML)Google Scholar
  11. 11.
    Liu Y-J, Sun D, Toh K-C (2012) An implementable proximal point algorithmic framework for nuclear norm minimization. Math Program 133(1–2):399–436Google Scholar
  12. 12.
    Ma S, Goldfarb D, Chen L (2011) Fixed point and Bregman iterative methods for matrix rank minimization. Math Program 128(1–2):321–353Google Scholar
  13. 13.
    Recht B (2011) A simpler approach to matrix completion. J Mach Learn Res 12:3413–3430zbMATHMathSciNetGoogle Scholar
  14. 14.
    Vishwanath S (2010) Information theoretic bounds for low-rank matrix completion. In: IEEE international symposium on information theory proceedings (ISIT), pp 1508–1512Google Scholar
  15. 15.
    Koltchinskii V, Lounici K, Tsybakov A (2011) Nuclear norm penalization and optimal rates for noisy low rank matrix completion. Ann Stat 39:2302–2329CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    Salakhutdinov R, Srebro N (2010) Collaborative filtering in a non-uniform world: learning with the weighted trace norm. Adv Neural Inf Process Syst (NIPS) 23:1–8Google Scholar
  17. 17.
    Pong TK, Tseng P, Ji S, Ye J (2010) Trace norm regularization: reformulations, algorithms, and multi-task learning. SIAM J Opt 20(6):3465–3489CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Nie F, Wang H, Cai X, Huang H, Ding C (2012) Robust matrix completion via joint schatten \(p\)-norm and \(l_p\)-norm minimization. In: IEEE international conference on data mining (ICDM), pp 566–574Google Scholar
  19. 19.
    Huang J, Nie F, Huang H, Lei Y, Ding C (2013) Social trust prediction using rank-k matrix recovery. In: 23rd international joint conference on artificial intelligenceGoogle Scholar
  20. 20.
    Huang J, Nie F, Huang H (2013) Robust discrete matrix completion. In: Twenty-seventh AAAI conference on artificial intelligence (AAAI-13), pp 424–430Google Scholar
  21. 21.
    Tan VY, Balzano L, Draper SC (2011) Rank minimization over finite fields. In: IEEE international symposium on information theory proceedings (ISIT), pp 1195–1199Google Scholar
  22. 22.
    Meka R, Jain P, Dhillon IS (2010) Guaranteed rank minimization via singular value projection. In: Conference on neural information processing systems (NIPS)Google Scholar
  23. 23.
    Gabidulin EM (1985) Theory of codes with maximum rank distance. Problemy Peredachi Informatsii 21(1):3–16MathSciNetGoogle Scholar
  24. 24.
    Loidreau P (2008) Properties of codes in rank metric. In: Eleventh international workshop on algebraic and combinatorial coding theory, pp 192–198Google Scholar
  25. 25.
    Fazel M, Hindi H, Boyd SP (2001) A rank minimization heuristic with application to minimum order system approximation. IEEE Am Control Conf 6:4734–4739CrossRefGoogle Scholar
  26. 26.
    Fazel M, Hindi H, Boyd SP (2003) Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices. IEEE Am Control Conf 3:2156–2162Google Scholar
  27. 27.
    Blomer J, Karp R, Welzl E (1997) The rank of sparse random matrices over finite fields. Random Struct Algorithms 10(4):407–420CrossRefMathSciNetGoogle Scholar
  28. 28.
    Ma S, Goldfarb D, Chen L (2011) Fixed point and Bregman iterative methods for matrix rank minimization. Math Program 128(1–2):321–353CrossRefzbMATHMathSciNetGoogle Scholar
  29. 29.
    Nie F, Huang H, Ding CHQ (2012) Low-rank matrix recovery via efficient schatten p-norm minimization. In: AAAI conference on artificial intelligenceGoogle Scholar
  30. 30.
    Nie F, Huang H, Cai X, Ding C (2010) Efficient and robust feature selection via joint \(\ell _{2,1}\)-norms minimization. In: Conference on neural information processing systems (NIPS)Google Scholar
  31. 31.
    Powell MJD (1969) A method for nonlinear constraints in minimization problems. In: Fletcher R (ed) Optimization. Academic Press, LondonGoogle Scholar
  32. 32.
    Hestenes MR (1969) Multiplier and gradient methods. J Opt Theory Appl 4:303–320CrossRefzbMATHMathSciNetGoogle Scholar
  33. 33.
    Bertsekas DP (1996) Constrained optimization and lagrange multiplier methods. Athena Scientific, BelmontGoogle Scholar
  34. 34.
    Gabay D, Mercier B (1969) A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput Math Appl 2(1):17–40CrossRefGoogle Scholar
  35. 35.
    Wright J, Ganesh A, Rao S, Ma Y (2009) Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: The proceedings of the conference on neural information processing systems. pp 1–9Google Scholar
  36. 36.
    Candes E, Plan Y (2010) Matrix completion with noise. Proc IEEE 98(6):925–936CrossRefGoogle Scholar
  37. 37.
    Salakhutdinov R, Mnih A (2008) Probabilistic matrix factorization. Adv Neural Inf Process Syst (NIPS) 20:1257–1264Google Scholar
  38. 38.
    Gu Q, Zhou J, Ding C (2010) Collaborative filtering: weighted nonnegative matrix factorization incorporating user and item graphs. In: Siam data mining conferenceGoogle Scholar
  39. 39.
    Leskovec J, Huttenlocher D, Kleinberg J (2010) Predicting positive and negative links in online social networks. In: International world wide web conference (WWW). ACM, pp 641–650Google Scholar
  40. 40.
    Leskovec J, Lang K, Dasgupta A, Mahoney M (2009) Community structure in large networks: natural cluster sizes and the absence of large well-defined clusters. Int Math 6(1):29–123zbMATHMathSciNetGoogle Scholar
  41. 41.
    Newman M (2001) Clustering and preferential attachment in growing networks. Phys Rev E 64(2):025102CrossRefGoogle Scholar
  42. 42.
    Billsus D, Pazzani M (1998) Learning collaborative information filters. In: International conference on machine learning (ICML), pp 46–54Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringUniversity of Texas at ArlingtonArlingtonUSA
  2. 2.Department of Electrical Engineering and Computer ScienceColorado School of MinesGoldenUSA

Personalised recommendations