A Probabilistic Approach to Robust Matrix Factorization

  • Naiyan Wang
  • Tiansheng Yao
  • Jingdong Wang
  • Dit-Yan Yeung
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7578)


Matrix factorization underlies a large variety of computer vision applications. It is a particularly challenging problem for large-scale applications and when there exist outliers and missing data. In this paper, we propose a novel probabilistic model called Probabilistic Robust Matrix Factorization (PRMF) to solve this problem. In particular, PRMF is formulated with a Laplace error and a Gaussian prior which correspond to an ℓ1 loss and an ℓ2 regularizer, respectively. For model learning, we devise a parallelizable expectation-maximization (EM) algorithm which can potentially be applied to large-scale applications. We also propose an online extension of the algorithm for sequential data to offer further scalability. Experiments conducted on both synthetic data and some practical computer vision applications show that PRMF is comparable to other state-of-the-art robust matrix factorization methods in terms of accuracy and outperforms them particularly for large data matrices.


Online Algorithm Laplace Distribution Structure From Motion Nuclear Norm Computer Vision Application 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. International Journal of Computer Vision 9(2), 137–154 (1992)CrossRefGoogle Scholar
  2. 2.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3d shape from image streams. In: CVPR, vol. 2, pp. 690–696 (2000)Google Scholar
  3. 3.
    Aanæs, H., Fisker, R., Astrom, K., Carstensen, J.M.: Robust factorization. IEEE Trans. Pattern Anal. Mach. Intelligence 24(9), 1215–1225 (2002)CrossRefGoogle Scholar
  4. 4.
    Ke, Q., Kanade, T.: Robust l1 norm factorization in the presence of outliers and missing data by alternative convex programming. In: CVPR, pp. 739–746 (2005)Google Scholar
  5. 5.
    Eriksson, A., van den Hengel, A.: Efficient computation of robust low-rank matrix approximations in the presence of missing data using the l1 norm. In: CVPR, pp. 771–778 (2010)Google Scholar
  6. 6.
    Salakhutdinov, R., Mnih, A.: Probabilistic matrix factorization. In: NIPS, vol. 20 (2008)Google Scholar
  7. 7.
    Lin, Z., Chen, M., Wu, L., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. Arxiv preprint arXiv:1009.5055 (2010)Google Scholar
  8. 8.
    Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58(3), 11 (2011)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Salakhutdinov, R., Mnih, A.: Bayesian probabilistic matrix factorization using markov chain monte carlo. In: ICML, pp. 880–887 (2008)Google Scholar
  10. 10.
    Lakshminarayanan, B., Bouchard, G., Archambeau, C.: Robust bayesian matrix factorisation. In: AISTATS (2011)Google Scholar
  11. 11.
    Mu, Y., Dong, J., Yuan, X., Yan, S.: Accelerated low-rank visual recovery by random projection. In: CVPR, pp. 2609–2616 (2011)Google Scholar
  12. 12.
    Zhou, T., Tao, D.: Godec: Randomized low-rank & sparse matrix decomposition in noisy case. In: ICML (2011)Google Scholar
  13. 13.
    He, J., Balzano, L., Szlam, A.: Incremental gradient on the grassmannian for online foreground and background separation in subsampled video. In: CVPR (2012)Google Scholar
  14. 14.
    Carin, L., Ding, X., He, L.: Bayesian robust principal component analysis. IEEE Transactions on Image Processing 20(12), 3419–3430 (2011)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Mazumder, R., Hastie, T., Tibshirani, R.: Spectral regularization algorithms for learning large incomplete matrices. The Journal of Machine Learning Research 11, 2287–2322 (2010)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Lange, K., Sinsheimer, J.S.: Normal/independent distributions and their applications in robust regression. Journal of Computational and Graphical Statistics, 175–198 (1993)Google Scholar
  17. 17.
    Jebara, T., Pentland, A.: Maximum conditional likelihood via bound maximization and the CEM algorithm. In: NIPS, vol. 11 (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Naiyan Wang
    • 1
  • Tiansheng Yao
    • 2
  • Jingdong Wang
    • 3
  • Dit-Yan Yeung
    • 1
  1. 1.Department of Computer Science and EngineeringHong Kong University of Science and TechnologyHong Kong, China
  2. 2.Computer Science DepartmentUniversity of CaliforniaLos AngelesUSA
  3. 3.Microsoft Research AsiaBeijingChina

Personalised recommendations