Advertisement

International Journal of Computer Vision

, Volume 111, Issue 3, pp 315–344 | Cite as

Practical Matrix Completion and Corruption Recovery Using Proximal Alternating Robust Subspace Minimization

  • Yu-Xiang WangEmail author
  • Choon Meng Lee
  • Loong-Fah Cheong
  • Kim-Chuan Toh
Article

Abstract

Low-rank matrix completion is a problem of immense practical importance. Recent works on the subject often use nuclear norm as a convex surrogate of the rank function. Despite its solid theoretical foundation, the convex version of the problem often fails to work satisfactorily in real-life applications. Real data often suffer from very few observations, with support not meeting the randomness requirements, ubiquitous presence of noise and potentially gross corruptions, sometimes with these simultaneously occurring. This paper proposes a Proximal Alternating Robust Subspace Minimization method to tackle the three problems. The proximal alternating scheme explicitly exploits the rank constraint on the completed matrix and uses the \(\ell _0\) pseudo-norm directly in the corruption recovery step. We show that the proposed method for the non-convex and non-smooth model converges to a stationary point. Although it is not guaranteed to find the global optimal solution, in practice we find that our algorithm can typically arrive at a good local minimizer when it is supplied with a reasonably good starting point based on convex optimization. Extensive experiments with challenging synthetic and real data demonstrate that our algorithm succeeds in a much larger range of practical problems where convex optimization fails, and it also outperforms various state-of-the-art algorithms.

Keywords

Low rank matrix completion Robust matrix factorization Non-convex optimization SfM Photometric stereo 

Notes

Acknowledgments

This work was supported by the Singapore PSF grant 1321202075.

References

  1. Attouch, H., Bolte, J., Redont, P., & Soubeyran, A. (2010). Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka–Łojasiewicz inequality. Mathematics of Operations Research, 35(2), 438–457.CrossRefzbMATHMathSciNetGoogle Scholar
  2. Balzano, L., Nowak, R., & Recht, B. (2010). Online identification and tracking of subspaces from highly incomplete information. In: 2010 48th Annual allerton conference on communication, control, and computing (Allerton) (pp. 704–711). IEEE.Google Scholar
  3. Beck, A., & Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1), 183–202.CrossRefzbMATHMathSciNetGoogle Scholar
  4. Becker, S. R., Candès, E. J., & Grant, M. C. (2011). Templates for convex cone problems with applications to sparse signal recovery. Mathematical Programming Computation, 3(3), 165–218.CrossRefzbMATHMathSciNetGoogle Scholar
  5. Becker, S. R., Candès, E. J., & Grant, M. C. (2012). TFOCS: Templates for first-order conic solvers. http://cvxr.com/tfocs/.
  6. Bennett, J., Lanning, S., & Netflix, N. (2007). The Netflix Prize. In In KDD Cup and Workshop in conjunction with KDD. Google Scholar
  7. Buchanan, A. M., & Fitzgibbon, A. W. (2005). Damped Newton algorithms for matrix factorization with missing data. IJCV, 2, 316– 322.Google Scholar
  8. Candès, E., & Plan, Y. (2010). Matrix completion with noise. Proceedings of the IEEE, 98(6), 925–936.CrossRefGoogle Scholar
  9. Candès, E., & Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6), 717–772.CrossRefzbMATHMathSciNetGoogle Scholar
  10. Candès, E., Li, X., Ma, Y., & Wright, J. (2011). Robust principal component analysis? Journal of the ACM, 58(3), 11.Google Scholar
  11. Candès, E. J., & Tao, T. (2010). The power of convex relaxation: Near-optimal matrix completion. IEEE Transactions on Information Theory, 56(5), 2053–2080.CrossRefGoogle Scholar
  12. Candès, E. J., Wakin, M. B., & Boyd, S. P. (2008). Enhancing sparsity by reweighted \(\ell _1\) minimization. Journal of Fourier Analysis and Applications, 14(5–6), 877–905.CrossRefzbMATHMathSciNetGoogle Scholar
  13. Chandrasekaran, V., Sanghavi, S., Parrilo, P., & Willsky, A. (2011). Rank-sparsity incoherence for matrix decomposition. SIAM Journal on Optimization, 21, 572–596.CrossRefzbMATHMathSciNetGoogle Scholar
  14. Chen, P. (2008). Optimization algorithms on subspaces: Revisiting missing data problem in low-rank matrix. IJCV, 80(1), 125–142.CrossRefGoogle Scholar
  15. Chen, P. (2011). Hessian matrix vs. Gauss–Newton hessian matrix. SIAM Journal on Numerical Analysis, 49(4), 1417–1435.CrossRefzbMATHMathSciNetGoogle Scholar
  16. Chen, Y., Jalali, A., Sanghavi, S., & Caramanis, C. (2011). Low-rank matrix recovery from errors and erasures. In: 2011 IEEE international symposium on information theory proceedings (ISIT) (pp. 2313–2317). IEEE.Google Scholar
  17. Clarke, F. H. (1990). Optimization and nonsmooth analysis (Vol. 5). SIAM.Google Scholar
  18. Del Bue, A., Xavier, J., Agapito, L., & Paladini, M. (2012). Bilinear modeling via augmented lagrange multipliers (balm). IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(8), 1496–1508.CrossRefGoogle Scholar
  19. Donoho, D. L. (1995). De-noising by soft-thresholding. IEEE Transactions on Information Theory, 41(3), 613–627.CrossRefzbMATHMathSciNetGoogle Scholar
  20. Eriksson, A., & Van Den Hengel, A. (2010). Efficient computation of robust low-rank matrix approximations in the presence of missing data using the L1 norm. In CVPR (pp 771–778).Google Scholar
  21. Friedland, S., Niknejad, A., Kaveh, M., & Zare, H. (2006). An algorithm for missing value estimation for DNA microarray data. In: 2006 IEEE international conference on acoustics, speech and signal processing, 2006. ICASSP 2006 proceedings (Vol. 2, p. II). IEEE.Google Scholar
  22. Funk, S. (2006). Netflix update: Try this at home. http://sifter.org/simon/journal/20061211.html.
  23. Gabriel, K. R., & Zamir, S. (1979). Lower rank approximation of matrices by least squares with any choice of weights. Technometrics, 21(4), 489–498.CrossRefzbMATHGoogle Scholar
  24. Ganesh, A., Wright, J., Li, X., Candès, E. J., & Ma, Y. (2010). Dense error correction for low-rank matrices via principal component pursuit. In: 2010 IEEE international symposium on information theory proceedings (ISIT) (pp. 1513–1517). IEEE.Google Scholar
  25. Grant, M., & Boyd, S. (2008). Graph implementations for nonsmooth convex programs. In V. Blondel, S. Boyd, & H. Kimura (Eds.), Recent advances in learning and control, lecture notes in control and information sciences (pp. 95–110). Berlin: Springer.Google Scholar
  26. Grant, M., & Boyd, S. (2012). CVX: Matlab software for disciplined convex programming, version 2.0 beta. http://cvxr.com/cvx.
  27. Hartley, R., & Schaffalitzky, F. (2003). Powerfactorization: 3d reconstruction with missing or uncertain data. In: Australia-Japan advanced workshop on computer vision (Vol. 74, pp. 76–85).Google Scholar
  28. He, J., Balzano, L. & Lui, J. (2011). Online robust subspace tracking from partial information. arXiv:1109.3827.
  29. Horn, B. K. (1990). Height and gradient from shading. International Journal of Computer Vision, 5(1), 37–75.CrossRefGoogle Scholar
  30. Jain, P., Netrapalli, P., Sanghavi, S. (2012). Low-rank matrix completion using alternating minimization. arXiv:1212.0467.
  31. Ke, Q., & Kanade, T. (2005). Robust l1 norm factorization in the presence of outliers and missing data by alternative convex programming. CVPR, 1, 739–746.Google Scholar
  32. Keshavan, R., Montanari, A., & Oh, S. (2009). Low-rank matrix completion with noisy observations: A quantitative comparison. In: Communication, control, and computing (pp. 1216–1222).Google Scholar
  33. Kiràly, F. , & Tomioka, R. (2012). A combinatorial algebraic approach for the identifiability of low-rank matrix completion. In: J. Langford & J. Pineau (Eds.), Proceedings of the 29th international conference on machine learning (ICML-12), Omnipress, New York, NY, USA. ICML ’12 (pp 967–974).Google Scholar
  34. Koren, Y. (2009). The Bellkor solution to the Netflix grand prize. Netflix prize documentation, 81.Google Scholar
  35. Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8), 30–37.CrossRefGoogle Scholar
  36. Lee, K. C., Ho, J., & Kriegman, D. J. (2005). Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5), 684–698.CrossRefGoogle Scholar
  37. Li, X. (2013). Compressed sensing and matrix completion with constant proportion of corruptions. Constructive Approximation, 37(1), 73–99.CrossRefzbMATHMathSciNetGoogle Scholar
  38. Liu, Y., Sun, D., & Toh, K. (2009). An implementable proximal point algorithmic framework for nuclear norm minimization. Mathematical Programming, 133(1–2), 1–38.MathSciNetGoogle Scholar
  39. Negahban, S., & Wainwright, M. J. (2012). Restricted strong convexity and weighted matrix completion: Optimal bounds with noise. The Journal of Machine Learning Research, 13, 1665–1697.zbMATHMathSciNetGoogle Scholar
  40. Nehab, D., Rusinkiewicz, S., Davis, J., & Ramamoorthi, R. (2005). Efficiently combining positions and normals for precise 3d geometry. In ACM transactions on graphics (TOG) (Vol. 24, pp. 536–543). ACM.Google Scholar
  41. Oh, S., Montanari, A., & Karbasi, A. (2010). Sensor network localization from local connectivity: Performance analysis for the mds-map algorithm. In: 2010 IEEE information theory workshop (ITW) (pp. 1–5). IEEE.Google Scholar
  42. Okatani, T., & Deguchi, K. (2007). On the wiberg algorithm for matrix factorization in the presence of missing components. IJCV, 72(3), 329–337.CrossRefGoogle Scholar
  43. Overton, M. L. (n.d.). NLCG: Nonlinear conjugate gradient. http://www.cs.nyu.edu/faculty/overton/software/nlcg/index.html.
  44. Paladini, M., Bue, A. D., Stosic, M., Dodig, M., Xavier, J., & Agapito, L. (2009). Factorization for non-rigid and articulated structure using metric projections. In CVPR (pp. 2898–2905).Google Scholar
  45. Rabaud. V. (n.d.). Vincent’s structure from motion toolbox. http://vision.ucsd.edu/vrabaud/toolbox/.
  46. Recht, B. (2009). A simpler approach to matrix completion. arXiv:0910.0651.
  47. Recht, B., Re, C., Wright, S., & Niu, F. (2011). Hogwild: A lock-free approach to parallelizing stochastic gradient descent. Advances in Neural Information Processing Systems, 24, 693–701. Google Scholar
  48. Rudin, W. (1987). Real analysis (Vol. 84). New York: McGraw-Hill.zbMATHGoogle Scholar
  49. Shi, X., & Yu, P. S. (2011). Limitations of matrix completion via trace norm minimization. ACM SIGKDD Explorations Newsletter, 12(2), 16–20.CrossRefGoogle Scholar
  50. Srebro, N., & Jaakkola, T. (2003). Weighted low-rank approximations. In: Proceedings of the 20th international conference on machine learning (ICML-2003) (Vol. 20, p. 720).Google Scholar
  51. Sturm, P., & Triggs, B. (1996). A factorization based algorithm for multi-image projective structure and motion. In 4th European conference on computer vision (pp. 709–720). Springer, Berlin.Google Scholar
  52. Toh, K., & Yun, S. (2010). An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pacific Journal of Optimization, 6, 615–640.Google Scholar
  53. Tomasi, C., & Kanade, T. (1992). Shape and motion from image streams under orthography: A factorization method. IJCV, 9(2), 137–154.CrossRefGoogle Scholar
  54. Wang, Y. X., & Xu, H. (2012). Stability of matrix factorization for collaborative filtering. In J. Langford & J. Pineau (Eds.), Proceedings of the 29th international conference on machine learning (ICML-12), Omnipress, New York, NY, USA (pp. 417–424). ICML ’12.Google Scholar
  55. Wen, Z., & Yin, W. (2013). A feasible method for optimization with orthogonality constraints. Mathematical Programming, 142(1–2), 397–434. Google Scholar
  56. Wu, B., Ding, C., Sun, D., & Toh, K. C. (2014). On the Moreau-Yosida regularization of the vector k-norm related functions. SIAM Journal on Optimization, 24(2), 766–794.Google Scholar
  57. Wu, L., Ganesh, A., Shi, B., Matsushita, Y., Wang, Y., & Ma, Y. (2011). Robust photometric stereo via low-rank matrix completion and recovery. In ACCV (pp. 703–717).Google Scholar
  58. Xiong, L., Chen, X., & Schneider, J. (2011). Direct robust matrix factorizatoin for anomaly detection. In: 2011 IEEE 11th international conference on data mining (ICDM) (pp. 844–853). IEEE.Google Scholar
  59. Yang, J., Sun, D., & Toh, K. C. (2013). A proximal point algorithm for log-determinant optimization with group Lasso regularization. SIAM Journal on Optimization, 23(2), 857–893.Google Scholar
  60. Yu, H. F., Hsieh, C. J., Si, S., & Dhillon, I. (2012). Scalable coordinate descent approaches to parallel matrix factorization for recommender systems. In 2012 IEEE 12th international conference on data mining (ICDM) (pp. 765–774). IEEE.Google Scholar
  61. Zhou, Z., Li, X., Wright, J., Candès, E., & Ma, Y. (2010). Stable principal component pursuit. In 2010 IEEE international symposium on information theory proceedings (ISIT) (pp. 1518–1522). IEEE.Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Yu-Xiang Wang
    • 1
    • 2
    Email author
  • Choon Meng Lee
    • 2
  • Loong-Fah Cheong
    • 2
  • Kim-Chuan Toh
    • 2
  1. 1.Carnegie Mellon UniversityPittsburghUSA
  2. 2.National University of SingaporeSingaporeSingapore

Personalised recommendations