Sparse and Low-Rank Methods

  • René Vidal
  • Yi Ma
  • S. Shankar Sastry
Part of the Interdisciplinary Applied Mathematics book series (IAM, volume 40)


The previous chapter studies a family of subspace clustering methods based on spectral clustering. In particular, we have studied both local and global methods for defining a subspace clustering affinity, and have noticed that we seem to be facing an important dilemma. On the one hand, local methods compute an affinity that depends only on the data points in a local neighborhood of each data point. Local methods can be rather efficient and somewhat robust to outliers, but they cannot deal well with intersecting subspaces. On the other hand, global methods utilize geometric information derived from the entire data set (or a large portion of it) to construct the affinity. Global methods might be immune to local mistakes, but they come with a big price: their computational complexity is often exponential in the dimension and number of subspaces. Moreover, none of the methods comes with a theoretical analysis that guarantees the correctness of clustering. Therefore, a natural question that arises is whether we can construct a subspace clustering affinity that utilizes global geometric relationships among all the data points, is computationally tractable when the dimension and number of subspaces are large, and is guaranteed to provide the correct clustering under certain conditions.


Sparse Subspace Clustering (SSC) Alternating Direction Method Of Multipliers (ADMM) Uncorrupted Data Disjoint Subspaces Outer Entrance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Amaldi, E., & Kann, V. (1998). On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theoretical Computer Science, 209, 237–260.MathSciNetCrossRefzbMATHGoogle Scholar
  2. Bertsekas, D. P. (1999). Nonlinear programming (2nd ed.). Optimization and computation (Vol. 2) Belmont: Athena Scientific.Google Scholar
  3. Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2010). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1), 1–122.CrossRefzbMATHGoogle Scholar
  4. Boyd, S., & Vandenberghe, L. (2004). Convex Optimization. Cambridge: Cambridge University Press.CrossRefzbMATHGoogle Scholar
  5. Candès, E., & Tao, T. (2005). Decoding by linear programming. IEEE Transactions on Information Theory, 51(12), 4203–4215.MathSciNetCrossRefzbMATHGoogle Scholar
  6. Costeira, J., & Kanade, T. (1998). A multibody factorization method for independently moving objects. International Journal of Computer Vision, 29(3), 159–179.CrossRefGoogle Scholar
  7. Donoho, D. L. (2005). Neighborly polytopes and sparse solution of underdetermined linear equations. Technical Report. Stanford University.Google Scholar
  8. Donoho, D. L. (2006). For most large underdetermined systems of linear equations the minimal 1-norm solution is also the sparsest solution. Communications on Pure and Applied Mathematics, 59(6), 797–829.MathSciNetCrossRefzbMATHGoogle Scholar
  9. Elhamifar, E., & Vidal, R. (2009). Sparse subspace clustering. In IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
  10. Elhamifar, E., & Vidal, R. (2010). Clustering disjoint subspaces via sparse representation. In IEEE International Conference on Acoustics, Speech, and Signal Processing.Google Scholar
  11. Elhamifar, E., & Vidal, R. (2013). Sparse subspace clustering: Algorithm, theory, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(11), 2765–2781.CrossRefGoogle Scholar
  12. Favaro, P., Vidal, R., & Ravichandran, A. (2011). A closed form solution to robust subspace estimation and clustering. In IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
  13. Goh, A., & Vidal, R. (2007). Segmenting motions of different types by unsupervised manifold clustering. In IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
  14. Kanatani, K. (2001). Motion segmentation by subspace separation and model selection. In IEEE International Conference on Computer Vision (Vol. 2, pp. 586–591).Google Scholar
  15. Kim, S. J., Koh, K., Lustig, M., Boyd, S., & Gorinevsky, D. (2007). An interior-point method for large-scale l1-regularized least squares. IEEE Journal on Selected Topics in Signal Processing, 1(4), 606–617.CrossRefGoogle Scholar
  16. Kontogiorgis, S., & Meyer, R. (1989). A variable-penalty alternating direction method for convex optimization. Mathematical Programming, 83, 29–53.MathSciNetzbMATHGoogle Scholar
  17. Lee, K.-C., Ho, J., & Kriegman, D. (2005). Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5), 684–698.CrossRefGoogle Scholar
  18. Lin, Z., Chen, M., Wu, L., & Ma, Y. (2011). The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:1009.5055v2.Google Scholar
  19. Lions, P., & Mercier, B. (1979). Splitting algorithms for the sum of two nonlinear operators. SIAM Journal on Numerical Analysis, 16(6), 964–979.MathSciNetCrossRefzbMATHGoogle Scholar
  20. Liu, G., Lin, Z., Yan, S., Sun, J., & Ma, Y. (2013). Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Analysis and Machine Intelligence, 35(1), 171–184.CrossRefGoogle Scholar
  21. Liu, G., Lin, Z., & Yu, Y. (2010). Robust subspace segmentation by low-rank representation. In International Conference on Machine Learning.Google Scholar
  22. Rao, S., Tron, R., Ma, Y., & Vidal, R. (2008). Motion segmentation via robust subspace separation in the presence of outlying, incomplete, or corrupted trajectories. In IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
  23. Rao, S., Tron, R., Vidal, R., & Ma, Y. (2010). Motion segmentation in the presence of outlying, incomplete, or corrupted trajectories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(10), 1832–1845.CrossRefGoogle Scholar
  24. Soltanolkotabi, M., & Candès, E. J. (2013). A geometric analysis of subspace clustering with outliers. Annals of Statistics, 40(4), 2195–2238.MathSciNetCrossRefzbMATHGoogle Scholar
  25. Soltanolkotabi, M., Elhamifar, E., & Candès, E. J. (2014). Robust subspace clustering. Annals of Statistics, 42(2), 669–699.MathSciNetCrossRefzbMATHGoogle Scholar
  26. Tibshirani, R. (1996). Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society B, 58(1), 267–288.MathSciNetzbMATHGoogle Scholar
  27. Vidal, R., & Favaro, P. (2014). Low rank subspace clustering (LRSC). Pattern Recognition Letters, 43, 47–61.CrossRefGoogle Scholar
  28. Vidal, R., Tron, R., & Hartley, R. (2008). Multiframe motion segmentation with missing data using PowerFactorization and GPCA. International Journal of Computer Vision, 79(1), 85–105.CrossRefGoogle Scholar
  29. Wang, Y.-X., & Xu, H. (2013). Noisy sparse subspace clustering. In International Conference on Machine learning.Google Scholar
  30. Wei, S., & Lin, Z. (2010). Analysis and improvement of low rank representation for subspace segmentation. Technical Report MSR-TR-2010-177, Microsoft Research Asia.Google Scholar
  31. Yuan, X., & Yang, J. (2009). Sparse and low-rank matrix decomposition via alternating direction methods. Preprint.Google Scholar

Copyright information

© Springer-Verlag New York 2016

Authors and Affiliations

  • René Vidal
    • 1
  • Yi Ma
    • 2
  • S. Shankar Sastry
    • 3
  1. 1.Center for Imaging Science Department of Biomedical EngineeringJohns Hopkins UniversityBaltimoreUSA
  2. 2.School of Information Science and Technology ShanghaiTech UniversityShanghaiChina
  3. 3.Department of Electrical Engineering and Computer ScienceUniversity of California BerkeleyBerkeleyUSA

Personalised recommendations