Advertisement

Sparse Additive Subspace Clustering

  • Xiao-Tong Yuan
  • Ping Li
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8691)

Abstract

In this paper, we introduce and investigate a sparse additive model for subspace clustering problems. Our approach, named SASC (Sparse Additive Subspace Clustering), is essentially a functional extension of the Sparse Subspace Clustering (SSC) of Elhamifar & Vidal [7] to the additive nonparametric setting. To make our model computationally tractable, we express SASC in terms of a finite set of basis functions, and thus the formulated model can be estimated via solving a sequence of grouped Lasso optimization problems. We provide theoretical guarantees on the subspace recovery performance of our model. Empirical results on synthetic and real data demonstrate the effectiveness of SASC for clustering noisy data points into their original subspaces.

Keywords

Cluster Accuracy Subspace Cluster Sparse Pattern Motion Segmentation Spectral Cluster Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

978-3-319-10578-9_42_MOESM1_ESM.pdf (57 kb)
Electronic Supplementary Material (PDF 58 KB)

References

  1. 1.
    Agrawal, R., Gehrke, J., Gunopulos, D., Raghavan, P.: Automatic subspace clustering of high dimensional data for data mining applications. In: Proceedings of the 1998 ACM SIGMOD International Conference on Management of Data (SIGMOD 1998), pp. 94–105 (1998)Google Scholar
  2. 2.
    Bako, L.: Identification of switched linear systems via sparse optimization. Automatica 47(4), 668–677 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Bradley, P.S., Mangasarian, O.L.: K-plane clustering. Journal of Global Optimization 16(1), 23–32 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Chen, G., Lerman, G.: Spectral curvature clustering (scc). International Journal of Computer Vision 81(3), 317–330 (2009)CrossRefGoogle Scholar
  5. 5.
    Cheng, B., Yang, J., Yan, S., Fu, Y., Huang, T.: Learning with ℓ1-graph for image analysis. IEEE Transactions on Image Processing 19(4), 858–866 (2010)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Costeira, J., Kanade, T.: A multibody factorization method for independently moving objects. International Journal of Computer Vision 29(3), 159–179 (1998)CrossRefGoogle Scholar
  7. 7.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering: Algorithm, theory, and applications. IEEE Transactions on Pattern Analysis And Machine Intelligence 35(11), 2765–2781 (2013)CrossRefGoogle Scholar
  8. 8.
    Eriksson, B., Balzano, L., Nowak, R.: High-rank matrix completion. In: Proceedings of the 15th International Conference on Artificial Intelligence and Statistics (AISTATS 2012), pp. 373–381 (2012)Google Scholar
  9. 9.
    Hastie, T., Tibshirani, R.: Generalized Additive Models. Chapman & Hall/CRC (1990)Google Scholar
  10. 10.
    Jalali, A., Chen, Y., Sanghavi, S., Xu, H.: Clustering partially observed graphs via convex optimization. In: Proceedings of the Twenty-Eighth International Conference on Machine Learning (ICML 2011). ACM (2011)Google Scholar
  11. 11.
    Kriegel, H.P., Kröger, P., Zimek, A.: Clustering high-dimensional data: A survey on subspace clustering, pattern-based clustering, and correlation clustering. ACM Transactions on Knowledge Discovery from Data (TKDD) 3, 1–58 (2009)CrossRefGoogle Scholar
  12. 12.
    Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. Transactions on Pattern Analysis and Machine Intelligence 35(1), 171–184 (2013)CrossRefGoogle Scholar
  13. 13.
    Liu, H., Han, F., Yuan, M., Lafferty, J., Wasserman, L.: High dimensional semiparametric gaussian copula graphical models. The Annals of Statistics 40(4), 2293–2326 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Liu, H., Lafferty, J., Wasserman, L.: The nonparanormal: Semiparametric estimation of high dimensional undirected graphs. Journal of Machine Learning Research 10, 2295–2328 (2009)zbMATHMathSciNetGoogle Scholar
  15. 15.
    Lu, C.-Y., Min, H., Zhao, Z.-Q., Zhu, L., Huang, D.-S., Yan, S.: Robust and efficient subspace segmentation via least squares regression. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VII. LNCS, vol. 7578, pp. 347–360. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  16. 16.
    Ma, Y., Vidal, R.: Identification of deterministic switched arx systems via identification of algebraic varieties. In: Morari, M., Thiele, L. (eds.) HSCC 2005. LNCS, vol. 3414, pp. 449–465. Springer, Heidelberg (2005)Google Scholar
  17. 17.
    McWilliams, B., Montana, G.: Subspace clustering of high-dimensional data: A predictive approach. Data Mining and Knowledge Discovery 28(3), 736–772 (2014)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Proceedings of the 16th Annual Conference on Neural Information Processing Systems, NIPS 2002 (2002)Google Scholar
  19. 19.
    Ni, Y., Sun, J., Yuan, X.T., Yan, S., Cheong, L.F.: Robust low-rank subspace segmentation with semidefinite guarantees. In: Proceedings of the Workshop on Optimization Based Methods for Emerging Data Mining Problems (OEDM 2010 in conjunction with ICDM 2010) (2010)Google Scholar
  20. 20.
    Parsons, L., Haque, E., Liu, H.: Subspace clustering for high dimensional data: A review. ACM SIGKDD Explorations Newsletter 6(1), 90–105 (2004)CrossRefGoogle Scholar
  21. 21.
    Patel, V., Nguyen, H., Vidal, R.: Latent space sparse subspace clustering. In: Proceedings of IEEE International Conference on Computer Vision, ICCV 2013 (2013)Google Scholar
  22. 22.
    Ravikumar, P., Lafferty, J., Liu, H., Wasserman, L.: Sparse additive models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) (JRSSB) 71(5), 1009–1030 (2009)MathSciNetGoogle Scholar
  23. 23.
    Soltanolkotabi, M., Candès, E.J.: A geometric analysis of subspace clustering with outliers. The Annals of Statistics 40(4), 2195–2238 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    Soltanolkotabi, M., Elhamifar, E., Candès, E.J.: Robust subspace clustering. The Annals of Statistics (to appear, 2014)Google Scholar
  25. 25.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision 9(2), 137–154 (1992)CrossRefGoogle Scholar
  26. 26.
    Tron, R., Vidal, R.: A benchmark for the comparison of 3-d motion segmentation algorithms. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, CVPR 2007 (2007)Google Scholar
  27. 27.
    Tseng, P.: On accelerated proximal gradient methods for convex-concave optimization. Submitted to SIAM Journal of Optimization (2008)Google Scholar
  28. 28.
    Vidal, R.: Subspace clustering. IEEE Signal Processing Magazine 28(3), 52–68 (2011)CrossRefMathSciNetGoogle Scholar
  29. 29.
    Vidal, R., Ma, Y., Sastry, S.: Generalized principal component analysis (gpca). Transactions on Pattern Analysis and Machine Intelligence 27(12), 1945–1959 (2005)CrossRefGoogle Scholar
  30. 30.
    Vidal, R., Tron, R., Hartley, R.: Multiframe motion segmentation with missing data using power factorization and gpca. International Journal of Computer Vision 79, 85–105 (2008)CrossRefGoogle Scholar
  31. 31.
    Wainwright, M.J.: Sharp thresholds for high-dimensional and noisy sparsity recovery using ℓ1-constrained quadratic programming (lasso). IEEE Transactions on Information Theory 55(5), 2183–2202 (2009)CrossRefMathSciNetGoogle Scholar
  32. 32.
    Wang, Y., Xu, H.: Noisy sparse subspace clustering. In: Proceedings of the 30 th International Conference on Machine Learning (ICML 2013), pp. 849–856 (2013)Google Scholar
  33. 33.
    Xue, L., Zou, H.: Regularized rank-based estimation of high-dimensional nonparanormal graphical models. The Annals of Statistics 40(5), 2541–2571 (2012)CrossRefMathSciNetGoogle Scholar
  34. 34.
    Yan, J., Pollefeys, M.: A general framework for motion segmentation: Independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 94–106. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  35. 35.
    Yang, A., Wright, J., Ma, Y., Sastry, S.: Unsupervised segmentation of natural images via lossy data compression. Computer Vision and Image Understanding 110, 212–225 (2008)CrossRefGoogle Scholar
  36. 36.
    Yin, Y., Chen, X., Xing, E.: Group sparse additive models. In: Proceedings of the Twenty-Ninth International Conference on Machine Learning, ICML 2012 (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Xiao-Tong Yuan
    • 1
    • 2
  • Ping Li
    • 2
  1. 1.S-mart LabNanjing University of Information Science and Technology NanjingChina
  2. 2.Department of Statistics and Biostatistics, Department of Computer ScienceRutgers UniversityPiscatawayUSA

Personalised recommendations