Multi-view spectral clustering via robust local subspace learning

Abstract

Because of the existence of multiple sources of datasets, multi-view clustering has a wide range of applications in data mining and pattern recognition. Multi-view could utilize complementary information that existed in multiple views to improve the performance of clustering. Recently, there have been multi-view clustering methods which improved the performance of clustering to some extent. However, they do not take local representation relationship into consideration and local representation relationship is a crucial technology in subspace learning. To solve this problem, we proposed a novel multi-view clustering algorithm via robust local representation. We consider that all the views are derived from a robust unified subspace and noisy. To get the robust similarity matrix we simultaneously take all the local reconstruction relationships into consideration and use L1-norm to guarantee the sparsity. We give an iteration solution for this problem and give the proof of correctness. We compare our method with a number of classical methods on real-world and synthetic datasets to show the efficacy of the proposed algorithm.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Banerjee A, Dhillon I, Ghosh J et al (2003) Generative model-based clustering of directional data. In: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 19–28

  2. Bickel S, Scheffer T (2004) Multi-view clustering. ICDM 4:19–26

    Google Scholar 

  3. Blaschko MB, Lampert CH (2008) Correlational spectral clustering. Computer Vision and Pattern Recognition CVPR 2008. IEEE Conference on. IEEE, pp 1–8

  4. Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings of the eleventh annual conference on Computational learning theory. ACM, pp 92–100

  5. Cai D, He X, Han J (2005) Document clustering using locality preserving indexing. IEEE Trans Knowl Data Eng 17(12):1624–1637

    Article  Google Scholar 

  6. Chaudhuri K, Kakade SM, Livescu K et al (2009) Multi-view clustering via canonical correlation analysis. In: Proceedings of the 26th annual international conference on machine learning. ACM, pp 129-136

  7. Cortes C, Mohri M, Rostamizadeh A (2009) Learning non-linear combinations of kernels. Adv Neural Inform Process Syst:396–404

  8. de Sa VR (2005) Spectral clustering with two views. ICML workshop on learning with multiple views, pp 20–27

  9. Deng C, Lv Z, Liu W et al (2015) Multi-view matrix decomposition: a new scheme for exploring discriminative information. In: Proceedings of the 24th International Conference on Artificial Intelligence. AAAI Press, pp 3438–3444

  10. Ding C, Li T, Jordan M (2010) Convex and semi-nonnegative matrix factorizations. IEEE Trans Pattern Anal Mach Intell 32(1):45–55

    Article  Google Scholar 

  11. Gui J, Tao D, Sun Z et al (2014) Group sparse multiview patch alignment framework with view consistency for image classification. IEEE Trans Image Process 23(7):3126–3137

    MathSciNet  Article  Google Scholar 

  12. Hubert L, Arabie P (1985) Comparing partitions. J Classif 2(1):193–218

    Article  MATH  Google Scholar 

  13. Karasuyama M, Mamitsuka H (2013) Manifold-based similarity adaptation for label propagation. Adv Neural Inform Process Syst:1547–1555

  14. Kong D, Ding CHQ, Huang H et al (2012) An iterative locally linear embedding algorithm. arXiv preprint arXiv:1206.6463

  15. Kumar A, Rai P, Daume H (2011) Co-regularized multi-view spectral clustering. Adv Neural Inform Process Syst 2011:1413–1421

    Google Scholar 

  16. Kumar A, Daum H (2011) A co-training approach for multi-view spectral clustering. Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp 393–400

  17. Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. Adv Neural Inform Process Syst:556–562

  18. Liu J, Wang C, Gao J et al (2013) Multi-view clustering via joint nonnegative matrix factorization. In: Proc. of SDM. vol 13, pp 252–260

  19. Long B, Philip SY, Zhongfei (Mark) Z (2008) A general model for multiple view unsupervised learning. SDM:822–833

  20. Lovsz L, Plummer MD (2009) Matching theory. In: American Mathematical Society

  21. Luo Y, Liu T, Tao D et al (2015) Multi-view matrix completion for multi-label image classification. IEEE Trans Image Process 24(8):2355–2368

  22. Manning CD, Raghavan P, Schtze H (2008) Introduction to information retrieval. Cambridge University Press, Cambridge

    Google Scholar 

  23. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  24. Sun S (2013) A survey of multi-view machine learning. Neural Comput Appl 23(7–8):2031–2038

    Article  Google Scholar 

  25. Sun S, Jin F (2011) Robust co-training. Int J Pattern Recogn Artif Intell 25(07):1113–1126

    MathSciNet  Article  Google Scholar 

  26. Tang W, Lu Z, Dhillon IS (2009) Clustering with multiple graphs, Data Mining. ICDM’09. Ninth IEEE International Conference on. IEEE, pp 1016–1021

  27. Tzortzis G, Likas A (2012) Kernel-based weighted multi-view clustering. Data Mining (ICDM), 2012 IEEE 12th International Conference on IEEE, pp 675–684

  28. Tzortzis GF, Likas AC (2010) Multiple view clustering using a weighted combination of exemplar-based mixture models. IEEE Trans Neural Netw 21(12):1925–1938

    Article  Google Scholar 

  29. Von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416

    MathSciNet  Article  Google Scholar 

  30. Wang W, Zhou ZH (2007) Analyzing co-training style algorithms. Machine Learning: ECML. Springer, Berlin Heidelberg, pp 454–465

  31. Wang W, Zhou ZH (2010) A new analysis of co-training. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp 1135–1142

  32. Xia T, Tao D, Mei T et al (2010) Multiview spectral embedding. IEEE Trans Syst Man Cybern Part B: Cybern 40(6):1438–1446

    Article  Google Scholar 

  33. Xia R, Pan Y, Du L et al (2014) Robust multi-view spectral clustering via low-rank and sparse decomposition. In: AAAI Conference on Artificial Intelligence, pp 2149–2155

  34. Xu C, Tao D, Xu C (2015a) Multi-view learning with incomplete views. IEEE Trans Image Process 24(12):5812–5824

    MathSciNet  Article  Google Scholar 

  35. Xu C, Tao D, Xu C (2015b) Multi-view intact space learning. IEEE Trans Pattern Anal Mach Intell 37(12):2531–2544

    Article  Google Scholar 

  36. Xu W, Liu X, Gong Y (2003) Document clustering based on non-negative matrix factorization. In: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval. ACM, pp 267–273

  37. Xu C, Tao D, Xu C (2013) A survey on multi-view learning. arXiv preprint: arXiv:1304.5634

  38. Xu C, Tao D, Xu C (2015c) Multi-view self-paced learning for clustering. In: Proceedings of the 24th International Conference on Artificial Intelligence. AAAI Press, pp 3974–3980

  39. Zhang Q, Zhang L, Zhang L et al (2015) Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding. Pattern Recogn 48(10):3102–3112

    Article  Google Scholar 

  40. Zhang X, Zhao L, Zong L et al (2014) Multi-view clustering via multi-manifold regularized nonnegative matrix factorization. Data Mining (ICDM), 2014 IEEE International Conference on IEEE, pp 1103–1108

  41. Zhuang J, Wang J, HOI CH et al (2011) Unsupervised multiple kernel learning. J Mach Learn Res (JMLR) 20:129–144

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Lin Feng.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Funding

This study funded by National Natural Science Foundation of People’s Republic of China 61173163, 61370200).

Additional information

Communicated by A. Di Nola.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Feng, L., Cai, L., Liu, Y. et al. Multi-view spectral clustering via robust local subspace learning. Soft Comput 21, 1937–1948 (2017). https://doi.org/10.1007/s00500-016-2120-3

Download citation

Keywords

  • Multi-view learning
  • Spectral clustering
  • Robust local subspace learning