Skip to main content
SpringerLink
Log in

The latest research progress on spectral clustering

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Spectral clustering is a clustering method based on algebraic graph theory. It has aroused extensive attention of academia in recent years, due to its solid theoretical foundation, as well as the good performance of clustering. This paper introduces the basic concepts of graph theory and reviews main matrix representations of the graph, then compares the objective functions of typical graph cut methods and explores the nature of spectral clustering algorithm. We also summarize the latest research achievements of spectral clustering and discuss several key issues in spectral clustering, such as how to construct similarity matrix and Laplacian matrix, how to select eigenvectors, how to determine cluster number, and the applications of spectral clustering. At last, we propose several valuable research directions in light of the deficiencies of spectral clustering algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Adefioye AA, Liu XH, Moor BD (2013) Multi-view spectral clustering and its chemical application. Int J Comput Biol Drug Des 6(1–2):32–49

    Article  Google Scholar 

  2. Alpert CJ, Kahng AB (1995) Multi-way partitioning via geometric embeddings, orderings and dynamic programming. IEEE Trans Comput-Aaid Des Integr Circuits Syst 14(11):1342–1358

    Article  Google Scholar 

  3. Alpert CJ, Yao SZ (1995) Spectral partitioning: the more eigenvectors, the better. In: Proceedings of the 32nd annual ACM/IEEE design automation conference. ACM, New York, pp 195–200

  4. Alzate C, Suykens JAK (2012) Hierarchical kernel spectral clustering. Neural Netw 35:21–30

    Article  MATH  Google Scholar 

  5. Bach FR, Jordan MI (2006) Learning spectral clustering, with application to speech separation. J Mach Learn Res 7:1963–2001

    MathSciNet  MATH  Google Scholar 

  6. Bames ER (1982) An algorithm for partitioning the nodes of a graph. SIAM J Algebraic Discrete Methods 17(5):541–550

    Google Scholar 

  7. Blekas K, Lagaris IE (2013) A spectral clustering approach based on Newton’s equations of motion. Int J Intell Syst 28(4):394–410

    Article  Google Scholar 

  8. Cai XY, Dai GZ, Yang LB (2008) Survey on spectral clustering algorithms. Comput Sci 35(7):14–18

    Google Scholar 

  9. Chasanis VT, Likas AC, Galatsanos NP (2009) Scene detection in videos using shot clustering and sequence alignment. IEEE Trans Multimed 11(1):89–100

    Article  Google Scholar 

  10. Chen WF, Feng GC (2012) Spectral clustering with discriminant cuts. Knowl-Based Syst 28:27–37

    Article  Google Scholar 

  11. Chen WF, Feng GC (2012) Spectral clustering: a semi-supervised approach. Neurocomputing 77(1):229–242

    Article  Google Scholar 

  12. Chen WY, Song YQ, Bai HJ et al (2011) Parallel spectral clustering in distributed systems. IEEE Trans Patt Anal Mach Intell 33(3):568–586

    Article  Google Scholar 

  13. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B Stat Methodol 39(1):1–38

    MathSciNet  MATH  Google Scholar 

  14. Ding CHQ, He X, Zha H et al (2001) A min-max cut algorithm for graph partitioning and data clustering. In: Proceedings of IEEE international conference on data mining (ICDM’ 2001), pp 107–114

  15. Ding L, Gonzalez-Longatt FM, Wall P, Terzija V (2013) Two-step spectral clustering controlled islanding algorithm. IEEE Trans Power Syst 28(1):75–84

    Article  Google Scholar 

  16. Ding SF, Jia HJ, Zhang LW et al (2012) Research of semi-supervised spectral clustering algorithm based on pairwise constraints. Neural Comput Appl. doi:10.1007/s00521-012-1207-8

    Google Scholar 

  17. Ding SF, Qi BJ, Jia HJ et al (2013) Research of semi-supervised spectral clustering based on constraints expansion. Neural Comput Appl 22(Suppl 1):S405–S410

    Article  Google Scholar 

  18. Donath WE, Hoffman AJ (1973) Lower bounds for the partitioning of graph. IBM J Res Dev 17(5):420–425

    Article  MathSciNet  MATH  Google Scholar 

  19. Dong XW, Frossard P, Vandergheynst P, Nefedov N (2012) Clustering with multi-layer graphs: a spectral perspective. IEEE Trans Sig Process 60(11):5820–5831

    Article  MathSciNet  Google Scholar 

  20. Driessche RV, Roose D (1995) An improved spectral bisection algorithm and its application to dynamic load balancing. Parallel Comput 21(1):29–48

    Article  MathSciNet  MATH  Google Scholar 

  21. Dunn JC (1974) Well-separated clusters and the optimal fuzzy partitions. J Cybern 4(1):95–104

    Article  MathSciNet  Google Scholar 

  22. Fang YX, Wang JH (2012) Selection of the number of clusters via the bootstrap method. Comput Stat Data Anal 56(3):468–477

    Article  MathSciNet  MATH  Google Scholar 

  23. Fiedler M (1973) Algebraic connectivity of graphs. Czechoslov Math J 23(2):298–305

    MathSciNet  Google Scholar 

  24. Frederix K, Van Barel M (2013) Sparse spectral clustering method based on the incomplete Cholesky decomposition. J Comput Appl Math 237(1):145–161

    Article  MathSciNet  MATH  Google Scholar 

  25. Hagen L, Kahng AB (1992) New spectral methods for radio cut partitioning and clustering. IEEE Trans Comput-aid Des Integr Circuits Syst 11(9):1074–1085

    Article  Google Scholar 

  26. Hamad D, Biela P (2008) Introduction to spectral clustering. In: 3rd International conference on information and communication technologies: from theory to applications, 1–5, pp 490–495

  27. Hendrickson B, Leland R (1995) An improved spectral graph partitioning algorithm for mapping parallel computations. SIAM J Sci Comput 16(2):452–459

    Article  MathSciNet  MATH  Google Scholar 

  28. Higham DJ, Kibble M (2004) A unified view of spectral clustering. In: University of Strathclyde Mathematics Research Report 02

  29. Huang Z (1997) A fast clustering algorithm to cluster very large categorical data sets in data mining. In: Proceedings of the SIGMOD workshop on research issues on data mining and knowledge discovery. Tucson, pp 146–151

  30. Huang Z (1998) Extensions to the k-means algorithm for clustering large data sets with categorical values. Data Min Knowl Discov 2(3):283–304

    Article  Google Scholar 

  31. Jia JH, Xiao X, Liu BX, Jiao LC (2011) Bagging-based spectral clustering ensemble selection. Patt Recogn Lett 32(10):1456–1467

    Article  Google Scholar 

  32. Jiao LC, Shang FH, Wang F, Liu YY (2012) Fast semi-supervised clustering with enhanced spectral embedding. Patt Recogn 45(12):4358–4369

    Article  MATH  Google Scholar 

  33. Kluger Y, Basri R, Chang JT et al (2003) Spectral biclustering of microarray data: coclustering genes and conditions. Genome Res 13(4):703–716

    Article  Google Scholar 

  34. Leicht EA, Newman MEJ (2008) Community structure in directed networks. Phys Rev Lett 100(11):118703

    Article  Google Scholar 

  35. Li JY, Zhou JG, Guan JH et al (2011) A survey of clustering algorithms based on spectra of graphs. CAAI Trans Intell Syst 6(5):405–414

    Google Scholar 

  36. Li XY, Guo LJ (2012) Constructing affinity matrix in spectral clustering based on neighbor propagation. Neurocomputing 97:125–130

    Article  Google Scholar 

  37. Liu HQ, Jiao LC, Zhao F (2010) Non-local spatial spectral clustering for image segmentation. Neurocomputing 74(1–3):461–471

    Article  Google Scholar 

  38. Liu HQ, Zhao F, Jiao LC (2012) Fuzzy spectral clustering with robust spatial information for image segmentation. Appl Soft Comput 12(11):3636–3647

    Article  Google Scholar 

  39. Luo DJ, Huang H, Ding C, Nie FP (2010) On the eigenvectors of p-Laplacian. Mach Learn 81(1):37–51

    Article  MathSciNet  Google Scholar 

  40. Luxburg U, Belkin M, Bousquet O (2008) Consistency of spectral clustering. Ann Stat 36(2):555–586

    Article  MATH  Google Scholar 

  41. MacQueen J (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of 5th Berkeley symposium on mathematical statistics, 1, pp 281–297

  42. Malik J, Belongie S, Leung T et al (2001) Contour and texture analysis for image segmentation. Int J Comput Vis 43(1):7–27

    Article  MATH  Google Scholar 

  43. Meila M, Shi JB (2001) Learning segmentation by random walks. Advances in neural information processing systems. MIT Press, Cambridge, pp 873–879

    Google Scholar 

  44. Michoel T, Nachtergaele B (2012) Alignment and integration of complex networks by hypergraph-based spectral clustering. Phys Rev E 86(5):056111

    Google Scholar 

  45. Mirkin B, Nascimento S (2012) Additive spectral method for fuzzy cluster analysis of similarity data including community structure and affinity matrices. Inf Sci 183(1):16–34

    Article  Google Scholar 

  46. Mohar B (1997) Some applications of Laplace eigenvalues of graphs. Graph Symmetry Algebraic Methods Appl 497(22):227–275

    MathSciNet  Google Scholar 

  47. Nascimento MCV, de Carvalho ACPLF (2011) Spectral methods for graph clustering: a survey. Eur J Oper Res 211(2):221–231

    Article  MATH  Google Scholar 

  48. Newman MEJ (2004) Analysis of weighted networks. Phys Rev E 70(5):056131

    Google Scholar 

  49. Newman MEJ (2006) Finding community structure in networks using the eigenvectors of matrices. Phys Rev E 74(3):036104

    Google Scholar 

  50. Newman MEJ (2006) Modularity and community structure in networks. Proc Nat Acad Sci US 103(23):8577–8582

    Article  Google Scholar 

  51. Ng AY, Jordan MI, Weiss Y (2002) On spectral clustering: analysis and an algorithm. Adv Neural Inf Process Syst 14:849–856

    Google Scholar 

  52. Paccanaro A, Chennubhotla C, Casbon JA (2006) Spectral clustering of protein sequences. Nucl Acids Res 34(5):1571–1580

    Article  Google Scholar 

  53. Rebagliati N, Verri A (2011) Spectral clustering with more than K eigenvectors. Neurocomputing 74(9):1391–1401

    Article  Google Scholar 

  54. Sarkar S, Soundararajan P (2000) Supervised learning of large perceptual organization: graph spectral partitioning and learning automata. IEEE Trans Patt Anal Mach Intell 22(5):504–525

    Article  Google Scholar 

  55. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Patt Anal Mach Intell 22(8):888–905

    Article  Google Scholar 

  56. Sun JG, Liu J, Zhao LY (2008) Clustering algorithms research. J Softw 19(1):48–61

    Article  MATH  Google Scholar 

  57. Tepper M, Muse P, Almansa A, Mejail M (2011) Automatically finding clusters in normalized cuts. Patt Recogn 44(7):1372–1386

    Article  MATH  Google Scholar 

  58. Tung F, Wong A, Clausi DA (2010) Enabling scalable spectral clustering for image segmentation. Patt Recogn 43(12):4069–4076

    Article  MATH  Google Scholar 

  59. Urquhart R (1982) Graph theoretical clustering based on limited neighborhood sets. Pattern Recogn 15(3):173–187

    Article  MathSciNet  MATH  Google Scholar 

  60. von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416

    Article  MathSciNet  Google Scholar 

  61. Wang JH (2010) Consistent selection of the number of clusters via cross validation. Biometrika 97(4):893–904

    Article  MathSciNet  MATH  Google Scholar 

  62. Wang L, Bo LF, Jiao LC (2007) Density-sensitive spectral clustering. Acta Electronica Sinica 35(8):1577–1581

    Google Scholar 

  63. Wang LJ, Dong M (2012) Multi-level low-rank approximation-based spectral clustering for image segmentation. Patt Recogn Lett 33(16):2206–2215

    Article  Google Scholar 

  64. Wang Y, Jiang Y, Wu Y, Zhou ZH (2011) Spectral clustering on multiple manifolds. IEEE Trans Neural Netw 22(7):1149–1161

    Article  Google Scholar 

  65. Wei YC, Cheng CK (1989) Toward efficient hierarchical designs by ratio cut partitioning. In: IEEE international conference on CAD. New York, pp 298–301

  66. Wu Z, Leahy R (1993) An optimal graph theoretic approach to data clustering: theory and its application to image segmentation. IEEE Trans Patt Anal Mach Intell 15(11):1101–1113

    Article  Google Scholar 

  67. Xiang T, Gong S (2008) Spectral clustering with eigenvector selection. Patt Recogn 41(3):1012–1029

    Article  MATH  Google Scholar 

  68. Xie B, Wang M, Tao DC (2011) Toward the optimization of normalized graph Laplacian. IEEE Trans Neural Netw 22(4):660–666

    Article  Google Scholar 

  69. Xie YK, Zhou YQ, Huang XJ (2009) A spectral clustering based conference resolution method. J Chin Inf Process 23(3):10–16

    Google Scholar 

  70. Yang P, Zhu QS, Huang B (2011) Spectral clustering with density sensitive similarity function. Knowl-Based Syst 24(5):621–628

    Article  Google Scholar 

  71. Yang Y, Xu D, Nie FP, Yan SC, Zhuang YT (2010) Image clustering using local discriminant models and global integration. IEEE Trans Image Process 19(10):2761–2773

    Article  MathSciNet  Google Scholar 

  72. Zahn CT (1971) Graph-theoretic methods for detecting and describing gestalt clusters. IEEE Trans Comput 20(1):68–86

    Article  MATH  Google Scholar 

  73. Zeng S, Sang N, Tong XJ (2011) Hand-written numeral recognition based on spectrum clustering. In: MIPPR 2011: pattern recognition and computer vision, Proceedings of SPIE, p 8004

  74. Zhang XC, Li JW, Yu H (2011) Local density adaptive similarity measurement for spectral clustering. Patt Recogn Lett 32(2):352–358

    Article  Google Scholar 

  75. Zhang XC, You QZ (2011) An improved spectral clustering algorithm based on random walk. Frontiers Comput Sci China 5(3):268–278

    Article  MathSciNet  MATH  Google Scholar 

  76. Zhang XR, Jiao LC, Liu F (2008) Spectral clustering ensemble applied to SAR image segmentation. IEEE Trans Geosci Rem Sens 46(7):2126–2136

    Article  Google Scholar 

  77. Zhao F, Jiao LC, Liu HQ et al (2010) Spectral clustering with eigenvector selection based on entropy ranking. Neurocomputing 73(10–12):1704–1717

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Key Basic Research Program of China (No.2013CB329502), and the Fundamental Research Funds for the Central Universities (No.2013XK10).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Hongjie Jia, Shifei Ding, Xinzheng Xu & Ru Nie

  2. Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Science, Beijing, 100190, China

    Shifei Ding

Authors
  1. Hongjie Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shifei Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinzheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ru Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shifei Ding.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jia, H., Ding, S., Xu, X. et al. The latest research progress on spectral clustering. Neural Comput & Applic 24, 1477–1486 (2014). https://doi.org/10.1007/s00521-013-1439-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-013-1439-2

Keywords

Search

Navigation