Subspace multi-clustering: a review

Survey Paper
  • 133 Downloads

Abstract

Clustering has been widely used to identify possible structures in data and help users to understand data in an unsupervised manner. Traditional clustering methods often provide a single partitioning of the data that groups similar data objects in one group while separates dissimilar ones into different groups. However, it has been well recognized that assuming only a single clustering for a data set can be too strict and cannot capture the diversity in applications. Multiple clustering structures can be hidden in the data, and each represents a unique perspective of the data. Different multi-clustering methods, which aim to discover multiple independent structures hidden in data, have been proposed in the recent decade. Although multi-clustering methods may provide more information for users, it is still challenging for users to efficiently and effectively understand each clustering structure. Subspace multi-clustering methods address this challenge by providing each clustering a feature subspace. Moreover, most subspace multi-clustering methods are especially scalable for high-dimensional data, which has become more and more popular in real applications due to the advances of big data technologies. In this paper, we focus on the subject of subspace multi-clustering, which has not been reviewed by any previous survey. We formulate the subspace multi-clustering problem and categorize the methodologies in different perspectives (e.g., de-coupled methods and coupled methods). We compare different methods on a series of specific properties (e.g., input parameters and different kinds of subspaces) and analyze the advantages and disadvantages. We also discuss several interesting and meaningful future directions.

Keywords

Multi-clustering Subspace multi-clustering De-coupled Coupled 

References

  1. 1.
    Abbasi AA, Younis M (2007) A survey on clustering algorithms for wireless sensor networks. Comput Commun 30(14):2826–2841CrossRefGoogle Scholar
  2. 2.
    Aggarwal CC, Procopiuc CM, Wolf JL, Yu PS, Park JS (1999) Fast algorithms for projected clustering. In: Proceedings of ACM SIGMOD international conference on management of data, Philadelphia, PA, pp 61–72Google Scholar
  3. 3.
    Aggarwal CC, Reddy CK (eds) (2014) Data clustering: algorithms and applications. CRC Press, Boca RatonMATHGoogle Scholar
  4. 4.
    Aggarwal CC, Yu PS (2000) Finding generalized projected clusters in high dimensional spaces. In: Proceedings of the 2000 ACM SIGMOD international conference on management of data, Dallas, TX, pp 70–81Google Scholar
  5. 5.
    Agrawal R, Gehrke J, Gunopulos D, Raghavan P (1998) Automatic subspace clustering of high dimensional data for data mining applications. In: Proceedings of the 1998 ACM SIGMOD international conference on management of data, Seattle, WA, pp 94–105Google Scholar
  6. 6.
    Assent I, Krieger R, Müller E, Seidl T (2007) DUSC: dimensionality unbiased subspace clustering. In: Proceedings of the 7th IEEE international conference on data mining, Omaha, NE, pp 409–414Google Scholar
  7. 7.
    Assent I, Krieger R, Müller E, Seidl T (2008) INSCY: indexing subspace clusters with in-process-removal of redundancy. In: Proceedings of the 8th IEEE international conference on data mining, Pisa, Italy, pp 719–724Google Scholar
  8. 8.
    Bae E, Bailey J (2006) COALA: a novel approach for the extraction of an alternate clustering of high quality and high dissimilarity. In: Proceedings of the 6th IEEE international conference on data mining, Hong Kong, China, pp 53–62Google Scholar
  9. 9.
    Bae E, Bailey J, Dong G (2010) A clustering comparison measure using density profiles and its application to the discovery of alternate clusterings. Data Min Knowl Disc 21(3):427–471MathSciNetCrossRefGoogle Scholar
  10. 10.
    Bailey J (2013) Alternative clustering analysis: a review. In: Data clustering: algorithms and applications, pp 535–550Google Scholar
  11. 11.
    Ben-Dor A, Shamir R, Yakhini Z (1999) Clustering gene expression patterns. J Comput Biol 6(3/4):281–297CrossRefGoogle Scholar
  12. 12.
    Berkhin P (2006) A survey of clustering data mining techniques. In: Grouping multidimensional data—recent advances in clustering, pp 25–71Google Scholar
  13. 13.
    Bickel S, Scheffer T (2004) Multi-view clustering. In: Proceedings of the 4th IEEE international conference on data mining, Brighton, UK, pp 19–26Google Scholar
  14. 14.
    Böhm C, Kailing K, Kriegel H-P, Kröger P (2004) Density connected clustering with local subspace preferences. In: Proceedings of the 4th IEEE international conference on data mining, Brighton, UK, pp 27–34Google Scholar
  15. 15.
    Böhm K, Keller F, Müller E, Nguyen HV, Vreeken J (2013) CMI: an information-theoretic contrast measure for enhancing subspace cluster and outlier detection. In: Proceedings of the 13th SIAM international conference on data mining, Austin, TX, pp 198–206Google Scholar
  16. 16.
    Caruana R, Elhawary MF, Nguyen N, Smith C (2006) Meta clustering. In: Proceedings of the 6th IEEE international conference on data mining, Hong Kong, China, pp 107–118Google Scholar
  17. 17.
    Cheng CH, Fu AW-C, Zhang Y (1999) Entropy-based subspace clustering for mining numerical data. In: Proceedings of the 5th ACM SIGKDD international conference on knowledge discovery and data mining, San Diego, CA, pp 84–93Google Scholar
  18. 18.
    Cordeiro RLF, Traina AJM, Faloutsos C, Traina C Jr (2010) Finding clusters in subspaces of very large, multi-dimensional datasets. In: Proceedings of the 26th international conference on data engineering, Long Beach, CA, pp 625–636Google Scholar
  19. 19.
    Cover TM, Thomas JA (2012) Elements of information theory. Wiley, HobokenMATHGoogle Scholar
  20. 20.
    Cui Y, Fern XZ, Dy JG (2007) Non-redundant multi-view clustering via orthogonalization. In: Proceedings of the 7th IEEE international conference on data mining, Omaha, NE, pp 133–142Google Scholar
  21. 21.
    Dang X, Bailey J (2015) A framework to uncover multiple alternative clusterings. Mach Learn 98(1–2):7–30MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Dang XH, Bailey J (2014) Generating multiple alternative clusterings via globally optimal subspaces. Data Min Knowl Disc 28(3):569–592MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Dasgupta S, Ng V (2010) Mining clustering dimensions. In: Proceedings of the 27th international conference on machine learning, Haifa, Israel, pp 263–270Google Scholar
  24. 24.
    Davidson I, Qi Z (2008) Finding alternative clusterings using constraints. In: Proceedings of the 8th IEEE international conference on data mining, Pisa, Italy, pp 773–778Google Scholar
  25. 25.
    Davies DL, Bouldin DW (1979) A cluster separation measure. IEEE Trans Pattern Anal Mach Intell 1(2):224–227CrossRefGoogle Scholar
  26. 26.
    Deza MM, Deza E (2009) Encyclopedia of distances. Springer, BerlinCrossRefMATHGoogle Scholar
  27. 27.
    Di Crescenzo A, Longobardi M (2009) On cumulative entropies. J Stat Plan Inference 139(12):4072–4087MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Domeniconi C, Al-Razgan M (2009) Weighted cluster ensembles: methods and analysis. ACM Trans Knowl Discov Data 2(4):17CrossRefGoogle Scholar
  29. 29.
    Dunn JC (1973) A fuzzy relative of the isodata process and its use in detecting compact well-separated clusters. J Cybern 3(4):32–57MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Ester M, Kriegel H-P, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the 2nd international conference on knowledge discovery and data mining, Portland, OR, pp 226–231Google Scholar
  31. 31.
    Fern XZ, Brodley CE (2003) Random projection for high dimensional data clustering: a cluster ensemble approach. In: Proceedings of the 20th international conference on machine learning, Washington, DC, pp 186–193Google Scholar
  32. 32.
    Fortunato S (2010) Community detection in graphs. Phys Rep 486(3):75–174MathSciNetCrossRefGoogle Scholar
  33. 33.
    Gentle JE (2007) Matrix transformations and factorizations. Matrix algebra: theory, computations, and applications in statistics, pp 173–200Google Scholar
  34. 34.
    Günnemann S, Färber I, Seidl T (2012) Multi-view clustering using mixture models in subspace projections. In: Proceedings of the 18th ACM SIGKDD international conference on knowledge discovery and data mining, Beijing, China, pp 132–140Google Scholar
  35. 35.
    Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182MATHGoogle Scholar
  36. 36.
    Hu J, Qian Q, Pei J, Jin R, Zhu S (2015) Finding multiple stable clusterings. In: Proceedings of the 15th IEEE international conference on data mining, Atlantic City, NJ, pp 171–180Google Scholar
  37. 37.
    Hu J, Qian Q, Pei J, Jin R, Zhu S (2017) Finding multiple stable clusterings. Knowl Inf Syst 51(3):991–1021CrossRefGoogle Scholar
  38. 38.
    Hua M, Pei J (2012) Clustering in applications with multiple data sources—a mutual subspace clustering approach. Neurocomputing 92:133–144CrossRefGoogle Scholar
  39. 39.
    Hubert L, Arabie P (1985) Comparing partitions. J Classif 2(1):193–218CrossRefMATHGoogle Scholar
  40. 40.
    Jain AK, Murty MN, Flynn PJ (1999) Data clustering: a review. ACM Comput Surv 31(3):264–323CrossRefGoogle Scholar
  41. 41.
    Jain P, Meka R, Dhillon IS (2008) Simultaneous unsupervised learning of disparate clusterings. In: Proceedings of SIAM international conference on data mining, Atlanta, GA, pp 858–869Google Scholar
  42. 42.
    Januzaj E, Kriegel H-P, Pfeifle M (2004) Scalable density-based distributed clustering. In: Proceedings of the 8th European conference on principles and practice of knowledge discovery in databases, Pisa, Italy, pp 231–244Google Scholar
  43. 43.
    Jolliffe I (2002) Principal component analysis. Wiley, HobokenMATHGoogle Scholar
  44. 44.
    Kailing K, Kriegel H-P, Kröger P (2004) Density-connected subspace clustering for high-dimensional data. In: Proceedings of the 4th SIAM international conference on data mining, Lake Buena Vista, FL, pp 246–256Google Scholar
  45. 45.
    Kailing K, Kriegel H-P, Kröger P, Wanka S (2003) Ranking interesting subspaces for clustering high dimensional data. In: Proceedings of the 7th European conference on principles and practice of knowledge discovery in databases, Cavtat-Dubrovnik, Croatia, pp 241–252Google Scholar
  46. 46.
    Kailing K, Kriegel H-P, Pryakhin A, Schubert M (2004) Clustering multi-represented objects with noise. In: Proceedings of the 8th Pacific-Asia conference on advances in knowledge discovery and data mining, Sydney, Australia, pp 394–403Google Scholar
  47. 47.
    Katiyar V, Chand N, Soni S (2010) Clustering algorithms for heterogeneous wireless sensor network: a survey. Int J Appl Eng Res 1(2):273Google Scholar
  48. 48.
    Kim S-Y, Jung T-S, Suh E-H, Hwang H-S (2006) Customer segmentation and strategy development based on customer lifetime value: a case study. Expert Syst Appl 31(1):101–107CrossRefGoogle Scholar
  49. 49.
    Kriegel H-P, Kröger P, Renz M, Wurst SHR (2005) A generic framework for efficient subspace clustering of high-dimensional data. In: Proceedings of the 5th IEEE international conference on data mining, Houston, TX, pp 250–257Google Scholar
  50. 50.
    Leskovec J, Lang KJ, Mahoney M (2010) Empirical comparison of algorithms for network community detection. In: Proceedings of the 19th international conference on World wide web, Raleigh, NC, pp 631–640Google Scholar
  51. 51.
    Lew MS, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: state of the art and challenges. ACM Trans Multimed Comput Commun Appl 2(1):1–19CrossRefGoogle Scholar
  52. 52.
    Long B, Yu PS, Zhang ZM (2008) A general model for multiple view unsupervised learning. In: Proceedings of SIAM international conference on data mining, Atlanta, GA, pp 822–833Google Scholar
  53. 53.
    Manning CD, Raghavan P, Schütze H (2008) Introduction to information retrieval. Cambridge University Press, CambridgeCrossRefMATHGoogle Scholar
  54. 54.
    Müller E, Assent I, Krieger R, Günnemann S, Seidl T (2009) DensEst: density estimation for data mining in high dimensional spaces. In: Proceedings of SIAM international conference on data mining, Sparks, NV, pp 175–186Google Scholar
  55. 55.
    Müller E, Günnemann S, Färber I, Seidl T (2012) Discovering multiple clustering solutions: grouping objects in different views of the data. In: Proceedings of the 28th IEEE international conference on data engineering, Washington, DC, pp 1207–1210Google Scholar
  56. 56.
    Nagesh HS, Goil S, Choudhary AN (2001) Adaptive grids for clustering massive data sets. In: Proceedings of the 1st SIAM international conference on data mining, Chicago, IL, pp 1–17Google Scholar
  57. 57.
    Newman MEJ (2006) Modularity and community structure in networks. Proc Natl Acad Sci 103(23):8577–8582CrossRefGoogle Scholar
  58. 58.
    Niu D, Dy JG, Jordan MI (2010) Multiple non-redundant spectral clustering views. In: Proceedings of the 27th international conference on machine learning, Haifa, Israel, pp 831–838Google Scholar
  59. 59.
    Parsons L, Haque E, Liu H (2004) Subspace clustering for high dimensional data: a review. SIGKDD Explor 6(1):90–105CrossRefGoogle Scholar
  60. 60.
    Procopiuc CM, Jones M, Agarwal PK, Murali TM (2002) A Monte Carlo algorithm for fast projective clustering. In: Proceedings of the 2002 ACM SIGMOD international conference on management of data, Madison, WI, pp 418–427Google Scholar
  61. 61.
    Qi Z, Davidson I (2009) A principled and flexible framework for finding alternative clusterings. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining, Paris, France, pp 717–726Google Scholar
  62. 62.
    Rand WM (1971) Objective criteria for the evaluation of clustering methods. J Am Stat Assoc 66(336):846–850CrossRefGoogle Scholar
  63. 63.
    Raychaudhuri S, Sutphin PD, Chang JT, Altman RB (2001) Basic microarray analysis: grouping and feature reduction. Trends Biotechnol 19(5):189–193CrossRefGoogle Scholar
  64. 64.
    Rényi A (1957) Representations for real numbers and their ergodic properties. Acta Math Hung 8(3–4):477–493MathSciNetCrossRefMATHGoogle Scholar
  65. 65.
    Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65CrossRefMATHGoogle Scholar
  66. 66.
    Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326CrossRefGoogle Scholar
  67. 67.
    Schwerdtfeger H (1950) Introduction to linear algebra and the theory of matrices. P. Noordhoff, GroningenMATHGoogle Scholar
  68. 68.
    Sequeira K, Zaki MJ (2005) SCHISM: a new approach to interesting subspace mining. Int J Bus Intell Data Min 1(2):137–160CrossRefGoogle Scholar
  69. 69.
    Shapiro L, Stockman GC (2001) Computer vision. Prentice Hall, Upper Saddle RiverGoogle Scholar
  70. 70.
    Szabó Z, Póczos B, Lörincz A (2012) Separation theorem for independent subspace analysis and its consequences. Pattern Recogn 45(4):1782–1791CrossRefMATHGoogle Scholar
  71. 71.
    Wiswedel B, Höppner F, Berthold MR (2010) Learning in parallel universes. Data Min Knowl Disc 21(1):130–152MathSciNetCrossRefGoogle Scholar
  72. 72.
    Xu D, Tian Y (2015) A comprehensive survey of clustering algorithms. Ann Data Sci 2(2):165–193MathSciNetCrossRefGoogle Scholar
  73. 73.
    Xu R, Wunsch D II (2005) Survey of clustering algorithms. IEEE Trans Neural Netw 16(3):645–678CrossRefGoogle Scholar
  74. 74.
    Yang L, Jin R (2006) Distance metric learning: a comprehensive survey, Michigan State University 2Google Scholar
  75. 75.
    Yang S, Zhang L (2017) Non-redundant multiple clustering by nonnegative matrix factorization. Mach Learn 106(5):695–712Google Scholar
  76. 76.
    Ye W, Maurus S, Hubig N, Plant C (2016) Generalized independent subspace clustering. In: Proceedings of the 16th IEEE international conference on data mining, Barcelona, Spain, pp 569–578Google Scholar
  77. 77.
    Yiu ML, Mamoulis N (2003) Frequent-pattern based iterative projected clustering. In: Proceedings of the 3rd IEEE international conference on data mining, Melbourne, FL, pp 689–692Google Scholar
  78. 78.
    Zhou D, Burges CJC (2007) Spectral clustering and transductive learning with multiple views. In: Proceedings of the 24th international conference on machine learning, Corvallis, OR, pp 1159–1166Google Scholar

Copyright information

© Springer-Verlag London Ltd. 2017

Authors and Affiliations

  1. 1.School of Computing ScienceSimon Fraser UniversityBurnabyCanada

Personalised recommendations