Data Mining and Knowledge Discovery

, Volume 29, Issue 1, pp 3–38 | Cite as

Clustering categorical data in projected spaces

  • Mohamed BouguessaEmail author


The problem of clustering categorical data has been widely investigated and appropriate approaches have been proposed. However, the majority of the existing methods suffer from one or more of the following limitations: (1) difficulty detecting clusters of very low dimensionality embedded in high-dimensional spaces, (2) lack of an automatic mechanism for identifying relevant dimensions for each cluster, (3) lack of an outlier detection mechanism and (4) dependence on a set of parameters that need to be properly tuned. Most of the existing approaches are inadequate for dealing with these four issues in a unified framework. This motivates our effort to propose a fully automatic projected clustering algorithm for high-dimensional categorical data which is capable of facing the four aforementioned issues in a single framework. Our algorithm comprises two phases: (1) outlier handling and (2) clustering in projected spaces. The first phase of the algorithm is based on a probabilistic approach that exploits the beta mixture model to identify and eliminate outlier objects from a data set in a systematic way. In the second phase, the clustering process is based on a novel quality function that allows the identification of projected clusters of low dimensionality embedded in a high-dimensional space without any parameter setting by the user. The suitability of our proposal is demonstrated through empirical studies using synthetic and real data sets.


Projected clustering Categorical data High dimensions  Mixture model 



The author gratefully thank Dr. Guiseppe Manco for providing the implementation of AT-DC, Dr. Tengke Xiong for providing the implementation of DHCC and Dr. Andy M. Yip for providing the Primate and Aging Human Brain data sets. The author also would like to thank the reviewers for their valuable comments and important suggestions. This work is supported by Research Grants from the Natural Sciences and Engineering Research Council of Canada (NSERC).


  1. Aggarwal CC, Yu PS (2002) Redefining clustering for high dimensional applications. IEEE Trans Knowl Data Eng 14(2):210–225CrossRefGoogle Scholar
  2. Aggarwal CC, Procopiuc C, Wolf JL, Yu PS, Park JS (1999) Fast algorithm for Projected clustering. In: Proceedings of the ACM SIGMOD’99 conference, pp 61–72Google Scholar
  3. Andritsos P, Tsaparas P, Miller RJ, Sevcik KC (2004) LIMBO: scalable clustering of categorical data. In: Proceedings of the 9th international conference on extending database technology (EDBT’04), pp 123–146Google Scholar
  4. Angiulli F, Pizzuti C (2005) Outlier mining in large high-dimensional data sets. IEEE Trans Knowl Data Eng 17(2):203–215CrossRefMathSciNetGoogle Scholar
  5. Bai L, Liang J, Dang C, Cao F (2011) A novel attribute weighting algorithm for clustering high-dimensional categorical data. Pattern Recognit 44(12):2843–2861CrossRefzbMATHGoogle Scholar
  6. Barbara D, Li Y, Couto J (2002) COOLCAT: an entropy-based algorithm for categorical clustering. In: Proceedings of the 11th ACM international conference on information and knowledge management (CIKM’02), pp 582–589Google Scholar
  7. Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum, New YorkCrossRefzbMATHGoogle Scholar
  8. Bouguessa M (2011) An unsupervised approach for identifying spammers in social networks. In: Proceedings of the 23rd IEEE international conference on tools with artificial intelligence (ICTAI’11), pp 832–840Google Scholar
  9. Bouguessa M, Wang S (2009) Mining projected clusters in high-dimensional spaces. IEEE Trans Knowl Data Eng 21(4):507–522CrossRefGoogle Scholar
  10. Bouguessa M, Wang S, Sun H (2006) An objective approach to cluster validation. Pattern Recognit Lett 27(13):1419–1430CrossRefGoogle Scholar
  11. Bouguila N, Ziou D, Monga E (2006) Practical Bayesian estimation of a finite beta mixture through Gibbs sampling and its applications. Stat Comput 16(2):215–225CrossRefMathSciNetGoogle Scholar
  12. Cesario E, Manco G, Ortale R (2007) Top-down parameter-free clustering of high-dimensional categorical data. IEEE Trans Knowl Data Eng 19(12):1607–1624CrossRefGoogle Scholar
  13. Das K, Schneider J (2007) Detecting anomalous records in categorical datasets. In: Proceedings of the 13th ACM SIGKDD international conference on knowledge discovery and data mining (KDD’07), pp 220–229Google Scholar
  14. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc B 39(1):1–38zbMATHMathSciNetGoogle Scholar
  15. Domeniconi C, Gunopulos D, Ma S, Yan B, Al-Razgan M, Papadopoulos D (2007) Locally adaptive metrics for clustering high dimensional data. Data Min Knowl Discov 14(1):63–97CrossRefMathSciNetGoogle Scholar
  16. Figueiredo MAT, Jain AK (2002) Unsupervised learning of finite mixture models. IEEE Trans Pattern Anal Mach Intell 24(3):381–396CrossRefGoogle Scholar
  17. Fisher DH (1987) Knowledge acquisition via incremental conceptual clustering. Mach Learn 2(2):139–172Google Scholar
  18. Gan G, Wu J (2004) Subspace clustering for high dimensional categorical data. ACM SIGKDD Explor Newsl 6(2):87–94CrossRefGoogle Scholar
  19. Ganti V, Gehrke J, Ramakrishnan R (1999) CACTUS: clustering categorical data using summaries. In: Proceedings of the 5th ACM SIGKDD international conference on knowledge discovery and data mining (KDD’99), pp 73–83Google Scholar
  20. Guha S, Rastogi R, Shim K (2000) ROCK: a robust clustering algorithm for categorical attributes. Inf Syst 25(5):345–366CrossRefGoogle Scholar
  21. He Z, Deng S, Xu X, Huang JZ (2006) A fast greedy algorithm for outlier mining. In: Proceedings of the 10th Pacific-Asia conference on advances in knowledge discovery and data mining (PAKDD’06), pp 567–576Google Scholar
  22. Ji Y, Wu C, Liu P, Wang J, Coombes KR (2005) Applications of beta-mixture models in bioinformatics. Bioinformatics 21(9):2118–2122CrossRefGoogle Scholar
  23. Jing L, Ng MK, Huang JZ (2007) An entropy weighting k-means algorithm for subspace clustering of high-dimensional sparse data. IEEE Trans Knowl Data Eng 19(8):1026–1041CrossRefGoogle Scholar
  24. Keogh E, Lonardi S, Ratanamahatana CA (2004) Towards parameter-free data mining. In: Proceedings of the 10th ACM SIGKDD international conference on knowledge discovery and data mining (KDD’04), pp 206–215Google Scholar
  25. Kim M, Ramakrishna RS (2006) Projected clustering for categorical datasets. Pattern Recognit Lett 27(12):1405–1417CrossRefGoogle Scholar
  26. Koufakou A, Georgiopoulos M (2010) A fast outlier detection strategy for distributed high-dimensional data sets with mixed attributes. Data Min Knowl Discov 20(2):259–289CrossRefMathSciNetGoogle Scholar
  27. Koufakou A, Ortiz EG, Georgiopoulos M, Anagnostopoulos GC, Reynolds KM (2007) A scalable and efficient outlier detection strategy for categorical data. In: Proceedings of the 19th IEEE international conference on tools with artificial intelligence (ICTAI’07), pp 210–217Google Scholar
  28. Kriegel HP, Kröger P, Zimek A (2009) Clustering high-dimensional data: a survey on subspace clustering, pattern-based clustering, and correlation clustering. ACM Trans Knowl Discov Data 3(1), art no 1Google Scholar
  29. Ma Z, Leijon A (2009) Beta mixture models and the application to image classification. In: Proceedings of the 16th IEEE international conference on image processing (ICIP’09), pp 2045–2048Google Scholar
  30. Moise G, Sander J, Ester M (2008) Robust projected clustering. Knowl Inf Syst 14(3):273–298CrossRefzbMATHGoogle Scholar
  31. Müller E, Günnemann S, Assent I, Seidl T (2009) Evaluating clustering in subspace projections of high dimensional data. Proc Very Large Databases Endow 2(1):1270–1281Google Scholar
  32. Otey ME, Ghoting A, Parthasarathy S (2006) Fast distributed outlier detection in mixed-attribute data sets. Data Min Knowl Discov 12(2–3):203–228CrossRefMathSciNetGoogle Scholar
  33. Rodriguez-Baena DS, Perez-Pulido AJ, Aguilar-Ruiz JS (2011) A biclustering algorithm for extracting bit-patterns from binary datasets. Bioinformatics 27(19):2738–2745CrossRefGoogle Scholar
  34. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464CrossRefzbMATHGoogle Scholar
  35. Smyth P (2000) Model selection for probabilistic clustering using cross-validated likelihood. Stat Comput 10(1):63–72CrossRefGoogle Scholar
  36. Wang K, Xu C, Liu B (1999) Clustering transactions using large items. In: Proceedings of the 8th ACM international conference on information and knowledge management (CIKM’99), pp 483–490Google Scholar
  37. Xiong T, Wang S, Mayers A, Monga E (2012) DHCC: divisive hierarchical clustering of categorical data. Data Min Knowl Discov 24(1):103–135CrossRefzbMATHMathSciNetGoogle Scholar
  38. Yip KY, Cheung DW, Ng MK (2004) HARP: A practical projected clustering algorithm. IEEE Trans Knowl Data Eng 16(11):1387–1397CrossRefGoogle Scholar
  39. Yip AM, Ng MK, Wu EH, Chan TF (2007) Strategies for identifying statistically significant dense regions in microarray data. IEEE/ACM Trans Comput Biol Bioinform 4(3):415–429CrossRefGoogle Scholar
  40. Ypma TJ (1995) Historical development of the Newton–Raphson method. SIAM Rev 37(4):531–551CrossRefzbMATHMathSciNetGoogle Scholar
  41. Zaki MJ, Peters M, Assent I, Seidl T (2007) CLICKS: an effective algorithm for mining subspace clusters in categorical datasets. Data Knowl Eng 60(1):51–70CrossRefGoogle Scholar

Copyright information

© The Author(s) 2013

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of Quebec at MontrealMontrealCanada

Personalised recommendations