Skip to main content
SpringerLink
Log in

A k-Means-Like Algorithm for Clustering Categorical Data Using an Information Theoretic-Based Dissimilarity Measure

  • Conference paper
  • First Online:
Foundations of Information and Knowledge Systems (FoIKS 2016)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9616))

Included in the following conference series:

Abstract

Clustering large datasets is one of the important research problems for many machine learning applications. The k-means is very popular and widely used due to its ease of implementation, linear time complexity in size of the data, and almost surely convergence to local optima. However, working only on numerical data prohibits it from being used for clustering categorical data. In this paper, we aim to introduce an extension of k-means algorithm for clustering categorical data. Basically, we propose a new dissimilarity measure based on an information theoretic definition of similarity that considers the amount of information of two values in the domain set. The definition of cluster centers is generalized using kernel density estimation approach. Then, the new algorithm is proposed by incorporating a feature weighting scheme that automatically measures the contribution of individual attributes for the clusters. In order to demonstrate the performance of the new algorithm, we conduct a series of experiments on real datasets from UCI Machine Learning Repository and compare the obtained results with several previously developed algorithms for clustering categorical data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aitchison, J., Aitken, C.G.G.: Multivariate binary discrimination by the kernel method. Biometrika 63(3), 413–420 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  2. Andritsos, P., Tsaparas, P., Miller, R.J., Sevcik, K.C.: LIMBO: A Scalable Algorithm to Cluster Categorical Data (2003)

    Google Scholar 

  3. Barbara, D., Couto, J., Li, Y.: COOLCAT: an entropy-based algorithm for categorical clustering. In: Proceedings of the Eleventh International Conference on Information and Knowledge Management, pp. 582–589 (2002)

    Google Scholar 

  4. Blake, C.L., Merz, C.J.: UCI Repository of Machine Learning Databases. Dept. of Information and Computer Science, University of California at Irvine (1998). http://www.ics.uci.edu/mlearn/MLRepository.html

  5. Boriah, S., Chandola, V., Kumar V.: Similarity measures for categorical data: a comparative evaluation. In: Proceedings of the SIAM International Conference on Data Mining, SDM 2008, pp. 243–254 (2008)

    Google Scholar 

  6. Chen, L., Wang, S.: Central clustering of categorical data with automated feature weighting. In: Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, pp. 1260–1266 (2013)

    Google Scholar 

  7. Ganti, V., Gehrke, J., Ramakrishnan, R.: CATUS–clustering categorical data using summaries. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining, San Diego, CA, pp. 73–83 (1999)

    Google Scholar 

  8. Gibson, D., Kleinberg, J., Raghavan, P.: Clustering categorical data: an approach based on dynamic systems. In: Proceedings of the 24th International Conference on Very Large Databases, New York, pp. 311–323 (1998)

    Google Scholar 

  9. Guha, S., Rastogi, R., Shim, K.: CURE: an efficient clustering algorithm for large databases. In: Proceedings of ACM SIGMOD International Conference on Management of Data, New York, pp. 73–84 (1998)

    Google Scholar 

  10. Guha, S., Rastogi, R., Shim, K.: ROCK: a robust clustering algorithm for categorical attributes. Inf. Syst. 25(5), 345–366 (2000)

    Article  Google Scholar 

  11. Han, J., Kamber, M.: Data Mining: Concepts and Techniques. Morgan Kaufmann Publishers, San Francisco (2001)

    MATH  Google Scholar 

  12. Hathaway, R.J., Bezdek, J.C.: Local convergence of the \(c\)-means algorithms. Pattern Recogn. 19, 477–480 (1986)

    Article  MATH  Google Scholar 

  13. Huang, Z.: Clustering large data sets with mixed numeric and categorical values. In: Lu, H., Motoda, H., Luu, H. (eds.) KDD: Techniques and Applications, pp. 21–34. World Scientific, Singapore (1997)

    Google Scholar 

  14. Huang, Z.: Extensions to the k-means algorithm for clustering large data sets with categorical values. Data Min. Knowl. Discov. 2, 283–304 (1998)

    Article  Google Scholar 

  15. Huang, Z., Ng, M.K.: A fuzzy \(k\)-modes algorithm for clustering categorical data. IEEE Trans. Fuzzy Syst. 7, 446–452 (1999)

    Article  Google Scholar 

  16. Huang, Z., Ng, M.K., Rong, H., Li, Z.: Automated variable weighting in \(k\)-means type clustering. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 657–668 (2005)

    Article  Google Scholar 

  17. Hubert, L., Arabie, P.: Comparing partitions. J. Classif. 2(1), 193–218 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  18. Ienco, D., Pensa, R.G., Meo, R.: Context-based distance learning for categorical data clustering. In: Adams, N.M., Robardet, C., Siebes, A., Boulicaut, J.-F. (eds.) IDA 2009. LNCS, vol. 5772, pp. 83–94. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  19. Ienco, D., Pensa, R.G., Meo, R.: From context to distance: learning dissimilarity for categorical data clustering. ACM Trans. Knowl. Discov. Data 6(1), 1–25 (2012)

    Article  Google Scholar 

  20. Lin, D.: An information-theoretic definition of similarity. In: Proceedings of the 15th International Conference on Machine Learning, pp. 296–304 (1998)

    Google Scholar 

  21. MacQueen, J.B.: Some methods for classification, analysis of multivariate observations. In: Proceedings of the Fifth Symposium on Mathematical Statistics and Probability, Berkelely, CA, vol. 1(AD 669871), pp. 281–297 (1967)

    Google Scholar 

  22. Ng, M.K., Li, M.J., Huang, J.Z., He, Z.: On the impact of dissimilarity measure in \(k\)-modes clustering algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 29, 503–507 (2007)

    Article  Google Scholar 

  23. Ralambondrainy, H.: A conceptual version of the \(k\)-means algorithm. Pattern Recog. Lett. 16, 1147–1157 (1995)

    Article  Google Scholar 

  24. San, O.M., Huynh, V.N., Nakamori, Y.: An alternative extension of the \(k\)-means algorithm for clustering categorical data. Int. J. Appl. Math. Comput. Sci. 14, 241–247 (2004)

    MathSciNet  MATH  Google Scholar 

  25. Selim, S.Z., Ismail, M.A.: k-Means-type algorithms: a generalized convergence theorem and characterization of local optimality. IEEE Trans. Pattern Anal. Mach. Intell. 6(1), 81–87 (1984)

    Article  MATH  Google Scholar 

  26. Strehl, A., Ghosh, J.: Cluster ensembles–a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3, 583–617 (2003)

    MathSciNet  MATH  Google Scholar 

  27. Titterington, D.M.: A comparative study of kernel-based density estimates for categorical data. Technometrics 22(2), 259–268 (1980)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Knowledge Science, Japan Advanced Institute of Science and Technology, Nomi, Japan

    Thu-Hien Thi Nguyen & Van-Nam Huynh

Authors
  1. Thu-Hien Thi Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Van-Nam Huynh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Van-Nam Huynh .

Editor information

Editors and Affiliations

  1. Faculteit Wetenschappen, Universiteit Hasselt, Hasselt, Belgium

    Marc Gyssens

  2. Dept. Ciencias Ingeniería Computación, Universidad Nacional del Sur, Bahía Blanca, Argentina

    Guillermo Simari

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Nguyen, TH.T., Huynh, VN. (2016). A k-Means-Like Algorithm for Clustering Categorical Data Using an Information Theoretic-Based Dissimilarity Measure. In: Gyssens, M., Simari, G. (eds) Foundations of Information and Knowledge Systems. FoIKS 2016. Lecture Notes in Computer Science(), vol 9616. Springer, Cham. https://doi.org/10.1007/978-3-319-30024-5_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-30024-5_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-30023-8

  • Online ISBN: 978-3-319-30024-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation