Average Cluster Consistency for Cluster Ensemble Selection
Various approaches to produce cluster ensembles and several consensus functions to combine data partitions have been proposed in order to obtain a more robust partition of the data. However, the existence of many approaches leads to another problem which consists in knowing which of these approaches to produce the cluster ensembles’ data and to combine these partitions best fits a given data set. In this paper, we propose a new measure to select the best consensus data partition, among a variety of consensus partitions, based on the concept of average cluster consistency between each data partition that belongs to the cluster ensemble and a given consensus partition. The experimental results obtained by comparing this measure with other measures for cluster ensemble selection in 9 data sets, showed that the partitions selected by our measure generally were of superior quality in comparison with the consensus partitions selected by other measures.
KeywordsConsistency Index Consensus Function Data Pattern Cluster Quality Data Partition
Unable to display preview. Download preview PDF.
- 4.Duarte, F.J., Fred, A.L.N., Rodrigues, M.F.C., Duarte, J.: Weighted evidence accumulation clustering using subsampling. In: Sixth International Workshop on Pattern Recognition in Information Systems (2006)Google Scholar
- 5.Fern, X., Brodley, C.: Solving cluster ensemble problems by bipartite graph partitioning. In: ICML 2004: Proceedings of the Twenty-First International Conference on Machine Learning, vol. 36. ACM, New York (2004)Google Scholar
- 6.Topchy, A.P., Jain, A.K., Punch, W.F.: A mixture model for clustering ensembles. In: Berry, M.W., Dayal, U., Kamath, C., Skillicorn, D.B. (eds.) SDM. SIAM, Philadelphia (2004)Google Scholar
- 7.Jouve, P., Nicoloyannis, N.: A new method for combining partitions, applications for distributed clustering. In: International Workshop on Paralell and Distributed Machine Learning and Data Mining (ECML/PKDD 2003), pp. 35–46 (2003)Google Scholar
- 9.Topchy, A., Jain, A.K., Punch, W.: Combining multiple weak clusterings, pp. 331–338 (2003)Google Scholar
- 11.Hubert, L., Arabie, P.: Comparing partitions. Journal of Classification (October 1985)Google Scholar
- 12.Kuncheva, L., Hadjitodorov, S.: Using diversity in cluster ensembles, vol. 2, pp. 1214–1219 (October 2004)Google Scholar
- 13.Duarte, F., Duarte, J., Fred, A., Rodrigues, F.: Cluster ensemble selection - using average cluster consistency. In: International Conference on Discovery and Information Retrieval (KDIR 2009), Funchal, October 6-8, pp. 85–95 (2009)Google Scholar
- 15.King, B.: Step-wise clustering procedures. Journal of the American Statistical Association (69), 86–101 (1973)Google Scholar
- 16.Macqueen, J.B.: Some methods of classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathemtical Statistics and Probability, pp. 281–297 (1967)Google Scholar
- 21.Ester, M., Kriegel, H.P., Jörg, S., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise (1996)Google Scholar
- 22.Wang, W., Yang, J., Muntz, R.R.: Sting: A statistical information grid approach to spatial data mining. In: VLDB 1997: Proceedings of the 23rd International Conference on Very Large Data Bases, pp. 186–195. Morgan Kaufmann Publishers Inc., San Francisco (1997)Google Scholar