A Method to Determine the Number of Clusters Based on Multi-validity Index
Cluster analysis is a method of unsupervised learning technology which is playing a more and more important role in data mining. However, one basic and difficult question for clustering is how to gain the number of clusters automatically. The traditional solution for the problem is to introduce a single validity index which may lead to failure because the index is bias to some specific condition. On the other hand, most of the existing clustering algorithms are based on hard partitioning which can not reflect the uncertainty of the data in the clustering process. To combat these drawbacks, this paper proposes a method to determine the number of clusters automatically based on three-way decision and multi-validity index which includes three parts: (1) the k-means clustering algorithm is devised to obtain the three-way clustering results; (2) multi-validity indexes are employed to evaluate the results and each evaluated result is weighed according to the mean similarity between the corresponding clustering result and the others based on the idea of the median partition in clustering ensemble; and (3) the comprehensive evaluation results are sorted and the best ranked k value is selected as the optional number of clusters. The experimental results show that the proposed method is better than the single evaluation method used in the fusion at determining the number of clusters automatically.
KeywordsClustering Uncertainty Three-way decisions Number of clusters Multi-validity index
This work was supported in part by the National Natural Science Foundation of China under Grant Nos. 61533020, 61751312 and 61379114.
- 1.Azimi, R., Ghayekhloo, M., Ghofrani, M., et al.: A novel clustering algorithm based on data transformation approaches. Expert Syst. Appl. Int. J. 76(C), 59–70 (2017)Google Scholar
- 2.Chen, H.P., Shen, X.J., Lv, Y.D.: A novel automatic fuzzy clustering algorithm based on soft partition and membership information. Neurocomputing 236, 104–112 (2016)Google Scholar
- 4.Gionis, A., Mannila, H., Tsaparas, P.: Clustering aggregation. In: International Conference on Data Engineering, 2005, ICDE 2005. Proceedings. IEEE, pp. 341–352 (2005)Google Scholar
- 5.Huang, D., Wang, C., Lai, J., et al.: Clustering ensemble by decision weighting. JCAAI Trans. Intell. Syst. 11(3), 418–424 (2016)Google Scholar
- 7.Ling, H.L., Wu, J.S., Zhou, Y., et al.: How many clusters? A robust PSO-based local density model. Neurocomputing 207(C), 264–275 (2016)Google Scholar
- 10.Singhbiostatistics, V.: Ensemble clustering using semidefiniteprogramming. Mach. Learn. 79(1–2), 177–200 (2008)Google Scholar