Core Clustering as a Tool for Tackling Noise in Cluster Labels
- 11 Downloads
Real-world data sets often contain mislabelled entities. This can be particularly problematic if the data set is being used by a supervised classification algorithm at its learning phase. In this case, the accuracy of this classification algorithm, when applied to unlabelled data, is likely to suffer considerably. In this paper, we introduce a clustering-based method capable of reducing the number of mislabelled entities in data sets. Our method can be summarised as follows: (i) cluster the data set; (ii) select the entities that have the most potential to be assigned to correct clusters; (iii) use the entities of the previous step to define the core clusters and map them to the labels using a confusion matrix; (iv) use the core clusters and our cluster membership criterion to correct the labels of the remaining entities. We perform numerous experiments to validate our method empirically using k-nearest neighbour classifiers as a benchmark. We experiment with both synthetic and real-world data sets with different proportions of mislabelled entities. Our experiments demonstrate that the proposed method produces promising results. Thus, it could be used as a preprocessing data correction step of a supervised machine learning algorithm.
KeywordsLabel noise Clustering k-means Core clustering Minkowski distance
BM thanks the Laboratory for Decision Choice and Analysis of the National Research University Higher School of Economics Moscow RF for partially supporting his work in the framework of the HSE University Basic Research Program funded by the Russian Academic Excellence Project ‘5-100’.
- Angluin, D., & Laird, P. (1988). Learning from noisy examples. Machine Learning, 2(4), 343–370.Google Scholar
- Arbelaitz, O., Gurrutxaga, I., Muguerza, J., Pérez, J. M., Perona, I. (2013). An extensive comparative study of cluster validity indices. Pattern Recognition, 46(1), 243–256.Google Scholar
- Ball, G.H., & Hall, D.J. (1967). A clustering technique for summarizing multivariate data. Behavioral Science, 12(2), 153–155.Google Scholar
- De Amorim, R.C., & Makarenkov, V. (2016). Applying subclustering and Lp distance in Weighted K-Means with distributed centroids. Neurocomputing, 173, 700–707.Google Scholar
- De Amorim, R.C., & Mirkin, B. (2011). Minkowski metric, feature weighting and anomalous cluster initializing in k-means clustering. Pattern Recognition, 45, 3.Google Scholar
- Grira, N., Crucianu, M., Boujemaa, N. (2004). Unsupervised and semisupervised clustering: a brief survey. A review of machine learning techniques for processing multimedia content, Report of the MUSCLE European Network of Excellence (FP6), pp. 1001–1030.Google Scholar
- Hughes, N.P., Roberts, S.J., Tarassenko, L. (2004). Semi-supervised learning of probabilistic models for ECG segmentation. In: Engineering in Medicine and Biology Society, 2004. IEMBS’04. 26th Annual International Conference of the IEEE. Vol. 1. IEEE, pp. 434–437.Google Scholar
- Jain, A., Jin, R., Chitta, R. (2014). Semi-supervised clustering. Handbook of Cluster Analysis, pp. 1–35.Google Scholar
- Jain, A.K. (2010). Data clustering: 50 years beyond K-means. Pattern Recognition Letters, 31(8), 651–666.Google Scholar
- Jones, E., Oliphant, T., Peterson, P., et al. (2001). SciPy: Open source scientific tools for Python. [Online; accessed 2016-11-28]. http://www.scipy.org/.
- Kaufman, L., & Rousseeuw, P.J. (1990). Finding groups in data: an introduction to cluster analysis. Vol. 39. Wiley Online Library.Google Scholar
- Lichman, M. (2013). UCI Machine Learning Repository. http://archive.ics.uci.edu/ml.
- Macqueen, J., & et al. (1967). Some methods for classification and analysis of multivariate observations. In: Proceedings of the fifth Berkeley symposium on mathematical statistics and probability. vol. 1. 281–297. California, USA, pp. 14.Google Scholar
- Maletic, J.I., & Marcus, A. (2000). Data cleansing: beyond integrity analysis. In: IQ. Citeseer, pp. 200–209.Google Scholar
- MATLAB. (2013). version 8.10.0 (R2013a). Natick. Massachusetts: The MathWorks Inc.Google Scholar
- Mirkin, B.G. (2016). Clustering for data mining: a data recovery approach Vol. 3. Boca Raton: CRC Press.Google Scholar
- Orr, K. (1998). Data quality and systems theory. Communications of the ACM, 41(2), 66–71.Google Scholar
- Pechenizkiy, M., Tsymbal, A., Puuronen, S., Pechenizkiy, O. (2006). Class noise and supervised learning in medical domains: the effect of feature extraction. In: 19th IEEE symposium on computer-based medical systems (CBMS’06). IEEE, pp. 708–713.Google Scholar
- Quinlan, J.R. (1986). Induction of decision trees. Machine Learning, 1(1), 81–106.Google Scholar
- Redman, T.C. (1998). The impact of poor data quality on the typical enterprise. Communications of the ACM, 41(2), 79–82.Google Scholar
- Saeys, Y., Inza, I., Larrañaga, P. (2007). A review of feature selection techniques in bioinformatics. Bioinformatics, 23(19), 2507–2517.Google Scholar
- Saáez, J.A., Galar, M., Luengo, J., Herrera, F. (2014). Analyzing the presence of noise in multi-class problems: alleviating its influence with the Onevs- One decomposition. Knowledge and Information Systems, 38(1), 179–206.Google Scholar
- Struyf, A., Hubert, M., Rousseeuw, P., et al. (1997). Clustering in an object-oriented environment. Journal of Statistical Software, 1(4), 1–30.Google Scholar
- Wishart, D. (1998). Clustan. http://www.clustan.com/ (visited on 11/28/2016).
- ZHU, X. (2006). Semi-supervised learning literature survey. Computer Science. University of Wisconsin-Madison, 2(3), 4.Google Scholar