Abstract
Data clustering is a basic data mining discipline that has been in center of interest of many research groups. This paper describes the formulation of the basic NP-hard optimization problem in data clustering which is approximated by many heuristic methods. The famous k-means clustering algorithm and its initialization is of a particular interest in this paper. A summary of the k-means variants and various initialization strategies is presented. Many initialization heuristics tend to search only through a fraction of the initial centroid space. The final clustering result is usually compared only to some other heuristic strategy. In this paper we compare the result to the solution provided by a brute-force experiment. Many instances of the k-means can be executed in parallel on the high performance computing infrastructure, which makes brute-force search for the best initial centroids possible. Solutions obtained by exact solvers [2, 11] of the clustering problem are used for verification of the brute-force approach. We present progress of the function optimization during the experiment for several benchmark data sets, including sparse document-term matrices.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aloise, D., Deshpande, A., Hansen, P., Popat, P.: Np-hardness of euclidean sum-of-squares clustering. Mach. Learn. 75(2), 245–248 (2009)
Aloise, D., Hansen, P., Liberti, L.: An improved column generation algorithm for minimum sum-of-squares clustering. Math. Program. 131(1–2), 195–220 (2012)
Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1027–1035. Society for Industrial and Applied Mathematics (2007)
Bagirov, A.M.: Modified global k-means algorithm for minimum sum-of-squares clustering problems. Pattern Recogn. 41(10), 3192–3199 (2008)
Bagirov, A.M., Ugon, J., Webb, D.: Fast modified global k-means algorithm for incremental cluster construction. Pattern Recogn. 44(4), 866–876 (2011)
Bradley, P.S., Fayyad, U.M.: Refining initial points for k-means clustering. In: ICML. vol. 98, pp. 91–99. Citeseer (1998)
Brusco, M.J.: A repetitive branch-and-bound procedure for minimum within-cluster sums of squares partitioning. Psychometrika 71(2), 347–363 (2006)
Celebi, M.E., Kingravi, H.A., Vela, P.A.: A comparative study of efficient initialization methods for the k-means clustering algorithm. Expert Syst. Appl. 40(1), 200–210 (2013)
Deng, G., Tao, J., Zhou, M., Xu, Y.: Improved k-means algorithm with better clustering centers based on density and variance. In: Industrial, Mechanical and Manufacturing Science: Proceedings of the 2014 International Conference on Industrial, Mechanical and Manufacturing Science (ICIMMS 2014), 12–13 June 2014, Tianjin, China. vol. 1, p. 147. CRC Press (2015)
Dhillon, I.S., Modha, D.S.: Concept decompositions for large sparse text data using clustering. Mach. Learn. 42(1–2), 143–175 (2001)
Du Merle, O., Hansen, P., Jaumard, B., Mladenovic, N.: An interior point algorithm for minimum sum-of-squares clustering. SIAM J. Sci. Comput. 21(4), 1485–1505 (1999)
Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Eugen. 7(2), 179–188 (1936)
Forgy, E.W.: Cluster analysis of multivariate data: efficiency versus interpretability of classifications. Biometrics 21, 768–769 (1965)
Hansen, P., Ngai, E., Cheung, B.K., Mladenovic, N.: Analysis of global k-means, an incremental heuristic for minimum sum-of-squares clustering. J. Classif. 22(2), 287–310 (2005)
Hartigan, J.A., Wong, M.A.: Algorithm as 136: a k-means clustering algorithm. J. Roy. Stat. Soc. Ser. C (Appl. Stat.) 28(1), 100–108 (1979)
Hornik, K., Feinerer, I., Kober, M., Buchta, C.: Spherical k-means clustering. J. Stat. Softw. 50(10), 1–22 (2012)
Jain, A.K.: Data clustering: 50 years beyond k-means. Pattern Recogn. Lett. 31(8), 651–666 (2010)
Likas, A., Vlassis, N., Verbeek, J.J.: The global k-means clustering algorithm. Pattern Recogn. 36(2), 451–461 (2003)
Liu, H., Fang, C., Wu, Y., Xu, K., Dai, T.: Improved k-means algorithm with the pretreatment of PCA dimension reduction. Int. J. Hybrid Inf. Technol. 8(6), 195–204 (2015)
Lloyd, S.: Least squares quantization in PCM. IEEE Trans. Inf. Theory 28(2), 129–137 (1982)
MacQueen, J., et al.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. vol. 1, pp. 281–297, Oakland, CA, USA (1967)
R Core Team R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2015). http://www.R-project.org/
Ruspini, E.H.: Numerical methods for fuzzy clustering. Inf. Sci. 2(3), 319–350 (1970)
Steinhaus, H.: Sur la division des corp materiels en parties. Bull. Acad. Polon. Sci 1(804), 801 (1956)
Steinley, D.: K-means clustering: a half-century synthesis. Br. J. Math. Stat. Psychol. 59(1), 1–34 (2006)
West, L.J., Hankin, R.K.: Set partitions in r. J. Stat. Softw. 23(02) (2007)
Acknowledgements
This work was supported by The Ministry of Education, Youth and Sports from the National Programme of Sustainability (NPU II) project ‘IT4Innovations excellence in science—LQ1602’ and co-financed by the internal grant agency of VŠB—Technical University of Ostrava, Czech Republic, under the projects no. SP2016/179 ‘HPC Usage for Transport Optimisation based on Dynamic Routing II’.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Golasowski, M., Martinovič, J., Slaninová, K. (2017). Comparison of K-means Clustering Initialization Approaches with Brute-Force Initialization. In: Chaki, R., Saeed, K., Cortesi, A., Chaki, N. (eds) Advanced Computing and Systems for Security. Advances in Intelligent Systems and Computing, vol 567. Springer, Singapore. https://doi.org/10.1007/978-981-10-3409-1_7
Download citation
DOI: https://doi.org/10.1007/978-981-10-3409-1_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-3408-4
Online ISBN: 978-981-10-3409-1
eBook Packages: EngineeringEngineering (R0)