Clustering Problems for More Useful Benchmarking of Optimization Algorithms

  • Marcus Gallagher
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8886)


This paper analyses the data clustering problem from the continuous black-box optimization point of view and proposes methodological guidelines for a standard benchmark of clustering problem instances. Clustering problems have been used many times in the literature to evaluate evolutionary, metaheuristic and other global optimization algorithms. However much of this work has occurred independently and the various experimental methodologies used have produced results which tend to be incomparable and provide little collective wisdom as to the difficulty of the problems used, or an objective measure for comparing and evaluating the performance of algorithms. This paper surveys some of the clustering literature and results to identify issues relevant for benchmarking. A set of 27 problem instances ranging from 4-D to 40-D and based on three well-known datasets is identified. To establish some pilot results on this benchmark set, experiments are presented for the Covariance Matrix Adaptation-Evolution Strategy and several other standard algorithms. A web-repository has also been created for this problem set to facilitate better experimental evaluations of algorithms.


Algorithm Benchmarking Continuous Black-box Optimization Clustering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blake, C., Keogh, E., Merz, C.J.: UCI repository of machine learning databases(1998), (retrieved)
  2. 2.
    Brimberg, J., Hansen, P., Mladenovic, N., Taillard, E.D.: Improvements and comparison of heuristics for solving the uncapacitated multisource Weber problem. Operations Research 48(3), 444–460 (2000)CrossRefGoogle Scholar
  3. 3.
    Chang, D.-X., Zhang, X.-D., Zheng, C.-W.: A genetic algorithm with gene rearrangement for k-means clustering. Pattern Recognition 42(7), 1210–1222 (2009)CrossRefGoogle Scholar
  4. 4.
    Du Merle, O., Hansen, P., Jaumard, B., Mladenovic, N.: An interior point algorithm for minimum sum-of-squares clustering. SIAM Journal on Scientific Computing 21(4), 1485–1505 (2000)CrossRefzbMATHGoogle Scholar
  5. 5.
    Fathian, M., Amiri, B., Maroosi, A.: Application of honey-bee mating optimization algorithm on clustering. Applied Mathematics and Computation 190(2), 1502–1513 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  6. 6.
    Kao, Y., Cheng, K.: An ACO-based clustering algorithm. In: Dorigo, M., Gambardella, L.M., Birattari, M., Martinoli, A., Poli, R., Stützle, T. (eds.) ANTS 2006. LNCS, vol. 4150, pp. 340–347. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Liu, R., Shen, Z., Jiao, L., Zhang, W.: Immunodomaince based clonal selection clustering algorithm. In: 2010 IEEE Congress on Evolutionary Computation (CEC), pp. 1–7 (2010)Google Scholar
  8. 8.
    Maulik, U., Bandyopadhyay, S.: Genetic algorithm-based clustering technique. Pattern Recognition 33(9), 1455–1465 (2000)CrossRefGoogle Scholar
  9. 9.
    Salhi, S., Gamal, M.D.H.: A genetic algorithm based approach for the uncapacitated continuous location-allocation problem. Annals of Operations Research 123, 230–222 (2003)Google Scholar
  10. 10.
    Shelokar, P.S., Jayaraman, V.K., Kulkarni, B.D.: An ant colony approach for clustering. Analytica Chimica Acta 509(2), 187–195 (2004)CrossRefGoogle Scholar
  11. 11.
    Steinley, D.: K-means clustering: A half-century synthesis. British Journal of Mathematical and Statistical Psychology 59, 1–34 (2006)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Taherdangkoo, M., Shirzadi, M.H., Yazdi, M., Bagheri, M.H.: A robust clustering method based on blind, naked mole-rats (bnmr) algorithm. Swarm and Evolutionary Computation 10, 1–11 (2013)CrossRefGoogle Scholar
  13. 13.
    Xu, R., Wunsch II., D.: Survey of clustering algorithms. IEEE Transactions on Neural Networks 16(3), 645–678 (2005)CrossRefGoogle Scholar
  14. 14.
    Ye, F., Chen, C.-Y.: Alternative kpso-clustering algorithm. Tamkang Journal of Science and Engineering 8(2), 165 (2005)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Marcus Gallagher
    • 1
  1. 1.School of Information Technology and Electrical EngineeringThe University of QueenslandBrisbaneAustralia

Personalised recommendations