Skip to main content
SpringerLink
Log in

Blending multiple algorithmic granular components: a recipe for clustering

  • Published:
Swarm Intelligence Aims and scope Submit manuscript

Abstract

Emerging trends in algorithm design have shown that hybrid algorithms, which combine or merge multiple algorithms, can create synergies to overcome the inherent limitations of the underlying individual algorithms. There are two broad types of hybridization: collaborative—individual algorithms tackle an instance of the problem sequentially or in parallel and exchange information accordingly while solving the problem; integrative—individual algorithms are dedicated to tackling different aspect(s) of the problem-solving process. In this research, we propose a schema for an enhanced form of integrative hybridization that blends granular algorithmic components from multiple algorithms to derive a new singular clustering algorithm. As a case study, we examine the ant clustering algorithm (a swarm intelligence algorithm that is based on the natural phenomenon of brood sorting in some species of ants); highlight the strengths and weaknesses of the algorithm; and present a blend of algorithmic components from Tabu search into the algorithm to improve its exploration strategy and solution quality. Empirical results from applying the blended algorithm to clustering benchmark datasets show improved clustering validation measures for the proposed blended hybrid algorithm compared to other hybridization of the same underlying individual algorithms. Besides, the quality of clusters uncovered by this hybrid algorithm competes favorably with those uncovered using popular clustering algorithms such as DBSCAN and mean shift.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  • Aldenderfer, M., Blashfield, R., & Blashfield, R. (1984). SAGE., and i. Sage Publications. Cluster analysis. Number no. 44 in Cluster Analysis. SAGE Publications. ISBN 9780803923768. URL https://books.google.ca/books?id=ZIARBoJQxzcC.

  • Al-Sultan, K. S. (1995). A tabu search approach to the clustering problem. Pattern Recognition, 28(9), 1443–1451.

    Article  Google Scholar 

  • Al-Sultan, K. S., & Fedjki, C. A. (1997). A tabu search-based algorithm for the fuzzy clustering problem. Pattern Recognition, 30(12), 2023–2030.

    Article  Google Scholar 

  • Blum, C., Puchinger, J., Raidl, G. R., & Roli, A. (2011). Hybrid metaheuristics in combinatorial optimization: A survey. Applied Soft Computing, 11(6), 4135–4151.

    Article  MATH  Google Scholar 

  • Boryczka, U. (2008). Ant clustering algorithm. In Intelligent information systems, (pp. 377–386).

  • Chen, Y.-F., Fattah, C. A., Liu, Y.-S., & Yan, G. (2004). Hdacc: A heuristic density-based ant colony clustering algorithm. In IAT, (pp. 397–400). IEEE Computer Society. ISBN 0-7695-2101-0.

  • Chiou, Y.-C., & Lan, L. W. (2001). Genetic clustering algorithms. European Journal of Operational Research, 135(2), 413–427.

    Article  MathSciNet  MATH  Google Scholar 

  • Chiu, C.-Y. & Lin, C.-H.(2007). Cluster analysis based on artificial immune system and ant algorithm. In J. Lei, J. Yao, and Q. Zhang, (Eds.), Third international conference on natural computation (ICNC 2007), (pp. 647–650). IEEE Computer Society.

  • Cohen, S. C. M. & de Castro, L. N.(2006). Data clustering with particle swarms. In IEEE congress on evolutionary computation, (pp. 1792–1798). IEEE. ISBN 0-7803-9487-9.

  • Comaniciu, D. & Meer, P. (May 2002). Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5), 603–619. ISSN 0162-8828.

  • Das, S., Abraham, A., & Konar, A. (2009). Metaheuristic clustering, (Vol 178). Springer Verlag.

  • Deneubourg, J.-L., Goss, S., Franks, N., Sendova-Franks, A., Detrain, C., & Chrétien, L. (1991). The dynamics of collective sorting robot-like ants and ant-like robots. In Proceedings of the first international conference on simulation of adaptive behavior on From animals to animats, (pp. 356–363).

  • der Merwe, D. V. & Engelbrecht, A. P. (2003). Data clustering using particle swarm optimization. In Evolutionary computation, 2003. CEC’03. The 2003 Congress on, (vol. 1, pp. 215–220). IEEE.

  • Dua, D. & Graff, C. (2017). UCI machine learning repository, http://archive.ics.uci.edu/ml

  • Duczmal, L., & Assunção, R. M. (2004). A simulated annealing strategy for the detection of arbitrarily shaped spatial clusters. Computational Statistics & Data Analysis, 45(2), 269–286.

    Article  MathSciNet  MATH  Google Scholar 

  • Duda, R. O., Hart, P. E., et al. (1973). Pattern classification and scene analysis (Vol. 3). Wiley.

    MATH  Google Scholar 

  • Esmin, Ahmed A. A.., & Matwin, Stan. (2012). Data clustering using hybrid particle swarm optimization. In Hujun Yin, José A. F.. Costa, & Guilherme Barreto (Eds.), Intelligent data engineering and automated learning - IDEAL 2012 (pp. 159–166). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-32639-4_20.

    Chapter  Google Scholar 

  • Ester, M., Kriegel, H.-P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters a density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the second international conference on knowledge discovery and data mining, KDD’96, (pp. 226–231). AAAI Press.

  • Estivill-Castro, V. (2002). Why so many clustering algorithms: a position paper. SIGKDD Explorations, 4(1), 65–75.

    Article  Google Scholar 

  • Everitt, B., Landau, S., Leese, M., & Stahl, D. (2011). Cluster analysis. Wiley, 5th edition.

  • Glover, F., & Laguna, M. (2013). Tabu search. Springer.

    MATH  Google Scholar 

  • Güngör, Z., & Ünler, A. (2007). K-harmonic means data clustering with simulated annealing heuristic. Applied Mathematics and Computation, 184(2), 199–209.

    Article  MathSciNet  MATH  Google Scholar 

  • Güngör, Z., & Ünler, A. (2008). K-harmonic means data clustering with tabu-search method. Applied Mathematical Modelling, 32(6), 1115–1125.

    Article  MATH  Google Scholar 

  • Hamdi, A., Monmarché, N., Alimi, M. A., & Slimane, M. (2008). Swarmclass: A novel data clustering approach by a hybridization of an ant colony with flying insects. In M. Dorigo, M. Birattari, C. Blum, M. Clerc, T. Stützle, and A. F. T. Winfield, (Eds.), Ant colony optimization and swarm intelligence, (pp. 411–412), Berlin, Heidelberg. Springer Berlin Heidelberg.

  • Hamdi, A., Slimane, M., Monmarché, N., & Alimi, A. M. (Nov 2016). Flyantclass: Intelligent move for ant based clustering algorithm. In 2016 IEEE/ACS 13th international conference of computer systems and applications (AICCSA), (pp. 1–8).

  • Handl, J. & Meyer, B.(2007). Ant-based and swarm-based clustering. Swarm intelligence, 1(2):95–113. ISSN 1935-3812.

  • Handl, J., Knowles, J. D., & Dorigo, M. (2003). On the performance of ant-based clustering. In A. Abraham, M. Köppen, and K. Franke, (Eds.), HIS, volume 105 of Frontiers in artificial intelligence and applications, (pp. 204–213). IOS Press. ISBN 1-58603-394-8.

  • Handl, J., Knowles, J., & Dorigo, M. (2006). Ant-based clustering and topographic mapping. Artificial Life, 12(1), 35–61.

    Article  Google Scholar 

  • Hartigan, J. A. (1975). Clustering algorithms. NY, USA: Wiley.

    MATH  Google Scholar 

  • Hasan, M. J. A., & Ramakrishnan, S. (2011). A survey: hybrid evolutionary algorithms for cluster analysis. Artif. Intell. Rev., 36(3), 179–204.

    Article  Google Scholar 

  • He, H., & Tan, Y. (2012). A two-stage genetic algorithm for automatic clustering. Neurocomputing, 81, 49–59.

    Article  Google Scholar 

  • Hofmann, T., Schölkopf, B., & Smola, A. J. (2008). Kernel methods in machine learning. The Annals of Statistics, pp. 1171–1220.

  • Hubert, L. & Arabie, P. (Dec 1985). Comparing partitions. Journal of Classification, 2(1):193–218. ISSN 1432-1343.

  • Jiang, L. & Xie, D.(2018). An efficient differential memetic algorithm for clustering problem. IAENG International Journal of Computer Science, 45(1).

  • Kanade, P. M. & Hall, L. O. (2007). Fuzzy ants and clustering. IEEE Transactions on Systems, Man, and Cybernetics, Part A, 37(5):758–769.

  • Krishna, K. & Murty, M. N. (1999). Genetic k-means algorithm. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 29(3):433–439.

  • Kuntz, P., Layzell, P., & Snyers, D. (1997). A colony of ant-like agents for partitioning in vlsi technology. In Proceedings of the fourth european conference on artificial life, (pp. 417–424). MIT Press.

  • Li, J., Fan, H., Yuan, D., & Zhang, C. (2008). Kernel function clustering based on ant colony algorithm. In Proceedings - 4th international conference on natural computation, ICNC 2008, (vol 7, pp. 645 – 649), 11.

  • Liu, Y., Li, Z., Xiong, H., Gao, X., & Wu, J. (2010). Understanding of internal clustering validation measures. In 2010 IEEE international conference on data mining, (pp. 911–916). IEEE.

  • Liu, Y. Y., Thulasiraman, P., & Thulasiram, R. K.(2016). Parallelizing active memory ants with mapreduce for clustering financial time series data. In Proceedings of IEEE international conferences on big data and cloud computing (BDCloud), social computing and networking (SocialCom), sustainable computing and communications (SustainCom)(BDCloud-SocialCom-SustainCom), (pp. 137–144). IEEE.

  • Lu, Y., Lu, S., Fotouhi, F., Deng, Y., & Brown, S. J. (2004). Fgka: a fast genetic k-means clustering algorithm. In H. Haddad, A. Omicini, R. L. Wainwright, and L. M. Liebrock, (Eds.), SAC, (pp. 622–623). ACM. ISBN 1-58113-812-1.

  • Lukashin, A. V., & Fuchs, R. (2001). Analysis of temporal gene expression profiles: clustering by simulated annealing and determining the optimal number of clusters. Bioinformatics, 17(5), 405–414.

    Article  Google Scholar 

  • Lumer, E. D. & Faieta, B. (1994). Diversity and adaptation in populations of clustering ants. In Proceedings of the third international conference on Simulation of adaptive behavior: from animals to animats 3: from animals to animats 3, (pp. 501–508). MIT Press.

  • Martí, R., Laguna, M., & Glover, F. W. (2007). Principles of tabu search. In Handbook of Approximation Algorithms and Metaheuristics.

  • Maulik, U., & Bandyopadhyay, S. (2000). Genetic algorithm-based clustering technique. Pattern Recognition, 33(9), 1455–1465.

    Article  Google Scholar 

  • Monmarché, N. (1999). On data clustering with artificial ants. In AAAI-99 & GECCO-99 workshop on data mining with evolutionary algorithms: research directions, (pp. 23–26).

  • Monmarché, N., Slimane, M., & Venturini, G. (1999). On improving clustering in numerical databases with artificial ants. In D. Floreano, J.-D. Nicoud, and F. Mondada, (Eds.), ECAL, volume 1674 of Lecture Notes in Computer Science, (pp. 626–635). Springer. ISBN 3-540-66452-1.

  • Murthy, C. A., & Chowdhury, N. (1996). In search of optimal clusters using genetic algorithms. Pattern Recognition Letters, 17(8), 825–832.

    Article  Google Scholar 

  • Ng, M.(2003). A parallel tabu search heuristic for clustering data sets. In Parallel processing workshops, 2003, international conference on. IEEE.

  • Ng, M. K., & Wong, J. C. (2002). Clustering categorical data sets using tabu search techniques. Pattern Recognition, 35(12), 2783–2790.

    Article  MATH  Google Scholar 

  • Niknam, T., Amiri, B., Olamaei, J., & Arefi, A. (2009). An efficient hybrid evolutionary optimization algorithm based on pso and sa for clustering. Journal of Zhejiang University-SCIENCE A, 10, 512–519.

    Article  MATH  Google Scholar 

  • Oduntan, O. I., Thulasiraman, P., & Thulasiram, R. (2014). Portfolio diversification using ant brood sorting clustering. In Sixth world congress on nature and biologically inspired computing, NaBIC 2014, Porto, Portugal, July 30 - August 1, 2014, (pp. 256–261). IEEE.

  • Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.

    MathSciNet  MATH  Google Scholar 

  • Pereira, A. L., De Castro, L., Hruschka, E., & Gudwin, R. (2005). Towards improving clustering ants: An adaptive ant clustering algorithm. Informatica (Slovenia), 29, 143–154.

    MATH  Google Scholar 

  • Qasem, M., Ying, Y., Wang, Z., Thulasiraman, P., & Thulasiram, R. (2018). Enhancing ant brood clustering with adaptive radius of perception and non-parametric estimation on multi-core architectures. In Advances in intelligent networking and collaborative systems. INCoS 2017. Springer.

  • Qu, J. & Liu, X.(2007). A quick ant clustering algorithm. In Fourth international conference on fuzzy systems and knowledge discovery (FSKD 2007), (vol. 1, pp. 722–725), Aug.

  • Rahman, M. A. & Islam, M. Z.(2014). A hybrid clustering technique combining a novel genetic algorithm with k-means. Knowledge-Based Systems, 71, 345 – 365. ISSN 0950-7051.

  • Rosenberg, A. & Hirschberg, J. (2007). V-measure: A conditional entropy-based external cluster evaluation measure. In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL), (pp. 410–420).

  • Rousseeuw, P. J. (1987). Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20, 53–65.

    Article  MATH  Google Scholar 

  • Selim, S. Z., & Alsultan, K. (1991). A simulated annealing algorithm for the clustering problem. Pattern Recognition, 24(10), 1003–1008.

    Article  MathSciNet  Google Scholar 

  • Shawe-Taylor, John, & Cristianini, Nello. (2004). Kernel methods for pattern analysis. Cambridge University Press.

    Book  MATH  Google Scholar 

  • Shirkhorshidi, A. S., Aghabozorgi, S., & Wah, T. Y. (2015). A comparison study on similarity and dissimilarity measures in clustering continuous data. PLOS ONE, 10(12), 1–20. https://doi.org/10.1371/journal.pone.0144059.

    Article  Google Scholar 

  • Sung, C. S., & Jin, H. W. (2000). A tabu-search-based heuristic for clustering. Pattern Recognition, 33(5), 849–858.

    Article  Google Scholar 

  • Turkensteen, M., & Andersen, K. A. (2009). A tabu search approach to clustering. In B. Fleischmann, K.-H. Borgwardt, R. Klein, and A. Tuma, (Eds), Operations research proceedings 2008, (pp. 475–480), Berlin, Heidelberg. Springer Berlin Heidelberg.

  • Tvrdik, J., & Křivỳ, I. (2015). Hybrid differential evolution algorithm for optimal clustering. Applied Soft Computing, 35, 502–512.

    Article  Google Scholar 

  • Vinh, N. X., Epps, J., & Bailey, J. (2010). Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance. Journal of Machine Learning Research, 11(Oct), 2837–2854.

  • Wang, F., Zhang, D., & Bao, N. (2009). Fuzzy document clustering based on ant colony algorithm. In W. Yu, H. He, and N. Zhang, (Eds.), Advances in neural networks – ISNN 2009, (pp. 709–716), Berlin, Heidelberg. Springer Berlin Heidelberg.

  • Weili, Z. (2009). An improved entropy-based ant clustering algorithm. In 2009 WASE international conference on information engineering, (vol. 2, pp. 41–44).

  • Welch, W. J. (1982). Algorithmic complexity: three np- hard problems in computational statistics. Journal of Statistical Computation and Simulation, 15(1), 17–25.

    Article  MathSciNet  MATH  Google Scholar 

  • Xu, D., & Tian, Y. (2015). A comprehensive survey of clustering algorithms. Annals of Data Science, 2(2), 165–193.

    Article  MathSciNet  Google Scholar 

  • Yaghini, M. & Ghazanfari, N. (2010). Tabu-km: a hybrid clustering algorithm based on tabu search approach. International Journal of Industrial Engineering, 21(2).

  • Yang, Y. & Kamel, M. S. (2006). An aggregated clustering approach using multi-ant colonies algorithms. Pattern Recognition, 39(7), 1278–1289. ISSN 0031-3203.

  • Yang, X.-S. (2013). Optimization and metaheuristic algorithms in engineering. In X.-S. Yang, A. H. Gandomi, S. Talatahari, and A. H. Alavi, (Eds.), Metaheuristics in water, geotechnical and transport engineering, (pp. 1–23). Elsevier, ISBN 978-0-12-398296-4.

  • Yang, L., & Jin, R. (2006). Distance metric learning: A comprehensive survey. Michigan State Universiy, 2(2), 4.

    Google Scholar 

  • Zhang, J., Marszałek, M., Lazebnik, S., & Schmid, C. (2007). Local features and kernels for classification of texture and object categories: A comprehensive study. International Journal of Computer Vision, 73(2), 213–238.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Manitoba, Winnipeg, MB, Canada

    Olayinka Idowu Oduntan & Parimala Thulasiraman

Authors
  1. Olayinka Idowu Oduntan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Parimala Thulasiraman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Olayinka Idowu Oduntan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Oduntan, O.I., Thulasiraman, P. Blending multiple algorithmic granular components: a recipe for clustering. Swarm Intell 16, 305–349 (2022). https://doi.org/10.1007/s11721-022-00219-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11721-022-00219-8

Keywords

Search

Navigation