Advertisement

Using Ant Colony Optimization to Build Cluster-Based Classification Systems

  • Khalid M. Salama
  • Ashraf M. AbdelbarEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9882)

Abstract

Learning cluster-based classification systems is the process of partitioning a training set into data subsets (clusters), and then building a local classifier for each data cluster. The class of a new instance is predicted by first assigning the instance to its nearest cluster, and then using that cluster’s local classification model to predict the instance’s class. In this paper, we use the Ant Colony Optimization (ACO) meta-heuristic to optimize the data clusters based on a given classification algorithm in an integrated cluster-with-learn manner. The proposed ACO algorithms use two different clustering solution representation approaches: instance-based and medoid-based, where in the latter the number of clusters is optimized as part of the ACO algorithm’s execution. In our experiments, we employ three widely-used classification algorithms, k-nearest neighbours, Ripper, and C4.5, and evaluate performance on 30 UCI benchmark datasets. We compare the ACO results to the traditional c-means clustering algorithm, where the data clusters are built prior to learning the local classifiers.

Keywords

Ant Colony Optimization (ACO) Data mining Classification Clustering Cluster-based classification system 

Notes

Acknowledgments

Partial support of a grant from the Brandon University Research Council is gratefully acknowledged.

References

  1. 1.
    Abdelbar, A.M., Salama, K.M.: Clustering with the ACOR algorithm. In: Swarm Intelligence, LNCS, vol. 9882, pp. 210–222 (2016)Google Scholar
  2. 2.
    Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, Cambridge (2004)zbMATHGoogle Scholar
  3. 3.
    Gan, G., Ma, C., Wu, J.: Data Clustering: Theory, Algorithms, and Applications. SIAM Press, Philadelphia (2007)CrossRefzbMATHGoogle Scholar
  4. 4.
    Jafar, M., Sivakumar, R.: Ant-based clustering algorithms: a brief survey. Int. J. Comput. Theor. Eng. 2, 787–796 (2010)CrossRefGoogle Scholar
  5. 5.
    Liao, T., Socha, K., de Montes Oca, M., Stützle, T., Dorigo, M.: Ant colony optimization for mixed-variable optimization problems. IEEE Trans. Evol. Comput. 18(4), 503–518 (2014)CrossRefGoogle Scholar
  6. 6.
    Liu, X.Y., Fu, H.: An effective clustering algorithm with ant colony. J. Comput. 5, 598–605 (2010)Google Scholar
  7. 7.
    Martens, D., De Backer, M., Haesen, R., Vanthienen, J., Snoeck, M., Baesens, B.: Classification with ant colony optimization. IEEE Trans. Evol. Comput. 11(5), 651–665 (2007)CrossRefGoogle Scholar
  8. 8.
    Martens, D., Baesens, B., Fawcett, T.: Editorial survey: swarm intelligence for data mining. Mach. Learn. 82(1), 1–42 (2011)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Otero, F.E., Freitas, A.A., Johnson, C.: A new sequential covering strategy for inducing classification rules with ant colony algorithms. IEEE Trans. Evol. Comput. 17(1), 64–74 (2013)CrossRefGoogle Scholar
  10. 10.
    Otero, F.E., Freitas, A.A., Johnson, C.G.: Handling continuous attributes in ant colony classification algorithms. In: IEEE Symposium on Computational Intelligence in Data Mining (CIDM 2009), pp. 225–231 (2009)Google Scholar
  11. 11.
    Otero, F.E., Freitas, A.A., Johnson, C.G.: Inducing decision trees with an ant colony optimization algorithm. Appl. Soft Comput. 12(11), 3615–3626 (2012)CrossRefGoogle Scholar
  12. 12.
    Parpinelli, R.S., Lopes, H.S., Freitas, A.A.: Data mining with an ant colony optimization algorithm. IEEE Trans. Evol. Comput. 6(4), 321–332 (2002)CrossRefzbMATHGoogle Scholar
  13. 13.
    Salama, K.M., Abdelbar, A.M.: Learning neural network structures with ant colony algorithms. Swarm Intell. 9(4), 229–265 (2015)CrossRefGoogle Scholar
  14. 14.
    Salama, K.M., Abdelbar, A.M., Anwar, I.M.: Data reduction for classification with ant colony optimization. Intelligent Data Analysis (2016, to appear)Google Scholar
  15. 15.
    Salama, K.M., Abdelbar, A.M., Freitas, A.A.: Multiple pheromone types and other extensions to the ant-miner classification rule discovery algorithm. Swarm Intell. 5(3–4), 149–182 (2011)CrossRefGoogle Scholar
  16. 16.
    Salama, K.M., Abdelbar, A.M., Helal, A.Z., Freitas, A.A.: Instance-based classification with ant colony optimization. Intelligent Data Analysis (accepted, 2016)Google Scholar
  17. 17.
    Salama, K.M., Freitas, A.A.: Clustering-based Bayesian multi-net classifier construction with ant colony optimization. In: IEEE Congress on Evolutionary Computation (IEEE CEC), pp. 3079–3086 (2013)Google Scholar
  18. 18.
    Salama, K.M., Freitas, A.A.: Learning Bayesian network classifiers using ant colony optimization. Swarm Intell. 7(2–3), 229–254 (2013)CrossRefGoogle Scholar
  19. 19.
    Salama, K.M., Freitas, A.A.: ABC-Miner+: constructing Markov blanket classifiers with ant colony algorithms. Memetic Comput. 6(3), 183–206 (2014)CrossRefGoogle Scholar
  20. 20.
    Salama, K.M., Freitas, A.A.: Classification with cluster-based Bayesian multi-nets using ant colony optimization. Swarm Evol. Comput. 18, 54–70 (2014)CrossRefGoogle Scholar
  21. 21.
    Salama, K.M., Freitas, A.A.: Ant colony algorithms for constructing Bayesian multi-net classifiers. Intell. Data Anal. 19(2), 233–257 (2015)Google Scholar
  22. 22.
    Salama, K.M., Otero, F.E.: Learning multi-tree classification models with ant colony optimization. In: 6th International Conference on Evolutionary Computation Theory and Applications (ECTA 2014), pp. 38–48 (2014)Google Scholar
  23. 23.
    Shelokar, P.S., Jayaraman, V.K., Kulkarni, B.D.: An ant colony approach for clustering. Anal. Chim. Acta 509(2), 187–195 (2004)CrossRefGoogle Scholar
  24. 24.
    Socha, K., Blum, C.: An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training. Neural Comput. Appl. 16, 235–247 (2007)CrossRefGoogle Scholar
  25. 25.
    Socha, K., Dorigo, M.: Ant colony optimization for continuous domains. Eur. J. Oper. Res. 185, 1155–1173 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Tan, P.N., Steinbach, M., Kumar, V.: Introduction to Data Mining, 2nd edn. Addison Wesley, Reading (2005)Google Scholar
  27. 27.
    Whitley, D., Dominic, S., Das, R., Anderson, C.: Genetic reinforcement learning for neurocontrol problems. Mach. Learn. 13(2–3), 259–284 (1993)CrossRefGoogle Scholar
  28. 28.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 3rd edn. Morgan Kaufmann, San Francisco (2010)zbMATHGoogle Scholar
  29. 29.
    Xu, R., Wunsch, D.: Clustering. Wiley-IEEE Press, Hoboken (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.School of ComputingUniversity of KentCanterburyUK
  2. 2.Department of Mathematics & Computer ScienceBrandon UniversityManitobaCanada

Personalised recommendations