# Ant colony classification mining algorithm based on pheromone attraction and exclusion

## Abstract

Ant colony optimization algorithms have been applied successfully in classification rule mining. However, basic ant colony classification mining algorithms generally suffer from problems, such as premature convergence and falling into local optimum easily. Simultaneously, the classification mining algorithms use sequential covering strategy to discover rules, and the interaction between rules is less considered. In this study, a new ant colony classification mining algorithm based on pheromone attraction and exclusion (Ant-Miner\(_\mathrm{PAE}\)) is proposed, in which a new pheromone calculation method is designed and the search is guided by the new probability transfer formula. By contrast,the basic algorithm structure is modified, and the order of the iteration is adjusted. Thus, the problem of rule interaction is mitigated. Ant-Miner\(_\mathrm{PAE}\) can balance the relation of exploration and development of constructing rules, which can make the ants in the search process initially explore and develop in the later period. Our experiments, which use 12 publicly available data sets, show that the predictive accuracy obtained by Ant-Miner\(_\mathrm{PAE}\) implementing the proposed pheromone attraction and exclusion strategy is statistically significantly higher than the predictive accuracy of other rule induction classification algorithms, such as CN2, C4.5 rules, PSO/ACO2, Ant-Miner, and *c*Ant-Miner\(_\mathrm{PB}\). Furthermore, the rules discovered by Ant-Miner\(_\mathrm{PAE}\) are considerably simpler than those discovered by its counterparts.

### Keywords

Data mining Classification rule Ant colony algorithm Pheromone attraction and exclusion### References

- Asuncion A, Newman DJ (2007) UCI machine learning repository. School of Information and Computer Science, University of California, Irvine. http://www.ics.uci.edu/~mlearn/mlrepository.html
- Chan A, Freitas AA (2006) A new ant colony algorithm for multi-label classification with applications in bioinfomatics. Proceedings of the 8th annual conference on genetic and evolutionary computation. ACM, New York, pp 27–34Google Scholar
- Chan A, Freitas AA (2006) A new classification-rule pruning procedure for an ant colony algorithm. Artificial evolution. Springer, New York, pp 25–36CrossRefGoogle Scholar
- Clark P, Boswell R (1991) Rule induction with CN2: some recent improvements. Machine learning EWSL-91. Springer, New York, pp 151–163CrossRefGoogle Scholar
- Clark P, Niblett T (1989) The CN2 induction algorithm. Mach Learn 3(4):261–283Google Scholar
- De López R, Mántaras M (1991) A distance-based attribute selection measure for decision tree induction. Mach Learn 6(1):81–92CrossRefGoogle Scholar
- Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetMATHGoogle Scholar
- Dhar V, Chou D, Provost F (2000) Discovering interesting patterns for investment decision making with glowera genetic learner overlaid with entropy reduction. Data Min Knowl Discov 4(4):251–280CrossRefMATHGoogle Scholar
- Dorigo M (1994) Learning by probabilistic boolean networks. IEEE world congress on computational intelligence, IEEE international conference on neural networks, vol 2. IEEE, New York, pp 887–891Google Scholar
- Dorigo M, Stützle T (2004) Ant colony optimization for NP-hard problems. Ant Colony Optimization. MIT press, pp 153–222Google Scholar
- Dorigo M, Maniezzo V, Colorni A (1996) Ant system: optimization by a colony of cooperating agents. IEEE Trans Syst Man Cybern Part B Cybern 26(1):29–41CrossRefGoogle Scholar
- Freitas AA, Wieser DC, Apweiler R (2010) On the importance of comprehensible classification models for protein function prediction. IEEE/ACM Trans Comput Biol Bioinform (TCBB) 7(1):172–182CrossRefGoogle Scholar
- Han J, Kamber M (2001) Data mining: concepts and technologies. Data Mining Concepts Models Methods & Algorithms, 2nd edn, 5(4):1–18Google Scholar
- Holden N, Freitas AA (2008) A hybrid PSO/ACO algorithm for discovering classification rules in data mining. J Artif Evol Appl 2008. doi:10.1122/2008/316145
- Jin P, Zhu Y, Hu K, Li S (2006) Classification rule mining based on ant colony optimization algorithm. Intelligent control and automation. Springer, New York, pp 654–663Google Scholar
- Kaltenbacher B (2001) On the regularizing properties of a full multigrid method for ill-posed problems. Inverse Probl 17(4):767–788MathSciNetCrossRefMATHGoogle Scholar
- Kaltenbacher B, Schicho J (2002) A multi-grid method with a priori and a posteriori level choice for the regularization of nonlinear ill-posed problems. Numer Math 93(1):77–107Google Scholar
- Keung YL, Zou J (2000) An efficient linear solver for nonlinear parameter identification problems. Siam J Sci Compt 22(5):1511–1526Google Scholar
- Kohavi R, Sahami M (1996) Error-based and entropy-based discretization of continuous features. In: KDD, pp 114–119Google Scholar
- Li J, Zou J (2007) A multilevel model correction method for parameter identification. Inverse Probl 23(5):1759–1786MathSciNetCrossRefMATHGoogle Scholar
- Liu B, Pan J (2007) Research of classification algorithm based on ant colony optimization. Comput Appl Softw 24(4):50–53Google Scholar
- Liu B, Abbass HA, McKay B (2002) Density-based heuristic for rule discovery with ant-miner. In: The 6th Australia-Japan joint workshop on intelligent and evolutionary system, vol 184, CiteseerGoogle Scholar
- Liu B, Abbass HA, McKay B (2003) Classification rule discovery with ant colony optimization. In: Null. IEEE, New York, p 83Google Scholar
- Martens D, De Backer M, Haesen R, Vanthienen J, Snoeck M, Baesens B (2007) Classification with ant colony optimization. IEEE Trans Evol Comput 11(5):651–665CrossRefGoogle Scholar
- Meyer F, Parpinelli RS (2012) ACO: public software. http://www.aco-metaheuristic.org/aco-code/public-software.html
- Otero FEB, Freitas AA, Johnson CG (2008) Cant-miner: an ant colony classification algorithm to cope with continuous attributes. Ant colony optimization and swarm intelligence. Springer, New York, pp 48–59CrossRefGoogle Scholar
- Otero FEB, Freitas A, Johnson CG et al (2009) Handling continuous attributes in ant colony classification algorithms. IEEE symposium on computational intelligence and data mining, CIDM’09. IEEE, New York, pp 225–231CrossRefGoogle Scholar
- Otero FEB, Freitas A, Johnson Colin G et al (2013) A new sequential covering strategy for inducing classification rules with ant colony algorithms. IEEE Trans Evol Comput 17(1):64–76CrossRefGoogle Scholar
- Parpinelli RS, Lopes HS, Freitas AA (2001) An ant colony based system for data mining: applications to medical data. In: Proceedings of the genetic and evolutionary computation conference (GECCO-2001), pp 791–797, CiteseerGoogle Scholar
- Parpinelli Rafael S, Lopes Heitor S, Freitas A et al (2002a) Data mining with an ant colony optimization algorithm. IEEE Trans Evol Comput 6(4):321–332CrossRefMATHGoogle Scholar
- Parpinelli RS, Lopes HS, Freitas AA (2002) An ant colony algorithm for classification rule discovery. In: Data mining: a heuristic approach, pp 191–208Google Scholar
- Quinlan JR (2014) C4.5: programs for machine learning. Elsevier, New YorkGoogle Scholar
- Smaldon J, Freitas AA (2006) A new version of the ant-miner algorithm discovering unordered rule sets. Proceedings of the 8th annual conference on genetic and evolutionary computation. ACM, New York, pp 43–50Google Scholar
- Wang Z, Feng B (2005) Classification rule mining with an improved ant colony algorithm. AI 2004: advances in artificial intelligence. Springer, New York, pp 357–367Google Scholar