Advertisement

A wrapper-filter feature selection technique based on ant colony optimization

  • Manosij Ghosh
  • Ritam Guha
  • Ram SarkarEmail author
  • Ajith Abraham
Hybrid Artificial Intelligence and Machine Learning Technologies
  • 53 Downloads

Abstract

Ant colony optimization (ACO) is a well-explored meta-heuristic algorithm, among whose many applications feature selection (FS) is an important one. Most existing versions of ACO are either wrapper based or filter based. In this paper, we propose a wrapper-filter combination of ACO, where we introduce subset evaluation using a filter method instead of using a wrapper method to reduce computational complexity. A memory to keep the best ants and feature dimension-dependent pheromone update has also been used to perform FS in a multi-objective manner. Our proposed approach has been evaluated on various real-life datasets, taken from UCI Machine Learning repository and NIPS2003 FS challenge, using K-nearest neighbors and multi-layer perceptron classifiers. The experimental outcomes have been compared to some popular FS methods. The comparison of results clearly shows that our method outperforms most of the state-of-the-art algorithms used for FS. For measuring the robustness of the proposed model, it has been additionally evaluated on facial emotion recognition and microarray datasets.

Keywords

Wrapper-filter method Ant colony optimization Feature selection NIPS2003 challenge 

Notes

References

  1. 1.
    Contributors W (2015) Curse of dimensionality. Wikipedia, Free EncyclGoogle Scholar
  2. 2.
    Ghosh M, Begum S, Sarkar R et al (2019) Recursive Memetic Algorithm for gene selection in microarray data. Expert Syst Appl 116:172–185.  https://doi.org/10.1016/j.eswa.2018.06.057 CrossRefGoogle Scholar
  3. 3.
    Liu H, Motoda H (2007) Computational methods of feature selection. CRC Press, Boca RatonzbMATHGoogle Scholar
  4. 4.
    Mitra P, Murthy CA, Pal SK (2002) Unsupervised feature selection using feature similarity. IEEE Trans Pattern Anal Mach Intell 24:301–312CrossRefGoogle Scholar
  5. 5.
    Shang W, Huang H, Zhu H et al (2007) A novel feature selection algorithm for text categorization. Expert Syst Appl 33:1–5CrossRefGoogle Scholar
  6. 6.
    Yang J, Honavar V (1998) Feature subset selection using a genetic algorithm. IEEE Intell Syst Appl 13:44–49CrossRefGoogle Scholar
  7. 7.
    Moradi P, Gholampour M (2016) A hybrid particle swarm optimization for feature subset selection by integrating a novel local search strategy. Appl Soft Comput J 43:117–130.  https://doi.org/10.1016/j.asoc.2016.01.044 CrossRefGoogle Scholar
  8. 8.
    Forsati R, Moayedikia A, Jensen R et al (2014) Enriched ant colony optimization and its application in feature selection. Neurocomputing 142:354–371.  https://doi.org/10.1016/j.neucom.2014.03.053 CrossRefGoogle Scholar
  9. 9.
    Duval B, Hao J-K, Hernandez Hernandez JC (2009) A memetic algorithm for gene selection and molecular classification of cancer. In: Proceedings of 11th annual conference genetic evolutionary computation—GECCO’09 201.  https://doi.org/10.1145/1569901.1569930
  10. 10.
    Zhu Z, Ong YS, Dash M (2007) Markov blanket-embedded genetic algorithm for gene selection. Pattern Recognit 40:3236–3248.  https://doi.org/10.1016/j.patcog.2007.02.007 CrossRefzbMATHGoogle Scholar
  11. 11.
    Fogel DB (1994) An introduction to simulated evolutionary optimization. IEEE Trans Neural Netw 5:3–14CrossRefGoogle Scholar
  12. 12.
    Gandomi AH, Yang X-S, Alavi AH (2013) Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Eng Comput 29:17–35CrossRefGoogle Scholar
  13. 13.
    Karaboga D, Akay B (2009) A comparative study of artificial bee colony algorithm. Appl Math Comput 214:108–132MathSciNetzbMATHGoogle Scholar
  14. 14.
    Forsati R, Moayedikia A, Keikha A, Shamsfard M (2012) A novel approach for feature selection based on the bee colony optimization. Int J Comput Appl 43:30–34Google Scholar
  15. 15.
    Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67CrossRefGoogle Scholar
  16. 16.
    Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61CrossRefGoogle Scholar
  17. 17.
    Jain M, Singh V, Rani A (2019) A novel nature-inspired algorithm for optimization: squirrel search algorithm. Swarm Evol Comput 44:148–175CrossRefGoogle Scholar
  18. 18.
    Li MD, Zhao H, Weng XW, Han T (2016) A novel nature-inspired algorithm for optimization: virus colony search. Adv Eng Softw 92:65–88CrossRefGoogle Scholar
  19. 19.
    Mafarja M, Mirjalili S (2018) Whale optimization approaches for wrapper feature selection. Appl Soft Comput 62:441–453CrossRefGoogle Scholar
  20. 20.
    Mafarja MM, Mirjalili S (2017) Hybrid whale optimization algorithm with simulated annealing for feature selection. Neurocomputing 260:302–312CrossRefGoogle Scholar
  21. 21.
    Emary E, Zawbaa HM, Hassanien AE (2016) Binary grey wolf optimization approaches for feature selection. Neurocomputing 172:371–381.  https://doi.org/10.1016/j.neucom.2015.06.083 CrossRefGoogle Scholar
  22. 22.
    Wei J, Zhang R, Yu Z et al (2017) A BPSO-SVM algorithm based on memory renewal and enhanced mutation mechanisms for feature selection. Appl Soft Comput J 58:176–192.  https://doi.org/10.1016/j.asoc.2017.04.061 CrossRefGoogle Scholar
  23. 23.
    Ghosh M, Guha R, Mondal R et al (2018) Feature selection using histogram-based multi-objective GA for handwritten Devanagari numeral recognition. In: Bhateja V, Coello Coello C, Satapathy S, Pattnaik P (eds) Intelligent engineering informatics. Advances in intelligent systems and computing. Springer, Singapore, pp 471–479.  https://doi.org/10.1007/978-981-10-7566-7_46 CrossRefGoogle Scholar
  24. 24.
    Dorigo M, Stützle T (2019) Ant colony optimization: overview and recent advances. In: Gendreau M, Potvin JY (eds) Handbook of metaheuristics. Springer, Cham, pp 311–351.  https://doi.org/10.1007/978-3-319-91086-4_10 CrossRefGoogle Scholar
  25. 25.
    Dorigo M, Maniezzo V, Colorni A (1996) Ant system: optimization by a colony of cooperating agents. IEEE Trans Syst Man Cybern Part B 26:29–41CrossRefGoogle Scholar
  26. 26.
    Gambardella LM, Dorigo M (1996) Solving symmetric and asymmetric TSPs by ant colonies. In: Proceedings of IEEE international conference on Evolutionary computation, 1996. IEEE, pp 622–627Google Scholar
  27. 27.
    Stützle T, Hoos HH (2000) MAX–MIN ant system. Futur Gener Comput Syst 16:889–914CrossRefzbMATHGoogle Scholar
  28. 28.
    Zhang Z, Feng Z (2012) Two-stage updating pheromone for invariant ant colony optimization algorithm. Expert Syst Appl 39:706–712CrossRefGoogle Scholar
  29. 29.
    Aghdam MH, Ghasem-Aghaee N, Basiri ME (2009) Text feature selection using ant colony optimization. Expert Syst Appl 36:6843–6853.  https://doi.org/10.1016/j.eswa.2008.08.022 CrossRefGoogle Scholar
  30. 30.
    Tabakhi S, Moradi P, Akhlaghian F (2014) An unsupervised feature selection algorithm based on ant colony optimization. Eng Appl Artif Intell 32:112–123.  https://doi.org/10.1016/j.engappai.2014.03.007 CrossRefGoogle Scholar
  31. 31.
    Tabakhi S, Moradi P (2015) Relevance-redundancy feature selection based on ant colony optimization. Pattern Recognit 48:2798–2811.  https://doi.org/10.1016/j.patcog.2015.03.020 CrossRefGoogle Scholar
  32. 32.
    Tabakhi S, Najafi A, Ranjbar R, Moradi P (2015) Gene selection for microarray data classification using a novel ant colony optimization. Neurocomputing 168:1024–1036.  https://doi.org/10.1016/j.neucom.2015.05.022 CrossRefGoogle Scholar
  33. 33.
    Markid HY, Dadaneh BZ, Moghaddam ME (2015) Bidirectional ant colony optimization for feature selection. In: The international symposium on artificial intelligence and signal processing (AISP). IEEE, Mashhad.  https://doi.org/10.1109/AISP.2015.7123519
  34. 34.
    Kashef S, Nezamabadi-pour H (2015) An advanced ACO algorithm for feature subset selection. Neurocomputing 147:271–279.  https://doi.org/10.1016/j.neucom.2014.06.067 CrossRefGoogle Scholar
  35. 35.
    Moradi P, Rostami M (2015) Integration of graph clustering with ant colony optimization for feature selection. Knowl Based Syst 84:144–161.  https://doi.org/10.1016/j.knosys.2015.04.007 CrossRefGoogle Scholar
  36. 36.
    Ghimatgar H, Kazemi K, Helfroush MS, Aarabi A (2018) An improved feature selection algorithm based on graph clustering and ant colony optimization. Knowl Based Syst 159:270–285CrossRefGoogle Scholar
  37. 37.
    Sreeja NK, Sankar A (2015) Pattern matching based classification using Ant Colony Optimization based feature selection. Appl Soft Comput J 31:91–102.  https://doi.org/10.1016/j.asoc.2015.02.036 CrossRefGoogle Scholar
  38. 38.
    (2018) Computational complexity of machine learning algorithms. https://www.thekerneltrip.com/machine/learning/computational-complexity-learning-algorithms/. Accessed 15 Feb 2019
  39. 39.
    Kabir MM, Shahjahan M, Murase K (2012) A new hybrid ant colony optimization algorithm for feature selection. Expert Syst Appl 39:3747–3763.  https://doi.org/10.1016/j.eswa.2011.09.073 CrossRefGoogle Scholar
  40. 40.
    Fallahzadeh O, Dehghani-Bidgoli Z, Assarian M (2018) Raman spectral feature selection using ant colony optimization for breast cancer diagnosis. Lasers Med Sci 33(8):1799–1806CrossRefGoogle Scholar
  41. 41.
    Sweetlin JD, Nehemiah HK, Kannan A (2018) Computer aided diagnosis of pulmonary hamartoma from CT scan images using ant colony optimization based feature selection. Alex Eng J 57:1557–1567CrossRefGoogle Scholar
  42. 42.
    Yin Z, Du C, Liu J et al (2018) Research on autodisturbance-rejection control of induction motors based on an ant colony optimization algorithm. IEEE Trans Ind Electron 65:3077–3094CrossRefGoogle Scholar
  43. 43.
    Parvin H, Moradi P, Esmaeili S (2019) TCFACO: trust-aware collaborative filtering method based on ant colony optimization. Expert Syst Appl 118:152–168CrossRefGoogle Scholar
  44. 44.
    Uthayakumar J, Metawa N, Shankar K, Lakshmanaprabu SK (2018) Financial crisis prediction model using ant colony optimization. Int J Inf Manag.  https://doi.org/10.1016/j.ijinfomgt.2018.12.001 Google Scholar
  45. 45.
    Langner O, Dotsch R, Bijlstra G et al (2010) Presentation and validation of the radboud faces database. Cognit Emot 24:1377–1388.  https://doi.org/10.1080/02699930903485076 CrossRefGoogle Scholar
  46. 46.
    Hamamoto Y, Uchimura S, Watanabe M et al (1998) A Gabor filter-based method for recognizing handwritten numerals. Pattern Recognit 31:395–400.  https://doi.org/10.1016/S0031-3203(97)00057-5 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Computer Science and Engineering DepartmentJadavpur UniversityKolkataIndia
  2. 2.Scientific Network for Innovation and Research ExcellenceMachine Intelligence Research Labs (MIR Labs)AuburnUSA

Personalised recommendations