Advertisement

Memetic Computing

, Volume 7, Issue 3, pp 181–201 | Cite as

A hybridization of clonal selection algorithm with iterated local search and variable neighborhood search for the feature selection problem

  • Magdalene Marinaki
  • Yannis Marinakis
Regular Research Paper

Abstract

Nature inspired methods are approaches that are used in various fields and for the solution of a number of problems. This study uses a hybridized version of the clonal selection algorithm, the clonal selection algorithm–iterated local search–variable neighborhood search (CSA–ILS–VNS), for the solution of the feature selection problem (FSP). The clonal selection algorithm is inspired by the clonal selection and affinity maturation process of B cells of the natural immune system once the immune system has detected a pathogen. The proposed clonal selection algorithm is combined with a number of nearest neighbour based classifiers and it is tested using various benchmark data sets from the UCI machine learning repository. The algorithm is compared with variants of the clonal selection algorithm [the classic clonal selection algorithm (CSA), the clonal selection algorithm–iterated local search (CSA–ILS) and the clonal selection algorithm–variable neighborhood search (CSA–VNS)], a particle swarm optimization algorithm, an ant colony optimization algorithm and a genetic algorithm.

Keywords

Artificial immune systems Clonal selection algorithm  Variable neighborhood search Feature selection problem  Iterated local search 

References

  1. 1.
    Aha DW, Bankert RL (1996) A comparative evaluation of sequential feature selection algorithms. In: Fisher D, Lenx J-H (eds) Artificial intelligence and statistics. Springer, New York, pp 199–206Google Scholar
  2. 2.
    Al-Ani A (2005) Feature subset selection using ant colony optimization. Int J Comput Intell 2(1):53–58Google Scholar
  3. 3.
    Al-Ani A (2005) Ant colony optimization for feature subset selection. Trans Eng Comput Technol 4:35–38Google Scholar
  4. 4.
    Brabazon A, O’Neill M (2006) Biologically inspired algorithms for financial modelling. Natural computing series. Springer, BerlinGoogle Scholar
  5. 5.
    Cantu-Paz E (2004) Feature subset selection, class separability, and genetic algorithms. In: Genetic and evolutionary computation conference, pp 959–970Google Scholar
  6. 6.
    Cantu-Paz E, Newsam S, Kamath C (2004) Feature selection in scientific application. In: Proceedings of the 2004 ACM SIGKDD international conference on knowledge discovery and data mining, pp 788–793Google Scholar
  7. 7.
    Carvalho DR, Freitas AA (2004) A hybrid decision tree/genetic algorithm method for data mining. Inf Sci 163(1–3):13–35Google Scholar
  8. 8.
    Casado Yusta S (2009) Different metaheuristic strategies to solve the feature selection problem. Pattern Recognit Lett 30:525–534Google Scholar
  9. 9.
    Casillas J, Cordon O, Del Jesus MJ, Herrera F (2001) Genetic feature selection in a fuzzy rule-based classification system learning process for high-dimensional problems. Inf Sci 136(1–4):135–157Google Scholar
  10. 10.
    Chen S-C, Lin S-W, Chou S-Y (2011) Enhancing the classification accuracy by scatter-search-based ensemble approach. Appl Soft Comput 11(1):1021–1028Google Scholar
  11. 11.
    Chen Y, Miao D, Wang R (2010) A rough set approach to feature selection based on ant colony optimization. Pattern Recognit Lett 31:226–233Google Scholar
  12. 12.
    Chuang L-Y, Yang C-H, Li J-C (2011) Chaotic maps based on binary particle swarm optimization for feature selection. Appl Soft Comput 11(1):239–248Google Scholar
  13. 13.
    Cotta C, Sloper C, Moscato P (2004) Evolutionary search of thresholds for robust feature set selection: application to the analysis of microarray data. Lect Notes Comput Sci 3005:21–30Google Scholar
  14. 14.
    Cuevas E, Osuna-Enciso V, Wario F, Zaldívar D, Pérez-Cisneros M (2012) Automatic multiple circle detection based on artificial immune systems. Expert Syst Appl 39:713–722Google Scholar
  15. 15.
    Dabrowski J (2008) Clonal selection algorithm for vehicle routing. In: Proceedings of the 2008 1st international conference on information technology, IT 2008, pp 19–21, May 2008, Gdansk, PolandGoogle Scholar
  16. 16.
    Daniel WW (1990) Applied nonparametric statistics. Duxbury Thomson Learning, Pacific GroveGoogle Scholar
  17. 17.
    Dasgupta D (ed) (1998) Artificial immune systems and their application. Springer, HeidelbergGoogle Scholar
  18. 18.
    Dasgupta D, Niño LF (2009) Immunological computation: theory and applications. CRC Press/Taylor and Francis Group, Boca Raton/LondonGoogle Scholar
  19. 19.
    De Castro LN, Timmis J (2002) Artificial immune systems: a new computational intelligence approach. Springer, HeidelbergGoogle Scholar
  20. 20.
    De Castro LN, Von Zuben FJ (2000) The clonal selection algorithm with engineering applications. In: Workshop on artificial immune systems and their applications (GECCO’00), Las Vegas, NV, pp 36–37Google Scholar
  21. 21.
    De Castro LN, Von Zuben FJ (2002) Learning and optimization using the clonal selection principle. IEEE Trans Evol Comput 6(3):239–251Google Scholar
  22. 22.
    Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. Mach Learn Res 7:1–30MathSciNetGoogle Scholar
  23. 23.
    Duda RO, Hart PE, Stork DG (2001) Pattern classification and scene analysis, 2nd edn. Wiley, New YorkGoogle Scholar
  24. 24.
    ElAlami ME (2009) A filter model for feature subset selection based on genetic algorithm. Knowl Based Syst 22:356–362Google Scholar
  25. 25.
    Engelbrecht AP (2007) Computational intelligence: an introduction, 2nd edn. Wiley, EnglandGoogle Scholar
  26. 26.
    Fleuret F (2004) Fast binary feature selection with conditional mutual information. J Mach Learn Res (JMLR) 5:1531–1555MathSciNetGoogle Scholar
  27. 27.
    Flower D, Timmis J (eds) (2007) In silico immunology. Springer, New YorkGoogle Scholar
  28. 28.
    Forrest S, Perelson A, Allen L, Cherukuri R (1994) Self-nonself discrimination in a computer. In: Proceedings of the 1994 IEEE symposium on research in security and privacy. IEEE Computer Society Press, Los Alamitos, pp 202–212Google Scholar
  29. 29.
    Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32:675–701Google Scholar
  30. 30.
    Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11:86–92Google Scholar
  31. 31.
    Garcia S, Herrera F (2008) An extension on ’statistical comparisons of classifiers over multiple data sets’ for all pairwise comparisons. Mach Learn Res 9:2677–2694Google Scholar
  32. 32.
    Garcia S, Fernandez A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180:2044–2064Google Scholar
  33. 33.
    Garcia Lopez F, Garcia Torres M, Melian Batista B, Moreno Perez JA, Moreno Vega JM (2006) Solving feature subset selection problem by a parallel scatter search. Eur J Oper Res 169:477–489Google Scholar
  34. 34.
    Gong M, Jiao L, Zhang L (2010) Baldwinian learning in clonal selection algorithm for optimization. Inf Sci 180:1218–1236Google Scholar
  35. 35.
    Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182Google Scholar
  36. 36.
    Hansen P, Mladenovic N (2001) Variable neighborhood search: principles and applications. Eur J Oper Res 130:449–467MathSciNetGoogle Scholar
  37. 37.
    Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65–70MathSciNetGoogle Scholar
  38. 38.
    Hsu WH (2004) Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning. Inf Sci 163(1–3):103–122Google Scholar
  39. 39.
    Huang CL (2009) ACO-based hybrid classification system with feature subset selection and model parameters optimization. Neurocomputing 73:438–448Google Scholar
  40. 40.
    Huang J, Cai Y, Xu X (2007) A hybrid genetic algorithm for feature selection wrapper based on mutual information. Pattern Recognit Lett 28:1825–1844Google Scholar
  41. 41.
    Iman RL, Davenport JM (1980) Approximations of the critical region of the Friedman statistic. Commun Stat 9(6):571–595Google Scholar
  42. 42.
    Jain A, Zongker D (1997) Feature selection: evaluation, application, and small sample performance. IEEE Trans Pattern Anal Mach Intell 19:153–158Google Scholar
  43. 43.
    Kira K, Rendell L (1992) A practical approach to feature selection. In: Proceedings of the ninth international conference on machine learning, Aberdeen, Scotland, pp 249–256Google Scholar
  44. 44.
    Kohavi R, John G (1997) Wrappers for feature subset selection. Artif Intell 97:273–324Google Scholar
  45. 45.
    Li F, Gao S, Wang W, Tang Z (2009) An adaptive clonal selection algorithm for edge linking problem. IJCSNS Int J Comput Sci Netw Secur 9(7):57–65Google Scholar
  46. 46.
    Lin SW, Chen SC (2009) PSOLDA: a particle swarm optimization approach for enhancing classification accurate rate of linear discriminant analysis. Appl Soft Comput 9:1008–1015Google Scholar
  47. 47.
    Lin SW, Lee ZJ, Chen SC, Tseng TY (2008) Parameter determination of support vector machine and feature selection using simulated annealing approach. Appl Soft Comput 8:1505–1512Google Scholar
  48. 48.
    Lin SW, Ying KC, Chen SC, Lee ZJ (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35:1817–1824Google Scholar
  49. 49.
    Lourenco HR, Martin O, Stützle T (2002) Iterated local search. Handbook of metaheuristics. Operations research and management science, vol 57. Kluwer Academic Publishers, Dordrecht, pp 321–353Google Scholar
  50. 50.
    Ma J, Shi G, Gao L (2009) An improved immune clonal selection algorithm and its applications for VRP. In: Proceedings of the IEEE international conference on automation and logistics Shenyang, China, August 2009, pp 2097–2100Google Scholar
  51. 51.
    Maldonado S, Weber R (2009) A wrapper method for feature selection using support vector machines. Inf Sci 179(13):2208–2217Google Scholar
  52. 52.
    Marinaki M, Marinakis Y (2014) An island memetic differential evolution algorithm for the feature selection problem. In: Terrazas G et al (ed) Nature inspired cooperative strategies for optimization—NICSO 2013. Studies in computational intelligence, vol 512. Springer International Publishing, Switzerland, pp 29–42Google Scholar
  53. 53.
    Marinakis Y, Marinaki M (2013) A hybridized particle swarm optimization with expanding neighborhood topology for the feature selection problem. In: Blesa MJ et al (eds) HM 2013. LNCS, vol 7919. Springer, Berlin, pp 37–51Google Scholar
  54. 54.
    Marinakis Y, Marinaki M, Doumpos M, Zopounidis C (2009) Ant colony and particle swarm optimization for financial classification problems. Expert Syst Appl 36(7):10604–10611Google Scholar
  55. 55.
    Marinakis Y, Marinaki M, Doumpos M, Matsatsinis N, Zopounidis C (2008) Optimization of nearest neighbor classifiers via metaheuristic algorithms for credit risk assessment. J Glob Optim 42:279–293MathSciNetGoogle Scholar
  56. 56.
    Martin O, Otto SW, Felten EW (1991) Large-step markov chains for the traveling salesman problem. Complex Syst 5(3):299–326MathSciNetGoogle Scholar
  57. 57.
    Meiri R, Zahavi J (2006) Using simulated annealing to optimize the feature selection problem in marketing applications. Eur J Oper Res 171:842–858Google Scholar
  58. 58.
    Narendra PM, Fukunaga K (1977) A branch and bound algorithm for feature subset selection. IEEE Trans Comput 26(9):917–922Google Scholar
  59. 59.
    Pacheco J, Casado S, Nunez L (2009) A variable selection method based on tabu search for logistic regression models. Eur J Oper Res 199:506–511MathSciNetGoogle Scholar
  60. 60.
    Panigrahi BK, Yadav SR, Agrawal S, Tiwari MK (2007) A clonal algorithm to solve economic load dispatch. Electr Power Syst Res 77:1381–1389Google Scholar
  61. 61.
    Parpinelli RS, Lopes HS, Freitas AA (2002) An ant colony algorithm for classification rule discovery. In: Abbas H, Sarker R, Newton C (eds) Data mining: a heuristic approach. Idea group publishing, London, pp 191–208Google Scholar
  62. 62.
    Pedrycz W, Park BJ, Pizzi NJ (2009) Identifying core sets of discriminatory features using particle swarm optimization. Expert Syst Appl 36:4610–4616Google Scholar
  63. 63.
    Pudil P, Novovicova J, Kittler J (1994) Floating search methods in feature selection. Pattern Recognit Lett 15:1119–1125Google Scholar
  64. 64.
    Rokach L (2008) Genetic algorithm-based feature set partitioning for classification problems. Pattern Recognit Lett 41:1676–1700Google Scholar
  65. 65.
    Shelokar PS, Jayaraman VK, Kulkarni BD (2004) An ant colony classifier system: application to some process engineering problems. Comput Chem Eng 28:1577–1584Google Scholar
  66. 66.
    Siedlecki W, Sklansky J (1988) On automatic feature selection. Int J Pattern Recognit Artif Intell 2(2):197–220Google Scholar
  67. 67.
    Siedlecki W, Sklansky J (1989) A note on genetic algorithms for large-scale feature selection. Pattern Recognit Lett 10:335–347Google Scholar
  68. 68.
    Srinivasa KG, Venugopal KR, Patnaik LM (2007) A self-adaptive migration model genetic algorithm for data mining applications. Inf Sci 177(20):4295–4313Google Scholar
  69. 69.
    Talbi E-G (2009) Metaheuristics: from design to implementation. Wiley, New YorkGoogle Scholar
  70. 70.
    Timmis J, Neal M (2000) A resource limited artificial immune system for data analysis. Research and development in intelligent systems, vol 14. Springer, Cambridge, pp 19–32Google Scholar
  71. 71.
    Ulutas BH, Islier AA (2007) Parameter setting for clonal selection algorithm in facility layout problems. In: Gervasi O, Gavrilova M (eds) ICCSA 2007. LNCS, vol 4705, Part I. Springer, Berlin, pp 886–899Google Scholar
  72. 72.
    Ulutas BH, Islier AA (2009) A clonal selection algorithm for dynamic facility layout problems. J Manuf Syst 28:123–131Google Scholar
  73. 73.
    Ulutas BH, Kulturel-Konak S (2012) An artificial immune system based algorithm to solve unequal area facility layout problem. Expert Syst Appl 39(5):5384–5395Google Scholar
  74. 74.
    Uncu O, Turksen IB (2007) A novel feature selection approach: combining feature wrappers and filters. Inf Sci 177(2):449–466MathSciNetGoogle Scholar
  75. 75.
    Unler A, Murat A (2010) A discrete particle swarm optimization method for feature selection in binary classification problems. Eur J Oper Res 206:528–539Google Scholar
  76. 76.
    Wang X, Yang J, Teng X, Xia W, Jensen R (2007) Feature selection based on rough sets and particle swarm optimization. Pattern Recognit Lett 28:459–471Google Scholar
  77. 77.
    Wang Y, Feng XY, Huang YX, Pu DB, Zhou WG, Liang YC, Zhou CG (2007) A novel quantum swarm evolutionary algorithm and its applications. Neurocomputing 70(4–6):633–640Google Scholar
  78. 78.
    Yang J-H, Sun L, Lee HP, Qian Y, Liang Y-C (2008) Clonal selection based memetic algorithm for job shop scheduling problems. J Bionic Eng 5:111–119Google Scholar
  79. 79.
    Yu L, Liu H (2004) Efficient feature selection via analysis of relevance and redundancy. J Mach Learn Res 5:1205–1224Google Scholar
  80. 80.
    Zhang C, Hu H (2005) Ant colony optimization combining with mutual information for feature selection in support vector machines. In: Zhang S, Jarvis R (eds) AI 2005. LNAI, vol 3809, pp 918–921Google Scholar
  81. 81.
    Zhu Y, Gao S, Dai H, Li F, Tang Z (2007) Improved clonal algorithm and its application to traveling salesman problem. IJCSNS Int J Comput Sci Netw Secur 7(8):109–113Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.School of Production Engineering and ManagementTechnical University of CreteChaniaGreece

Personalised recommendations