Skip to main content

Advertisement

SpringerLink
Log in

A hybridization of clonal selection algorithm with iterated local search and variable neighborhood search for the feature selection problem

  • Regular Research Paper
  • Published:
Memetic Computing Aims and scope Submit manuscript

Abstract

Nature inspired methods are approaches that are used in various fields and for the solution of a number of problems. This study uses a hybridized version of the clonal selection algorithm, the clonal selection algorithm–iterated local search–variable neighborhood search (CSA–ILS–VNS), for the solution of the feature selection problem (FSP). The clonal selection algorithm is inspired by the clonal selection and affinity maturation process of B cells of the natural immune system once the immune system has detected a pathogen. The proposed clonal selection algorithm is combined with a number of nearest neighbour based classifiers and it is tested using various benchmark data sets from the UCI machine learning repository. The algorithm is compared with variants of the clonal selection algorithm [the classic clonal selection algorithm (CSA), the clonal selection algorithm–iterated local search (CSA–ILS) and the clonal selection algorithm–variable neighborhood search (CSA–VNS)], a particle swarm optimization algorithm, an ant colony optimization algorithm and a genetic algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aha DW, Bankert RL (1996) A comparative evaluation of sequential feature selection algorithms. In: Fisher D, Lenx J-H (eds) Artificial intelligence and statistics. Springer, New York, pp 199–206

    Google Scholar 

  2. Al-Ani A (2005) Feature subset selection using ant colony optimization. Int J Comput Intell 2(1):53–58

    Google Scholar 

  3. Al-Ani A (2005) Ant colony optimization for feature subset selection. Trans Eng Comput Technol 4:35–38

    Google Scholar 

  4. Brabazon A, O’Neill M (2006) Biologically inspired algorithms for financial modelling. Natural computing series. Springer, Berlin

    Google Scholar 

  5. Cantu-Paz E (2004) Feature subset selection, class separability, and genetic algorithms. In: Genetic and evolutionary computation conference, pp 959–970

  6. Cantu-Paz E, Newsam S, Kamath C (2004) Feature selection in scientific application. In: Proceedings of the 2004 ACM SIGKDD international conference on knowledge discovery and data mining, pp 788–793

  7. Carvalho DR, Freitas AA (2004) A hybrid decision tree/genetic algorithm method for data mining. Inf Sci 163(1–3):13–35

    Google Scholar 

  8. Casado Yusta S (2009) Different metaheuristic strategies to solve the feature selection problem. Pattern Recognit Lett 30:525–534

    Google Scholar 

  9. Casillas J, Cordon O, Del Jesus MJ, Herrera F (2001) Genetic feature selection in a fuzzy rule-based classification system learning process for high-dimensional problems. Inf Sci 136(1–4):135–157

    Google Scholar 

  10. Chen S-C, Lin S-W, Chou S-Y (2011) Enhancing the classification accuracy by scatter-search-based ensemble approach. Appl Soft Comput 11(1):1021–1028

    Google Scholar 

  11. Chen Y, Miao D, Wang R (2010) A rough set approach to feature selection based on ant colony optimization. Pattern Recognit Lett 31:226–233

    Google Scholar 

  12. Chuang L-Y, Yang C-H, Li J-C (2011) Chaotic maps based on binary particle swarm optimization for feature selection. Appl Soft Comput 11(1):239–248

    Google Scholar 

  13. Cotta C, Sloper C, Moscato P (2004) Evolutionary search of thresholds for robust feature set selection: application to the analysis of microarray data. Lect Notes Comput Sci 3005:21–30

    Google Scholar 

  14. Cuevas E, Osuna-Enciso V, Wario F, Zaldívar D, Pérez-Cisneros M (2012) Automatic multiple circle detection based on artificial immune systems. Expert Syst Appl 39:713–722

    Google Scholar 

  15. Dabrowski J (2008) Clonal selection algorithm for vehicle routing. In: Proceedings of the 2008 1st international conference on information technology, IT 2008, pp 19–21, May 2008, Gdansk, Poland

  16. Daniel WW (1990) Applied nonparametric statistics. Duxbury Thomson Learning, Pacific Grove

  17. Dasgupta D (ed) (1998) Artificial immune systems and their application. Springer, Heidelberg

    Google Scholar 

  18. Dasgupta D, Niño LF (2009) Immunological computation: theory and applications. CRC Press/Taylor and Francis Group, Boca Raton/London

    Google Scholar 

  19. De Castro LN, Timmis J (2002) Artificial immune systems: a new computational intelligence approach. Springer, Heidelberg

    Google Scholar 

  20. De Castro LN, Von Zuben FJ (2000) The clonal selection algorithm with engineering applications. In: Workshop on artificial immune systems and their applications (GECCO’00), Las Vegas, NV, pp 36–37

  21. De Castro LN, Von Zuben FJ (2002) Learning and optimization using the clonal selection principle. IEEE Trans Evol Comput 6(3):239–251

    Google Scholar 

  22. Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. Mach Learn Res 7:1–30

    MathSciNet  Google Scholar 

  23. Duda RO, Hart PE, Stork DG (2001) Pattern classification and scene analysis, 2nd edn. Wiley, New York

    Google Scholar 

  24. ElAlami ME (2009) A filter model for feature subset selection based on genetic algorithm. Knowl Based Syst 22:356–362

    Google Scholar 

  25. Engelbrecht AP (2007) Computational intelligence: an introduction, 2nd edn. Wiley, England

    Google Scholar 

  26. Fleuret F (2004) Fast binary feature selection with conditional mutual information. J Mach Learn Res (JMLR) 5:1531–1555

    MathSciNet  Google Scholar 

  27. Flower D, Timmis J (eds) (2007) In silico immunology. Springer, New York

    Google Scholar 

  28. Forrest S, Perelson A, Allen L, Cherukuri R (1994) Self-nonself discrimination in a computer. In: Proceedings of the 1994 IEEE symposium on research in security and privacy. IEEE Computer Society Press, Los Alamitos, pp 202–212

  29. Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32:675–701

    Google Scholar 

  30. Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11:86–92

    Google Scholar 

  31. Garcia S, Herrera F (2008) An extension on ’statistical comparisons of classifiers over multiple data sets’ for all pairwise comparisons. Mach Learn Res 9:2677–2694

    Google Scholar 

  32. Garcia S, Fernandez A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180:2044–2064

    Google Scholar 

  33. Garcia Lopez F, Garcia Torres M, Melian Batista B, Moreno Perez JA, Moreno Vega JM (2006) Solving feature subset selection problem by a parallel scatter search. Eur J Oper Res 169:477–489

  34. Gong M, Jiao L, Zhang L (2010) Baldwinian learning in clonal selection algorithm for optimization. Inf Sci 180:1218–1236

    Google Scholar 

  35. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182

    Google Scholar 

  36. Hansen P, Mladenovic N (2001) Variable neighborhood search: principles and applications. Eur J Oper Res 130:449–467

    MathSciNet  Google Scholar 

  37. Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65–70

    MathSciNet  Google Scholar 

  38. Hsu WH (2004) Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning. Inf Sci 163(1–3):103–122

    Google Scholar 

  39. Huang CL (2009) ACO-based hybrid classification system with feature subset selection and model parameters optimization. Neurocomputing 73:438–448

    Google Scholar 

  40. Huang J, Cai Y, Xu X (2007) A hybrid genetic algorithm for feature selection wrapper based on mutual information. Pattern Recognit Lett 28:1825–1844

    Google Scholar 

  41. Iman RL, Davenport JM (1980) Approximations of the critical region of the Friedman statistic. Commun Stat 9(6):571–595

    Google Scholar 

  42. Jain A, Zongker D (1997) Feature selection: evaluation, application, and small sample performance. IEEE Trans Pattern Anal Mach Intell 19:153–158

    Google Scholar 

  43. Kira K, Rendell L (1992) A practical approach to feature selection. In: Proceedings of the ninth international conference on machine learning, Aberdeen, Scotland, pp 249–256

  44. Kohavi R, John G (1997) Wrappers for feature subset selection. Artif Intell 97:273–324

    Google Scholar 

  45. Li F, Gao S, Wang W, Tang Z (2009) An adaptive clonal selection algorithm for edge linking problem. IJCSNS Int J Comput Sci Netw Secur 9(7):57–65

    Google Scholar 

  46. Lin SW, Chen SC (2009) PSOLDA: a particle swarm optimization approach for enhancing classification accurate rate of linear discriminant analysis. Appl Soft Comput 9:1008–1015

    Google Scholar 

  47. Lin SW, Lee ZJ, Chen SC, Tseng TY (2008) Parameter determination of support vector machine and feature selection using simulated annealing approach. Appl Soft Comput 8:1505–1512

    Google Scholar 

  48. Lin SW, Ying KC, Chen SC, Lee ZJ (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35:1817–1824

    Google Scholar 

  49. Lourenco HR, Martin O, Stützle T (2002) Iterated local search. Handbook of metaheuristics. Operations research and management science, vol 57. Kluwer Academic Publishers, Dordrecht, pp 321–353

  50. Ma J, Shi G, Gao L (2009) An improved immune clonal selection algorithm and its applications for VRP. In: Proceedings of the IEEE international conference on automation and logistics Shenyang, China, August 2009, pp 2097–2100

  51. Maldonado S, Weber R (2009) A wrapper method for feature selection using support vector machines. Inf Sci 179(13):2208–2217

    Google Scholar 

  52. Marinaki M, Marinakis Y (2014) An island memetic differential evolution algorithm for the feature selection problem. In: Terrazas G et al (ed) Nature inspired cooperative strategies for optimization—NICSO 2013. Studies in computational intelligence, vol 512. Springer International Publishing, Switzerland, pp 29–42

  53. Marinakis Y, Marinaki M (2013) A hybridized particle swarm optimization with expanding neighborhood topology for the feature selection problem. In: Blesa MJ et al (eds) HM 2013. LNCS, vol 7919. Springer, Berlin, pp 37–51

  54. Marinakis Y, Marinaki M, Doumpos M, Zopounidis C (2009) Ant colony and particle swarm optimization for financial classification problems. Expert Syst Appl 36(7):10604–10611

    Google Scholar 

  55. Marinakis Y, Marinaki M, Doumpos M, Matsatsinis N, Zopounidis C (2008) Optimization of nearest neighbor classifiers via metaheuristic algorithms for credit risk assessment. J Glob Optim 42:279–293

    MathSciNet  Google Scholar 

  56. Martin O, Otto SW, Felten EW (1991) Large-step markov chains for the traveling salesman problem. Complex Syst 5(3):299–326

    MathSciNet  Google Scholar 

  57. Meiri R, Zahavi J (2006) Using simulated annealing to optimize the feature selection problem in marketing applications. Eur J Oper Res 171:842–858

    Google Scholar 

  58. Narendra PM, Fukunaga K (1977) A branch and bound algorithm for feature subset selection. IEEE Trans Comput 26(9):917–922

    Google Scholar 

  59. Pacheco J, Casado S, Nunez L (2009) A variable selection method based on tabu search for logistic regression models. Eur J Oper Res 199:506–511

    MathSciNet  Google Scholar 

  60. Panigrahi BK, Yadav SR, Agrawal S, Tiwari MK (2007) A clonal algorithm to solve economic load dispatch. Electr Power Syst Res 77:1381–1389

    Google Scholar 

  61. Parpinelli RS, Lopes HS, Freitas AA (2002) An ant colony algorithm for classification rule discovery. In: Abbas H, Sarker R, Newton C (eds) Data mining: a heuristic approach. Idea group publishing, London, pp 191–208

  62. Pedrycz W, Park BJ, Pizzi NJ (2009) Identifying core sets of discriminatory features using particle swarm optimization. Expert Syst Appl 36:4610–4616

    Google Scholar 

  63. Pudil P, Novovicova J, Kittler J (1994) Floating search methods in feature selection. Pattern Recognit Lett 15:1119–1125

    Google Scholar 

  64. Rokach L (2008) Genetic algorithm-based feature set partitioning for classification problems. Pattern Recognit Lett 41:1676–1700

    Google Scholar 

  65. Shelokar PS, Jayaraman VK, Kulkarni BD (2004) An ant colony classifier system: application to some process engineering problems. Comput Chem Eng 28:1577–1584

    Google Scholar 

  66. Siedlecki W, Sklansky J (1988) On automatic feature selection. Int J Pattern Recognit Artif Intell 2(2):197–220

    Google Scholar 

  67. Siedlecki W, Sklansky J (1989) A note on genetic algorithms for large-scale feature selection. Pattern Recognit Lett 10:335–347

    Google Scholar 

  68. Srinivasa KG, Venugopal KR, Patnaik LM (2007) A self-adaptive migration model genetic algorithm for data mining applications. Inf Sci 177(20):4295–4313

  69. Talbi E-G (2009) Metaheuristics: from design to implementation. Wiley, New York

    Google Scholar 

  70. Timmis J, Neal M (2000) A resource limited artificial immune system for data analysis. Research and development in intelligent systems, vol 14. Springer, Cambridge, pp 19–32

    Google Scholar 

  71. Ulutas BH, Islier AA (2007) Parameter setting for clonal selection algorithm in facility layout problems. In: Gervasi O, Gavrilova M (eds) ICCSA 2007. LNCS, vol 4705, Part I. Springer, Berlin, pp 886–899

  72. Ulutas BH, Islier AA (2009) A clonal selection algorithm for dynamic facility layout problems. J Manuf Syst 28:123–131

    Google Scholar 

  73. Ulutas BH, Kulturel-Konak S (2012) An artificial immune system based algorithm to solve unequal area facility layout problem. Expert Syst Appl 39(5):5384–5395

    Google Scholar 

  74. Uncu O, Turksen IB (2007) A novel feature selection approach: combining feature wrappers and filters. Inf Sci 177(2):449–466

    MathSciNet  Google Scholar 

  75. Unler A, Murat A (2010) A discrete particle swarm optimization method for feature selection in binary classification problems. Eur J Oper Res 206:528–539

    Google Scholar 

  76. Wang X, Yang J, Teng X, Xia W, Jensen R (2007) Feature selection based on rough sets and particle swarm optimization. Pattern Recognit Lett 28:459–471

    Google Scholar 

  77. Wang Y, Feng XY, Huang YX, Pu DB, Zhou WG, Liang YC, Zhou CG (2007) A novel quantum swarm evolutionary algorithm and its applications. Neurocomputing 70(4–6):633–640

    Google Scholar 

  78. Yang J-H, Sun L, Lee HP, Qian Y, Liang Y-C (2008) Clonal selection based memetic algorithm for job shop scheduling problems. J Bionic Eng 5:111–119

    Google Scholar 

  79. Yu L, Liu H (2004) Efficient feature selection via analysis of relevance and redundancy. J Mach Learn Res 5:1205–1224

    Google Scholar 

  80. Zhang C, Hu H (2005) Ant colony optimization combining with mutual information for feature selection in support vector machines. In: Zhang S, Jarvis R (eds) AI 2005. LNAI, vol 3809, pp 918–921

  81. Zhu Y, Gao S, Dai H, Li F, Tang Z (2007) Improved clonal algorithm and its application to traveling salesman problem. IJCSNS Int J Comput Sci Netw Secur 7(8):109–113

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Production Engineering and Management, Technical University of Crete, 73100, Chania, Greece

    Magdalene Marinaki & Yannis Marinakis

Authors
  1. Magdalene Marinaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yannis Marinakis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yannis Marinakis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marinaki, M., Marinakis, Y. A hybridization of clonal selection algorithm with iterated local search and variable neighborhood search for the feature selection problem. Memetic Comp. 7, 181–201 (2015). https://doi.org/10.1007/s12293-015-0161-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12293-015-0161-2

Keywords

Search

Navigation