Abstract
To obtain the optimal set of features in feature selection problems is the most challenging and prominent problem in machine learning. Very few human-related metaheuristic algorithms were developed and solved this type of problem. It motivated us to check the performance of recently developed gaining–sharing knowledge-based optimization algorithm (GSK), which is based on the concept of gaining and sharing knowledge of humans throughout their lifespan. It depends on two stages: beginners–intermediate gaining and sharing stage and intermediate–experts gaining and sharing stage. In this study, two approaches are proposed to solve feature selection problems: FS-BGSK: a novel binary version of GSK algorithm that relies on these two stages with knowledge factor 1 and FS-pBGSK: a population reduction technique that is employed on BGSK algorithm to enhance the exploration and exploitation quality of FS-BGSK. The proposed approaches are checked on twenty two feature selection benchmark datasets from UCI repository that contains small, medium and large dimensions datasets. The obtained results are compared with seven state-of-the-art metaheuristic algorithms; binary differential evolution, binary particle swarm optimization algorithm, binary bat algorithm, binary grey wolf optimizer, binary ant lion optimizer, binary dragonfly algorithm and binary salp swarm algorithm. It concludes that FS-pBGSK and FS-BGSK outperform other algorithms in terms of accuracy, convergence and robustness in most of the datasets.
Similar content being viewed by others
References
Akinyelu AA, Ezugwu AE, Adewumi AO (2019) Ant colony optimization edge selection for support vector machine speed optimization. Neural Comput Appl 1–33
Al-Madi N, Faris H, Mirjalili S (2019) Binary multi-verse optimization algorithm for global optimization and discrete problems. Int J Mach Learn Cybern 10(12):3445–3465
Allam M, Nandhini M (2018) Optimal feature selection using binary teaching learning based optimization algorithm. J King Saud Univ Comput Inform Sci. https://doi.org/10.1016/j.jksuci.2018.12.001
Beheshti Z (2018) Bmnabc: Binary multi-neighborhood artificial bee colony for high-dimensional discrete optimization problems. Cybern Syst 49(7–8):452–474
Brest J, Maučec MS (2011) Self-adaptive differential evolution algorithm using population size reduction and three strategies. Soft Comput 15(11):2157–2174
Chandrashekar G, Sahin F (2014) A survey on feature selection methods. Comput Electr En 40(1):16–28
Chantar H, Mafarja M, Alsawalqah H, Heidari AA, Aljarah I, Faris H (2019) Feature selection using binary grey wolf optimizer with elite-based crossover for arabic text classification. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04368-6
Cheng J, Zhang G, Neri F (2013) Enhancing distributed differential evolution with multicultural migration for global numerical optimization. Inf Sci 247:72–93
Chuang LY, Chang HW, Tu CJ, Yang CH (2008) Improved binary pso for feature selection using gene expression data. Comput Biol Chem 32(1):29–38
Dassios I, Baleanu D (2018) Optimal solutions for singular linear systems of caputo fractional differential equations. Math Methods Appl Sci. https://doi.org/10.1002/mma.5410
Dassios I, Fountoulakis K, Gondzio J (2015) A preconditioner for a primal-dual newton conjugate gradient method for compressed sensing problems. SIAM J Sci Comput 37(6):A2783–A2812
Dassios IK (2019) Analytic loss minimization: theoretical framework of a second order optimization method. Symmetry 11(2):136
De Souza RCT, dos Santos Coelho L, De Macedo CA, Pierezan J (2018) A v-shaped binary crow search algorithm for feature selection. In: 2018 IEEE congress on evolutionary computation (CEC), pp 1–8. IEEE
Diao R, Shen Q (2012) Feature selection with harmony search. IEEE Trans Syst Man Cybern Part B42(6):1509–1523
Ding S, Zhang N, Zhang X, Wu F (2017) Twin support vector machine: theory, algorithm and applications. Neural Comput Appl 28(11):3119–3130
Emary E, Zawbaa HM, Hassanien AE (2016) Binary ant lion approaches for feature selection. Neurocomputing 213:54–65
Emary E, Zawbaa HM, Hassanien AE (2016) Binary grey wolf optimization approaches for feature selection. Neurocomputing 172:371–381
Faris H, Hassonah MA, Ala’M AZ, Mirjalili S, Aljarah I (2018) A multi-verse optimizer approach for feature selection and optimizing svm parameters based on a robust system architecture. Neural Comput Appl 30(8):2355–2369
Faris H, Mafarja MM, Heidari AA, Aljarah I, Ala’M AZ, Mirjalili S, Fujita H (2018) An efficient binary salp swarm algorithm with crossover scheme for feature selection problems. Knowl-Based Syst 154:43–67
Frank A, Asuncion A et al (2011) UCI machine learning repository, 2010. http://archive.ics.uci.edu/ml 15, 22
Gao WF, Yen GG, Liu SY (2014) A dual-population differential evolution with coevolution for constrained optimization. IEEE Trans Cybern 45(5):1108–1121
García S, Molina D, Lozano M, Herrera F (2009) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the cec’2005 special session on real parameter optimization. J Heurist 15(6):617
Gavrilis D, Tsoulos IG, Dermatas E (2008) Selecting and constructing features using grammatical evolution. Pattern Recogn Lett 29(9):1358–1365
Hafez AI, Hassanien AE, Zawbaa HM, Emary E (2015) Hybrid monkey algorithm with krill herd algorithm optimization for feature selection. In: 2015 11th international computer engineering conference (ICENCO), pp 273–277. IEEE
Hafez AI, Zawbaa HM, Emary E, Hassanien AE (2016) Sine cosine optimization algorithm for feature selection. In: 2016 international symposium on innovations in intelligent systems and applications (INISTA), pp. 1–5. IEEE
Hafez AI, Zawbaa HM, Emary E, Mahmoud HA, Hassanien AE (2015) An innovative approach for feature selection based on chicken swarm optimization. In: 2015 7th international conference of soft computing and pattern recognition (SoCPaR), pp 19–24. IEEE
He X, Zhang Q, Sun N, Dong Y (2009) Feature selection with discrete binary differential evolution. In: 2009 international conference on artificial intelligence and computational intelligence, vol 4, pp 327–330. IEEE
Hichem H, Elkamel M, Rafik M, Mesaaoud MT, Ouahiba C (2019) A new binary grasshopper optimization algorithm for feature selection problem. J King Saud Univ Compute Inform Sci
Hu B, Dai Y, Su Y, Moore P, Zhang X, Mao C, Chen J, Xu L (2016) Feature selection for optimized high-dimensional biomedical data using an improved shuffled frog leaping algorithm. IEEE/ACM Trans Comput Biol Bioinform 15(6):1765–1773
Ibrahim HT, Mazher WJ, Ucan ON, Bayat O (2019) A grasshopper optimizer approach for feature selection and optimizing SVM parameters utilizing real biomedical data sets. Neural Comput Appl 31(10):5965–5974
Jabbar A, Zainudin S (2014) Water cycle algorithm for attribute reduction problems in rough set theory. J Theor Appl Inform Technol 61(1):107–117
Leardi R (1994) Application of a genetic algorithm to feature selection under full validation conditions and to outlier detection. J Chemom 8(1):65–79
Lin KC, Zhang KY, Huang YH, Hung JC, Yen N (2016) Feature selection based on an improved cat swarm optimization algorithm for big data classification. J Supercomput 72(8):3210–3221
Liu H, Motoda H (1998) Feature extraction, construction and selection: a data mining perspective, vol 453. Springer, Berlin
Mafarja MM, Eleyan D, Jaber I, Hammouri A, Mirjalili S (2017) Binary dragonfly algorithm for feature selection. In: 2017 international conference on new trends in computing sciences (ICTCS), pp 12–17. IEEE
Mafarja MM, Mirjalili S (2017) Hybrid whale optimization algorithm with simulated annealing for feature selection. Neurocomputing 260:302–312
Mohamed AK, Mohamed AW, Elfeky EZ, Saleh M (2018) Enhancing AGDE algorithm using population size reduction for global numerical optimization. In: International conference on advanced machine learning technologies and applications, pp 62–72. Springer
Mohamed AW (2016) A new modified binary differential evolution algorithm and its applications. Appl Math Inform Sci 10(5):1965–1969
Mohamed AW, Hadi AA, Mohamed AK (2020) Gaining-sharing knowledge based algorithm for solving optimization problems: a novel nature-inspired algorithm. Int J Mach Learn Cybern 11:1501–1529
Mohamed AW, Sabry HZ (2012) Constrained optimization based on modified differential evolution algorithm. Inf Sci 194:171–208
Muni DP, Pal NR, Das J (2006) Genetic programming for simultaneous feature selection and classifier design. IEEE Trans Syst Man Cybern Part B 36(1):106–117
Nakamura RY, Pereira LA, Costa KA, Rodrigues D, Papa JP, Yang XS (2012) BBA: a binary bat algorithm for feature selection. In: 2012 25th SIBGRAPI conference on graphics, patterns and images, pp 291–297. IEEE
Panwar LK, Reddy S, Verma A, Panigrahi BK, Kumar R (2018) Binary grey wolf optimizer for large scale unit commitment problem. Swarm Evolut Comput 38:251–266
Pashaei E, Aydin N (2017) Binary black hole algorithm for feature selection and classification on biological data. Appl Soft Comput 56:94–106
Rad SM, Tab FA, Mollazade K (2012) Application of imperialist competitive algorithm for feature selection: a case study on bulk rice classification. Int J Comput Appl 40(16):41–48
Rashedi E, Nezamabadi-pour H (2014) Feature subset selection using improved binary gravitational search algorithm. J Intell Fuzzy Syst 26(3):1211–1221
Rodrigues D, Pereira LA, Almeida T, Papa JP, Souza A, Ramos CC, Yang XS (2013) BCS: a binary cuckoo search algorithm for feature selection. In: 2013 IEEE international symposium on circuits and systems (ISCAS2013), pp 465–468. IEEE
Sasirekha K, Thangavel K (2019) Optimization of k-nearest neighbor using particle swarm optimization for face recognition. Neural Comput Appl 31(11):7935–7944
Schiezaro M, Pedrini H (2013) Data feature selection based on artificial bee colony algorithm. EURASIP J Image Video Process 2013(1):47
Shen L, Chen H, Yu Z, Kang W, Zhang B, Li H, Yang B, Liu D (2016) Evolving support vector machines using fruit fly optimization for medical data classification. Knowl-Based Syst 96:61–75
Singh T (2020) A chaotic sequence-guided Harris Hawks optimizer for data clustering. Neural Comput Appl. https://doi.org/10.1007/s00521-020-04951-2
Tawhid MA, Dsouza KB (2018) Hybrid binary bat enhanced particle swarm optimization algorithm for solving feature selection problems. Appl Comput Inform. https://doi.org/10.1016/j.aci.2018.04.001
Wan Y, Wang M, Ye Z, Lai X (2016) A feature selection method based on modified binary coded ant colony optimization algorithm. Appl Soft Comput 49:248–258
Xue B, Zhang M, Browne WN (2012) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Cybern 43(6):1656–1671
Yan C, Ma J, Luo H, Patel A (2019) Hybrid binary coral reefs optimization algorithm with simulated annealing for feature selection in high-dimensional biomedical datasets. Chemometr Intell Lab Syst 184:102–111
Yan Z, Yuan C (2004) Ant colony optimization for feature selection in face recognition. In: International conference on biometric authentication, pp 221–226. Springer, Berlin
Zawbaa HM, Emary E, Parv B, Sharawi M (2016) Feature selection approach based on moth-flame optimization algorithm. In: 2016 IEEE congress on evolutionary computation (CEC), pp 4612–4617. IEEE
Zhang C, Zhou J, Li C, Fu W, Peng T (2017) A compound structure of elm based on feature selection and parameter optimization using hybrid backtracking search algorithm for wind speed forecasting. Energy Convers Manag 143:360–376
Zhang H, Sun G (2002) Feature selection using tabu search method. Pattern Recogn 35(3):701–711
Zhang M, Shao C, Li F, Gan Y, Sun J (2006) Evolving neural network classifiers and feature subset using artificial fish swarm. In: 2006 international conference on mechatronics and automation, pp 1598–1602. IEEE
Zhang N, Ding S (2017) Unsupervised and semi-supervised extreme learning machine with wavelet kernel for high dimensional data. Memet Comput 9(2):129–139
Zhang N, Ding S, Sun T, Liao H, Wang L, Shi Z (2020) Multi-view rbm with posterior consistency and domain adaptation. Inf Sci 516:142–157
Zhang N, Ding S, Zhang J, Xue Y (2018) An overview on restricted boltzmann machines. Neurocomputing 275:1186–1199
Zhang WQ, Zhang Y, Peng C (2019) Brain storm optimization for feature selection using new individual clustering and updating mechanism. Appl Intell 49(12):4294–4302
Zhang Y, Song XF, Gong DW (2017) A return-cost-based binary firefly algorithm for feature selection. Inform Sci 418:561–574
Zhang Z, Ding S, Jia W (2019) A hybrid optimization algorithm based on cuckoo search and differential evolution for solving constrained engineering problems. Eng Appl Artif Intell 85:254–268
Zhang Z, Ding S, Sun Y (2020) A support vector regression model hybridized with chaotic krill herd algorithm and empirical mode decomposition for regression task. Neurocomputing. https://doi.org/10.1016/j.neucom.2020.05.075
Zhu Z, Ong YS, Dash M (2007) Wrapper-filter feature selection algorithm using a memetic framework. IEEE Trans Syst Man Cybern Part B Cybern 37(1):70–76
Acknowledgements
The authors would like to acknowledge the editors and anonymous reviewers for providing their valuable comments and suggestions. The authors would also like to thank the Ministry of Human Resource Development (MHRD Govt. of India) for providing the financial assistantship to the first author.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Agrawal, P., Ganesh, T. & Mohamed, A.W. A novel binary gaining–sharing knowledge-based optimization algorithm for feature selection. Neural Comput & Applic 33, 5989–6008 (2021). https://doi.org/10.1007/s00521-020-05375-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-020-05375-8