Skip to main content

Advertisement

SpringerLink
Log in

A novel binary gaining–sharing knowledge-based optimization algorithm for feature selection

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

To obtain the optimal set of features in feature selection problems is the most challenging and prominent problem in machine learning. Very few human-related metaheuristic algorithms were developed and solved this type of problem. It motivated us to check the performance of recently developed gaining–sharing knowledge-based optimization algorithm (GSK), which is based on the concept of gaining and sharing knowledge of humans throughout their lifespan. It depends on two stages: beginners–intermediate gaining and sharing stage and intermediate–experts gaining and sharing stage. In this study, two approaches are proposed to solve feature selection problems: FS-BGSK: a novel binary version of GSK algorithm that relies on these two stages with knowledge factor 1 and FS-pBGSK: a population reduction technique that is employed on BGSK algorithm to enhance the exploration and exploitation quality of FS-BGSK. The proposed approaches are checked on twenty two feature selection benchmark datasets from UCI repository that contains small, medium and large dimensions datasets. The obtained results are compared with seven state-of-the-art metaheuristic algorithms; binary differential evolution, binary particle swarm optimization algorithm, binary bat algorithm, binary grey wolf optimizer, binary ant lion optimizer, binary dragonfly algorithm and binary salp swarm algorithm. It concludes that FS-pBGSK and FS-BGSK outperform other algorithms in terms of accuracy, convergence and robustness in most of the datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Akinyelu AA, Ezugwu AE, Adewumi AO (2019) Ant colony optimization edge selection for support vector machine speed optimization. Neural Comput Appl 1–33

  2. Al-Madi N, Faris H, Mirjalili S (2019) Binary multi-verse optimization algorithm for global optimization and discrete problems. Int J Mach Learn Cybern 10(12):3445–3465

    Google Scholar 

  3. Allam M, Nandhini M (2018) Optimal feature selection using binary teaching learning based optimization algorithm. J King Saud Univ Comput Inform Sci. https://doi.org/10.1016/j.jksuci.2018.12.001

    Article  Google Scholar 

  4. Beheshti Z (2018) Bmnabc: Binary multi-neighborhood artificial bee colony for high-dimensional discrete optimization problems. Cybern Syst 49(7–8):452–474

    Google Scholar 

  5. Brest J, Maučec MS (2011) Self-adaptive differential evolution algorithm using population size reduction and three strategies. Soft Comput 15(11):2157–2174

    Google Scholar 

  6. Chandrashekar G, Sahin F (2014) A survey on feature selection methods. Comput Electr En 40(1):16–28

    Google Scholar 

  7. Chantar H, Mafarja M, Alsawalqah H, Heidari AA, Aljarah I, Faris H (2019) Feature selection using binary grey wolf optimizer with elite-based crossover for arabic text classification. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04368-6

    Article  Google Scholar 

  8. Cheng J, Zhang G, Neri F (2013) Enhancing distributed differential evolution with multicultural migration for global numerical optimization. Inf Sci 247:72–93

    MathSciNet  MATH  Google Scholar 

  9. Chuang LY, Chang HW, Tu CJ, Yang CH (2008) Improved binary pso for feature selection using gene expression data. Comput Biol Chem 32(1):29–38

    MATH  Google Scholar 

  10. Dassios I, Baleanu D (2018) Optimal solutions for singular linear systems of caputo fractional differential equations. Math Methods Appl Sci. https://doi.org/10.1002/mma.5410

    Article  Google Scholar 

  11. Dassios I, Fountoulakis K, Gondzio J (2015) A preconditioner for a primal-dual newton conjugate gradient method for compressed sensing problems. SIAM J Sci Comput 37(6):A2783–A2812

    MathSciNet  MATH  Google Scholar 

  12. Dassios IK (2019) Analytic loss minimization: theoretical framework of a second order optimization method. Symmetry 11(2):136

    MATH  Google Scholar 

  13. De Souza RCT, dos Santos Coelho L, De Macedo CA, Pierezan J (2018) A v-shaped binary crow search algorithm for feature selection. In: 2018 IEEE congress on evolutionary computation (CEC), pp 1–8. IEEE

  14. Diao R, Shen Q (2012) Feature selection with harmony search. IEEE Trans Syst Man Cybern Part B42(6):1509–1523

    Google Scholar 

  15. Ding S, Zhang N, Zhang X, Wu F (2017) Twin support vector machine: theory, algorithm and applications. Neural Comput Appl 28(11):3119–3130

    Google Scholar 

  16. Emary E, Zawbaa HM, Hassanien AE (2016) Binary ant lion approaches for feature selection. Neurocomputing 213:54–65

    Google Scholar 

  17. Emary E, Zawbaa HM, Hassanien AE (2016) Binary grey wolf optimization approaches for feature selection. Neurocomputing 172:371–381

    Google Scholar 

  18. Faris H, Hassonah MA, Ala’M AZ, Mirjalili S, Aljarah I (2018) A multi-verse optimizer approach for feature selection and optimizing svm parameters based on a robust system architecture. Neural Comput Appl 30(8):2355–2369

    Google Scholar 

  19. Faris H, Mafarja MM, Heidari AA, Aljarah I, Ala’M AZ, Mirjalili S, Fujita H (2018) An efficient binary salp swarm algorithm with crossover scheme for feature selection problems. Knowl-Based Syst 154:43–67

    Google Scholar 

  20. Frank A, Asuncion A et al (2011) UCI machine learning repository, 2010. http://archive.ics.uci.edu/ml 15, 22

  21. Gao WF, Yen GG, Liu SY (2014) A dual-population differential evolution with coevolution for constrained optimization. IEEE Trans Cybern 45(5):1108–1121

    Google Scholar 

  22. García S, Molina D, Lozano M, Herrera F (2009) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the cec’2005 special session on real parameter optimization. J Heurist 15(6):617

    MATH  Google Scholar 

  23. Gavrilis D, Tsoulos IG, Dermatas E (2008) Selecting and constructing features using grammatical evolution. Pattern Recogn Lett 29(9):1358–1365

    Google Scholar 

  24. Hafez AI, Hassanien AE, Zawbaa HM, Emary E (2015) Hybrid monkey algorithm with krill herd algorithm optimization for feature selection. In: 2015 11th international computer engineering conference (ICENCO), pp 273–277. IEEE

  25. Hafez AI, Zawbaa HM, Emary E, Hassanien AE (2016) Sine cosine optimization algorithm for feature selection. In: 2016 international symposium on innovations in intelligent systems and applications (INISTA), pp. 1–5. IEEE

  26. Hafez AI, Zawbaa HM, Emary E, Mahmoud HA, Hassanien AE (2015) An innovative approach for feature selection based on chicken swarm optimization. In: 2015 7th international conference of soft computing and pattern recognition (SoCPaR), pp 19–24. IEEE

  27. He X, Zhang Q, Sun N, Dong Y (2009) Feature selection with discrete binary differential evolution. In: 2009 international conference on artificial intelligence and computational intelligence, vol 4, pp 327–330. IEEE

  28. Hichem H, Elkamel M, Rafik M, Mesaaoud MT, Ouahiba C (2019) A new binary grasshopper optimization algorithm for feature selection problem. J King Saud Univ Compute Inform Sci

  29. Hu B, Dai Y, Su Y, Moore P, Zhang X, Mao C, Chen J, Xu L (2016) Feature selection for optimized high-dimensional biomedical data using an improved shuffled frog leaping algorithm. IEEE/ACM Trans Comput Biol Bioinform 15(6):1765–1773

    Google Scholar 

  30. Ibrahim HT, Mazher WJ, Ucan ON, Bayat O (2019) A grasshopper optimizer approach for feature selection and optimizing SVM parameters utilizing real biomedical data sets. Neural Comput Appl 31(10):5965–5974

    Google Scholar 

  31. Jabbar A, Zainudin S (2014) Water cycle algorithm for attribute reduction problems in rough set theory. J Theor Appl Inform Technol 61(1):107–117

    Google Scholar 

  32. Leardi R (1994) Application of a genetic algorithm to feature selection under full validation conditions and to outlier detection. J Chemom 8(1):65–79

    Google Scholar 

  33. Lin KC, Zhang KY, Huang YH, Hung JC, Yen N (2016) Feature selection based on an improved cat swarm optimization algorithm for big data classification. J Supercomput 72(8):3210–3221

    Google Scholar 

  34. Liu H, Motoda H (1998) Feature extraction, construction and selection: a data mining perspective, vol 453. Springer, Berlin

    MATH  Google Scholar 

  35. Mafarja MM, Eleyan D, Jaber I, Hammouri A, Mirjalili S (2017) Binary dragonfly algorithm for feature selection. In: 2017 international conference on new trends in computing sciences (ICTCS), pp 12–17. IEEE

  36. Mafarja MM, Mirjalili S (2017) Hybrid whale optimization algorithm with simulated annealing for feature selection. Neurocomputing 260:302–312

    Google Scholar 

  37. Mohamed AK, Mohamed AW, Elfeky EZ, Saleh M (2018) Enhancing AGDE algorithm using population size reduction for global numerical optimization. In: International conference on advanced machine learning technologies and applications, pp 62–72. Springer

  38. Mohamed AW (2016) A new modified binary differential evolution algorithm and its applications. Appl Math Inform Sci 10(5):1965–1969

    Google Scholar 

  39. Mohamed AW, Hadi AA, Mohamed AK (2020) Gaining-sharing knowledge based algorithm for solving optimization problems: a novel nature-inspired algorithm. Int J Mach Learn Cybern 11:1501–1529

    Google Scholar 

  40. Mohamed AW, Sabry HZ (2012) Constrained optimization based on modified differential evolution algorithm. Inf Sci 194:171–208

    Google Scholar 

  41. Muni DP, Pal NR, Das J (2006) Genetic programming for simultaneous feature selection and classifier design. IEEE Trans Syst Man Cybern Part B 36(1):106–117

    Google Scholar 

  42. Nakamura RY, Pereira LA, Costa KA, Rodrigues D, Papa JP, Yang XS (2012) BBA: a binary bat algorithm for feature selection. In: 2012 25th SIBGRAPI conference on graphics, patterns and images, pp 291–297. IEEE

  43. Panwar LK, Reddy S, Verma A, Panigrahi BK, Kumar R (2018) Binary grey wolf optimizer for large scale unit commitment problem. Swarm Evolut Comput 38:251–266

    Google Scholar 

  44. Pashaei E, Aydin N (2017) Binary black hole algorithm for feature selection and classification on biological data. Appl Soft Comput 56:94–106

    Google Scholar 

  45. Rad SM, Tab FA, Mollazade K (2012) Application of imperialist competitive algorithm for feature selection: a case study on bulk rice classification. Int J Comput Appl 40(16):41–48

    Google Scholar 

  46. Rashedi E, Nezamabadi-pour H (2014) Feature subset selection using improved binary gravitational search algorithm. J Intell Fuzzy Syst 26(3):1211–1221

    Google Scholar 

  47. Rodrigues D, Pereira LA, Almeida T, Papa JP, Souza A, Ramos CC, Yang XS (2013) BCS: a binary cuckoo search algorithm for feature selection. In: 2013 IEEE international symposium on circuits and systems (ISCAS2013), pp 465–468. IEEE

  48. Sasirekha K, Thangavel K (2019) Optimization of k-nearest neighbor using particle swarm optimization for face recognition. Neural Comput Appl 31(11):7935–7944

    Google Scholar 

  49. Schiezaro M, Pedrini H (2013) Data feature selection based on artificial bee colony algorithm. EURASIP J Image Video Process 2013(1):47

    Google Scholar 

  50. Shen L, Chen H, Yu Z, Kang W, Zhang B, Li H, Yang B, Liu D (2016) Evolving support vector machines using fruit fly optimization for medical data classification. Knowl-Based Syst 96:61–75

    Google Scholar 

  51. Singh T (2020) A chaotic sequence-guided Harris Hawks optimizer for data clustering. Neural Comput Appl. https://doi.org/10.1007/s00521-020-04951-2

    Article  Google Scholar 

  52. Tawhid MA, Dsouza KB (2018) Hybrid binary bat enhanced particle swarm optimization algorithm for solving feature selection problems. Appl Comput Inform. https://doi.org/10.1016/j.aci.2018.04.001

    Article  Google Scholar 

  53. Wan Y, Wang M, Ye Z, Lai X (2016) A feature selection method based on modified binary coded ant colony optimization algorithm. Appl Soft Comput 49:248–258

    Google Scholar 

  54. Xue B, Zhang M, Browne WN (2012) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Cybern 43(6):1656–1671

    Google Scholar 

  55. Yan C, Ma J, Luo H, Patel A (2019) Hybrid binary coral reefs optimization algorithm with simulated annealing for feature selection in high-dimensional biomedical datasets. Chemometr Intell Lab Syst 184:102–111

    Google Scholar 

  56. Yan Z, Yuan C (2004) Ant colony optimization for feature selection in face recognition. In: International conference on biometric authentication, pp 221–226. Springer, Berlin

  57. Zawbaa HM, Emary E, Parv B, Sharawi M (2016) Feature selection approach based on moth-flame optimization algorithm. In: 2016 IEEE congress on evolutionary computation (CEC), pp 4612–4617. IEEE

  58. Zhang C, Zhou J, Li C, Fu W, Peng T (2017) A compound structure of elm based on feature selection and parameter optimization using hybrid backtracking search algorithm for wind speed forecasting. Energy Convers Manag 143:360–376

    Google Scholar 

  59. Zhang H, Sun G (2002) Feature selection using tabu search method. Pattern Recogn 35(3):701–711

    MATH  Google Scholar 

  60. Zhang M, Shao C, Li F, Gan Y, Sun J (2006) Evolving neural network classifiers and feature subset using artificial fish swarm. In: 2006 international conference on mechatronics and automation, pp 1598–1602. IEEE

  61. Zhang N, Ding S (2017) Unsupervised and semi-supervised extreme learning machine with wavelet kernel for high dimensional data. Memet Comput 9(2):129–139

    Google Scholar 

  62. Zhang N, Ding S, Sun T, Liao H, Wang L, Shi Z (2020) Multi-view rbm with posterior consistency and domain adaptation. Inf Sci 516:142–157

    Google Scholar 

  63. Zhang N, Ding S, Zhang J, Xue Y (2018) An overview on restricted boltzmann machines. Neurocomputing 275:1186–1199

    Google Scholar 

  64. Zhang WQ, Zhang Y, Peng C (2019) Brain storm optimization for feature selection using new individual clustering and updating mechanism. Appl Intell 49(12):4294–4302

    Google Scholar 

  65. Zhang Y, Song XF, Gong DW (2017) A return-cost-based binary firefly algorithm for feature selection. Inform Sci 418:561–574

    Google Scholar 

  66. Zhang Z, Ding S, Jia W (2019) A hybrid optimization algorithm based on cuckoo search and differential evolution for solving constrained engineering problems. Eng Appl Artif Intell 85:254–268

    Google Scholar 

  67. Zhang Z, Ding S, Sun Y (2020) A support vector regression model hybridized with chaotic krill herd algorithm and empirical mode decomposition for regression task. Neurocomputing. https://doi.org/10.1016/j.neucom.2020.05.075

    Article  Google Scholar 

  68. Zhu Z, Ong YS, Dash M (2007) Wrapper-filter feature selection algorithm using a memetic framework. IEEE Trans Syst Man Cybern Part B Cybern 37(1):70–76

    Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the editors and anonymous reviewers for providing their valuable comments and suggestions. The authors would also like to thank the Ministry of Human Resource Development (MHRD Govt. of India) for providing the financial assistantship to the first author.

Author information

Authors and Affiliations

  1. Department of Mathematics & Scientific Computing, National Institute of Technology Hamirpur, Himachal Pradesh, 177005, India

    Prachi Agrawal & Talari Ganesh

  2. Operations Research Department, Faculty of Graduate Studies for Statistical Research, Cairo University, Giza, 12613, Egypt

    Ali Wagdy Mohamed

  3. Wireless Intelligent Networks Center (WINC), School of Engineering and Applied Sciences, Nile University, Giza, Egypt

    Ali Wagdy Mohamed

Authors
  1. Prachi Agrawal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Talari Ganesh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ali Wagdy Mohamed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ali Wagdy Mohamed.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Agrawal, P., Ganesh, T. & Mohamed, A.W. A novel binary gaining–sharing knowledge-based optimization algorithm for feature selection. Neural Comput & Applic 33, 5989–6008 (2021). https://doi.org/10.1007/s00521-020-05375-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05375-8

Keywords

Search

Navigation