Skip to main content

Advertisement

SpringerLink
Log in

A novel adaptive memetic binary optimization algorithm for feature selection

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Feature selection (FS) determines the beneficial features in data and decreases the disadvantages of the curse of dimensionality. This work proposes a novel adaptive memetic binary optimization (AMBO) algoraaithm for FS. FS is an NP-Hard binary optimization problem. AMBO is a pure binary optimization algorithm that works in binary discrete search space. New candidate individuals are adaptively created by a single point, double point, uniform crossovers, and canonical mutation mechanism. Local improvement for the best and worst individuals is provided with a new binary logic-gate based memetic smart local search mechanism. The balance between exploration and exploitation is achieved by adaptively. A diverse dimension dataset experimental setup is provided for determining the success of the proposed method. AMBO firstly was compared with binary particle swarm optimization (BPSO), a genetic algorithm with a random wheel selection strategy (GARW), a genetic algorithm with a tournaments selection strategy (GATS), and a genetic algorithm with a random selection strategy (GARS). AMBO outperformed the opponents on 11 datasets, especially the largest one. Wilcoxon signed-rank test and Friedman’s test were conducted to show the statistical significance of AMBO. For an additional experiment with state-of-art metaheuristic algorithms in the literature, Population reduction binary gaining sharing knowledge-based algorithm with V-4 shaped transfer function (PbGSK-V4), binary salp swarm algorithm (BSSA), binary differential evolution algorithm (BDE), binary dragonfly algorithm (BDA), binary particle swarm optimization algorithm (BPSO), binary bat algorithm (BBA), binary ant lion optimization (BALO) and binary grey wolf optimizer (BGWO) are used in experiments with 21 datasets. The experimental results of the proposed AMBO algorithm are significantly better than the state-of-art algorithms, in terms of classification error rate, fitness function, and average selected features.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

Similar content being viewed by others

References

  • Abu Zaher A, Berretta R, Noman N, Moscato P (2019) An adaptive memetic algorithm for feature selection using proximity graphs. Computational Intell 35:156–183

    Article  MathSciNet  Google Scholar 

  • Agrawal P, Ganesh T, Oliva D, Mohamed AW (2022) S-shaped and v-shaped gaining-sharing knowledge-based algorithm for feature selection. Appl Intell. https://doi.org/10.1007/s10489-021-02233-5

    Article  Google Scholar 

  • Al-Tashi Q, Kadir SJA, Rais HM, Mirjalili S, Alhussian H (2019) Binary optimization using hybrid grey wolf optimization for feature selection. IEEE Access 7:39496–39508

    Article  Google Scholar 

  • Al-Tashi Q, Abdulkadir SJ, Rais HM, Mirjalili S, Alhussian H (2020) Approaches to multi-objective feature selection: A systematic literature review. IEEE Access 8:125076–125096

    Article  Google Scholar 

  • Alweshah M, Al Khalaileh S, Gupta BB, Almomani A, Hammouri AI, Al-Betar MA (2020) The monarch butterfly optimization algorithm for solving feature selection problems. Neural Comput Applic. https://doi.org/10.1007/s00521-020-05210-0

    Article  Google Scholar 

  • A.A. Altun, N. Allahverdi, 2007 Neural network based recognition by using genetic algorithm for feature selection of enhanced fingerprints, in: International Conference on Adaptive and Natural Computing Algorithms, Springer, pp. 467–476.

  • Albrecht AA (2006) Stochastic local search for the feature set problem, with applications to microarray data. Appl Math Comput 183:1148–1164

    MathSciNet  MATH  Google Scholar 

  • Altun AA, Kocer HE, Allahverdi N (2008) Genetic algorithm based feature selection level fusion using fingerprint and iris biometrics. Int J Pattern Recognit Artif Intell 22:585–600

    Article  Google Scholar 

  • Arora S, Anand P (2019) Binary butterfly optimization approaches for feature selection. Expert Syst Appl 116:147–160

    Article  Google Scholar 

  • A. Asuncion, D. Newman, UCI machine learning repository, in, Irvine, CA, USA, 2007.

  • Awadallah MA, Hammouri AI, Al-Betar MA, Braik MS, Abd Elaziz M (2022) Binary Horse herd optimization algorithm with crossover operators for feature selection. Computers in Biol Med 141:105152

    Article  Google Scholar 

  • Awadallah MA, Al-Betar MA, Hammouri AI, Alomari OA (2020) Binary JAYA algorithm with adaptive mutation for feature selection. Arab J Sci Eng. https://doi.org/10.1007/s13369-020-04871-2

    Article  Google Scholar 

  • Awadallah MA, Al-Betar MA, Braik MS, Hammouri AI, Doush IA, Zitar RA (2022a) An enhanced binary Rat Swarm Optimizer based on local-best concepts of PSO and collaborative crossover operators for feature selection. Comput Biol Med 105675.

  • A. Babalık, Ö.K. Baykan, H. İşcan, İ. Babaoğlu, O. Fındık, 2010 Effects of Feature Selection Using Binary Particle Swarm Optimization on Wheat Variety Classification, in: International Conference on Advances in Information Technology, Springer, pp. 11–17

  • Babaoglu İ, Findik O, Ülker E (2010) A comparison of feature selection models utilizing binary particle swarm optimization and genetic algorithm in determining coronary artery disease using support vector machine. Expert Syst Appl 37:3177–3183

    Article  Google Scholar 

  • F. Chen, Z. Ye, C. Wang, L. Yan, R. Wang, A feature selection approach for network intrusion detection based on tree-seed algorithm and K-nearest neighbor, in: 2018 IEEE 4th International Symposium on Wireless Systems within the International Conferences on Intelligent Data Acquisition and Advanced Computing Systems (IDAACS-SWS), IEEE, 2018, pp. 68-72

  • Chuang L-Y, Chang H-W, Tu C-J, Yang C-H (2008) Improved binary PSO for feature selection using gene expression data. Comput Biol Chem 32:29–38

    Article  MATH  Google Scholar 

  • Emine B, Ülker E (2020) An efficient binary social spider algorithm for feature selection problem. Expert Syst Appl 146:113185

    Article  Google Scholar 

  • Emary E, Zawbaa HM, Hassanien AE (2016a) Binary ant lion approaches for feature selection. Neurocomputing 213:54–65

    Article  Google Scholar 

  • Emary E, Zawbaa HM, Hassanien AE (2016b) Binary grey wolf optimization approaches for feature selection. Neurocomputing 172:371–381

    Article  Google Scholar 

  • Ertuğrul ÖF, Tağluk ME (2017) A fast feature selection approach based on extreme learning machine and coefficient of variation. Turk J Electr Eng Comput Sci 25:3409–3420

    Article  Google Scholar 

  • Faris H, Mafarja MM, Heidari AA, Aljarah I, Ala’MMirjaliliFujita AZSH (2018) An efficient binary salp swarm algorithm with crossover scheme for feature selection problems. Knowledge-Based Syst 154:43–67

    Article  Google Scholar 

  • Gao Y, Zhou Y, Luo Q (2020) An efficient binary equilibrium optimizer algorithm for feature selection. IEEE Access 8:140936–140963

    Article  Google Scholar 

  • Gabardo AC, Berretta R, Moscato P (2020) M-Link: a link clustering memetic algorithm for overlapping community detection. Memetic Comput 12:1–13

    Article  Google Scholar 

  • Ghosh M, Guha R, Sarkar R, Abraham A (2019) A wrapper-filter feature selection technique based on ant colony optimization. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04171-3

    Article  Google Scholar 

  • M.N. Haque, L. Mathieson, P. Moscato, A memetic algorithm approach to network alignment: mapping the classification of mental disorders of DSM-IV with ICD-10, in: Proceedings of the Genetic and Evolutionary Computation Conference, 2019, pp. 258–265.

  • X. He, Q. Zhang, N. Sun, Y. Dong, 2009 Feature selection with discrete binary differential evolution, in: 2009 international conference on artificial intelligence and computational intelligence, IEEE, pp. 327-330.

  • J.H. Holland, Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence, MIT press, 1992.

  • Jia H, Li J, Song W, Peng X, Lang C, Li Y (2019) Spotted hyena optimization algorithm with simulated annealing for feature selection. IEEE Access 7:71943–71962

    Article  Google Scholar 

  • Kundu R, Mallipeddi R (2022) HFMOEA: a hybrid framework for multi-objective feature selection. J Comput Design and Eng 9:949–965

    Article  Google Scholar 

  • Lee J, Kim D-W (2015) Memetic feature selection algorithm for multi-label classification. Inf Sci 293:80–96

    Article  Google Scholar 

  • Liu H, Zhou M, Liu Q (2019) An embedded feature selection method for imbalanced data classification. IEEE/CAA J Automatica Sinica 6:703–715

    Article  Google Scholar 

  • Maldonado S, López J (2018) Dealing with high-dimensional class-imbalanced datasets: embedded feature selection for SVM classification. Appl Soft Comput 67:94–105

    Article  Google Scholar 

  • M.M. Mafarja, D. Eleyan, I. Jaber, A. Hammouri, S. Mirjalili, 2017 Binary dragonfly algorithm for feature selection, in: 2017 International conference on new trends in computing sciences (ICTCS), IEEE, pp. 12-17

  • Mafarja M, Mirjalili S (2018) Whale optimization approaches for wrapper feature selection. Appl Soft Comput 62:441–453

    Article  Google Scholar 

  • Mafarja M, Aljarah I, Heidari AA, Hammouri AI, Faris H, Ala’M AZ, Mirjalili S (2018) Evolutionary population dynamics and grasshopper optimization approaches for feature selection problems. Knowledge-Based Syst 145:25–45

    Article  Google Scholar 

  • Mafarja M, Aljarah I, Faris H, Hammouri AI, Ala’M AZ, Mirjalili S (2019) Binary grasshopper optimisation algorithm approaches for feature selection problems. Expert Syst with Appl 117:267–286

    Article  Google Scholar 

  • S. Mirjalili, S. Mirjalili, Genetic algorithm, Evolutionary Algorithms and Neural Networks: Theory and Applications, (2019) 43–55.

  • Mlakar U, Fister I, Brest J, Potočnik B (2017) Multi-objective differential evolution for feature selection in facial expression recognition systems. Expert Syst Appl 89:129–137

    Article  Google Scholar 

  • P. Moscato, C. Cotta, A gentle introduction to memetic algorithms, in: Handbook of metaheuristics, Springer, 2003, pp. 105–144.

  • P. Moscato, C. Cotta, A. Mendes, Memetic algorithms, in: New optimization techniques in engineering, Springer, 2004, pp. 53–85.

  • P. Moscato, H. Sun, M.N. Haque, (2019) Analytic Continued Fractions for Regression: A Memetic Algorithm Approach, arXiv preprint arXiv:2001.00624

  • P. Moscato, C. Cotta, A modern introduction to memetic algorithms, in: Handbook of metaheuristics, Springer, 2010, pp. 141–183.

  • R.Y. Nakamura, L.A. Pereira, K.A. Costa, D. Rodrigues, J.P. Papa, X.-S. Yang, 2012 BBA: a binary bat algorithm for feature selection, in: 2012 25th SIBGRAPI conference on graphics, patterns and images, IEEE, pp. 291-297

  • Neri F, Cotta C (2012) Memetic algorithms and memetic computing optimization: a literature review. Swarm Evol Comput 2:1–14

    Article  Google Scholar 

  • F. Neri, C. Cotta, P. Moscato, Handbook of memetic algorithms, Springer, 2011.

  • Nekkaa M, Boughaci D (2015) A memetic algorithm with support vector machine for feature selection and classification. Memetic Computing 7:59–73

    Article  Google Scholar 

  • Nguyen BH, Xue B, Zhang M (2020) A survey on swarm intelligence approaches to feature selection in data mining. Swarm Evol Comput 54:100663

    Article  Google Scholar 

  • Nouri-Moghaddam B, Ghazanfari M, Fathian M (2021) A novel multi-objective forest optimization algorithm for wrapper feature selection. Expert Syst Appl 175:114737

    Article  Google Scholar 

  • Ouadfel S, Abd Elaziz M (2020) Enhanced crow search algorithm for feature selection. Expert Syst with Applic 159:113572

    Article  Google Scholar 

  • H. Sun, P. Moscato, 2019 A Memetic Algorithm for Symbolic Regression, in: 2019 IEEE Congress on Evolutionary Computation (CEC), IEEE, pp. 2167-2174

  • Too J, Abdullah AR, Mohd Saad N, Tee W (2019) EMG feature selection and classification using a Pbest-guide binary particle swarm optimization. Computation 7:12

    Article  Google Scholar 

  • Too J, Abdullah AR (2020) Chaotic Atom Search Optimization for Feature Selection. Arab J Sci Eng 45(8):6063–6079

  • Tubishat M, Ja’afar S, Alswaitti M, Mirjalili S, Idris N, Ismail MA, Omar MS (2020) Dynamic salp swarm algorithm for feature selectio. Expert Syst with Appl. https://doi.org/10.1016/j.eswa.2020.113873

    Article  Google Scholar 

  • Xue B, Zhang M, Browne WN (2012) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Transac on Cybernet 43:1656–1671

    Article  Google Scholar 

  • Zawbaa HM, Emary E, Grosan C, Snasel V (2018) Large-dimensionality small-instance set feature selection: a hybrid bio-inspired heuristic approach. Swarm Evol Comput 42:29–42

    Article  Google Scholar 

  • Zhu Z, Ong Y-S, Dash M (2007) Wrapper–filter feature selection algorithm using a memetic framework. IEEE Transac Syst, Man, Cybernet Part B (cybernetics) 37:70–76

    Article  Google Scholar 

  • Zhang Y, Gong D-W, Gao X-Z, Tian T, Sun X-Y (2020) Binary differential evolution with self-learning for multi-objective feature selection. Inf Sci 507:67–85

    Article  MathSciNet  MATH  Google Scholar 

  • Zhu Z, Jia S, Ji Z (2010) Towards a memetic feature selection paradigm [application notes]. IEEE Comput Intell Mag 5:41–53

    Article  Google Scholar 

  • J. Zhong, X. Hu, J. Zhang, M. Gu, 2005 Comparison of performance between different selection strategies on simple genetic algorithms, in: International conference on computational intelligence for modelling, control and automation and international conference on intelligent agents, web technologies and internet commerce (CIMCA-IAWTIC'06), IEEE, , pp. 1115–1121

Download references

Acknowledgements

The authors wish to thank Scientific Research Projects Coordinatorship at Selcuk University and The Scientific and Technological Research Council of Turkey for their institutional supports.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Faculty of Technology, Selçuk University, Konya, Turkey

    Ahmet Cevahir Cinar

Authors
  1. Ahmet Cevahir Cinar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Only one author prepared this manuscript.

Corresponding author

Correspondence to Ahmet Cevahir Cinar.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1

Appendix 1

Experimental results of all datasets.

See Tables 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, and 25

Table 15 The experimental results for the Ionosphere dataset
Full size table
Table 16 The experimental results for the BreastEW dataset
Full size table
Table 17 The experimental results for the Lymphography dataset
Full size table
Table 18 The experimental results for the SonarEW dataset
Full size table
Table 19 The experimental results for the SpectEW dataset
Full size table
Table 20 The experimental results for the Tic-tac-toe dataset
Full size table
Table 21 The experimental results for the WineEW dataset
Full size table
Table 22 The experimental results for the Zoo dataset
Full size table
Table 23 The experimental results for the Yale dataset
Full size table
Table 24 The experimental results for the ORL dataset
Full size table
Table 25 The experimental results for the COIL20 dataset
Full size table

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cinar, A.C. A novel adaptive memetic binary optimization algorithm for feature selection. Artif Intell Rev 56, 13463–13520 (2023). https://doi.org/10.1007/s10462-023-10482-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-023-10482-8

Keywords

Search

Navigation