Investigating Multi-Operator Differential Evolution for Feature Selection

  • Essam DebieEmail author
  • Saber Mohammed Elsayed
  • Daryl L. Essam
  • Ruhul A. Sarker
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9592)


Performance issues when dealing with a large number of features are well-known for classification algorithms. Feature selection aims at mitigating these issues by reducing the number of features in the data. Hence, in this paper, a feature selection approach based on a multi-operator differential evolution algorithm is proposed. The algorithm partitions the initial population into a number of sub-populations evolving using a pool of distinct mutation strategies. Periodically, the sub-populations exchange information to enhance their diversity. This multi-operator approach reduces the sensitivity of the standard differential evolution to the selection of an appropriate mutation strategy. Two classifiers, namely decision trees and k-nearest neighborhood, are used to evaluate the generated subsets of features. Experimental analysis has been conducted on several real data sets using a 10-fold cross validation. The analysis shows that the proposed algorithm successfully determines efficient feature subsets, which can improve the classification accuracy of the classifiers under consideration. The usefulness of the proposed method on large scale data set has been demonstrated using the KDD Cup 1999 intrusion data set, where the proposed method can effectively remove irrelevant features from the data.


Feature Selection Classification Accuracy Differential Evolution Intrusion Detection Feature Subset 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported by the Australian Centre for Cyber Security Research Funding Program, under a grant no. PS38135.


  1. 1.
    Bazi, Y., Melgani, F.: Toward an optimal SVM classification system for hyperspectral remote sensing images. IEEE Trans. Geosci. Remote Sens. 44(11), 3374–3385 (2006). 00252CrossRefGoogle Scholar
  2. 2.
    Caruana, R., Freitag, D.: Greedy Attribute Selection, pp. 28–36. Citeseer (1994). 00536Google Scholar
  3. 3.
    Debie, E., Shafi, K., Lokan, C., Merrick, K.: Performance analysis of rough set ensemble of learning classifier systems with differential evolution based rule discovery. Evol. Intell. 6(2), 109–126 (2013). 00001CrossRefGoogle Scholar
  4. 4.
    Elsayed, S.M., Sarker, R.A., Essam, D.L.: Multi-operator based evolutionary algorithms for solving constrained optimization problems. Comput. Oper. Res. 38(12), 1877–1896 (2011). CrossRefMathSciNetzbMATHGoogle Scholar
  5. 5.
    Frank, A., Asuncion, A.: UCI Machine Learning Repository. University of California, Irvine, School of Information and Computer Sciences (2010).
  6. 6.
    Gadat, S., Younes, L.: A stochastic algorithm for feature selection in pattern recognition. J. Mach. Learn. Res. 8, 509–547 (2007)zbMATHGoogle Scholar
  7. 7.
    Garcia-Nieto, J., Alba, E., Apolloni, J.: Hybrid DE-SVM approach for feature selection: application to gene expression datasets, pp. 1–6. IEEE (2009)Google Scholar
  8. 8.
    Hall, M.A.: Correlation-based Feature Selection for Machine Learning. Ph.D. thesis (1999)Google Scholar
  9. 9.
    John, G.H., Kohavi, R., Pfleger, K.: Irrelevant Features and the Subset Selection Problem. In: Proceedings, pp. 121–129. Morgan Kaufmann (1994)Google Scholar
  10. 10.
    Khushaba, R.N., Al-Ani, A., Al-Jumaily, A.: Feature subset selection using differential evolution and a statistical repair mechanism. Expert Syst. Appl. 38(9), 11515–11526 (2011). CrossRefGoogle Scholar
  11. 11.
    Kononenko, I.: Estimating attributes: analysis and extensions of RELIEF. In: Bergadano, F., De Raedt, L. (eds.) ECML-94. LNCS, vol. 784, pp. 171–182. Springer, Heidelberg (1994). CrossRefGoogle Scholar
  12. 12.
    Maimon, O., Rokach, L.: Improving supervised learning by feature decomposition. In: Eiter, T., Schewe, K.-D. (eds.) FoIKS 2002. LNCS, vol. 2284, pp. 178–196. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  13. 13.
    Mallipeddi, R., Suganthan, P.N., Pan, Q.K., Tasgetiren, M.F.: Differential evolution algorithm with ensemble of parameters and mutation strategies. Appl. Soft Comput. 11(2), 1679–1696 (2011)CrossRefGoogle Scholar
  14. 14.
    Martinoyić, G., Bajer, D., Zorić, B.: A differential evolution approach to dimensionality reduction for classification needs. Int. J. Appl. Math. Comput. Sci. 24(1), 111 (2014). MathSciNetGoogle Scholar
  15. 15.
    Oh, I.S., Lee, J.S., Moon, B.R.: Hybrid genetic algorithms for feature selection. IEEE Trans. Pattern Anal. Mach. Intell. 26(11), 1424–1437 (2004)CrossRefGoogle Scholar
  16. 16.
    Price, K., Storn, R., Lampinen, J.: Differential Evolution: A Practical Approach to Global Optimization. Springer, New York (2005)Google Scholar
  17. 17.
    Raymer, M.L., Punch, W.F., Goodman, E.D., Kuhn, L., Jain, A.K.: Dimensionality reduction using genetic algorithms. IEEE Trans. Evol. Comput. 4(2), 164–171 (2000)CrossRefGoogle Scholar
  18. 18.
    Storn, R., Price, K.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997). 09676CrossRefMathSciNetzbMATHGoogle Scholar
  19. 19.
    Tasoulis, D.K., Plagianakos, V.P., Vrahatis, M.N.: Differential evolution algorithms for finding predictive gene subsets in microarray data. In: Maglogiannis, I., Karpouzis, K., Bramer, M. (eds.) AIAI. IFIP, vol. 204, pp. 484–491. Springer, New York (2006)CrossRefGoogle Scholar
  20. 20.
    Tušar, T., Filipič, B.: Differential evolution versus genetic algorithms in multiobjective optimization. In: Obayashi, S., Deb, K., Poloni, C., Hiroyasu, T., Murata, T. (eds.) EMO 2007. LNCS, vol. 4403, pp. 257–271. Springer, Heidelberg (2007). CrossRefGoogle Scholar
  21. 21.
    Yu, L., Liu, H.: Feature selection for high-dimensional data: a fast correlation-based filter solution, pp. 856–863 (2003).

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Essam Debie
    • 1
    Email author
  • Saber Mohammed Elsayed
    • 1
  • Daryl L. Essam
    • 1
  • Ruhul A. Sarker
    • 1
  1. 1.School of Engineering and Information TechnologyUniversity of New South WalesCanberraAustralia

Personalised recommendations