Recent Developments on Evolutionary Computation Techniques to Feature Construction

  • Idheba Mohamad Ali O. SwesiEmail author
  • Azuraliza Abu Bakar
Part of the Studies in Computational Intelligence book series (SCI, volume 830)


The quality of the search space is an important factor that influences the performance of any machine learning algorithm including its classification. The attributes that define the search space can be poorly understood or inadequate, thereby making it difficult to discover high quality knowledge and understanding. Feature construction (FC) and feature selection (FS) are two pre-processing steps that can be used to improve the feature space quality, by enhancing the classifier performance in terms of accuracy, complexity, speed and interpretability. While FS aims to choose a set of informative features for improving the performance, FC can enhance the classification performance by evolving new features out of the original ones. The evolved features are expected to have more predictive value than the originals that make them up. Over the past few decades, several evolutionary computation (EC) methods have been proposed in the area of FC. This paper gives an overview of the literature on EC for FC. Here, we focus mainly on filter, wrapper and embedded methods, in which the contributions of these different methods are identified. Furthermore, some open challenges and current issues are also discussed in order to identify promising areas for future research.


Classification Evolutionary computation Feature selection Feature construction 



This work is supported by University Kebangsaan Malaysia, under grant number FRGS/1/2016/ICT02/UKM/01/2.


  1. 1.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 3rd edn. Prentice Hall Press, USA (2009)zbMATHGoogle Scholar
  2. 2.
    Blum, A.L., Langley, P.: Selection of relevant features and examples in machine learning. Artif. Intell. 97(1–2), 245–271 (1997)MathSciNetCrossRefGoogle Scholar
  3. 3.
    John, G.H., Kohavi, R., Pfleger, K.: Irrelevant features and subset selection problem. In: Proceedings of 11th International Conference on Machine Learning, pp. 121–129 (1994)Google Scholar
  4. 4.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publisher (1993)Google Scholar
  5. 5.
    Liu, H., Motada, H.: Feature Extraction, Construction and Selection: A Data Mining Perspective. Kluwer Academic Publishers, Norwell (1998)CrossRefGoogle Scholar
  6. 6.
    Xue, B., Zhang, M.: Evolutionary computation for feature manipulation: key challenges and future directions. In: Evolutionary Computation (CEC), pp. 3061–3067 (2016)Google Scholar
  7. 7.
    Xue, B., Zhang, M., Browne, W.N., Yao, X.: A survey on evolutionary computation approaches to feature selection. IEEE Trans. Evol. Comput. 20(4), 606–626 (2016)CrossRefGoogle Scholar
  8. 8.
    Liu, H., Yu, L.: Toward integrating feature selection algorithms for classification and clustering. IEEE Trans. Knowl. Data Eng. 17(4), 491–502 (2005)CrossRefGoogle Scholar
  9. 9.
    Tran, B., Zhang, M., Xue, B.: Multiple feature construction in classification on high-dimensional data using GP. In: SSCI, pp. 1–8 (2016)Google Scholar
  10. 10.
    Dai, Y., Xue, B., Zhang, M.: New representations in PSO for feature construction in classification. In: European Conference on the Applications of Evolutionary Computation, pp. 476–488. Springer, Berlin (2014)Google Scholar
  11. 11.
    Nguyen, H.B., Xue, B., Andreae, P.: A hybrid GA-GP method for feature reduction in classification. In: Asia-Pacific Conference on Simulated Evolution and Learning, pp. 591–604. Springer, Cham (2017)Google Scholar
  12. 12.
    Fogel, D.B.: Introduction to Evolutionary Computation. Wiley, New York (2007)Google Scholar
  13. 13.
    Vafaie, H., De Jong, K.: Genetic algorithms as a tool for restructuring feature space representations. In: Proceedings of 7th International Conference of Tools with AI, pp. 8–11 (1995)Google Scholar
  14. 14.
    Ahmed, S., Zhang, M., Peng, L., Xue, B.: Multiple feature construction for effective biomarker identification and classification using genetic programming. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, pp. 249–256 (2014a)Google Scholar
  15. 15.
    Eiben, A.E., Smith, J.: From evolutionary computation to the evolution of things. Nature 521(7553), 476 (2015)CrossRefGoogle Scholar
  16. 16.
    Sondhi, P.: Feature construction methods: a survey. sifaka. uiuc. edu. 69, 70–71 (2009)Google Scholar
  17. 17.
    Lillywhite, K., Lee, D.J., Tippetts, B., Archibald, J.: A feature construction method for general object recognition. Pattern Recogn. 46(12), 3300–3314 (2013)CrossRefGoogle Scholar
  18. 18.
    Drozdz, K., Kwasnicka, H.: Feature set reduction by evolutionary selection and construction. In: KES International Symposium on Agent and Multi-agent Systems: Technologies and Applications (pp. 140–149). Springer, Berlin (2010)Google Scholar
  19. 19.
    Shafti, L.S., Pérez, E.: Evolutionary multi-feature construction for data reduction: a case study. Appl. Soft Comput. 9(4), 1296–1303 (2009)CrossRefGoogle Scholar
  20. 20.
    Larsen, O., Freitas, A.A., Nievola, J.C.: Constructing X-of-N attributes with a genetic algorithm. In: GECCO Late Breaking Papers, pp. 316–322 (2002)Google Scholar
  21. 21.
    García, D., González, A., Pérez, R.: A two-step approach of feature construction for a genetic learning algorithm. In: Fuzzy Systems (FUZZ), pp. 1255–1262 (2011)Google Scholar
  22. 22.
    García, D., González, A., Pérez, R.: A feature construction approach for genetic iterative rule learning algorithm. J. Comput. Syst. Sci. 80(1), 101–117 (2014)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Alfred, R.: DARA: data summarisation with feature construction. In: Second Asia International Conference on Modeling & Simulation, pp. 830–835. AICMS 08 (2008)Google Scholar
  24. 24.
    Firpi, H., Goodman, E., Echauz, J.: On prediction of epileptic seizures by computing multiple genetic programming artificial features. In: European Conference on Genetic Programming, pp. 321–330. Springer, Berlin (2005)Google Scholar
  25. 25.
    Chen, Q., Zhang, M., Xue, B.: Genetic programming with embedded feature construction for high-dimensional symbolic regression. In: Intelligent and Evolutionary Systems: The 20th Asia Pacific Symposium, IES 2016, Canberra, Australia, Nov 2016, pp. 87–102. Proceedings (2017)Google Scholar
  26. 26.
    Krawiec, K.: Genetic programming-based construction of features for machine learning and knowledge discovery tasks. Genet. Program Evol. Mach. 3(4), 329–343 (2002)CrossRefGoogle Scholar
  27. 27.
    Yazdani, S., Shanbehzadeh, J., Hadavandi, E.: MBCGP-FE: a modified balanced cartesian genetic programming feature extractor. Knowl.-Based Syst. 135, 89–98 (2017)CrossRefGoogle Scholar
  28. 28.
    Ahmed, S., Zhang, M., Peng, L.: A new GP-based wrapper feature construction approach to classification and biomarker identification. In: 2014 IEEE Congress on Evolutionary Computation (CEC), pp. 2756–2763 (2014b)Google Scholar
  29. 29.
    Elola, A., Del Ser, J., Bilbao, M.N., Perfecto, C., Alexandre, E., Salcedo-Sanz, S.: Hybridizing cartesian genetic programming and harmony search for adaptive feature construction in supervised learning problems. Appl. Soft Comput. 52, 760–770 (2017)CrossRefGoogle Scholar
  30. 30.
    Suganuma, M., Tsuchiya, D., Shirakawa, S., Nagao, T.: Hierarchical feature construction for image classification using genetic programming. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 001423–001428 (2016)Google Scholar
  31. 31.
    Tariq, H., Eldridge, E., Welch, I.: An efficient approach for feature construction of high-dimensional microarray data by random projections. PLoS ONE 13(4), e0196385 (2018)CrossRefGoogle Scholar
  32. 32.
    Tran, C.T., Zhang, M., Andreae, P., Xue, B.: Genetic programming based feature construction for classification with incomplete data. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1033–1040 (2017)Google Scholar
  33. 33.
    Muharram, M., Smith, G.: Evolutionary constructive induction. IEEE Trans. Knowl. Data Eng. 17, 1518–1528 (2005)CrossRefGoogle Scholar
  34. 34.
    Guo, H., Nandi, A.K.: Breast cancer diagnosis using genetic programming generated feature. Pattern Recogn. 39(5), 980–987 (2006)CrossRefGoogle Scholar
  35. 35.
    Liu, Q., Qiao, M., Sung, A.H.: Distance metric learning and support vector machines for classification of mass spectrometry proteomics data. Int. J. Knowl. Eng. Soft Data Paradig. 1(3), 216–226 (2009)CrossRefGoogle Scholar
  36. 36.
    Muharram, M.A., Smith, G.D.: Evolutionary feature construction using information gain and gini index. In: European Conference on Genetic Programming, pp. 379–388. Springer, Berlin (2004)Google Scholar
  37. 37.
    Neshatian, K., Zhang, M., Andreae, P.: A filter approach to multiple feature construction for symbolic learning classifiers using genetic programming. IEEE Trans. Evol. Comput. 16(5), 645–661 (2012)CrossRefGoogle Scholar
  38. 38.
    Liang, Y., Zhang, M., Browne, W.N.: Feature construction using genetic programming for figure-ground image segmentation. In: Intelligent and Evolutionary Systems, pp. 237–250. Springer, Cham (2017)Google Scholar
  39. 39.
    Tran, B., Xue, B., Zhang, M.: Genetic programming for feature construction and selection in classification on high-dimensional data. Memet. Comput. 8(1), 3–15 (2015)CrossRefGoogle Scholar
  40. 40.
    Ahmed, S., Zhang, M., Peng, L., Xue, B.: A Multi-objective genetic programming biomarker detection approach in mass spectrometry data. In: European Conference on the Applications of Evolutionary Computation, pp. 106–122. Springer, Cham (2016)Google Scholar
  41. 41.
    Cano, A., Ventura, S., Cios, K.J.: Multi-objective genetic programming for feature extraction and data visualization. Soft Comput. 21(8), 2069–2089 (2017)CrossRefGoogle Scholar
  42. 42.
    Xue, B., Zhang, M., Dai, Y., Browne, W.N.: PSO for feature construction and binary classification. In: Proceeding of the Fifteenth Annual Conference on Genetic and Evolutionary Computation Conference, GECCO, pp. 137–144 (2013)Google Scholar
  43. 43.
    Mahanipour, A., Nezamabadi-pour, H.: Improved PSO-based feature construction algorithm using feature selection methods. In: Swarm Intelligence and Evolutionary Computation (CSIEC), pp. 1–5 (2017)Google Scholar
  44. 44.
    Tran, B., Xue, B., Zhang, M.: Using feature clustering for GP-based feature construction on high-dimensional data. In: European Conference on Genetic Programming, pp. 210–226. Springer, Cham (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Idheba Mohamad Ali O. Swesi
    • 1
    Email author
  • Azuraliza Abu Bakar
    • 1
  1. 1.Faculty of Information Science and Technology, Centre for Artificial Intelligence TechnologyUniversity Kebangsaan MalaysiaBangiMalaysia

Personalised recommendations