A Hybrid GA-GP Method for Feature Reduction in Classification

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10593)


Feature reduction is an important pre-processing step in classification and other artificial intelligent applications. Its aim is to improve the quality of feature sets. There are two main types of feature reduction: feature construction and feature selection. Most current feature reduction algorithms focus on just one of the two types because they require different representations. This paper proposes a new representation which supports a feature reduction algorithm that combines feature selection and feature construction. The algorithm uses new genetic operators to update the new representation. The proposed algorithm is compared with two conventional feature selection algorithms, a genetic algorithms-based feature selection algorithm, and a genetic programming-based algorithm which evolves feature sets containing both original and high-level features. The experimental results on 10 different datasets show that the new representation can help to produce a smaller number of features and improve the classification accuracy over using all features on most datasets. In comparison with other feature selection or construction algorithms, the proposed algorithm achieves similar or better classification performance on all datasets.


  1. 1.
    Lones, M.A., Smith, S.L., Alty, J.E., Lacy, S.E., Possin, K.L., Jamieson, D.S., Tyrrell, A.M.: Evolving classifiers to recognize the movement characteristics of Parkinson’s disease patients. IEEE Trans. Evol. Comput. 18(4), 559–576 (2014)CrossRefGoogle Scholar
  2. 2.
    Xue, B., Zhang, M., Browne, W.N., Yao, X.: A survey on evolutionary computation approaches to feature selection. IEEE Trans. Evol. Comput. 20(4), 606–626 (2016)CrossRefGoogle Scholar
  3. 3.
    Nguyen, H.B., Xue, B., Andreae, P.: Mutual information for feature selection: estimation or counting? Evol. Intel. 9(3), 95–110 (2016)CrossRefGoogle Scholar
  4. 4.
    Neshatian, K., Zhang, M., Andreae, P.: A filter approach to multiple feature construction for symbolic learning classifiers using genetic programming. IEEE Trans. Evol. Comput. 16(5), 645–661 (2012)CrossRefGoogle Scholar
  5. 5.
    Tran, B., Xue, B., Zhang, M.: Genetic programming for feature construction and selection in classification on high-dimensional data. Memet. Comput. 8(1), 3–15 (2015)CrossRefGoogle Scholar
  6. 6.
    Goldberg, D.E., Holland, J.H.: Genetic algorithms and machine learning. Mach. Learn. 3(2), 95–99 (1988)CrossRefGoogle Scholar
  7. 7.
    Koza, J.R.: Genetic Programming II: Automatic Discovery of Reusable Subprograms. MIT Press, Cambridge (1994)zbMATHGoogle Scholar
  8. 8.
    Niu, G.: Feature selection optimization. Data-Driven Technology for Engineering Systems Health Management, pp. 139–171. Springer, Singapore (2017). doi: 10.1007/978-981-10-2032-2_6 CrossRefGoogle Scholar
  9. 9.
    De Paula, L.C., Soares, A.S., de Lima, T.W., Coelho, C.J.: Feature selection using genetic algorithm: an analysis of the bias-property for one-point crossover. In: GECCO 2016 Companion, pp. 1461–1462 (2016)Google Scholar
  10. 10.
    Stefano, C.D., Fontanella, F., Marrocco, C., di Freca, A.S.: A GA-based feature selection approach with an application to handwritten character recognition. Pattern Recogn. Lett. 35, 130–141 (2014). Frontiers in Handwriting ProcessingCrossRefGoogle Scholar
  11. 11.
    Li, N.J., Wang, W.J., Hsu, C.C.J.: Hybrid particle swarm optimization incorporating fuzzy reasoning and weighted particle. Neurocomputing 167, 488–501 (2015)CrossRefGoogle Scholar
  12. 12.
    Mistry, K., Zhang, L., Neoh, S.C., Lim, C.P., Fielding, B.: A micro-GA embedded PSO feature selection approach to intelligent facial emotion recognition. IEEE Trans. Cybern. 47(6), 1496–1509 (2017)CrossRefGoogle Scholar
  13. 13.
    Bharti, K.K., Singh, P.K.: Opposition chaotic fitness mutation based adaptive inertia weight BPSO for feature selection in text clustering. Appl. Soft Comput. 43, 20–34 (2016)CrossRefGoogle Scholar
  14. 14.
    Xue, B., Zhang, M., Browne, W.N.: Particle swarm optimisation for feature selection in classification: novel initialisation and updating mechanisms. Appl. Soft Comput. 18, 261–276 (2014)CrossRefGoogle Scholar
  15. 15.
    Vieira, S.M., Mendonça, L.F., Farinha, G.J., Sousa, J.M.: Modified binary PSO for feature selection using SVM applied to mortality prediction of septic patients. Appl. Soft Comput. 13(8), 3494–3504 (2013)CrossRefGoogle Scholar
  16. 16.
    Nguyen, H.B., Xue, B., Liu, I., Andreae, P., Zhang, M.: Gaussian transformation based representation in particle swarm optimisation for feature selection. In: Mora, A.M., Squillero, G. (eds.) EvoApplications 2015. LNCS, vol. 9028, pp. 541–553. Springer, Cham (2015). doi: 10.1007/978-3-319-16549-3_44 Google Scholar
  17. 17.
    Nguyen, H.B., Xue, B., Liu, I., Zhang, M.: PSO and statistical clustering for feature selection: a new representation. In: Dick, G., et al. (eds.) SEAL 2014. LNCS, vol. 8886, pp. 569–581. Springer, Cham (2014). doi: 10.1007/978-3-319-13563-2_48 Google Scholar
  18. 18.
    Nguyen, H.B., Xue, B., Andreae, P.: Surrogate-model based particle swarm optimisation with local search for feature selection in classification. In: Squillero, G., Sim, K. (eds.) EvoApplications 2017. LNCS, vol. 10199, pp. 487–505. Springer, Cham (2017). doi: 10.1007/978-3-319-55849-3_32 CrossRefGoogle Scholar
  19. 19.
    Guo, H., Nandi, A.K.: Breast cancer diagnosis using genetic programming generated feature. Pattern Recogn. 39(5), 980–987 (2006)CrossRefGoogle Scholar
  20. 20.
    Lichman, M.: UCI machine learning repository (2013)Google Scholar
  21. 21.
    Fortin, F.A., De Rainville, F.M., Gardner, M.A., Parizeau, M., Gagné, C.: DEAP: evolutionary algorithms made easy. J. Mach. Learn. Res. 13, 2171–2175 (2012)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of Engineering and Computer ScienceVictoria University of WellingtonWellingtonNew Zealand

Personalised recommendations