Automated Design of Genetic Programming Classification Algorithms Using a Genetic Algorithm

  • Thambo Nyathi
  • Nelishia Pillay
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10200)


There is a large scale initiative by the machine learning community to automate the design of machine learning techniques to remove reliance on the human expert, providing out of the box software that can be used by novices. In this study the automated design of genetic programming classification algorithms is proposed. A number of design decisions have to be considered by algorithm designers during the design process and this is usually a time consuming task. Our automated design approach uses a genetic algorithm to automatically configure a genetic programming classification algorithm. The genetic algorithm determines parameter values and sets the flow control for the classification algorithm. The proposed system is tested on real world problems and the results indicate that induced classifiers perform better than manually designed classifiers.


Data classification Automated machine learning Genetic programming Genetic algorithms 


  1. 1.
    Back, T.: Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms. Oxford University Press, Oxford (1996)zbMATHGoogle Scholar
  2. 2.
    Banzhaf, W., Nordin, P., Keller, R.E., Francone, F.D.: Genetic Programming: An Introduction, vol. 1. Morgan Kaufmann, San Francisco (1998)CrossRefzbMATHGoogle Scholar
  3. 3.
    Espejo, P.G., Ventura, S., Herrera, F.: A survey on the application of genetic programming to classification. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 40(2), 121–144 (2010)CrossRefGoogle Scholar
  4. 4.
    Eiben, Á.E., Hinterding, R., Michalewicz, Z.: Parameter control in evolutionary algorithms. IEEE Trans. Evol. Comput. 3(2), 124–141 (1999)CrossRefGoogle Scholar
  5. 5.
    Sabar, N.R., Ayob, M., Kendall, G., Qu, R.: Automatic design of a hyper-heuristic framework with gene expression programming for combinatorial optimization problems. IEEE Trans. Evol. Comput. 19(3), 309–325 (2015)CrossRefGoogle Scholar
  6. 6.
    Han, J., Pei, J., Kamber, M.: Data Mining: Concepts and Techniques. Elsevier, USA (2011)zbMATHGoogle Scholar
  7. 7.
    Koza, J.R.: Concept formation and decision tree induction using the genetic programming paradigm. In: Schwefel, H.-P., Männer, R. (eds.) PPSN 1990. LNCS, vol. 496, pp. 124–128. Springer, Heidelberg (1991). doi: 10.1007/BFb0029742CrossRefGoogle Scholar
  8. 8.
    Bojarczuk, C.C., Lopes, H.S., Freitas, A.A.: Discovering comprehensible classification rules using genetic programming: a case study in a medical domain. In: Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation, vol. 2, pp. 953–958. Morgan Kaufmann Publishers Inc. (1999)Google Scholar
  9. 9.
    Ong, C.S., Huang, J.J., Tzeng, G.H.: Building credit scoring models using genetic programming. Expert Syst. Appl. 29(1), 41–47 (2005)CrossRefGoogle Scholar
  10. 10.
    Souffriau, W., Vansteenwegen, P., Berghe, G.V., Van Oudheusden, D.: Automated parameterisation of a metaheuristic for the orienteering problem. In: Cotta, C., Sevaux, M., Sörensen, K. (eds.) Adaptive and Multilevel Metaheuristics. SCI, vol. 136, pp. 255–269. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Barros, R.C., Basgalupp, M.P., de Carvalho, A.C., Freitas, A.A.: Automatic design of decision-tree algorithms with evolutionary algorithms. Evol. Comput. 21(4), 659–684 (2013)CrossRefGoogle Scholar
  12. 12.
    Diosan, L.S., Oltean, M.: Evolving evolutionary algorithms using evolutionary algorithms. In: Proceedings of the 9th Annual Conference Companion on Genetic and Evolutionary Computation, pp. 2442–2449. ACM (2007)Google Scholar
  13. 13.
    Goldberg, D.E.: Genetic Algorithms. Pearson Education India, New Delhi (2006)Google Scholar
  14. 14.
    Eiben, A.E., Smith, J.E., et al.: Introduction to Evolutionary Computing, vol. 53. Springer, Heidelberg (2003)CrossRefzbMATHGoogle Scholar
  15. 15.
    Aitkenhead, M.: A co-evolving decision tree classification method. Expert Syst. Appl. 34(1), 18–25 (2008)CrossRefGoogle Scholar
  16. 16.
    Agnelli, D., Bollini, A., Lombardi, L.: Image classification: an evolutionary approach. Pattern Recogn. Lett. 23(1), 303–309 (2002)CrossRefzbMATHGoogle Scholar
  17. 17.
    Cios, K.J., Swiniarski, R.W., Pedrycz, W., Kurgan, L.A.: The knowledge discovery process. In: Data Mining, pp. 9–24. Springer, New York (2007)Google Scholar
  18. 18.
    Bhowan, U., Zhang, M., Johnston, M.: Genetic programming for classification with unbalanced data. In: Esparcia-Alcázar, A.I., Ekárt, A., Silva, S., Dignum, S., Uyar, A.Ş. (eds.) EuroGP 2010. LNCS, vol. 6021, pp. 1–13. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-12148-7_1CrossRefGoogle Scholar
  19. 19.
    Frank, A., Asuncion, A., et al.: Uci machine learning repository (2010)Google Scholar
  20. 20.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Yao, X., Liu, Y.: Ensemble structure of evolutionary artificial neural networks. In: Proceedings of IEEE International Conference on Evolutionary Computation, pp. 659–664. IEEE (1996)Google Scholar
  22. 22.
    Cano, A., Ventura, S., Cios, K.J.: Multi-objective genetic programming for feature extraction and data visualization. Soft Comput., 1–21 (2015)Google Scholar
  23. 23.
    Raymer, M.L., Punch, W.F., Goodman, E.D., Kuhn, L.A., Jain, A.K.: Dimensionality reduction using genetic algorithms. IEEE Trans. Evol. Comput. 4(2), 164–171 (2000)CrossRefGoogle Scholar
  24. 24.
    Bojarczuk, C.C., Lopes, H.S., Freitas, A.A., Michalkiewicz, E.L.: A constrained-syntax genetic programming system for discovering classification rules: application to medical data sets. Artif. Intell. Med. 30(1), 27–48 (2004)CrossRefGoogle Scholar
  25. 25.
    Akay, M.F.: Support vector machines combined with feature selection for breast cancer diagnosis. Expert Syst. Appl. 36(2), 3240–3247 (2009)CrossRefGoogle Scholar
  26. 26.
    Bylander, T.: Estimating generalization error on two-class datasets using out-of-bag estimates. Mach. Learn. 48(1–3), 287–297 (2002)CrossRefzbMATHGoogle Scholar
  27. 27.
    Espejo, P.G., Romero, C., Ventura, S., Hervás, C.: Induction of classification rules with grammar-based genetic programming. In: Conference on Machine Intelligence, pp. 596–601 (2005)Google Scholar
  28. 28.
    Kahramanli, H., Allahverdi, N.: Design of a hybrid system for the diabetes and heart diseases. Expert Syst. Appl. 35(1), 82–89 (2008)CrossRefGoogle Scholar
  29. 29.
    Cao, V.L., Le-Khac, N.-A., O’Neill, M., Nicolau, M., McDermott, J.: Improving fitness functions in genetic programming for classification on unbalanced credit card data. In: Squillero, G., Burelli, P. (eds.) EvoApplications 2016. LNCS, vol. 9597, pp. 35–45. Springer, Cham (2016). doi: 10.1007/978-3-319-31204-0_3CrossRefGoogle Scholar
  30. 30.
    West, D.: Neural network credit scoring models. Comput. Oper. Res. 27(11), 1131–1152 (2000)CrossRefzbMATHGoogle Scholar
  31. 31.
    Jabeen, H., Baig, A.R.: Depthlimited crossover in GP for classifier evolution. Comput. Hum. Behav. 27(5), 1475–1481 (2011)CrossRefGoogle Scholar
  32. 32.
    Raymer, M.L., Doom, T.E., Kuhn, L.A., Punch, W.F.: Knowledge discovery in medical and biological datasets using a hybrid bayes classifier/evolutionary algorithm. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 33(5), 802–813 (2003)Google Scholar
  33. 33.
    Mangasarian, O.L., Musicant, D.R.: Lagrangian support vector machines. J. Mach. Learn. Res. 1, 161–177 (2001)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of Mathematics, Statistics and Computer ScienceUniversity of KwaZulu-NatalPietermaritzburgSouth Africa

Personalised recommendations