Automated Design of Genetic Programming Classification Algorithms Using a Genetic Algorithm

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10200)

Abstract

There is a large scale initiative by the machine learning community to automate the design of machine learning techniques to remove reliance on the human expert, providing out of the box software that can be used by novices. In this study the automated design of genetic programming classification algorithms is proposed. A number of design decisions have to be considered by algorithm designers during the design process and this is usually a time consuming task. Our automated design approach uses a genetic algorithm to automatically configure a genetic programming classification algorithm. The genetic algorithm determines parameter values and sets the flow control for the classification algorithm. The proposed system is tested on real world problems and the results indicate that induced classifiers perform better than manually designed classifiers.

Keywords

Data classification Automated machine learning Genetic programming Genetic algorithms 

References

  1. 1.
    Back, T.: Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms. Oxford University Press, Oxford (1996)MATHGoogle Scholar
  2. 2.
    Banzhaf, W., Nordin, P., Keller, R.E., Francone, F.D.: Genetic Programming: An Introduction, vol. 1. Morgan Kaufmann, San Francisco (1998)CrossRefMATHGoogle Scholar
  3. 3.
    Espejo, P.G., Ventura, S., Herrera, F.: A survey on the application of genetic programming to classification. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 40(2), 121–144 (2010)CrossRefGoogle Scholar
  4. 4.
    Eiben, Á.E., Hinterding, R., Michalewicz, Z.: Parameter control in evolutionary algorithms. IEEE Trans. Evol. Comput. 3(2), 124–141 (1999)CrossRefGoogle Scholar
  5. 5.
    Sabar, N.R., Ayob, M., Kendall, G., Qu, R.: Automatic design of a hyper-heuristic framework with gene expression programming for combinatorial optimization problems. IEEE Trans. Evol. Comput. 19(3), 309–325 (2015)CrossRefGoogle Scholar
  6. 6.
    Han, J., Pei, J., Kamber, M.: Data Mining: Concepts and Techniques. Elsevier, USA (2011)MATHGoogle Scholar
  7. 7.
    Koza, J.R.: Concept formation and decision tree induction using the genetic programming paradigm. In: Schwefel, H.-P., Männer, R. (eds.) PPSN 1990. LNCS, vol. 496, pp. 124–128. Springer, Heidelberg (1991). doi: 10.1007/BFb0029742CrossRefGoogle Scholar
  8. 8.
    Bojarczuk, C.C., Lopes, H.S., Freitas, A.A.: Discovering comprehensible classification rules using genetic programming: a case study in a medical domain. In: Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation, vol. 2, pp. 953–958. Morgan Kaufmann Publishers Inc. (1999)Google Scholar
  9. 9.
    Ong, C.S., Huang, J.J., Tzeng, G.H.: Building credit scoring models using genetic programming. Expert Syst. Appl. 29(1), 41–47 (2005)CrossRefGoogle Scholar
  10. 10.
    Souffriau, W., Vansteenwegen, P., Berghe, G.V., Van Oudheusden, D.: Automated parameterisation of a metaheuristic for the orienteering problem. In: Cotta, C., Sevaux, M., Sörensen, K. (eds.) Adaptive and Multilevel Metaheuristics. SCI, vol. 136, pp. 255–269. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Barros, R.C., Basgalupp, M.P., de Carvalho, A.C., Freitas, A.A.: Automatic design of decision-tree algorithms with evolutionary algorithms. Evol. Comput. 21(4), 659–684 (2013)CrossRefGoogle Scholar
  12. 12.
    Diosan, L.S., Oltean, M.: Evolving evolutionary algorithms using evolutionary algorithms. In: Proceedings of the 9th Annual Conference Companion on Genetic and Evolutionary Computation, pp. 2442–2449. ACM (2007)Google Scholar
  13. 13.
    Goldberg, D.E.: Genetic Algorithms. Pearson Education India, New Delhi (2006)Google Scholar
  14. 14.
    Eiben, A.E., Smith, J.E., et al.: Introduction to Evolutionary Computing, vol. 53. Springer, Heidelberg (2003)CrossRefMATHGoogle Scholar
  15. 15.
    Aitkenhead, M.: A co-evolving decision tree classification method. Expert Syst. Appl. 34(1), 18–25 (2008)CrossRefGoogle Scholar
  16. 16.
    Agnelli, D., Bollini, A., Lombardi, L.: Image classification: an evolutionary approach. Pattern Recogn. Lett. 23(1), 303–309 (2002)CrossRefMATHGoogle Scholar
  17. 17.
    Cios, K.J., Swiniarski, R.W., Pedrycz, W., Kurgan, L.A.: The knowledge discovery process. In: Data Mining, pp. 9–24. Springer, New York (2007)Google Scholar
  18. 18.
    Bhowan, U., Zhang, M., Johnston, M.: Genetic programming for classification with unbalanced data. In: Esparcia-Alcázar, A.I., Ekárt, A., Silva, S., Dignum, S., Uyar, A.Ş. (eds.) EuroGP 2010. LNCS, vol. 6021, pp. 1–13. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-12148-7_1CrossRefGoogle Scholar
  19. 19.
    Frank, A., Asuncion, A., et al.: Uci machine learning repository (2010)Google Scholar
  20. 20.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetMATHGoogle Scholar
  21. 21.
    Yao, X., Liu, Y.: Ensemble structure of evolutionary artificial neural networks. In: Proceedings of IEEE International Conference on Evolutionary Computation, pp. 659–664. IEEE (1996)Google Scholar
  22. 22.
    Cano, A., Ventura, S., Cios, K.J.: Multi-objective genetic programming for feature extraction and data visualization. Soft Comput., 1–21 (2015)Google Scholar
  23. 23.
    Raymer, M.L., Punch, W.F., Goodman, E.D., Kuhn, L.A., Jain, A.K.: Dimensionality reduction using genetic algorithms. IEEE Trans. Evol. Comput. 4(2), 164–171 (2000)CrossRefGoogle Scholar
  24. 24.
    Bojarczuk, C.C., Lopes, H.S., Freitas, A.A., Michalkiewicz, E.L.: A constrained-syntax genetic programming system for discovering classification rules: application to medical data sets. Artif. Intell. Med. 30(1), 27–48 (2004)CrossRefGoogle Scholar
  25. 25.
    Akay, M.F.: Support vector machines combined with feature selection for breast cancer diagnosis. Expert Syst. Appl. 36(2), 3240–3247 (2009)CrossRefGoogle Scholar
  26. 26.
    Bylander, T.: Estimating generalization error on two-class datasets using out-of-bag estimates. Mach. Learn. 48(1–3), 287–297 (2002)CrossRefMATHGoogle Scholar
  27. 27.
    Espejo, P.G., Romero, C., Ventura, S., Hervás, C.: Induction of classification rules with grammar-based genetic programming. In: Conference on Machine Intelligence, pp. 596–601 (2005)Google Scholar
  28. 28.
    Kahramanli, H., Allahverdi, N.: Design of a hybrid system for the diabetes and heart diseases. Expert Syst. Appl. 35(1), 82–89 (2008)CrossRefGoogle Scholar
  29. 29.
    Cao, V.L., Le-Khac, N.-A., O’Neill, M., Nicolau, M., McDermott, J.: Improving fitness functions in genetic programming for classification on unbalanced credit card data. In: Squillero, G., Burelli, P. (eds.) EvoApplications 2016. LNCS, vol. 9597, pp. 35–45. Springer, Cham (2016). doi: 10.1007/978-3-319-31204-0_3CrossRefGoogle Scholar
  30. 30.
    West, D.: Neural network credit scoring models. Comput. Oper. Res. 27(11), 1131–1152 (2000)CrossRefMATHGoogle Scholar
  31. 31.
    Jabeen, H., Baig, A.R.: Depthlimited crossover in GP for classifier evolution. Comput. Hum. Behav. 27(5), 1475–1481 (2011)CrossRefGoogle Scholar
  32. 32.
    Raymer, M.L., Doom, T.E., Kuhn, L.A., Punch, W.F.: Knowledge discovery in medical and biological datasets using a hybrid bayes classifier/evolutionary algorithm. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 33(5), 802–813 (2003)Google Scholar
  33. 33.
    Mangasarian, O.L., Musicant, D.R.: Lagrangian support vector machines. J. Mach. Learn. Res. 1, 161–177 (2001)MathSciNetMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of Mathematics, Statistics and Computer ScienceUniversity of KwaZulu-NatalPietermaritzburgSouth Africa

Personalised recommendations