Genetic Programming and Evolvable Machines

, Volume 16, Issue 4, pp 499–530 | Cite as

Prudent alignment and crossover of decision trees in genetic programming

Article

Abstract

Crossover is the central search operator responsible for navigating through unknown problem landscapes while at the same time the main conservation operator, which is supposed to preserve the already learned lessons. This paper is about a novel homologous decision tree crossover operator. Contrary to other tree crossover operators it defines the context for a decision tree node and elaborates a fast one-sample-based tree alignment procedure. The idea is to replace a sub-tree with a better one from the same context, as defined by the decision tree training process. This operator does not rely on the topological properties of the tree but rather on its behavioral properties. During empirical testing the new operator showed the best generalization capabilities.

Keywords

Genetic programming Decision trees Crossover  Context Alignment 

References

  1. 1.
    W. Banzhaf, P. Nordin, R. Keller, F. Francone, Genetic Programming—An Introduction (Morgan Kaufmann, San Francisco, 1998)MATHCrossRefGoogle Scholar
  2. 2.
    R. Barros, M. Basgalupp, A. de Carvalho, A. Freitas, A survey of evolutionary algorithms for decision-tree induction. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 42(3), 291–311 (2012)CrossRefGoogle Scholar
  3. 3.
    K. Bache, M. Lichman, UCI Machine Learning Repository (School of Information and Computer Sciences, University of California, Irvine, 2013). http://archive.ics.uci.edu/ml
  4. 4.
    J. Bongard, A probabilistic functional crossover operator for genetic programming, in Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, GECCO ’10 (ACM, New York, NY, 2010), pp. 925–932Google Scholar
  5. 5.
    L. Booker, D. Goldberg, J. Holland, Classifier systems and genetic algorithms. Artif. Intell. 40(1–3), 235–282 (1989)CrossRefGoogle Scholar
  6. 6.
    L. Breiman, J. Friedman, R. Olshen, C. Stone, Classification and Regression Trees (Wadsworth, Monterrey, CA, 1984)MATHGoogle Scholar
  7. 7.
    S.H. Cha, C. Tappert, A genetic algorithm for constructing compact binary decision trees. J. Pattern Recognit. Res. 4(1), 1–13 (2009)CrossRefGoogle Scholar
  8. 8.
    M. Defoin-Platel, M. Clergue, P. Collard, Maximum homologous crossover for linear genetic programming, in Genetic Programming, 6th European Conference, EuroGP 2003, Lecture Notes in Computer Science, vol. 2610, ed. by C. Ryan, T. Soule, M. Keijzer, E. Tsang, R. Poli, E. Costa (Springer, Berlin, 2003), pp. 194–203Google Scholar
  9. 9.
    K. DeJong, The Analysis of the Behaviour of a Class of Genetic adaptive Systems. Ph.D. thesis, (Department of Computer Science, University of Michigan, Ann Arbor, Michigan, 1975)Google Scholar
  10. 10.
    L. Devroye, A note on the height of binary search trees. J. ACM 33(3), 489–498 (1986)MATHMathSciNetCrossRefGoogle Scholar
  11. 11.
    P. D’haeseleer, Context preserving crossover in genetic programming, in Proceedings of the 1994 IEEE World Congress on Computational Intelligence, vol. 1 (IEEE Press, New York, 1994), pp. 256–261Google Scholar
  12. 12.
    J. Hernandez, B. Duval, J.K. Hao, A study of crossover operators for gene selection of microarray data, in Artificial Evolution, Lecture Notes in Computer Science, vol. 4926 (2008), pp. 243–254Google Scholar
  13. 13.
    H. Kennedy, C. Chinniah, P. Bradbeer, L. Morss, The construction and evaluation of decision trees: a comparison of evolutionary and concept learning methods, in Evolutionary Computing, AISB Workshop, Lecture Notes in Computer Science, vol. 1305, ed. by D. Corne, J. Shapiro (Springer, Berlin, 1997), pp. 147–162Google Scholar
  14. 14.
    J. Koza, Genetic Programming: On the Programming of Computers by Natural Selection (MIT Press, Cambridge, MA, 1992)MATHGoogle Scholar
  15. 15.
    W.B. Langdon, Size fair and homologous tree crossovers for tree genetic programming, in Genetic Programming and Evolvable Machines, vol. 1 (Kluwer, Boston, 2000), pp. 95–119Google Scholar
  16. 16.
    S. Luke, L. Panait, Is the perfect the enemy of the good, in Genetic and Evolutionary Computation Conference (Morgan Kaufmann, Los Altos, CA, 2002), pp. 820–828Google Scholar
  17. 17.
    R. MacCallum, Introducing a perl genetic programming system—and can meta-evolution solve the bloat problem? in Genetic Programming, Proceedings of EuroGP, Lecture Notes in Computer Science, vol. 2610 (Springer, Berlin, 2003), pp. 369–378Google Scholar
  18. 18.
    A. Moraglio, Towards a Geometric Unification of Evolutionary Algorithms. Ph.D. thesis (Department of Computer Science, University of Essex, 2007)Google Scholar
  19. 19.
    A. Moraglio, One-point geometric crossover, in Parallel Problem Solving from Nature, PPSN XI, Lecture Notes in Computer Science, vol. 6238 (Springer, Berlin, 2010), pp. 83–93Google Scholar
  20. 20.
    A. Moraglio, R. Poli, Topological interpretation of crossover. GECCO 1, 1377–1388 (2004)Google Scholar
  21. 21.
    N. Paterson, Genetic Programming with Context-Sensitive Grammars. Ph.D. thesis (School of Computer Science, University of St Andrews, Scotland, 2002)Google Scholar
  22. 22.
    R. Poli, W. Langdon, N. McPhee, A Field Guide to Genetic Programming (Lulu Enterprises, UK Ltd, 2008)Google Scholar
  23. 23.
    J. Quinlan, C4.5: Programs for Machine Learning (Morgan Kaufmann, San Francisco, 1993)Google Scholar
  24. 24.
    J. Quinlan, Decision trees and instance-based classifiers, in The Computer Science and Engineering Handbook (1996), pp. 521–535Google Scholar
  25. 25.
    K. Rasheed, H. Hirsh, Informed operators: speeding up genetic-algorithm-based design optimization using reduced models, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO) (Morgan Kaufmann, Los Altos, CA, 2000), pp. 628–635Google Scholar
  26. 26.
    W. Spears, Simple subpopulation schemes, in Proceedings of the Evolutionary Programming Conference (World Scientific, Singapore, 1994), pp. 296–307Google Scholar
  27. 27.
    M. Šprogar, P. Kokol, Š.H. Babič, V. Podgorelec, M. Zorman, Vector decision trees. Intell. Data Anal. 4, 305–321 (2000)MATHGoogle Scholar
  28. 28.
    W. Tackett, Recombination, selection, and the genetic construction of computer programs. Ph.D. thesis, (University of Southern California, Department of Electrical Engineering Systems, 1994)Google Scholar
  29. 29.
    A. Teller, M. Veloso, Pado: Learning Tree Structured Algorithms for Orchestration into an Object Recognition System. Technical Report (Department of Computer Science, Carnegie Mellon University, Pittsburgh, USA, 1995)Google Scholar
  30. 30.
    D. Wolpert, W. Macready, No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Faculty of Electrical Engineering and Computer ScienceUniversity of MariborMariborSlovenia

Personalised recommendations