Genetic Programming and Evolvable Machines

, Volume 12, Issue 2, pp 91–119

Semantically-based crossover in genetic programming: application to real-valued symbolic regression

  • Nguyen Quang Uy
  • Nguyen Xuan Hoai
  • Michael O’Neill
  • R. I. McKay
  • Edgar Galván-López
Article

Abstract

We investigate the effects of semantically-based crossover operators in genetic programming, applied to real-valued symbolic regression problems. We propose two new relations derived from the semantic distance between subtrees, known as semantic equivalence and semantic similarity. These relations are used to guide variants of the crossover operator, resulting in two new crossover operators—semantics aware crossover (SAC) and semantic similarity-based crossover (SSC). SAC, was introduced and previously studied, is added here for the purpose of comparison and analysis. SSC extends SAC by more closely controlling the semantic distance between subtrees to which crossover may be applied. The new operators were tested on some real-valued symbolic regression problems and compared with standard crossover (SC), context aware crossover (CAC), Soft Brood Selection (SBS), and No Same Mate (NSM) selection. The experimental results show on the problems examined that, with computational effort measured by the number of function node evaluations, only SSC and SBS were significantly better than SC, and SSC was often better than SBS. Further experiments were also conducted to analyse the perfomance sensitivity to the parameter settings for SSC. This analysis leads to a conclusion that SSC is more constructive and has higher locality than SAC, NSM and SC; we believe these are the main reasons for the improved performance of SSC.

Keywords

Genetic programming Semantics Crossover Symbolic regression locality 

References

  1. 1.
    C. Alan, Meaning and language: an introduction to semantics and pragmatics. (Oxford Textbooks in Linguistics, Cambridge, 2004)Google Scholar
  2. 2.
    L. Altenberg, in Advances in Genetic Programming, ed. by K. E. Kinnear, Jr., The evolution of evolvability in genetic programming, chap. 3 (MIT Press, Cambridge, 1994), pp. 47–74Google Scholar
  3. 3.
    C. Baier, J.P. Katoen, Principles of Model Checking (MIT Press, Cambridge, 2008).MATHGoogle Scholar
  4. 4.
    L. Beadle, C. Johnson, Semantically driven crossover in genetic programming, in Proceedings of the IEEE World Congress on Computational Intelligence (IEEE Press, New York, 2008), pp. 111–116Google Scholar
  5. 5.
    L. Beadle, C.G. Johnson, Semantic analysis of program initialisation in genetic programming. Genet. Program. Evol. Mach. 10(3), 307–337 (2009)CrossRefGoogle Scholar
  6. 6.
    L. Beadle, C. G. Johnson, Semantically driven mutation in genetic programming. in 2009 IEEE Congress on Evolutionary Computation, ed. by A. Tyrrell (IEEE Computational Intelligence Society, IEEE Press, Trondheim, Norway, 18–21 May 2009), pp. 1336–1342Google Scholar
  7. 7.
    R.E. Bryant, Graph-based algorithms for Boolean function manipulation. IEEE Trans. Comp. C-35, 677–691 (1986)CrossRefGoogle Scholar
  8. 8.
    E.K. Burke, S. Gustafson, G. Kendall, Diversity in genetic programming: an analysis of measures and correlation with fitness. IEEE Trans. Evol. Comput. 8(1), 47–62 (2004)CrossRefGoogle Scholar
  9. 9.
    R. Cleary, M. O’Neill, Solving knapsack problems with attribute grammars, in Proceedings of the Grammatical Evolution Workshop, 2004Google Scholar
  10. 10.
    R. Cleary, M. O’Neill, in Proceedings of the Evolutionary Computation in Combinatorial Optimization. An attribute grammar decoder for the 01 multi-constrained knapsack problem (Springer, Berlin, 2005), pp. 34–45Google Scholar
  11. 11.
    A.M. Collins, M.R. Quillian, Retrieval time from semantic memory. J. Verbal Learn.Verbal Behav. 8, 240–247 (1969)CrossRefGoogle Scholar
  12. 12.
    J.M. Daida, D.S. Ampy, M. Ratanasavetavadhana, H. Li, O. Chaudhri, in Proceedings of the Genetic and Evolutionary Computation Conference, (GECCO’1999). Challenges with verification, repeatability, and meaningful comparison in genetic programming: Gibson’s magic (Morgan Kaufmann, 1999), pp. 1851–1858Google Scholar
  13. 13.
    M. de la Cruz Echeand’a, A.O. de la Puente, M. Alfonseca, in Proceedings of the IWINAC 2005. Attribute grammar evolution (Springer, Berlin, 2005), pp. 182–191Google Scholar
  14. 14.
    K. Deb, H.G. Beyer, in Proceedings of the Genetic and Evolutionary Computation Conference. Self-adaptation in real-parameter genetic algorithms with simulated binary crossover (Morgan Kaufmann, July 1999), pp. 172–179Google Scholar
  15. 15.
    E. Galvan-Lopez, M. O’Neill, in CIG. On the effects of locality in a permutation problem: the sudoku problem (IEEE, 2009)Google Scholar
  16. 16.
    E. Galvan-Lopez, M. O’Neill, in MICAI, Lecture Notes in Computer Science. Towards understanding the effects of locality in genetic programming (Springer, Berlin, 2009)Google Scholar
  17. 17.
    J. Gottlieb, G. Raidl, in Proceedings of the Genetic and Evolutionary Computation Conference. The effects of locality on the dynamics of decoder-based evolutionary search (ACM, 2000), p. 283–290Google Scholar
  18. 18.
    S. Gustafson, E.K. Burke, N. Krasnogor, in Proceedings of the 2005 IEEE Congress on Evolutionary Computation. On improving genetic programming for symbolic regression, vol. 1. (IEEE Press, Edinburgh, 2005), pp. 912–919Google Scholar
  19. 19.
    S. Hengpraprohm, P. Chongstitvatana, in Proceedings of ISCIT International Symposium on Communications and Information Technologies. Selective crossover in genetic programming, Nov 2001, pp. 14–16Google Scholar
  20. 20.
    N.X. Hoai, R. McKay, D. Essam, in Proceedings of the 2002 Congress on Evolutionary Computation (CEC2002). Solving the symbolic regression problem with tree-adjunct grammar guided genetic programming: the comparative results (IEEE Press, 2002), pp. 1326–1331Google Scholar
  21. 21.
    N.X. Hoai, R.I. McKay, D. Essam, Representation and structural difficulty in genetic programming. IEEE Trans. Evol. Comput. 10(2), 157–166 (2006)CrossRefGoogle Scholar
  22. 22.
    N.X. Hoai, R.I.B. McKay, D. Essam, H. Abbass, in Genetic Programming 7th European Conference, EuroGP 2004, Proceedings, vol. 3003 of LNCS, ed. by M. Keijzer, U.-M. O’Reilly, S.M. Lucas, E. Costa, T. Soule, Toward an alternative comparison between different genetic programming systems (Springer, Berlin, 2004), pp. 67–77Google Scholar
  23. 23.
    T.-H. Hoang, D. Essam, R.I.B. McKay, X.H. Nguyen, in Proceedings of the 2007 International Symposium on Intelligent Computation and Applications (ISICA). Building on success in genetic programming:adaptive variation & developmental evaluation (China University of Geosciences Press, Wuhan, China, Sep 2007)Google Scholar
  24. 24.
    T. Ito, H. Iba, S. Sato, in Proceedings of the 1998 IEEE World Congress on Computational Intelligence. Depth-dependent crossover for genetic programming (IEEE Press, May 1998), pp. 775–780Google Scholar
  25. 25.
    T. Ito, H. Iba, S. Sato, in Advances in Genetic Programming. A self-tuning mechanism for depth-dependent crossover (IEEE Press, June 1999), pp. 377–399Google Scholar
  26. 26.
    C. Johnson, in Proceedings of the 4th European Conference on Genetic Programming (EuroGP2002). Deriving genetic programming fitness properties by static analysis (Springer, Berlin, 2002), pp. 299–308Google Scholar
  27. 27.
    C. Johnson, in Recent Advances in Soft Computing. Genetic programming with guaranteed constraints (The Nottingham Trent University, UK, 2002), pp. 134–140Google Scholar
  28. 28.
    C. Johnson, in Proceedings of the UK Workshop on Computational Intelligence. What can automatic programming learn from theoretical computer science (University of Birmingham, Birmingham, 2002)Google Scholar
  29. 29.
    C. Johnson, in Proceedings of the 10th European Conference on Genetic Programming (EuroGP2002). Genetic programming with fitness based on model checking (Springer, Berlin, 2007), pp. 114–124Google Scholar
  30. 30.
    C. Johnson, in Proceedings of the 12th European Conference on Genetic Programming (EuroGP2009). Genetic programming crossover: Does it cross over? (Springer, Berlin, 2009), pp. 97–108Google Scholar
  31. 31.
    G. Katz, D. Peled, Genetic programming and model checking: Synthesizing new mutual exclusion algorithms. Automated technology for verification and analysis. Lect. Notes Comput. Sci. 5311, 33–47 (2008)CrossRefGoogle Scholar
  32. 32.
    G. Katz, D. Peled, Model checking-based genetic programming with an application to mutual exclusion. Tools Algorithm. Constr. Anal. Syst. 4963, 141–156 (2008)CrossRefGoogle Scholar
  33. 33.
    M. Keijzer, in Proceedings of EuroGP’2003. Improving symbolic regression with interval arithmetic and linear scaling, Springer, Berlin, April 2003), pp. 70–82Google Scholar
  34. 34.
    D. Knuth, Semantics of context-free languages. Math. Syst. Theory. 2 95 (1968)MathSciNetCrossRefGoogle Scholar
  35. 35.
    J. Koza, Genetic Programming: On the Programming of Computers by Natural Selection (MIT Press, Cambridge, 1992)MATHGoogle Scholar
  36. 36.
    J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (The MIT Press, Cambridge, 1992)MATHGoogle Scholar
  37. 37.
    K. Krawiec, P. Lichocki, in Genetic and Evolutionary Computation Conference, GECCO 2009, Proceedings, Montreal, Québec, Canada, July 8–12, 2009, ed. by F. Rothlauf. Approximating geometric crossover in semantic space (ACM, New York, 2009), pp. 987–994Google Scholar
  38. 38.
    K. Krawiec, B. Wieloch, in GECCO ’09: Proceedings of the 11th Annual conference on Genetic and evolutionary computation. Functional modularity for genetic programming (ACM, Montreal, July 2009), pp. 995–1002Google Scholar
  39. 39.
    W.B. Langdon, in Proceedings of the Genetic and Evolutionary Computation Conference. Size fair and homologous tree genetic programming crossovers (Morgan Kaufmann, July 1999), pp. 1092–1097Google Scholar
  40. 40.
    W. B. Langdon, R. Poli, Foundations of Genetic Programming (Springer, Berlin, 2002)MATHGoogle Scholar
  41. 41.
    H. Majeed, C. Ryan, in Proceedings of the 9th European Conference on Genetic Programming. A less destructive, context-aware crossover operator for gp, Lecture Notes in Computer Science (Springer, Berlin, April 2006), pp. 36–48Google Scholar
  42. 42.
    H. Majeed, C. Ryan, in Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation (GECCO). On the constructiveness of context-aware crossover (ACM Press, New York, July 2007), pp. 1659–1666Google Scholar
  43. 43.
    N. McPhee, B. Ohs, T. Hutchison, in Proceedings of 11th European Conference on Genetic Programming. Semantic building blocks in genetic programming (Springer, Berlin, 2008) , pp. 134–145Google Scholar
  44. 44.
    N. Mori, B. McKay, N.X. Hoai, D. Essam, S. Takeuchi, A new method for simplifying algebraic expressions in genetic programming called equivalent decision simplification. J. Adv. Comput. Intell. Intell. Inform. 13(3), 237–244 (2009)Google Scholar
  45. 45.
    F. Nielson, H.R. Nielson, C. Hankin, Principles of Program Analysis. (Springer, Berlin, 2005)MATHGoogle Scholar
  46. 46.
    H.R. Nielson, F. Nielson, Semantics with Applications: An Appetizer (Springer, London, 2007)MATHCrossRefGoogle Scholar
  47. 47.
    U.M. O’Reilly, F. Oppacher, Program search with a hierarchical variable length representation: genetic programming, simulated annealing and hill climbing. Lect. Notes Comput. Sci. 866(1), 397–406 (1994)Google Scholar
  48. 48.
    R. Poli, W.B. Langdon, in Proceedings of Soft Computing in Engineering Design and Manufacturing Conference. Genetic programming with one-point crossover (Springer, Berlin, June 1997), pp. 180–189Google Scholar
  49. 49.
    R. Poli, W.B. Langdon, N.F. McPhee, A Field Guide to Genetic Programming. Published via http://lulu.com and freely available at http://www.gp-field-guide.org.uk, 2008. (With contributions by J. R. Koza).
  50. 50.
    B.J. Ross, Logic-based genetic programming with definite clause translation grammars. New Gen. Comput. 19(4), 313–337 (2001)MATHCrossRefGoogle Scholar
  51. 51.
    F. Rothlauf, Representations for Genetic and Evolutionary Algorithms, 2nd edn. (Springer, Berlin, 2006)Google Scholar
  52. 52.
    F. Rothlauf, D. Goldberg, Redundant representations in evolutionary algorithms. Evol. Comput. 11(4), 381–415 (2003)CrossRefGoogle Scholar
  53. 53.
    F. Rothlauf, M. Oetzel, in Proceedings of the 9th European Conference on Genetic Programming. On the locality of grammatical evolution, lecture notes in computer science (Springer, Berlin, April 2006), pp. 320–330Google Scholar
  54. 54.
    R.P. Salustowicz, J. Schmidhuber, Probabilistic incremental program evolution. Evol. Comput. 5(2), 123–141 (1997)CrossRefGoogle Scholar
  55. 55.
    W.A. Tackett, Selection, and the Genetic Construction of Computer Programs. PhD thesis, University of Southern California, USA, 1994Google Scholar
  56. 56.
    W.A. Tackett, A. Carmi, in Proceedings of the 1994 IEEE World Congress on Computational Intelligence. The unique implications of brood selection for genetic programming (IEEE Press, Orlando, Florida, USA, 27–29 June 1994)Google Scholar
  57. 57.
    N.Q. Uy, N.X. Hoai, M. O’Neill, in Proceedings of EuroGP09. Semantic aware crossover for genetic programming: the case for real-valued function regression (Springer, Berlin, April 2009), pp. 292–302.Google Scholar
  58. 58.
    M.L. Wong, K.S. Leung, in Proceedings of the 7th IEEE International Conference on Tools with Artificial Intelligence. An induction system that learns programs in different programming languages using genetic programming and logic grammars (1995)Google Scholar
  59. 59.
    M.L. Wong, K.S. Leung, in Proceedings of the Fourth Congress of the Italian Association for Artificial Intelligence. Learning programs in different paradigms using genetic programming (Springer, Berlin, 1995)Google Scholar
  60. 60.
    P. Wong, M. Zhang, 2008 IEEE World Congress on Computational Intelligence, ed. by J. Wang. SCHEME: caching subtrees in genetic programming (IEEE Computational Intelligence Society, IEEE Press, Hong Kong, 1–6 June 2008)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Nguyen Quang Uy
    • 1
  • Nguyen Xuan Hoai
    • 2
  • Michael O’Neill
    • 1
  • R. I. McKay
    • 3
  • Edgar Galván-López
    • 1
  1. 1.Complex & Adaptive Systems Lab, School of Computer Science & InformaticsUniversity College DublinDublinIreland
  2. 2.Department of Computer ScienceLe Quy Don UniversityHanoiVietnam
  3. 3.School of Computer Science and EngineeringSeoul National UniversitySeoulKorea

Personalised recommendations