Genetic Programming and Evolvable Machines

, Volume 9, Issue 3, pp 203–228

Evolving encapsulated programs as shared grammars

Original Paper

Abstract

Facilitating the discovery and reuse of modular building blocks is generally regarded as the key to achieving better scalability in genetic programming (GP). A precedent for this exists in biology, where complex designs are the product of developmental processes that can also be abstractly modeled as generative grammars. We introduce shared grammar evolution (SGE), which aligns grammatical development with the common application of grammars in GP as a means of establishing declarative bias. Programs are derived from and represented by a global context-free grammar that is transformed and extended according to another, user-defined grammar. Grammatical productions and the subroutines they encapsulate are shared between programs, which enables their reuse without reevaluation and can significantly reduce total evaluation time for large programs and populations. Several variants of SGE employing different strategies for controlling solution size and diversity are tested on classic GP problems. Results compare favorably against GP and newer techniques, with the best results obtained by promoting diversity between derived programs.

Keywords

Genetic programming Grammatical evolution Shared grammars Developmental systems Encapsulation Modularity Memoization 

References

  1. 1.
    H.A. Abbass, N.X. Hoai, R.I. McKay, Anttag: a new method to compose computer programs using colonies of ants, in Proceedings of the 2002 IEEE Congress on Evolutionary Computation (IEEE Press, 2002), pp. 1654–1659Google Scholar
  2. 2.
    A. Agapitos, S.M. Lucas, Learning recursive functions with object oriented genetic programming, in Proceedings of the 9th European Conference on Genetic Programming. Lecture Notes in Computer Science, vol. 3905 (Springer, 2006), pp. 166–177 Google Scholar
  3. 3.
    P.J. Angeline, J.B. Pollack, Evolutionary module acquisition, in Proceedings of the 2nd Annual Conference on Evolutionary Programming (MIT Press, 1993), pp. 154–163Google Scholar
  4. 4.
    K.M.S. Badran, P.I. Rockett, The roles of diversity preservation and mutation in preventing population collapse in multiobjective genetic programming, in Proceedings of the 2007 Genetic and Evolutionary Computation Conference, vol. 2 (ACM Press, 2007), pp. 1551–1558Google Scholar
  5. 5.
    W. Banzhaf, Genotype-phenotype-mapping and neutral variation—a case study in genetic programming. In Parallel Problem Solving from Nature III. Lecture Notes in Computer Science, vol. 866 (Springer-Verlag, 1994), pp. 322–332Google Scholar
  6. 6.
    T. Bersano-Begey, Controlling exploration, diversity and escaping local optima in gp. In Late Breaking Papers at the Genetic Programming Conference (MIT Press, 1997), pp. 7–10Google Scholar
  7. 7.
    E.J.W. Boers, I.G. Sprinkhuizen-Kuyper, in Advances in the Evolutionary Synthesis of Intelligent Agents, chapter 6, ed. by M.J. Patel, V. Honavar, K. Balakrishnan, (MIT Press, Cambridge, MA, 2001), pp. 153–183Google Scholar
  8. 8.
    E.K. Burke, S.M. Gustafson, G. Kendall, N. Krasnogor, Is increased diversity in genetic programming beneficial? An analysis of lineage selection, in Proceedings of the 2003 IEEE Congress on Evolutionary Computation (IEEE Press, 2003), pp. 1398–1405Google Scholar
  9. 9.
    Y.-P. Chen, T.-L. Yu, K. Sastry, D. E. Goldberg, A survey of linkage learning techniques in genetic and evolutionary algorithms. IlliGAL Report 2007014 (Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, USA, 2007)Google Scholar
  10. 10.
    J.M. Daida, R.R. Bertram, S.A. Stanhope, J.C. Khoo, S.A. Chaudhary, O.A. Chaudhri, J.A. Polito II, What makes a problem GP-hard? Analysis of a tunably difficult problem in genetic programming. Genet. Program. Evolvable Mach. 2(2), 165–191 (2001)CrossRefMATHGoogle Scholar
  11. 11.
    E.D. De Jong, J.B. Pollack, Multi-objective methods for tree size control. Genet. Program. Evolvable Mach. 4(3), 211–233 (2003)CrossRefGoogle Scholar
  12. 12.
    E.D. De Jong, R.A. Watson, J.B. Pollack, Reducing bloat and promoting diversity using multiobjective methods, in Proceedings of the 2001 Genetic and Evolutionary Computation Conference (Morgan Kaufmann, 2001), pp. 11–18Google Scholar
  13. 13.
    K. Deb, Multi-objective Optimization Using Evolutionary Algorithms (Wiley, Chichester, 2001)MATHGoogle Scholar
  14. 14.
    K. Deb, S. Agrawal, A. Pratab, T. Meyarivan, A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II, in Proceedings of the Parallel Problem Solving from Nature VI Conference. Lecture Notes in Computer Science, vol. 1917 (Springer-Verlag, 2000), pp. 849–858Google Scholar
  15. 15.
    M. Dorigo, L.M. Gambardella, Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Trans. Evol. Comput. 1(1), 53–66 (1997)CrossRefGoogle Scholar
  16. 16.
    R.M. Downing, Neutrality and gradualism: encouraging exploration and exploitation simultaneously with binary decision diagrams, in Proceedings of the 2006 IEEE Congress on Evolutionary Computation (IEEE Press, 2006), pp. 615–622Google Scholar
  17. 17.
    M. Ebner, P. Langguth, J. Albert, S. Mark, R. Shipman, On neutral networks and evolvability, in Proceedings of the 2001 IEEE Congress on Evolutionary Computation (IEEE Press, 2001), pp. 1–8Google Scholar
  18. 18.
    A. Ekárt, S.Z. Németh, Selection based on the pareto nondomination criterion for controlling code growth in genetic programming. Genet. Program. Evolvable Mach. 2(1), 61–73 (2001)CrossRefMATHGoogle Scholar
  19. 19.
    A. Ekárt, S.Z. Németh, Maintaining the diversity of genetic programs, in Proceedings of the 5th European Conference on Genetic Programming. LNCS, vol. 2278. (Springer-Verlag, 2002), pp. 162–171Google Scholar
  20. 20.
    D.J. Futuyma, Evolutionary Biology, vol. 3rd (Sinauer Associates, Inc., Sunderland, MA, 1998)Google Scholar
  21. 21.
    D.E. Goldberg, J. Richardson, Genetic algorithms with sharing for multimodal function optimization, in Proceedings of the 2nd International Conference on Genetic Algorithms and their Applications (Lawrence Erlbaum Associates, 1987), pp. 41–49Google Scholar
  22. 22.
    S.M. Gustafson, An Analysis of Diversity in Genetic Programming. PhD Thesis, University of Nottingham, 2004Google Scholar
  23. 23.
    T.F. Hansen, The evolution of genetic architecture. Ann. Rev. Ecol. Evol. Syst. 37(1), 123–157 (2006)CrossRefGoogle Scholar
  24. 24.
    I. Harvey, A. Thompson, Through the labyrinth evolution finds a way: a silicon ridge, in Proceedings of the First International Conference on Evolvable Systems: From Biology to Hardware (Springer-Verlag, 1996), pp. 406–422Google Scholar
  25. 25.
    N.X. Hoai, R.I. McKay, H.A. Abbass, Tree adjoining grammars, language bias, and genetic programming, in Proceedings of the 6th European Conference on Genetic Programming. Lecture Notes in Computer Science, vol. 2610 (Springer-Verlag, 2003), pp. 335–344 Google Scholar
  26. 26.
    J. Holland, Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence, vol. 2 (MIT Press, Cambridge, 1992)Google Scholar
  27. 27.
    G.S. Hornby, Generative Representations for Evolutionary Design Automation. PhD Thesis, Brandeis University, 2003Google Scholar
  28. 28.
    D. Howard, in Genetic Programming Theory and Practise, chapter 10, ed. by R.L. Riolo, B. Worzel (Kluwer, 2003), pp. 155–172Google Scholar
  29. 29.
    N. Jakobi, in On Growth, Form and Computers, ed. by P.J. Bentley, S. Kumar, (Academic Press, London, 2003), pp. 392–404CrossRefGoogle Scholar
  30. 30.
    M. Keijzer, C. Ryan, M. Cattolico, Run transferable libraries—learning functional bias in problem domains, in Proceedings of the 2004 Genetic and Evolutionary Computation Conference. Lecture Notes in Computer Science, vol. 3103 (Springer, 2004), pp. 531–542 Google Scholar
  31. 31.
    H. Kitano, Designing neural networks using genetic algorithms with graph generation systems. Complex Syst. 4(4), 461–476 (1990)MATHGoogle Scholar
  32. 32.
    J.D. Knowles, R.A. Watson, On the utility of redundant encodings in mutation-based evolutionary search. In Parallel Problem Solving from Nature VII. Lecture Notes in Computer Science, vol. 2439 (Springer-Verlag, 2002), pp. 88–98Google Scholar
  33. 33.
    J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (The MIT Press, Cambridge, 1992)MATHGoogle Scholar
  34. 34.
    J.R. Koza, Genetic Programming II: Automatic Discovery of Reusable Programs (MIT Press, Cambridge, 1994)MATHGoogle Scholar
  35. 35.
    J.R. Koza, Evolving the architecture of a multi-part program in genetic programming using architecture-altering operations, in Proceedings of the 4th Annual Conference on Evolutionary Programming (MIT Press, 1995), pp. 695–717Google Scholar
  36. 36.
    J.R. Koza, F.H. Bennett III, D. Andre, M.A. Keane, Genetic Programming III. Darwinian Invention and Problem Solving (Morgan Kaufmann, San Francisco, 1999)MATHGoogle Scholar
  37. 37.
    W.B. Langdon, R. Poli, Fitness causes bloat. In Second On-line World Conference on Soft Computing in Engineering Design and Manufacturing (Springer-Verlag, 1997), pp. 13–22 Google Scholar
  38. 38.
    W.B. Langdon, R. Poli, Foundations of Genetic Programming (Springer-Verlag, London, 2002)CrossRefMATHGoogle Scholar
  39. 39.
    A. Lindenmayer. Mathematical models for cellular interaction in development, parts I and II. J. Theor. Biol. 18, 280–315 (1968)CrossRefGoogle Scholar
  40. 40.
    M.H. Luerssen, Phenotype diversity objectives for graph grammar evolution. In Recent Advances in Artificial Life, ed. by H.A. Abbass, T. Bossamaier, J. Wiles, Advances in Natural Computation, vol. 3, chapter 12 (World Scientific, Singapore, 2005), pp. 159–170CrossRefGoogle Scholar
  41. 41.
    M.H. Luerssen, D.M.W. Powers, Evolvability and redundancy in shared grammar evolution, in Proceedings of the 2007 IEEE Congress on Evolutionary Computation (IEEE Press, 2007), pp. 370–377Google Scholar
  42. 42.
    R.I. McKay, Fitness sharing in genetic programming, in Proceedings of the 2000 Genetic and Evolutionary Computation Conference (Morgan Kaufmann, 2000), pp. 435–442Google Scholar
  43. 43.
    R.I. McKay, H.A. Abbass, Anticorrelation measures in genetic programming. In Australasia-Japan Workshop on Intelligent and Evolutionary Systems (2001), pp. 45–51Google Scholar
  44. 44.
    N.F. McPhee, J.D. Miller, Accurate replication in genetic programming, in Proceedings of the 6th International Conference on Genetic Algorithms (Morgan Kaufmann, 1995), pp. 303–309Google Scholar
  45. 45.
    J.F. Miller, What bloat? Cartesian genetic programming on Boolean problems. In Late Breaking Papers of the 2001 Genetic and Evolutionary Computation Conference (ISGEC Press, 2001), pp. 295–302Google Scholar
  46. 46.
    J.F. Miller, S.L. Smith, Redundancy and computational efficiency in cartesian genetic programming. IEEE Trans. Evol. Comput. 10(2), 167–174 (2006)CrossRefGoogle Scholar
  47. 47.
    J.F. Miller, P. Thomson, Cartesian genetic programming, in Proceedings of the 3rd European Conference on Genetic Programming. Lecture Notes in Computer Science, vol. 1802 (Springer-Verlag, 2000), pp. 121–132Google Scholar
  48. 48.
    V.B. Mountcastle, The columnar organization of the neocortex. Brain, 120, 701–722 (1997)CrossRefGoogle Scholar
  49. 49.
    H. Mühlenbein, G. Paaß, From recombination of genes to the estimation of distributions I, binary parameters. In Parallel Problem Solving from Nature IV. Lecture Notes in Computer Science, vol. 1411 (Springer, 1996), pp. 178–187Google Scholar
  50. 50.
    M. O’Neill, mGGA: The meta-grammar genetic algorithm, in Proceedings of the 8th European Conference on Genetic Programming, Lausanne, Switzerland. Lecture Notes in Computer Science, vol. 3447 (Springer Verlag, 2005), pp. 311–320Google Scholar
  51. 51.
    M. O’Neill, C. Ryan, Grammar based function definition in grammatical evolution, in Proceedings of the 5th Annual Conference on Genetic Programming (MIT Press, 2000), pp. 485–490Google Scholar
  52. 52.
    M. O’Neill, C. Ryan, Grammatical evolution by grammatical evolution: the evolution of grammar and genetic code, in Proceedings of the 7th European Conference on Genetic Programming. Lecture Notes in Computer Science, vol. 3003 (Springer, 2004), pp. 138–149Google Scholar
  53. 53.
    M. O’Neill, C. Ryan, M. Keijzer, M. Cattolico, Crossover in grammatical evolution. Genet. Program. Evolvable Mach. 4(1), 67–93 (2003)CrossRefMATHGoogle Scholar
  54. 54.
    U.-M. O’Reilly, Using a distance metric on genetic programs to understand genetic operators. In IEEE International Conference on Systems, Man, and Cybernetics, Computational Cybernetics and Simulation, vol. 5 (1997), pp. 4092–4097Google Scholar
  55. 55.
    B. Pakkenberg, H.J.G. Gundersen, Neocortical neuron number in humans: effect of sex and age, J. Comp. Neurol. 384(2), 312–320 (1997)CrossRefGoogle Scholar
  56. 56.
    N. Paterson, M. Livesey, Evolving caching algorithms in c by genetic programming, in Proceedings of the 2nd Annual Conference on Genetic Programming (Morgan Kaufmann, 1997), pp. 262–267Google Scholar
  57. 57.
    S.C. Roberts, D. Howard, J.R. Koza, Evolving modules in genetic programming by subtree encapsulation, in Proceedings of the 4th European Conference on Genetic Programming. Lecture Notes in Computer Science, vol. 2038 (Springer, 2001), pp. 160–175Google Scholar
  58. 58.
    E. Rodrigues, A. Pozo, Grammar-guided genetic programming and automatically defined functions. In Advances in Artificial Intelligence: Proceedings of the 16th Brazilian Symposium on Artificial Intelligence. Lecture Notes in Artificial Intelligence, vol. 2507 (2002) pp. 324–333Google Scholar
  59. 59.
    J.P. Rosca, Entropy-driven adaptive representation, in Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications (1995), pp. 23–32Google Scholar
  60. 60.
    J.P. Rosca, Towards automatic discovery of building blocks in genetic programming. In Working Notes for the AAAI Symposium on Genetic Programming (1995), pp. 78–85Google Scholar
  61. 61.
    C. Ryan, J.J. Collins, M. O’Neill, Grammatical evolution: evolving programs for an arbitrary language, in Proceedings of the 1st European Workshop on Genetic Programming. Lecture Notes in Computer Science, vol. 1391 (Springer-Verlag, 1998), pp. 83–95Google Scholar
  62. 62.
    C. Ryan, M. Keijzer, M. Cattolico, in Genetic Programming Theory and Practice II, chapter 7, ed. by U.-M. O’Reilly, T. Yu, R. L. Riolo, B. Worzel (Springer, Ann Arbor, 2005), pp. 103–120Google Scholar
  63. 63.
    Y. Shan, R.I. McKay, H.A. Abbass, D. Essam, Program evolution with explicit learning: a new framework for program automatic synthesis, in Proceedings of the 2003 IEEE Congress on Evolutionary Computation (IEEE Press, 2003), pp. 1639–1646Google Scholar
  64. 64.
    Y. Shan, R.I. McKay, R. Baxter, H.A. Abbass, D. Essam, H.X. Nguyen, Grammar model-based program evolution, in Proceedings of the 2004 IEEE Congress on Evolutionary Computation, vol. 1 (2004), pp. 478–485Google Scholar
  65. 65.
    R. Shipman, M. Schackleton, M. Ebner, R. Watson, Neutral search spaces for artificial evolution: a lesson from life. In Artificial Life: Proceedings of the Seventh International Conference on Artificial Life (MIT Press, 2000), pp. 162–169Google Scholar
  66. 66.
    S. Silva, J. Almeida, Dynamic maximum tree depth—a simple technique for avoiding bloat in tree-based GP, in Proceedings of the 2003 Genetic and Evolutionary Computation Conference. Lecture Notes in Computer Science, vol. 2724 (Springer-Verlag, 2003), pp. 1776–1787Google Scholar
  67. 67.
    H.A. Simon, The Sciences of the Artificial, 3rd edn. (MIT Press, Cambridge, 1996)Google Scholar
  68. 68.
    T. Smith, P. Husbands, M. O’Shea, Neutral networks in an evolutionary robotics search space, in Proceedings of the 2001 IEEE Congress on Evolutionary Computation (IEEE Press, 2001), pp. 136–143Google Scholar
  69. 69.
    M. Toussaint, C. Igel, Neutrality: a necessity for self-adaptation, in Proceedings of the 2002 IEEE Congress on Evolutionary Computation (2002), pp. 1354–1359Google Scholar
  70. 70.
    J.C. Venter et al., The sequence of the human genome. Science, 291(5507), 1304–1351 (2001)CrossRefGoogle Scholar
  71. 71.
    N. Wagner, Z. Michalewicz, Genetic programming with efficient population control for financial times series prediction. In Late Breaking Papers of the 2001 Genetic and Evolutionary Computation Conference, vol. 1 (ISGEC Press, 2001), pp. 458–462Google Scholar
  72. 72.
    R.A. Watson, Compositional Evolution: Interdisciplinary Investigations in Evolvability, Modularity, and Symbiosis. PhD Thesis, Brandeis University, 2002Google Scholar
  73. 73.
    P.A. Whigham, Grammatically-based genetic programming, in Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications (Morgan Kaufmann Publishers, 1995), pp. 33–41Google Scholar
  74. 74.
    M.L. Wong, Evolving recursive programs by using adaptive grammar based genetic programming. Genet. Program. Evolvable Mach. 6(4), 421–455 (2005)CrossRefGoogle Scholar
  75. 75.
    T. Yu, An Analysis of the Impact of Functional Programming Techniques on Genetic Programming. PhD Thesis, University College London, 1999Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.School of Informatics and EngineeringFlinders University of South AustraliaAdelaideAustralia

Personalised recommendations