Genetic Programming and Evolvable Machines

, Volume 10, Issue 3, pp 307–337 | Cite as

Semantic analysis of program initialisation in genetic programming

Original Paper

Abstract

Population initialisation in genetic programming is both easy, because random combinations of syntax can be generated straightforwardly, and hard, because these random combinations of syntax do not always produce random and diverse program behaviours. In this paper we perform analyses of behavioural diversity, the size and shape of starting populations, the effects of purely semantic program initialisation and the importance of tree shape in the context of program initialisation. To achieve this, we create four different algorithms, in addition to using the traditional ramped half and half technique, applied to seven genetic programming problems. We present results to show that varying the choice and design of program initialisation can dramatically influence the performance of genetic programming. In particular, program behaviour and evolvable tree shape can have dramatic effects on the performance of genetic programming. The four algorithms we present have different rates of success on different problems.

Keywords

Genetic programming Program initialisation Program semantics Program structure 

References

  1. 1.
    W.F. Punch, D. Zongker, E.D. Goodman, The royal tree problem, a benchmark for single and multiple population genetic programming, in Advances in Genetic Programming 2, Chap. 15, ed. by P.J. Angeline, K.E. Kinnear, Jr. (MIT Press, Cambridge, 1996), pp. 299–316Google Scholar
  2. 2.
    J.M. Daida, H. Li, R. Tang, A.M. Hilss, What makes a problem GP-hard? Validating a hypothesis of structural causes, in Genetic and Evolutionary Computation—GECCO-2003, Chicago, IL, ed. by E. Cantú-Paz, J.A. Foster, K. Deb, D. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M.A. Potter, A.C. Schultz, K. Dowsland, N. Jonoska, J. Miller. Lecture Notes in Computer Science, vol. 2724 (Springer-Verlag, Berlin, 2003), pp. 1665–1677Google Scholar
  3. 3.
    S. Gustafson, E.K. Burke, N. Krasnogor, The tree-string problem: an artificial domain for structure and content search, in Proceedings of the 8th European Conference on Genetic Programming, Lausanne, Switzerland, 30 March–1 April 2005, ed. by M. Keijzer, A. Tettamanzi, P. Collet, J.I. van Hemert, M. Tomassini. Lecture Notes in Computer Science, vol. 3447 (Springer, Berlin, 2005), pp. 215–226Google Scholar
  4. 4.
    J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (MIT Press, Cambridge, 1992)MATHGoogle Scholar
  5. 5.
    P. Nordin, F. Francone, W. Banzhaf, Explicitly defined introns and destructive crossover in genetic programming, in Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, Tahoe City, CA, 9 July 1995, ed. by J.P. Rosca, pp. 6–22Google Scholar
  6. 6.
    P.A. Whigham, Inductive bias and genetic programming, in First International Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications, GALESIA, Sheffield, UK, 12–14 September 1995, ed. by A.M.S. Zalzala, vol. 414 (IEE, Piscataway, 1995), pp. 461–466Google Scholar
  7. 7.
    P.A. Whigham, Grammatically-based genetic programming, in Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, Tahoe City, CA, 9 July 1995, ed. by J.P. Rosca, pp. 33–41Google Scholar
  8. 8.
    P.A. Whigham, Search bias, language bias, and genetic programming, in Genetic Programming 1996: Proceedings of the First Annual Conference, ed. by J.R. Koza, D.E. Goldberg, D.B. Fogel, R.L. Riolo, Stanford University, CA, 28–31 July 1996 (MIT Press, Cambridge), pp. 230–237Google Scholar
  9. 9.
    H. Iba, Random tree generation for genetic programming. Technical Report ETL-TR-95-35, 14 November 1995 (ElectroTechnical Laboratory (ETL), Tsukuba, Japan, 1995)Google Scholar
  10. 10.
    W. Bohm, A. Geyer-Schulz, Exact uniform initialization for genetic programming, in Foundations of Genetic Algorithms IV, University of San Diego, San Diego, CA, 3–5 August 1996, ed. by R.K. Belew, M. Vose (Morgan Kaufmann, San Francisco, 1996), pp. 379–407Google Scholar
  11. 11.
    S. Luke, L. Panait, A survey and comparison of tree generation algorithms, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), San Francisco, CA, 7–11 July 2001, ed. by L. Spector, E.D. Goodman, A. Wu, W.B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M.H. Garzon, E. Burke (Morgan Kaufmann, San Francisco, 2001), pp. 81–88Google Scholar
  12. 12.
    W.B. Langdon, Size fair and homologous tree genetic programming crossovers. Genet. Program. Evol. Mach. 1(1/2), 95–119 (2000)MATHCrossRefGoogle Scholar
  13. 13.
    K. Chellapilla, Evolving computer programs without subtree crossover. IEEE Trans. Evol. Comput. 1(3), 209–216 (1997)CrossRefGoogle Scholar
  14. 14.
    S. Luke, Two fast tree-creation algorithms for genetic programming. IEEE Trans. Evol. Comput. 4(3), 274–283 (2000)CrossRefGoogle Scholar
  15. 15.
    M. Looks, On the behavioral diversity of random programs, in GECCO ’07: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, London, 7–11 July 2007, vol. 2, ed. by D. Thierens, H.-G. Beyer, J. Bongard, J. Branke, J.A. Clark, D. Cliff, C.B. Congdon, K. Deb, B. Doerr, T. Kovacs, S. Kumar, J.F. Miller, J. Moore, F. Neumann, M. Pelikan, R. Poli, K. Sastry, K.O. Stanley, T. Stutzle, R.A. Watson, I. Wegener (ACM Press, New York, 2007), pp. 1636–1642Google Scholar
  16. 16.
    S. Gustafson, E.K. Burke, G. Kendall, Sampling of unique structures and behaviours in genetic programming, in Proceedings of the 7th European Conference on Genetic Programming, EuroGP 2004, Coimbra, Portugal, 5–7 April 2004, ed. by M. Keijzer, U.-M. O’Reilly, S.M. Lucas, E. Costa, T. Soule. Lecture Notes in Computer Science, vol. 3003 (Springer-Verlag, Berlin, 2004), pp. 279–288Google Scholar
  17. 17.
    E.K. Burke, S. Gustafson, G. Kendall, Diversity in genetic programming: an analysis of measures and correlation with fitness. IEEE Trans. Evol. Comput. 8(1), 47–62 (2004)CrossRefGoogle Scholar
  18. 18.
    S. Gustafson, An analysis of diversity in genetic programming. PhD thesis, School of Computer Science and Information Technology, University of Nottingham, Nottingham, England, 2004)Google Scholar
  19. 19.
    R. Poli, W.B. Langdon, On the search properties of different crossover operators in genetic programming, in Genetic Programming 1998: Proceedings of the Third Annual Conference, University of Wisconsin, Madison, WI, 22–25 July 1998, ed. by J.R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D.B. Fogel, M.H. Garzon, D.E. Goldberg, H. Iba, R. Riolo (Morgan Kaufmann, San Francisco, 1998), pp. 293–301Google Scholar
  20. 20.
    U.-M. O’Reilly, F. Oppacher, Program search with a hierarchical variable length representation: genetic programming, simulated annealing and hill climbing, in Parallel Problem Solving from Nature—PPSN III, Jerusalem, 9–14 October 1994, ed. by Y. Davidor, H.-P. Schwefel, R. Manner. Lecture Notes in Computer Science, vol. 866 (Springer-Verlag, Berlin, 1994), pp. 397–406Google Scholar
  21. 21.
    W.B. Langdon, T. Soule, R. Poli, J.A. Foster, The evolution of size and shape, in Advances in Genetic Programming 3, Chap. 8, ed. by L. Spector, W.B. Langdon, U.-M. O’Reilly, P.J. Angeline (MIT Press, Cambridge, 1999), pp. 163–190Google Scholar
  22. 22.
    J.M. Daida, A.M. Hilss, Identifying structural mechanisms in standard genetic programming, in Genetic and Evolutionary Computation—GECCO-2003, Chicago, 12–16 July 2003, ed. by E. Cantú-Paz, J.A. Foster, K. Deb, D. Davis, R. Roy, U.-M. O’Reilly, H.-G. Beyer, R. Standish, G. Kendall, S. Wilson, M. Harman, J. Wegener, D. Dasgupta, M.A. Potter, A.C. Schultz, K. Dowsland, N. Jonoska, J. Miller. Lecture in Computer Science, vol. 2724 (Springer-Verlag, Berlin, 2003), pp. 1639–1651Google Scholar
  23. 23.
    S. Luke, Code growth is not caused by introns, in Late Breaking Papers at the 2000 Genetic and Evolutionary Computation Conference, Las Vegas, NV, 8 July 2000, ed. by D. Whitley, pp. 228–235Google Scholar
  24. 24.
    T. Soule, Exons and code growth in genetic programming, in Proceedings of the 5th European Conference on Genetic Programming, EuroGP 2002, Kinsale, Ireland, 3–5 April 2002, ed. by J.A. Foster, E. Lutton, J. Miller, C. Ryan, A.G.B. Tettamanzi. Lecture Notes in Computer Science, vol. 2278 (Springer-Verlag, Berlin, 2002), pp. 142–151Google Scholar
  25. 25.
    W. Banzhaf, W.B. Langdon, Some considerations on the reason for bloat. Genet. Program. Evol. Mach. 3(1), 81–91 (2002)MATHCrossRefGoogle Scholar
  26. 26.
    T. Soule, R.B. Heckendorn, An analysis of the causes of code growth in genetic programming. Genet. Program. Evol. Mach. 3(3), 283–309 (2002)MATHCrossRefGoogle Scholar
  27. 27.
    W. Banzhaf, P. Nordin, R.E. Keller, F.D. Francone, Genetic Programming—An Introduction: On the Automatic Evolution of Computer Programs and its Applications (Morgan Kaufmann, San Francisco, 1998)MATHGoogle Scholar
  28. 28.
    R.M. Downing, Neutrality and gradualism: encouraging exploration and exploitation simultaneously with binary decision diagrams, in Proceedings of the 2006 IEEE Congress on Evolutionary Computation, Vancouver, Canada, 6–21 July 2006 (IEEE Press, Piscataway, 2006), pp. 615–622Google Scholar
  29. 29.
    W.B. Langdon, R. Poli, Foundations of Genetic Programming (Springer-Verlag, Berlin, 2002)Google Scholar
  30. 30.
    L. Beadle, Epoch X—Genetic Programming Analysis Software. http://www.epochx.com/epochx/default.asp, 2007–2008. Accessed 2 Mar 2009
  31. 31.
    F. Somenzi, Cudd: CU Decision Diagram Package release. http://vlsi.colorado.edu/ fabio/CUDD/, 1998. Accessed 2 Mar 2009
  32. 32.
    J. Whaley, JavaBDD. http://javabdd.sourceforge.net/, 2007. Accessed 2 Mar 2009
  33. 33.
    R.E. Bryant, Graph-based algorithms for boolean function manipulation. IEEE Trans. Comput. 35(8), 677–691 (1986)MATHCrossRefGoogle Scholar
  34. 34.
    S. Luke, Modification point depth and genome growth in genetic programming. Evol. Comput. 11(1), 67–106 (2003)CrossRefGoogle Scholar
  35. 35.
    S. Dignum, R. Poli, Generalisation of the limiting distribution of program sizes in tree-based genetic programming and analysis of its effects on bloat, in GECCO ’07: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, London, 7–11 July 2007, vol. 2, ed. by D. Thierens, H.-G. Beyer, J. Bongard, J. Branke, J.A. Clark, D. Cliff, C.B. Congdon, K. Deb, B. Doerr, T. Kovacs, S. Kumar, J.F. Miller, J. Moore, F. Neumann, M. Pelikan, R. Poli, K. Sastry, K.O. Stanley, T. Stutzle, R.A. Watson, I. Wegener (ACM Press, New York, 2007), pp. 1588–1595Google Scholar
  36. 36.
    R. Poli, N.F. McPhee, General schema theory for genetic programming with subtree-swapping crossover: Part I. Evol. Comput. 11(1), 53–66 (2003)CrossRefGoogle Scholar
  37. 37.
    R. Poli, N.F. McPhee, General schema theory for genetic programming with subtree-swapping crossover: Part II. Evol. Comput. 11(2), 169–206 (2003)CrossRefGoogle Scholar
  38. 38.
    R.P. Salustowicz, J. Schmidhuber, Probabilistic incremental program evolution. Evol. Comput. 5(2), 123–141 (1997)CrossRefGoogle Scholar
  39. 39.
    N.F. McPhee, B. Ohs, T. Hutchison, Semantic building blocks in genetic programming, in Proceedings of the 11th European Conference on Genetic Programming, EuroGP 2008, Naples, Italy, 26–28 March 2008, ed. by M. O’Neill, L. Vanneschi, S. Gustafson, A.I.E. Alcazar, I. De Falco, A.D. Cioppa, E. Tarantino. Lecture Notes in Computer Science, vol. 4971 (Springer, Berlin, 2008), pp. 134–145Google Scholar
  40. 40.
    L. Beadle, C.G. Johnson, Semantically driven crossover in genetic programming, in Proceedings of the IEEE World Congress on Computational Intelligence, Hong Kong, 1–6 June 2008 (IEEE, Piscataway, 2008), pp. 111–116Google Scholar
  41. 41.
    P.J. Angeline, Subtree crossover: building block engine or macromutation?, in Genetic Programming 1997: Proceedings of the Second Annual Conference, Stanford University, Stanford, CA, 13–16 July 1997, ed. by J.R. Koza, K. Deb, M. Dorigo, D.B. Fogel, M. Garzon, H. Iba, R.L. Riolo (Morgan Kaufmann, San Francisco, 1997), pp. 9–17Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Computing LaboratoryUniversity of KentCanterburyUK

Personalised recommendations