Genetic Programming and Evolvable Machines

, Volume 18, Issue 4, pp 467–507 | Cite as

Understanding grammatical evolution: initialisation

Article
  • 428 Downloads

Abstract

Grammatical evolution is one of the most used variants of genetic programming, and ever since its introduction, several improvements have been suggested. One of these concerns the routine used to create the initial population. In this study, several proposed initialisation routines are compared; based on a detailed analysis of the generated initial populations, and subsequent results obtained on a large set of experiments, a variant of the PTC2 algorithm is shown to consistently outperform all other routines, while a variant of random initialisation provides a good compromise between efficiency and ease of implementation.

Keywords

Grammatical evolution Initialisation Representation bias Tree creation Symbolic regression Classification Design 

References

  1. 1.
    J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (MIT Press, Cambridge, 1992)MATHGoogle Scholar
  2. 2.
    C. Ryan, J. Collins, M. O’Neill, Grammatical evolution: Evolving programs for an arbitrary language, in Genetic Programming, First European Workshop, EuroGP 1998, Paris, France, April 14–15, 1998, Proceedings, LNCS, vol. 1391, ed. by W. Banzhaf, R. Poli, M. Schoenauer, T.C. Fogarty (Springer, Berlin, 1998), pp. 83–95Google Scholar
  3. 3.
    R. Harper, A. Blair, A structure preserving crossover in grammatical evolution, in IEEE Congress on Evolutionary Computation, CEC 2005, Edinburgh, UK, September 2–5, 2005, Proceedings, vol. 3, (2005), pp. 2537–2544Google Scholar
  4. 4.
    J. Byrne, M. O’Neill, J. McDermott, A. Brabazon, An analysis of the behaviour of mutation in grammatical evolution, in Genetic Programming, 13th European Conference, EuroGP 2010, Istanbul, Turkey, April 7–9, 2010, Proceedings, LNCS, vol. 6021, ed. by A.I. Esparcia-Alcázar, A. Ekárt, S. Silva, S. Dignum, A.S. Uyar (Springer, Berlin, 2010), pp. 14–25Google Scholar
  5. 5.
    C. Ryan, A. Azad, Sensible initialisation in grammatical evolution, in Genetic and Evolutionary Computation—GECCO 2003, Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 12–16, 2004, Workshops, Proceedings, ed. by E. Cantú-Paz, J.A. Foster, K. Deb, L. Davis, R. Roy, U.M. O’Reilly, H.G. Beyer, R.K. Standish, G. Kendall, S.W. Wilson, M. Harman, J. Wegener, D. Dasgupta, M.A. Potter, A.C. Schultz, K.A. Dowsland, N. Jonoska, J.F. Miller (AAAI, 2003)Google Scholar
  6. 6.
    D. Fagan, M. Fenton, M. O’Neill, Exploring position independent initialisation in grammatical evolution, in IEEE Congress on Evolutionary Computation, CEC 2016, Vancouver, Canada, July 24–29, 2016, Proceedings (2016)Google Scholar
  7. 7.
    S. Forstenlechner, M. Nicolau, D. Fagan, M. O’Neill, Grammar design for derivation tree based genetic programming systems, in Genetic Programming, 19th European Conference, EuroGP 2016, Porto, Portugal, March 30–April 1, 2016, Proceedings, LNCS, vol. 9594, ed. by M.I. Heywood, J. McDermott, M. Castelli, E. Costa, K. Smith (Springer, Berlin, 2016), pp. 199–214Google Scholar
  8. 8.
    R. Harper, GE, explosive grammars and the lasting legacy of bad initialisation, in IEEE Congress on Evolutionary Computation, CEC 2010, Barcelona, Spain, July 18–23, 2010, Proceedings (2010), pp. 2602–2609Google Scholar
  9. 9.
    D.J. Montana, Strongly typed genetic programming. Evolut. Comput. 3(2), 199–230 (1995)Google Scholar
  10. 10.
    P.A. Whigham, Grammatically-based genetic programming, in Workshop on Genetic Programming: From Theory to Real-World Applications, Tahoe City, CA, USA, July 9, 1995, Proceedings, vol. 414, ed. by J.P. Rosca (Morgan Kaufmann, Burlington, 1995), pp. 33–41Google Scholar
  11. 11.
    M. O’Neill, C. Ryan, M. Keijzer, M. Cattolico, Crossover in grammatical evolution. Genet. Program. Evolv. Mach. 4(1), 67–93 (2003)CrossRefMATHGoogle Scholar
  12. 12.
    M. Nicolau, Automatic grammar complexity reduction in grammatical evolution, in Genetic and Evolutionary Computation—GECCO 2004, Genetic and Evolutionary Computation Conference, Seattle, WA, USA, June 26–30, 2004, Workshops, Proceedings, ed. by R. Poli, S. Cagnoni, M. Keijzer, E. Costa, F. Pereira, G. Raidl, S.C. Upton, D.E. Goldberg, H. Lipson, E. de Jong, J.R. Koza, H. Suzuki, H. Sawai, I. Parmee, M. Pelikan, K. Sastry, D. Thierens, W. Stolzmann, P.L. Lanzi, S.W. Wilson, M. O’Neill, C. Ryan, T. Yu, J.F. Miller, I. Garibay, G. Holifield, A.S. Wu, T. Riopka, M.M. Meysenburg, A.W. Wright, N. Richter, J.H. Moore, M.D. Ritchie, L. Davis, R. Roy, M. Jakiela (2004)Google Scholar
  13. 13.
    M. Nicolau, M. Fenton, Managing repetition in grammar-based genetic programming. In: Genetic and Evolutionary Computation—GECCO 2016, Genetic and Evolutionary Computation Conference, Denver, CO, USA, July 20–24, 2016, Proceedings, ed. by T. Friedrich (ACM, 2016), pp. 765–772Google Scholar
  14. 14.
    A. Thorhauer, On the non-uniform redundancy in grammatical evolution, in Parallel Problem Solving from Nature—PPSN XIV, 14th International Conference, Edinburgh, UK, September 17–21, 2016, Proceedings, LNCS, vol. 9921, ed. by J. Handl, E. Hart, P.R. Lewis, M. López-Ibáñez, G. Ochoa, B. Paechter (Springer, Berlin, 2016), pp. 292–302Google Scholar
  15. 15.
    M. Fenton, C. McNally, J. Byrne, E. Hemberg, J. McDermott, M. O’Neill, Discrete planar truss optimization by node position variation using grammatical evolution. IEEE Trans. Evolut. Comput. 20(4), 577–589 (2016)CrossRefGoogle Scholar
  16. 16.
    M. O’Neill, C. Ryan, Grammatical Evolution—Evolutionary Automatic Programming in an Arbitrary Language, Genetic Programming, vol. 4 (Kluwer, Dordrecht, 2003)MATHGoogle Scholar
  17. 17.
    I. Tsoulos, D. Gavrilis, E. Glavas, Neural network construction and training using grammatical evolution. Neurocomputing 72(1–3), 269–277 (2008)CrossRefGoogle Scholar
  18. 18.
    M. Nicolau, M. O’Neill, A. Brabazon, Termination in grammatical evolution: Grammar design, wrapping, and tails, in IEEE Congress on Evolutionary Computation, CEC 2012, Brisbane, Australia, June 10–15, 2012, Proceedings (2012), pp. 1–8Google Scholar
  19. 19.
    E.F. Crane, N.F. McPhee, The effects of size and depth limits on tree based genetic programming, in Genetic Programming Theory and Practice III, ed. by T. Yu, R. Riolo, B. Worzel (Springer, Boston, 2006), pp. 223–240CrossRefGoogle Scholar
  20. 20.
    W.B. Langdon, Size fair and homologous tree crossovers for tree genetic programming. Genet. Program. Evolv. Mach. 1(1), 95–119 (2000)CrossRefMATHGoogle Scholar
  21. 21.
    T. Soule, J.E. Foster, Code size and depth flows in genetic programming, in Genetic Programming 1997: Second Annual Conference, Stanford, USA, July 13–16, 1997, Proceedings, ed. by J.R. Koza, K. Deb, M. Dorigo, D.B. Fogel, M. Garzon, H. Iba, R.L. Riolo (Morgan Kaufmann, Burlington, 1997), pp. 313–320Google Scholar
  22. 22.
    S. Luke, Two fast tree-creation algorithms for genetic programming. IEEE Trans. Evolut. Comput. 4(3), 274–283 (2000)CrossRefGoogle Scholar
  23. 23.
    D.R. White, J. McDermott, M. Castelli, L. Manzoni, B.W. Goldman, G. Kronberger, W. Jaśkowski, U.M. O’Reilly, S. Luke, Better GP benchmarks: community survey results and proposals. Genet. Program. Evolv. Mach. 14(1), 3–29 (2013)CrossRefGoogle Scholar
  24. 24.
    E. Hemberg, An exploration of grammars in grammatical evolution. Ph.D. Thesis, University College Dublin, University College Dublin (2010)Google Scholar
  25. 25.
    M. O’Neill, A. Brabazon, Grammatical swarm, in Genetic and Evolutionary Computation - GECCO 2004, Genetic and Evolutionary Computation Conference, Seattle, WA, USA, June 26–30, 2004, Proceedings, Part I, LNCS, vol. 3102, ed. by K. Deb, R. Poli, W. Banzhaf, H.G. Beyer, E.K. Burke, P.J. Darwen, D. Dasgupta, D. Floreano, J.A. Foster, M. Harman, O. Holland, P.L. Lanzi, L. Spector, A. Tettamanzi, D. Thierens, A.M. Tyrrell (Springer, Berlin, 2004), pp. 163–174Google Scholar
  26. 26.
    J. O’Sullivan, An investigation into the use of different search engines with grammatical evolution. Master’s Thesis, University of Limerick, University of Limerick (2001)Google Scholar
  27. 27.
    D. Fagan, M. O’Neill, E. Galván-López, A. Brabazon, S. McGarraghy, An analysis of genotype-phenotype maps in grammatical evolution, in Genetic Programming, 13th European Conference, EuroGP 2010, Istanbul, Turkey, April 7–9, 2010, Proceedings, LNCS, vol. 6021, ed. by A.I. Esparcia-Alcázar, A. Ekárt, S. Silva, S. Dignum, A.S. Uyar (Springer, Berlin, 2010), pp. 62–73Google Scholar
  28. 28.
    M. Keijzer, M. O’Neill, C. Ryan, M. Cattolico, Grammatical evolution rules: The mod and the bucket rule, in Genetic Programming, 5th European Conference, EuroGP 2002, Kinsale, Ireland, April 3–5, 2002, Proceedings, LNCS, vol. 2278, ed. by J.A. Foster, E. Lutton, J. Miller, C. Ryan, A.G. Tettamanzi (Springer, Berlin, 2002), pp. 123–130Google Scholar
  29. 29.
    N. Lourenço, F.B. Pereira, E. Costa, Unveiling the properties of structured grammatical evolution. Genet. Program. Evolv. Mach. 17(3), 251–289 (2016)CrossRefGoogle Scholar
  30. 30.
    R.M.A. Azad, A.R. Ansari, C. Ryan, M. Walsh, T. McGloughlin, An evolutionary approach to wall shear stress prediction in a grafted artery. Appl. Soft Comput. 4(2), 139–148 (2004)CrossRefGoogle Scholar
  31. 31.
    A. Brabazon, M. O’Neill, Biologically Inspired Algorithms for Financial Modelling (Springer, Berlin, 2006)MATHGoogle Scholar
  32. 32.
    J. McDermott, J. Byrne, J.M. Swafford, M. Hemberg, C. McNally, E. Shotton, E. Hemberg, M. Fenton, M. O’Neill, String-rewriting grammars for evolutionary architectural design. Environ. Plan. B: Plan. Des. 39(4), 713–731 (2012)CrossRefGoogle Scholar
  33. 33.
    R. Harper, Evolving robocode tanks for evo robocode. Genet. Program. Evolv. Mach. 15(4), 403–431 (2014)CrossRefGoogle Scholar
  34. 34.
    M. Nicolau, D. Perez-Liebana, M. O’Neill, A. Brabazon, Evolutionary behavior tree approaches for navigating platform games. IEEE Trans. Comput. Intell. AI Games 99 (2016). doi: 10.1109/TCIAIG.2016.2543661
  35. 35.
    M. Hemberg, U.M. O’Reilly, Extending grammatical evolution to evolve digital surfaces with genr8, in Genetic Programming, 7th European Conference, EuroGP 2004, Coimbra, Portugal, April 5–7, 2004, Proceedings, LNCS, vol. 3003, ed. by M. Keijzer, U.M. O’Reilly, S.M. Lucas, E. Costa, T. Soule (Springer, Berlin, 2004), pp. 299–308Google Scholar
  36. 36.
    J.E. Murphy, M. O’Neill, H. Carr, Exploring grammatical evolution for horse gait optimisation, in Genetic Programming, 12th European Conference, EuroGP 2009, Tübingen, Germany, April 15–17, 2009, Proceedings, LNCS, vol. 5481, ed. by L. Vanneschi, S. Gustafson, A. Moraglio, I.D. Falco, M. Ebner (Springer, Berlin, 2009), pp. 183–194Google Scholar
  37. 37.
    M. Nicolau, M. Saunders, M. O’Neill, B. Osborne, A. Brabazon, Evolving interpolating models of net ecosystem CO2 exchange using grammatical evolution, in Genetic Programming, 15th European Conference, EuroGP 2012, Malaga, Spain, April 11–13, 2012, Proceedings, LNCS, vol. 7244, ed. by A. Moraglio, S. Silva, K. Krawiec, P. Machado, C. Cotta (Springer, Berlin, 2012), pp. 134–145Google Scholar
  38. 38.
    J. Tavares, F.B. Pereira, Automatic design of ant algorithms with grammatical evolution, in Genetic Programming, 15th European Conference, EuroGP 2012, Malaga, Spain, April 11–13, 2012, Proceedings, LNCS, vol. 7244, ed. by A. Moraglio, S. Silva, K. Krawiec, P. Machado, C. Cotta (Springer, Berlin, 2012), pp. 206–217Google Scholar
  39. 39.
    C. Ryan, M. Keijzer, M. Nicolau, On the avoidance of fruitless wraps in grammatical evolution, in Genetic and Evolutionary Computation—GECCO 2003, Genetic and Evolutionary Computation Conference, Chicago, IL, USA, July 12–16, 2003. Proceedings, Part II, LNCS, vol. 2724, ed. by E. Cantú-Paz, J.A. Foster, K. Deb, L. Davis, R. Roy, U.M. O’Reilly, H.G. Beyer, R.K. Standish, G. Kendall, S.W. Wilson, M. Harman, J. Wegener, D. Dasgupta, M.A. Potter, A.C. Schultz, K.A. Dowsland, N. Jonoska, J.F. Miller (Springer, Berlin, 2003), pp. 1752–1763Google Scholar
  40. 40.
    F. Rothlauf, M. Oetzel, On the locality of grammatical evolution, in Genetic Programming, 9th European Conference, EuroGP 2006, Budapest, Hungary, April 10–12, 2006, Proceedings, LNCS, vol. 3905, ed. by P. Collet, M. Tomassini, M. Ebner, S. Gustafson, A. Ekárt (Springer, Berlin, 2006), pp. 320–330Google Scholar
  41. 41.
    A. Thorhauer, F. Rothlauf, On the locality of standard search operators in grammatical evolution, in Parallel Problem Solving from Nature - PPSN XIII, 13th International Conference, Ljubljana, Slovenia, September 13–17, 2014, Proceedings, LNCS, vol. 8672, ed. by T. Bartz-Beielstein, J. Branke, B. Filipič J. Smith (Springer, 2014), pp. 465–475Google Scholar
  42. 42.
    N. Lourenço, J. Ferrer, F.B. Pereira, E. Costa, A comparative study of different grammar-based genetic programming approaches, in Genetic Programming, 20th European Conference, EuroGP 2017, Amsterdam, The Netherlands, April 19–21, 2017, Proceedings, LNCS, vol. 10196, ed. by J. McDermott, M. Castelli, L. Sekanina, E. Haasdijk, P. García-Sánchez (Springer, Berlin, 2017), pp. 311–325Google Scholar
  43. 43.
    P.A. Whigham, G. Dick, J. Maclaurin, C.A. Owen, Examining the “best of both worlds” of grammatical evolution, in Genetic and Evolutionary Computation—GECCO 2015, Genetic and Evolutionary Computation Conference, Madrid, Spain, July 11–15, 2015, Proceedings, ed. by S. Silva (ACM, 2015), pp. pp. 1111–1118Google Scholar
  44. 44.
    A. Brabazon, M. O’Neill, S. McGarraghy, Natural Computing Algorithms (Springer, Berlin, 2015)CrossRefMATHGoogle Scholar
  45. 45.
    M. Keijzer, Improving symbolic regression with interval arithmetic and linear scaling, in Genetic Programming, 6th European Conference, EuroGP 2003, Essex, UK, April 14–16, 2003, Proceedings, LNCS, vol. 2610, ed. by C. Ryan, T. Soule, M. Keijzer, E. Tsang, R. Poli, E. Costa (Springer, Berlin, 2003), pp. 70–82Google Scholar
  46. 46.
    L. Pagie, P. Hogeweg, Evolutionary consequences of coevolving targets. Evolut. Comput. 5(4), 401–418 (1997)CrossRefGoogle Scholar
  47. 47.
    E.J. Vladislavleva, G.F. Smits, D. den Hertog, Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Trans. Evolut. Comput. 13(2), 333–349 (2009)CrossRefGoogle Scholar
  48. 48.
    M.F. Korns, Accuracy in symbolic regression, in Genetic Programming Theory and Practice IX, Genetic and Evolutionary Computation, ed. by R. Riolo, E. Vladislavleva, J.H. Moore (Springer, New York, 2011), pp. 129–151CrossRefGoogle Scholar
  49. 49.
    P. Cortez, A. Morais, A data mining approach to predict forest fires using meteorological data, in New Trends in Artificial Intelligence, Portuguese Conference on Artificial Intelligence, EPIA 2007, Guimaraes, Portugal, December 2007, Proceedings, ed. by J. Neves, M.F. Santos, J. Machado (APPIA, New York, 2007), pp. 512–523Google Scholar
  50. 50.
    M. Lichman, UCI machine learning repository (2013). http://archive.ics.uci.edu/ml
  51. 51.
    P. Cortez, A. Cerdeira, F. Almeida, T. Matos, J. Reis, Modeling wine preferences by data mining from physicochemical properties. Decis. Support Syst. 47(4), 547–553 (2009)CrossRefGoogle Scholar
  52. 52.
    M. O’Neill, J.M. Swafford, J. McDermott, J. Byrne, A. Brabazon, E. Shotton, C. McNally, M. Hemberg, Shape grammars and grammatical evolution for evolutionary design, in Genetic and Evolutionary Computation—GECCO 2009, Genetic and Evolutionary Computation Conference, Montreal, Canada, July 8–12, 2009, Proceedings, ed. by G.R. et al. (ACN, 2009), pp. 1035–1042Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.College of BusinessUniversity College DublinDublinIreland

Personalised recommendations