Designing automatically a representation for grammatical evolution

Abstract

A long-standing problem in evolutionary computation consists in how to choose an appropriate representation for the solutions. In this work we investigate the feasibility of synthesizing a representation automatically, for the large class of problems whose solution spaces can be defined by a context-free grammar. We propose a framework based on a form of meta-evolution in which individuals are candidate representations expressed with an ad hoc language that we have developed to this purpose. Individuals compete and evolve according to an evolutionary search aimed at optimizing such representation properties as redundancy, uniformity of redundancy, and locality. We assessed experimentally three variants of our framework on established benchmark problems and compared the resulting representations to human-designed representations commonly used (e.g., classical grammatical evolution). The results are promising as the evolved representations indeed exhibit better properties than the human-designed ones. Furthermore, the evolved representations compare favorably with the human-designed baselines in search effectiveness as well. Specifically, we select a best evolved representation as the representation with best search effectiveness on a set of learning problems and assess its effectiveness on a separate set of challenging validation problems. For each of the three proposed variants of our framework, the best evolved representation exhibits an average fitness rank on the set of validation problems that is better than the average fitness rank of the human-designed baselines on the same problems.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Notes

  1. 1.

    https://github.com/ericmedvet/evolved-ge.

References

  1. 1.

    C. Ryan, A rebuttal to Whigham, Dick, and Maclaurin by one of the inventors of grammatical evolution: commentary on “On the mapping of genotype to phenotype in evolutionary algorithms” by Peter A. Whigham, Grant Dick, and James Maclaurin. Genet. Program. Evolvable Mach. 18(3), 385–389 (2017). https://doi.org/10.1007/s10710-017-9294-z

    Article  Google Scholar 

  2. 2.

    L. Altenberg, Probing the axioms of evolutionary algorithm design: commentary on “On the mapping of genotype to phenotype in evolutionary algorithms” by Peter A. Whigham, Grant Dick, and James Maclaurin. Genet. Program. Evolvable Mach. 18(3), 363–367 (2017). https://doi.org/10.1007/s10710-017-9290-3

    Article  Google Scholar 

  3. 3.

    A. Bartoli, A. De Lorenzo, E. Medvet, F. Tarlao, Learning text patterns using separate-and-conquer genetic programming, in European Conference on Genetic Programming (Springer, 2015), pp. 16–27

  4. 4.

    A. Bartoli, A. De Lorenzo, E. Medvet, F. Tarlao, Syntactical similarity learning by means of grammatical evolution, in Parallel Problem Solving from Nature—PPSN XIV: 14th International Conference, Edinburgh, UK, September 17–21, 2016, Proceedings (Springer, Cham, 2016), pp. 260–269. https://doi.org/10.1007/978-3-319-45823-6_24

  5. 5.

    M. Castelli, L. Manzoni, L. Vanneschi, S. Silva, A. Popovič, Self-tuning geometric semantic genetic programming. Genet. Program. Evolvable Mach. 17(1), 55–74 (2016)

    Article  Google Scholar 

  6. 6.

    M.B. Correia, A study of redundancy and neutrality in evolutionary optimization. Evol. Comput. 21(3), 413–443 (2013)

    Article  Google Scholar 

  7. 7.

    A.F. Cruz-Salinas, J.G. Perdomo, Self-adaptation of genetic operators through genetic programming techniques, in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO’17 (ACM, New York, NY, USA, 2017), pp. 913–920. https://doi.org/10.1145/3071178.3071214

  8. 8.

    K. De Jong, Parameter setting in EAS: a 30 year perspective, in Parameter Setting in Evolutionary Algorithms, ed. by F. G. Lobo, C. F. Lima, Z. Michalewicz (Springer, Berlin, Heidelberg, 2007), pp. 1–18. https://doi.org/10.1007/978-3-540-69432-8_1

    Google Scholar 

  9. 9.

    D.B. Fogel, L.J. Fogel, J.W. Atmar, Meta-evolutionary programming, in Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems and Computers, 1991 (IEEE, 1991), pp. 540–545

  10. 10.

    J.A. Foster, Taking “biology” just seriously enough: commentary on “On the mapping of genotype to phenotype in evolutionary algorithms” by Peter A. Whigham, Grant Dick, and James Maclaurin. Genet. Program. Evolvable Mach. 18(3), 395–398 (2017). https://doi.org/10.1007/s10710-017-9296-x

    Article  Google Scholar 

  11. 11.

    L. Hong, J.H. Drake, J.R. Woodward, E. Özcan, A hyper-heuristic approach to automated generation of mutation operators for evolutionary programming. Appl. Soft Comput. 62, 162–175 (2017)

    Article  Google Scholar 

  12. 12.

    M. Keijzer, Improving symbolic regression with interval arithmetic and linear scaling, in Genetic Programming (2003), pp. 275–299

  13. 13.

    N. Lourenço, F.B. Pereira, E. Costa, SGE: a structured representation for grammatical evolution, in International Conference on Artificial Evolution (Evolution Artificielle) (Springer, 2015), pp. 136–148

  14. 14.

    S.W. Mahfoud, Niching methods for genetic algorithms. Urbana 51(95001), 62–94 (1995)

    Google Scholar 

  15. 15.

    R.I. Mckay, N.X. Hoai, P.A. Whigham, Y. Shan, M. O’Neill, Grammar-based genetic programming: a survey. Genet. Program. Evolvable Mach. 11(3–4), 365–396 (2010)

    Article  Google Scholar 

  16. 16.

    E. Medvet, A comparative analysis of dynamic locality and redundancy in grammatical evolution, in Genetic Programming: 20th European Conference, EuroGP 2017, Amsterdam, Netherlands, April 19–21, 2017, Proceedings (Springer, Cham, 2017) (to appear)

  17. 17.

    E. Medvet, Hierarchical grammatical evolution, in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO (2017)

  18. 18.

    E. Medvet, A. Bartoli, On the automatic design of a representation for grammar-based genetic programming, in Genetic Programming, ed. by M. Castelli, L. Sekanina, M. Zhang, S. Cagnoni, P. García-Sánchez (Springer, Cham, 2018), pp. 101–117

    Google Scholar 

  19. 19.

    E. Medvet, A. Bartoli, G. Squillero, An effective diversity promotion mechanism in grammatical evolution, in Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO’17 (ACM, New York, NY, USA, 2017), pp. 247–248. https://doi.org/10.1145/3067695.3076057

  20. 20.

    E. Medvet, F. Daolio, D. Tagliapietra, Evolvability in grammatical evolution, in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO’17 (ACM, New York, NY, USA, 2017), pp. 977–984. https://doi.org/10.1145/3071178.3071298

  21. 21.

    J.F. Miller, S.L. Smith, Redundancy and computational efficiency in cartesian genetic programming. IEEE Trans. Evol. Comput. 10(2), 167–174 (2006)

    Article  Google Scholar 

  22. 22.

    M. O’Neill, A. Brabazon, M. Nicolau, S.M. Garraghy, P. Keenan, \(\pi\) Grammatical Evolution (Springer, Berlin, 2004), pp. 617–629. https://doi.org/10.1007/978-3-540-24855-2_70

  23. 23.

    M. O’Neill, C. Ryan, Genetic code degeneracy: implications for grammatical evolution and beyond, in European Conference on Artificial Life (Springer, 1999), pp. 149–153

  24. 24.

    M. O’Neill, C. Ryan, Grammatical evolution by grammatical evolution: the evolution of grammar and genetic code, in Genetic Programming (2004), pp. 138–149

  25. 25.

    L. Pagie, P. Hogeweg, Evolutionary consequences of coevolving targets. Evol. Comput. 5(4), 401–418 (1997)

    Article  Google Scholar 

  26. 26.

    G.L. Pappa, G. Ochoa, M.R. Hyde, A.A. Freitas, J. Woodward, J. Swan, Contrasting meta-learning and hyper-heuristic research: the role of evolutionary algorithms. Genet. Program. Evolvable Mach. 15(1), 3–35 (2014). https://doi.org/10.1007/s10710-013-9186-9

    Article  Google Scholar 

  27. 27.

    A.K. Qin, V.L. Huang, P.N. Suganthan, Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Tran. Evol. Comput. 13(2), 398–417 (2009)

    Article  Google Scholar 

  28. 28.

    F. Rothlauf, Representations for genetic and evolutionary algorithms, in Representations for Genetic and Evolutionary Algorithms (Springer, Berlin, 2006), pp. 9–32

  29. 29.

    F. Rothlauf, D.E. Goldberg, Redundant representations in evolutionary computation. Evol. Comput. 11(4), 381–415 (2003)

    Article  Google Scholar 

  30. 30.

    C. Ryan, J. Collins, M.O. Neill, Grammatical Evolution: Evolving Programs for an Arbitrary Language (Springer, Berlin, 1998), pp. 83–96. https://doi.org/10.1007/BFb0055930

    Google Scholar 

  31. 31.

    N. Saravanan, D.B. Fogel, K.M. Nelson, A comparison of methods for self-adaptation in evolutionary algorithms. BioSystems 36(2), 157–166 (1995)

    Article  Google Scholar 

  32. 32.

    E.O. Scott, J.K. Bassett, Learning genetic representations for classes of real-valued optimization problems, in Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation (ACM, 2015), pp. 1075–1082

  33. 33.

    L.F. Simões, D. Izzo, E. Haasdijk, A.E. Eiben, Self-adaptive genotype–phenotype maps: neural networks as a meta-representation, in International Conference on Parallel Problem Solving from Nature (Springer, 2014), pp. 110–119

  34. 34.

    L. Spector, Introduction to the peer commentary special section on “On the mapping of genotype to phenotype in evolutionary algorithms” by Peter A. Whigham, Grant Dick, and James Maclaurin. Genet. Program. Evolvable Mach. 18(3), 351–352 (2017). https://doi.org/10.1007/s10710-017-9287-y

    Article  Google Scholar 

  35. 35.

    G. Squillero, A. Tonda, (Over-)Realism in evolutionary computation: commentary on “On the mapping of genotype to phenotype in evolutionary algorithms” by Peter A. Whigham, Grant Dick, and James Maclaurin. Genet. Program. Evolvable Mach. 1–3 (2017). https://doi.org/10.1007/s10710-017-9295-y

  36. 36.

    G. Squillero, A.P. Tonda, Divergence of character and premature convergence: a survey of methodologies for promoting diversity in evolutionary optimization. Inf. Sci. 329, 782–799 (2016). https://doi.org/10.1016/j.ins.2015.09.056

    Article  Google Scholar 

  37. 37.

    A. Thorhauer, On the non-uniform redundancy in grammatical evolution, in International Conference on Parallel Problem Solving from Nature (Springer, 2016), pp. 292–302

  38. 38.

    A. Thorhauer, F. Rothlauf, On the locality of standard search operators in grammatical evolution, in International Conference on Parallel Problem Solving from Nature (Springer, 2014), pp. 465–475

  39. 39.

    N.Q. Uy, N.X. Hoai, M. O’Neill, R.I. McKay, E. Galván-López, Semantically-based crossover in genetic programming: application to real-valued symbolic regression. Genet. Program. Evolvable Mach. 12(2), 91–119 (2011)

    Article  Google Scholar 

  40. 40.

    L. Vanneschi, M. Castelli, L. Manzoni, The k landscapes: a tunably difficult benchmark for genetic programming, in Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation (ACM, 2011), pp. 1467–1474

  41. 41.

    P.A. Whigham, G. Dick, J. Maclaurin, On the mapping of genotype to phenotype in evolutionary algorithms. Genet. Program. Evolvable Mach. (2017). https://doi.org/10.1007/s10710-017-9288-x

    Article  Google Scholar 

  42. 42.

    P.A. Whigham, G. Dick, J. Maclaurin, C.A. Owen, Examining the best of both worlds of grammatical evolution, in Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation (ACM, 2015), pp. 1111–1118

  43. 43.

    P.A. Whigham et al., Grammatically-based genetic programming, in Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, vol. 16 (1995), pp. 33–41

  44. 44.

    D.R. White, J. Mcdermott, M. Castelli, L. Manzoni, B.W. Goldman, G. Kronberger, W. Jaśkowski, U.M. O’Reilly, S. Luke, Better gp benchmarks: community survey results and proposals. Genet. Program. Evolvable Mach. 14(1), 3–29 (2013)

    Article  Google Scholar 

  45. 45.

    P.K. Wong, M.L. Wong, K.S. Leung, Hierarchical knowledge in self-improving grammar-based genetic programming, in International Conference on Parallel Problem Solving from Nature (Springer, 2016), pp. 270–280

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Eric Medvet.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Medvet, E., Bartoli, A., De Lorenzo, A. et al. Designing automatically a representation for grammatical evolution. Genet Program Evolvable Mach 20, 37–65 (2019). https://doi.org/10.1007/s10710-018-9327-2

Download citation

Keywords

  • Genotype–phenotype mapping
  • Grammatical evolution
  • Meta-evolution