Improving Genetic Programming with Novel Exploration - Exploitation Control

  • Jonathan KellyEmail author
  • Erik Hemberg
  • Una-May O’Reilly
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11451)


Low population diversity is recognized as a factor in premature convergence of evolutionary algorithms. We investigate program synthesis performance via grammatical evolution. We focus on novelty search – substituting the conventional search objective – based on synthesis quality, with a novelty objective. This prompts us to introduce a new selection method named knobelty. It parametrically balances exploration and exploitation by creating a mixed population of parents. One subset is chosen based on performance quality and the other subset is chosen based on diversity. Three versions of this method, two that adaptively tune balance during evolution solve program synthesis problems more accurately, faster and with less duplication than grammatical evolution with lexicase selection.


Program synthesis Novelty Diversity 



This material is based upon work supported by DARPA. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements either expressed or implied of Applied Communication Services, or the US Government.


  1. 1.
    Helmuth, T., Spector, L.: General program synthesis benchmark suite. In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 1039–1046. ACM (2015)Google Scholar
  2. 2.
    Spector, L.: Autoconstructive evolution: push, pushGP, and pushpop. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), vol. 137 (2001)Google Scholar
  3. 3.
    Forstenlechner, S., Fagan, D., Nicolau, M., O’Neill, M.: Towards understanding and refining the general program synthesis benchmark suite with genetic programming. In: 2018 IEEE Congress on Evolutionary Computation (CEC), pp. 1–6. IEEE (2018)Google Scholar
  4. 4.
    Helmuth, T., McPhee, N.F., Spector, L.: Lexicase selection for program synthesis: a diversity analysis. In: Riolo, R., Worzel, B., Kotanchek, M., Kordon, A. (eds.) Genetic Programming Theory and Practice XIII. GEC, pp. 151–167. Springer, Cham (2016). Scholar
  5. 5.
    Lehman, J., Stanley, K.O.: Exploiting open-endedness to solve problems through the search for novelty. In: ALIFE, pp. 329–336 (2008)Google Scholar
  6. 6.
    López-López, V.R., Trujillo, L., Legrand, P.: Novelty search for software improvement of a slam system. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 1598–1605. ACM (2018)Google Scholar
  7. 7.
    Doucette, J., Heywood, M.I.: Novelty-based fitness: an evaluation under the Santa Fe Trail. In: Esparcia-Alcázar, A.I., Ekárt, A., Silva, S., Dignum, S., Uyar, A.Ş. (eds.) EuroGP 2010. LNCS, vol. 6021, pp. 50–61. Springer, Heidelberg (2010). Scholar
  8. 8.
    Naredo, E.: Genetic programming based on novelty search. Ph.D. thesis, ITT, Instituto tecnologico de Tijuana (2016)Google Scholar
  9. 9.
    Ryan, C., O’Neill, M., Collins, J.J.: Introduction to 20 Years of Grammatical Evolution. In: Ryan, C., O’Neill, M., Collins, J.J. (eds.) Handbook of Grammatical Evolution, pp. 1–21. Springer, Cham (2018). Scholar
  10. 10.
    Thorhauer, A., Rothlauf, F.: On the locality of standard search operators in grammatical evolution. In: Bartz-Beielstein, T., Branke, J., Filipič, B., Smith, J. (eds.) PPSN 2014. LNCS, vol. 8672, pp. 465–475. Springer, Cham (2014). Scholar
  11. 11.
    Booth, T.L.: Sequential machines and automata theory (1967)Google Scholar
  12. 12.
    O’Neill, M., Ryan, C.: Evolving multi-line compilable C programs. In: Poli, R., Nordin, P., Langdon, W.B., Fogarty, T.C. (eds.) EuroGP 1999. LNCS, vol. 1598, pp. 83–92. Springer, Heidelberg (1999). Scholar
  13. 13.
    Lucas, S.M.: Exploiting reflection in object oriented genetic programming. In: Keijzer, M., O’Reilly, U.-M., Lucas, S., Costa, E., Soule, T. (eds.) EuroGP 2004. LNCS, vol. 3003, pp. 369–378. Springer, Heidelberg (2004). Scholar
  14. 14.
    Agapitos, A., Lucas, S.M.: Learning recursive functions with object oriented genetic programming. In: Collet, P., Tomassini, M., Ebner, M., Gustafson, S., Ekárt, A. (eds.) EuroGP 2006. LNCS, vol. 3905, pp. 166–177. Springer, Heidelberg (2006). Scholar
  15. 15.
    Yu, T., Clack, C.: Recursion, lambda-abstractions and genetic programming. In: Poli, R., Langdon, W.B., Schoenauer, M., Fogarty, T., Banzhaf, W. (eds.) Late Breaking Papers at EuroGP 1998: The First European Workshop on Genetic Programming, pp. 26–30. CSRP-98-10, The University of Birmingham, UK, Paris, 14–15 April 1998Google Scholar
  16. 16.
    Wan, M., Weise, T., Tang, K.: Novel loop structures and the evolution of mathematical algorithms. In: Silva, S., Foster, J.A., Nicolau, M., Machado, P., Giacobini, M. (eds.) EuroGP 2011. LNCS, vol. 6621, pp. 49–60. Springer, Heidelberg (2011). Scholar
  17. 17.
    Weise, T., Tang, K.: Evolving distributed algorithms with genetic programming. IEEE Trans. Evol. Comput. 16(2), 242–265 (2012). Scholar
  18. 18.
    Weise, T., Wan, M., Tang, K., Yao, X.: Evolving exact integer algorithms with genetic programming. In: 2014 IEEE Congress on Evolutionary Computation (CEC), pp. 1816–1823, July 2014Google Scholar
  19. 19.
    Krawiec, K.: Behavioral Program Synthesis with Genetic Programming. SCI, vol. 618, pp. 1–19. Springer, Cham (2016). Scholar
  20. 20.
    Helmuth, T., McPhee, N.F., Spector, L.: Program synthesis using uniform mutation by addition and deletion. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1127–1134. ACM (2018)Google Scholar
  21. 21.
    Forstenlechner, S., Fagan, D., Nicolau, M., O’Neill, M.: A grammar design pattern for arbitrary program synthesis problems in genetic programming. In: McDermott, J., Castelli, M., Sekanina, L., Haasdijk, E., García-Sánchez, P. (eds.) EuroGP 2017. LNCS, vol. 10196, pp. 262–277. Springer, Cham (2017). Scholar
  22. 22.
    De Jong, K.A.: Analysis of the behavior of a class of genetic adaptive systems (1975)Google Scholar
  23. 23.
    Mengshoel, O.J., Goldberg, D.E.: The crowding approach to niching in genetic algorithms. Evol. Comput. 16(3), 315–354 (2008)CrossRefGoogle Scholar
  24. 24.
    Hornby, G.S.: ALPS: the age-layered population structure for reducing the problem of premature convergence. In: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, pp. 815–822. ACM (2006)Google Scholar
  25. 25.
    Mitchell, M., Thomure, M.D., Williams, N.L.: The role of space in the success of coevolutionary learning. In: Artificial Life X: Proceedings of the Tenth International Conference on the Simulation and Synthesis of Living Systems, pp. 118–124. MIT Press, Cambridge (2006)Google Scholar
  26. 26.
    Gomez, F.J.: Sustaining diversity using behavioral information distance. In: Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, GECCO 2009, pp. 113–120. ACM, New York (2009).
  27. 27.
    Burke, E.K., Gustafson, S., Kendall, G.: Diversity in genetic programming: an analysis of measures and correlation with fitness. IEEE Trans. Evol. Comput. 8(1), 47–62 (2004)CrossRefGoogle Scholar
  28. 28.
    Sudholt, D.: The benefits of population diversity in evolutionary algorithms: a survey of rigorous runtime analyses. arXiv preprint arXiv:1801.10087 (2018)
  29. 29.
    Burks, A.R., Punch, W.F.: An analysis of the genetic marker diversity algorithm for genetic programming. Genet. Program. Evolvable Mach. 18(2), 213–245 (2017)CrossRefGoogle Scholar
  30. 30.
    Affenzeller, M., Winkler, S.M., Burlacu, B., Kronberger, G., Kommenda, M., Wagner, S.: Dynamic observation of genotypic and phenotypic diversity for different symbolic regression GP variants. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 1553–1558. ACM (2017)Google Scholar
  31. 31.
    Shahrzad, H., Fink, D., Miikkulainen, R.: Enhanced optimization with composite objectives and novelty selection. arXiv preprint arXiv:1803.03744 (2018)
  32. 32.
    Goldsby, H.J., Cheng, B.H.C.: Automatically discovering properties that specify the latent behavior of UML models. In: Petriu, D.C., Rouquette, N., Haugen, Ø. (eds.) MODELS 2010. LNCS, vol. 6394, pp. 316–330. Springer, Heidelberg (2010). Scholar
  33. 33.
    Cuccu, G., Gomez, F.: When novelty is not enough. In: Di Chio, C., et al. (eds.) EvoApplications 2011. LNCS, vol. 6624, pp. 234–243. Springer, Heidelberg (2011). Scholar
  34. 34.
    Fenton, M., McDermott, J., Fagan, D., Forstenlechner, S., O’Neill, M., Hemberg, E.: PonyGE2: grammatical evolution in python. CoRR abs/1703.08535 (2017).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Jonathan Kelly
    • 1
    Email author
  • Erik Hemberg
    • 1
  • Una-May O’Reilly
    • 1
  1. 1.MITCambridgeUSA

Personalised recommendations