Advertisement

Soft Computing

, Volume 21, Issue 15, pp 4399–4416 | Cite as

The training set and generalization in grammatical evolution for autonomous agent navigation

  • Enrique Naredo
  • Paulo Urbano
  • Leonardo Trujillo
Methodologies and Application

Abstract

Over recent years, evolutionary computation research has begun to emphasize the issue of generalization. Instead of evolving solutions that are optimized for a particular problem instance, the goal is to evolve solutions that can generalize to various different scenarios. This paper compares objective-based search and novelty search on a set of generalization oriented experiments for a navigation task using grammatical evolution (GE). In particular, this paper studies the impact that the training set has on the generalization of evolved solutions, considering: (1) the training set size; (2) the manner in which the training set is chosen (random or manual); and (3) if the training set is fixed throughout the run or dynamically changed every generation. Experimental results suggest that novelty search outperforms objective-based search in terms of evolving navigation behaviors that are able to cope with different initial conditions. The traditional objective-based search requires larger training sets and its performance degrades when the training set is not fixed. On the other hand, novelty search seems to be robust to different training sets, finding general solutions in almost all of the studied conditions with almost perfect generalization in many scenarios.

Keywords

Novelty search Generalization Grammatical evolution Autonomous agent navigation 

Notes

Acknowledgments

The authors acknowledge the following projects. First author was supported by CONACYT (México) scholarship No. 232288. Funding for this work was provided by FCT project EXPL/EEI-SII/1861/2013, and by CONACYT Basic Science Research Project No. 178323, DGEST (Mexico) Research Project 5414.14-P, and FP7-PEOPLE-2013-IRSES project ACOBSEC financed by the European Commission with contract No. 612689.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

References

  1. Banzhaf W, Francone FD, Nordin P (1996) The effect of extensive use of the mutation operator on generalization in genetic programming using sparse data sets. In: In parallel problem solving from nature IV, proceedings of the international conference on evolutionary computation, edited by, Springer, Berlin, pp 300–309Google Scholar
  2. Bishop CM (2006) Pattern recognition and machine learning (information science and statistics). Springer, New YorkzbMATHGoogle Scholar
  3. Burke EK, Gustafson S, Kendall G, Krasnogor N (2004) Is increased diversity in genetic programming beneficial? an analysis of lineage selection. Ph.D. thesis, University of Nottingham, UKGoogle Scholar
  4. Castelli M, Manzoni L, Silva S, Vanneschi L (2010) A comparison of the generalization ability of different genetic programming frameworks. In: Evolutionary Computation (CEC), 2010 IEEE Congress on, pp 1–8Google Scholar
  5. Castelli M, Manzoni L, Silva S, Vanneschi L (2011) A quantitative study of learning and generalization in genetic programming. In: Silva S, Foster JA, Nicolau M, Machado P, Giacobini M (eds) EuroGP., Lecture notes in computer scienceSpringer, Berlin, pp 25–36Google Scholar
  6. Dempsey I, O’Neill M, Brabazon A (2009) Foundations in grammatical evolution for dynamic environments, vol 194., Studies in computational intelligenceSpringer, BerlinGoogle Scholar
  7. Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18CrossRefGoogle Scholar
  8. Doucette J, Heywood MI (2010) Novelty-Based Fitness: An Evaluation under the Santa Fe Trail. In: Esparcia-Alcázar AI, Ekárt A, Silva S, Dignum S, Uyar AS (eds) Genetic Programming: 13th European Conference, EuroGP 2010, Istanbul, Turkey, April 7-9, 2010. Proceedings. Springer, Berlin, Heidelberg, pp 50–61Google Scholar
  9. Duda RO, Hart PE, Stork DG (2000) Pattern classification, 2nd edn. Wiley, New JerseyzbMATHGoogle Scholar
  10. Francone FD, Nordin P, Banzhaf W (1996) Benchmarking the generalization capabilities of a compiling genetic programming system using sparse data sets. Proceedings of the 1st annual conference on genetic programming. MIT Press, Cambridge, pp 72–80Google Scholar
  11. Gathercole C, Ross P (1994) Dynamic training subset selection for supervised learning in genetic programming. In: Proceedings of the international conference on evolutionary computation. The third conference on parallel problem solving From nature: parallel problem solving from nature, PPSN III, Springer, London, pp 312–321Google Scholar
  12. Georgiou L (2012) Constituent grammatical evolution. Ph.D. thesis, School of computer science, Bangor University, BangorGoogle Scholar
  13. Georgiou L, Teahan WJ (2006) jge–a java implementation of grammatical evolution. 10th WSEAS international conference on systems. Greece, Athens, pp 534–869Google Scholar
  14. Georgiou L, Teahan WJ (2010) Grammatical evolution and the santa fe trail problem. International conference on evolutionary computation (ICEC). SciTePress, Valencia, Spain, pp 10–19Google Scholar
  15. Goldberg DE (1987) Simple genetic algorithms and the minimal, deceptive problem. In: Davis L (ed) Genetic algorithms and simulated annealing., Research notes in artificial intelligencePitman, London, pp 74–88Google Scholar
  16. Gomes J, Mariano P, Christensen AL (2015) Devising effective novelty search algorithms: a comprehensive empirical study. In: Proceedings of the 2015 annual conference on genetic and evolutionary computation, GECCO’15, ACM, New York (2015), pp 943–950Google Scholar
  17. Gomes J, Urbano P, Christensen A (2013) Evolution of swarm robotics systems with novelty search. Swarm Intell 7(2–3):115–144CrossRefGoogle Scholar
  18. Gonçalves I, Silva S (2011) Experiments on controlling overfitting in genetic programming. In: 15th Portuguese conference on artificial intelligence (EPIA 2011)Google Scholar
  19. Gonçalves I, Silva S (2011) Experiments on controlling overfitting in genetic programming. In: 15th Portuguese conference on artificial intelligence. EPIA 2011Google Scholar
  20. Gonçalves I, Silva S (2013) Balancing learning and overfitting in genetic programming with interleaved sampling of training data. In: Krawiec K, Moraglio A, Hu T, Etaner-Uyar A, Hu B (eds) Genetic programming. Lecture notes in computer science, vol 7831. Springer, Berlin, pp 73–84Google Scholar
  21. Gonçalves I, Silva S, Fonseca C (2015) On the generalization ability of geometric semantic genetic programming. In: 18th European conference on genetic programming (EuroGP 2015). n/aGoogle Scholar
  22. Gonçalves I, Silva S, Melo J, Carreiras JAMB (2012) Random sampling technique for overfitting control in genetic programming. In: Moraglio A, Silva S, Krawiec K, Machado P, Cotta C (eds) Genetic programming. Lecture notes in computer science, vol 7244. Springer, Berlin, pp 218–229Google Scholar
  23. Kaelbling LP, Littman ML, Moore AP (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285Google Scholar
  24. Koza JR (1992) Genetic programming–on the programming of computers by means of natural selection. MIT Press, Cambridge, MA, USAGoogle Scholar
  25. Kushchu I (2002) An evaluation of evolutionary generalisation in genetic programming. Artif Intell Rev 18(1):3–14CrossRefzbMATHGoogle Scholar
  26. Kushchu I (2002) Genetic programming and evolutionary generalization. IEEE Trans Evol Comput 6(5):431–442CrossRefzbMATHGoogle Scholar
  27. Langdon W, Poli R (2001) Foundations of genetic programming. Springer, BerlinzbMATHGoogle Scholar
  28. Lehman J, Stanley K (2008) Exploiting open-endedness to solve problems through the search for novelty. In: Bullock S, Noble J, Watson R, Bedau MA (eds) Artificial life XI: proceedings of the eleventh international conference on the simulation and synthesis of living systems. MIT Press, Cambridge, pp 329–336Google Scholar
  29. Lehman J, Stanley KO (2010) Efficiently evolving programs through the search for novelty. In: Pelikan M, Branke J (eds) GECCO. ACM, New York, pp 837–844Google Scholar
  30. Lehman J, Stanley KO (2011) Abandoning objectives: evolution through the search for novelty alone. Evol Comput 19(2):189–223CrossRefGoogle Scholar
  31. Mahler S, Robilliard D, Fonlupt C (2005) Tarpeian bloat control and generalization accuracy. In: Keijzer M, Tettamanzi A, Collet P, van Hemert JI, Tomassini M (eds) Proceedings of the 8th European conference on genetic programming, vol 3447., Lecture notes in computer scienceSpringer, Lausanne, pp 203–214Google Scholar
  32. Martinez Y, Trujillo L, Naredo E, Legrand P (2014) A comparison of fitness-case sampling methods for symbolic regression with genetic programming. In: EVOLVE 2014, Beijing, ChinaGoogle Scholar
  33. Martnez Y, Naredo E, Trujillo L, Lpez EG (2013) Searching for novel regression functions. In: IEEE congress on evolutionary computation, pp 16–23Google Scholar
  34. Mouret JB, Doncieux S (2012) Encouraging behavioral diversity in evolutionary robotics: an empirical study. Evol Comput 20(1):91–133CrossRefGoogle Scholar
  35. Naik TR, Dabhi VK (2013) Improving generalization ability of genetic programming: comparative study. CoRR abs/1304.3779Google Scholar
  36. Naredo E, Trujillo L (2013) Searching for novel clustering programs. Proceedings of the 15th annual conference on genetic and evolutionary computation. GECCO’13, ACM, New York, pp 1093–1100Google Scholar
  37. Nelson AL, Barlow GJ, Doitsidis L (2009) Fitness functions in evolutionary robotics: a survey and analysis. Robot. Auton. Syst. 57(4):345–370CrossRefGoogle Scholar
  38. Nicoară ES (2009) Mechanisms to avoid the premature convergence of genetic algorithms. Pet—Gas Univ Ploiesti Bull, Math-Inform-Phys Ser 61(1):87–96Google Scholar
  39. Nolfi S, Floreano D (2000) Evolutionary robotics: the biology, intelligence, and technology. MIT Press, CambridgeGoogle Scholar
  40. O’Neill M, Ryan C (2001) Grammatical evolution. IEEE Trans Evol Comput 5(4):349–358CrossRefGoogle Scholar
  41. Robilliard D, Mahler S, Verhaghe D, Fonlupt C (2006) Santa fe trail hazards. In: Talbi EG, Liardet P, Collet P, Lutton E, Schoenauer M (eds) 7th international conference on artificial evolution EA 2005, vol 3871., Lecture notes in computer scienceSpringer, Lille, pp 1–12Google Scholar
  42. Rosca J (1996) Generality versus size in genetic programming. In: Koza JR, Goldberg DE, Fogel DB, Riolo RL (eds) Genetic programming 1996: proceedings of the first annual conference. MIT Press, Stanford University, CA, pp 381–387Google Scholar
  43. Shorten D, Nitschke G (2015) Evolving generalised maze solvers. In: Mora AM, Squillero G (eds) Applications of evolutionary computation. Lecture notes in computer science, vol 9028. Springer, Berlin, pp 783–794Google Scholar
  44. Spector L (2012) Assessment of problem modality by differential performance of lexicase selection in genetic programming: a preliminary report. Proceedings of the 14th annual conference companion on genetic and evolutionary computation. GECCO’12, ACM, New York, pp 401–408Google Scholar
  45. Trujillo L, Olague G, Lutton E, de Vega FF (2008) Behavior-based speciation for evolutionary robotics. In: GECCO, pp 297–298Google Scholar
  46. Trujillo L, Olague G, Lutton E, de Vega FF, Dozal L, Clemente E (2011) Speciation in behavioral space for evolutionary robotics. J Intell Robot Syst 64(3–4):323–351CrossRefGoogle Scholar
  47. Trujillo L, Silva S, Legrand P, Vanneschi L (2011) An empirical study of functional complexity as an indicator of overfitting in genetic programming. In: Silva S, Foster JA, Nicolau M, Machado P, Giacobini M (eds) EuroGP. Lecture notes in computer science, vol 6621. Springer, Berlin, pp 262–273Google Scholar
  48. Urbano P, Loukas G (2013) Improving grammatical evolution in santa fe trail using novelty search. In: Advances in artificial life, ECAL, pp 917–924Google Scholar
  49. Urbano P, Naredo E, Trujillo L (2014) Generalization in maze navigation using grammatical evolution and novelty search. In: Dediu AH, Lozano M, Martn-Vide C (eds) Theory and practice of natural computing. Lecture notes in computer science, vol 8890. Springer, Berlin, pp 35–46Google Scholar
  50. Uy NQ, Hien NT, Hoai NX, O’Neill M (2010) Improving the generalisation ability of genetic programming with semantic similarity based crossover. In: Proceedings of the 13th European conference on genetic programming. EuroGP’10, Springer, Berlin, pp 184–195Google Scholar
  51. Vanneschi L, Castelli M, Silva S (2010) Measuring bloat, overfitting and functional complexity in genetic programming. Proceedings of the 12th annual conference on genetic and evolutionary computation, GECCO’10. ACM, New York, pp 877–884CrossRefGoogle Scholar
  52. Velez R, Clune J (2014) Novelty search creates robots with general skills for exploration. Proceedings of the 2014 conference on genetic and evolutionary computation, GECCO’14. ACM, New York, pp 737–744Google Scholar
  53. Wilensky U (1999) Netlogo, Evanston, IL: Center for connected learning and computer-based modeling. http://ccl.northwestern.edu/netlogo. Accessed 27 Nov 2015

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Tree-LabInstituto Tecnológico de TijuanaTijuanaMexico
  2. 2.University of Lisboa, Faculty of Sciences, BioISI Biosystems and Integrative Sciences InstituteCampo Grande, LisboaPortugal

Personalised recommendations