Soft Computing

, Volume 17, Issue 5, pp 753–767 | Cite as

An examination of different fitness and novelty based selection methods for the evolution of neural networks

  • Benjamin Inden
  • Yaochu Jin
  • Robert Haschke
  • Helge Ritter
  • Bernhard Sendhoff
Methodologies and Application

Abstract

It has been suggested recently that it is a reasonable abstraction of evolutionary processes to use evolutionary algorithms that select individuals based on the novelty of their behavior instead of their fitness. Here we study the performance of fitness- and novelty-based search on several neuroevolution tasks. We also propose several new algorithms that select both for fit and for novel individuals, but without weighting these two criteria directly against each other. We find that behavioral speciation, behavioral near neutral speciation, and behavioral novelty speciation perform best on most tasks. Pure novelty search, as well as a number of hybrid methods without speciation mechanism, do not perform well on most tasks. Using behavioral criteria for speciation often yields better results than using genetic criteria.

Keywords

Neuroevolution Selection Novelty search Evolutionary robotics NEAT 

References

  1. Alba E, Troya JM (2002) Improving flexibility and efficiency by adding parallelism to genetic algorithms. Stat Comput 12:91–114MathSciNetCrossRefGoogle Scholar
  2. Banzhaf W, Beslon G, Christensen S, Foster JA, Kps F, Lefort V, Miller JF, Radman M, Ramsden JJ (2006) From artificial evolution to computational evolution: a research agenda. Nat Rev Genet 7:729–735CrossRefGoogle Scholar
  3. Barton N, Partridge L (2000) Limits to natural selection. BioEssays 22:1075–1084CrossRefGoogle Scholar
  4. Buason G, Bergfeldt N, Ziemke T (2005) Brains, bodies and beyond: competitive co-evolution of robot controllers, morphologies and environments. Genet Program Evolvable Mach 6:25–51CrossRefGoogle Scholar
  5. Clune J, Beckmann BE, Ofria C, Pennock RT (2009) Evolving coordinated quadruped gaits with the hyperneat generative encoding. In: Proceedings of the IEEE congress on evolutionary computingGoogle Scholar
  6. Cuccu G, Gomez F (2011) When novelty is not enough. In: Di Chio C, Cagnoni S, Cotta C, Ebner M, Ekárt A, Esparcia- Alcázar AI, Merelo JJ, Neri F, Preuss M, Richter H, Togelius J, Yannakakis GN (eds) Applications of evolutionary computation. Lecture notes in computer science, vol 6624. Springer, Heidelberg, pp 234-243Google Scholar
  7. Cuccu G, Gomez F, Glasmachers T (2011) Novelty restarts for evolution strategies. In: Proceedings of the IEEE congres on evolutionary computationGoogle Scholar
  8. Darwen P, Yao X (1995) A dilemma for fitness sharing with a scaling function. In: Proceedings of the 1995 conference on evolutionary computationGoogle Scholar
  9. De Jong KA (2006) Evolutionary computation—a unified approach. MIT Press, CambridgeGoogle Scholar
  10. Doncieux S, Mouret JB (2010) Behavioral diversity measures for evolutionary robotics. In: IEEE congress on evolutionary computationGoogle Scholar
  11. Floreano D, Drr P, Mattiussi C (2008) Neuroevolution: from architectures to learning. Evol Intel 1:47–62CrossRefGoogle Scholar
  12. Futuyma DJ (2005) Evolution. Sinauer AssociatesGoogle Scholar
  13. Goldberg D, Richardson J (1987) Genetic algorithms with sharing for multimodal optimization. In: Proceedings of the second international conference on genetic algorithmsGoogle Scholar
  14. Gomez F, Miikkulainen R (1997) Incremental evolution of complex general behavior. Adapt Behav 5:317–342CrossRefGoogle Scholar
  15. Gomez FJ (2009) Sustaining diversity using behavioral information distance. In: Proceedings of the genetic and evolutionary computation conferenceGoogle Scholar
  16. Gould SJ, Lewontin RC (1979) The spandrels of san marco and the panglossian paradigm: a critique of the adaptionist programme. Proc R Soc Lond B 205:581–598CrossRefGoogle Scholar
  17. Harding S, Banzhaf W (2008) Organic computing, chap. Artificial development. Springer, New YorkGoogle Scholar
  18. Inden B (2008) Neuroevolution and complexifying genetic architectures for memory and control tasks. Theory Biosci 127:187–194CrossRefGoogle Scholar
  19. Inden B, Jin Y, Haschke R, Ritter H (2010) Neatfields: evolution of neural fields. In: Proceedings of the conference on genetic and evolutionary computationGoogle Scholar
  20. Inden B, Jin Y, Haschke R, Ritter H (2011a) Evolution of multisensory integration in large neural fields. In: Tenth international conference on artificial evolutionGoogle Scholar
  21. Inden B, Jin Y, Haschke R, Ritter H (2011b) How evolved neural fields can exploit inherent regularity in multilegged robot locomotion tasks. In: Third world congres on nature and biologically inspired computationGoogle Scholar
  22. Inden B, Jin Y, Haschke R, Ritter H (2012) Evolving neural fields for problems with large input and output spaces. Neural Netw 28:24–39CrossRefGoogle Scholar
  23. Kauffman SA (1993) The origins of order—self-organization and selection in evolution. Oxford University Press, OxfordGoogle Scholar
  24. Lehman J, Stanley KO (2008) Exploiting open-endedness to solve problems through the search for novelty. In: Proceedings of the eleventh international conference on artificial lifeGoogle Scholar
  25. Lehman J, Stanley KO (2010) Revising the evolutionary computation abstraction: minimal criteria novelty search. In: Proceedings of the genetic and evolutionary computation conferenceGoogle Scholar
  26. Lehman J, Stanley KO (2011) Evolving a diversity of creatures through novelty search and local competition. In: Proceedings of the genetic and evolutionary computation conferenceGoogle Scholar
  27. Lynch M (2007) The frailty of adaptive hypotheses for the origins of organismal complexity. Proc Natl Acad Sci 104:8597–8604CrossRefGoogle Scholar
  28. Mattiussi C, Floreano D (2007) Analog genetic encoding for the evolution of circuits and networks. IEEE Trans Evol Comput 11:596–607CrossRefGoogle Scholar
  29. Miconi T (2009) Why coevolution doesn’t "work": superiority and progress in coevolution. In: Proceedings of the EuroGP conferenceGoogle Scholar
  30. Mouret JB (2009) Novelty-based multiobjectivization. In: Proceedings of the workshop on exploring new horizons in evolutionary design of robots, 2009 IEEE/RSJ international conference on intelligent robots and systemsGoogle Scholar
  31. Mouret JB, Doncieux S (2008) Incremental evolution of animat’s behaviors as a multi-objective optimization. In: Simulation of adaptive behaviorGoogle Scholar
  32. Nolfi S, Floreano D (2000) Evolutionary robotics—the biology, intelligence, and technology of self-organizing Machines. MIT Press, CambridgeGoogle Scholar
  33. Poli R, Langdon WB, McPhee NF (2008) A field guide to genetic programming. Published via http://lulu.com, http://www.gp-field-guide.org.uk
  34. Sareni B, Krähenbühl L (1998) Fitness sharing and niching methods revisited. IEEE Trans Evol Comput 2:97–106CrossRefGoogle Scholar
  35. Stanley K (2004) Efficient evolution of neural networks through complexification. PhD thesis, Report AI-TR-04-314, University of Texas at AustinGoogle Scholar
  36. Stanley K (2007) Compositional pattern producing networks: a novel abstraction of development. Genet Program Evolvable Mach 8:131–162Google Scholar
  37. Stanley K, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10:99–127CrossRefGoogle Scholar
  38. Stanley K, Miikkulainen R (2003) A taxonomy for artificial embryogeny. Artif Life 9:93–130CrossRefGoogle Scholar
  39. Tomassini M (2005) Spatially structured evolutionary algorithms—artificial evolution in space and time. Springer, BerlinGoogle Scholar
  40. Wieland AP (1991) Evolving controls for unstable systems. In: Touretzky D (ed) Connectionist models: proceedings of the 1990 Summer SchoolGoogle Scholar
  41. Yao X (1999) Evolving artificial neural networks. Proceedings of the IEEE 87:1423–1447CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Benjamin Inden
    • 1
  • Yaochu Jin
    • 2
  • Robert Haschke
    • 3
  • Helge Ritter
    • 3
  • Bernhard Sendhoff
    • 4
  1. 1.Research Institute for Cognition and RoboticsBielefeld UniversityBielefeldGermany
  2. 2.Department of ComputingUniversity of SurreyGuildfordUK
  3. 3.Neuroinformatics GroupBielefeld UniversityBielefeldGermany
  4. 4.Honda Research Institute EuropeOffenbach/MainGermany

Personalised recommendations