Skip to main content

Advertisement

SpringerLink
Log in

An examination of different fitness and novelty based selection methods for the evolution of neural networks

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

It has been suggested recently that it is a reasonable abstraction of evolutionary processes to use evolutionary algorithms that select individuals based on the novelty of their behavior instead of their fitness. Here we study the performance of fitness- and novelty-based search on several neuroevolution tasks. We also propose several new algorithms that select both for fit and for novel individuals, but without weighting these two criteria directly against each other. We find that behavioral speciation, behavioral near neutral speciation, and behavioral novelty speciation perform best on most tasks. Pure novelty search, as well as a number of hybrid methods without speciation mechanism, do not perform well on most tasks. Using behavioral criteria for speciation often yields better results than using genetic criteria.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  • Alba E, Troya JM (2002) Improving flexibility and efficiency by adding parallelism to genetic algorithms. Stat Comput 12:91–114

    Article  MathSciNet  Google Scholar 

  • Banzhaf W, Beslon G, Christensen S, Foster JA, Kps F, Lefort V, Miller JF, Radman M, Ramsden JJ (2006) From artificial evolution to computational evolution: a research agenda. Nat Rev Genet 7:729–735

    Article  Google Scholar 

  • Barton N, Partridge L (2000) Limits to natural selection. BioEssays 22:1075–1084

    Article  Google Scholar 

  • Buason G, Bergfeldt N, Ziemke T (2005) Brains, bodies and beyond: competitive co-evolution of robot controllers, morphologies and environments. Genet Program Evolvable Mach 6:25–51

    Article  Google Scholar 

  • Clune J, Beckmann BE, Ofria C, Pennock RT (2009) Evolving coordinated quadruped gaits with the hyperneat generative encoding. In: Proceedings of the IEEE congress on evolutionary computing

  • Cuccu G, Gomez F (2011) When novelty is not enough. In: Di Chio C, Cagnoni S, Cotta C, Ebner M, Ekárt A, Esparcia- Alcázar AI, Merelo JJ, Neri F, Preuss M, Richter H, Togelius J, Yannakakis GN (eds) Applications of evolutionary computation. Lecture notes in computer science, vol 6624. Springer, Heidelberg, pp 234-243

  • Cuccu G, Gomez F, Glasmachers T (2011) Novelty restarts for evolution strategies. In: Proceedings of the IEEE congres on evolutionary computation

  • Darwen P, Yao X (1995) A dilemma for fitness sharing with a scaling function. In: Proceedings of the 1995 conference on evolutionary computation

  • De Jong KA (2006) Evolutionary computation—a unified approach. MIT Press, Cambridge

  • Doncieux S, Mouret JB (2010) Behavioral diversity measures for evolutionary robotics. In: IEEE congress on evolutionary computation

  • Floreano D, Drr P, Mattiussi C (2008) Neuroevolution: from architectures to learning. Evol Intel 1:47–62

    Article  Google Scholar 

  • Futuyma DJ (2005) Evolution. Sinauer Associates

  • Goldberg D, Richardson J (1987) Genetic algorithms with sharing for multimodal optimization. In: Proceedings of the second international conference on genetic algorithms

  • Gomez F, Miikkulainen R (1997) Incremental evolution of complex general behavior. Adapt Behav 5:317–342

    Article  Google Scholar 

  • Gomez FJ (2009) Sustaining diversity using behavioral information distance. In: Proceedings of the genetic and evolutionary computation conference

  • Gould SJ, Lewontin RC (1979) The spandrels of san marco and the panglossian paradigm: a critique of the adaptionist programme. Proc R Soc Lond B 205:581–598

    Article  Google Scholar 

  • Harding S, Banzhaf W (2008) Organic computing, chap. Artificial development. Springer, New York

  • Inden B (2008) Neuroevolution and complexifying genetic architectures for memory and control tasks. Theory Biosci 127:187–194

    Article  Google Scholar 

  • Inden B, Jin Y, Haschke R, Ritter H (2010) Neatfields: evolution of neural fields. In: Proceedings of the conference on genetic and evolutionary computation

  • Inden B, Jin Y, Haschke R, Ritter H (2011a) Evolution of multisensory integration in large neural fields. In: Tenth international conference on artificial evolution

  • Inden B, Jin Y, Haschke R, Ritter H (2011b) How evolved neural fields can exploit inherent regularity in multilegged robot locomotion tasks. In: Third world congres on nature and biologically inspired computation

  • Inden B, Jin Y, Haschke R, Ritter H (2012) Evolving neural fields for problems with large input and output spaces. Neural Netw 28:24–39

    Article  Google Scholar 

  • Kauffman SA (1993) The origins of order—self-organization and selection in evolution. Oxford University Press, Oxford

  • Lehman J, Stanley KO (2008) Exploiting open-endedness to solve problems through the search for novelty. In: Proceedings of the eleventh international conference on artificial life

  • Lehman J, Stanley KO (2010) Revising the evolutionary computation abstraction: minimal criteria novelty search. In: Proceedings of the genetic and evolutionary computation conference

  • Lehman J, Stanley KO (2011) Evolving a diversity of creatures through novelty search and local competition. In: Proceedings of the genetic and evolutionary computation conference

  • Lynch M (2007) The frailty of adaptive hypotheses for the origins of organismal complexity. Proc Natl Acad Sci 104:8597–8604

    Article  Google Scholar 

  • Mattiussi C, Floreano D (2007) Analog genetic encoding for the evolution of circuits and networks. IEEE Trans Evol Comput 11:596–607

    Article  Google Scholar 

  • Miconi T (2009) Why coevolution doesn’t "work": superiority and progress in coevolution. In: Proceedings of the EuroGP conference

  • Mouret JB (2009) Novelty-based multiobjectivization. In: Proceedings of the workshop on exploring new horizons in evolutionary design of robots, 2009 IEEE/RSJ international conference on intelligent robots and systems

  • Mouret JB, Doncieux S (2008) Incremental evolution of animat’s behaviors as a multi-objective optimization. In: Simulation of adaptive behavior

  • Nolfi S, Floreano D (2000) Evolutionary robotics—the biology, intelligence, and technology of self-organizing Machines. MIT Press, Cambridge

  • Poli R, Langdon WB, McPhee NF (2008) A field guide to genetic programming. Published via http://lulu.com, http://www.gp-field-guide.org.uk

  • Sareni B, Krähenbühl L (1998) Fitness sharing and niching methods revisited. IEEE Trans Evol Comput 2:97–106

    Article  Google Scholar 

  • Stanley K (2004) Efficient evolution of neural networks through complexification. PhD thesis, Report AI-TR-04-314, University of Texas at Austin

  • Stanley K (2007) Compositional pattern producing networks: a novel abstraction of development. Genet Program Evolvable Mach 8:131–162

    Google Scholar 

  • Stanley K, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10:99–127

    Article  Google Scholar 

  • Stanley K, Miikkulainen R (2003) A taxonomy for artificial embryogeny. Artif Life 9:93–130

    Article  Google Scholar 

  • Tomassini M (2005) Spatially structured evolutionary algorithms—artificial evolution in space and time. Springer, Berlin

  • Wieland AP (1991) Evolving controls for unstable systems. In: Touretzky D (ed) Connectionist models: proceedings of the 1990 Summer School

  • Yao X (1999) Evolving artificial neural networks. Proceedings of the IEEE 87:1423–1447

    Article  Google Scholar 

Download references

Acknowledgments

Benjamin Inden gratefully acknowledges the financial support from Honda Research Institute Europe for the project “Co-Evolution of Neural and Morphological Development for Grasping in Changing Environments”. Pole balancing code was adapted from the NEAT implementation by Kenneth Stanley.

Author information

Authors and Affiliations

  1. Research Institute for Cognition and Robotics, Bielefeld University, Universitätsstr. 25, 33615, Bielefeld, Germany

    Benjamin Inden

  2. Department of Computing, University of Surrey, Guildford, UK

    Yaochu Jin

  3. Neuroinformatics Group, Bielefeld University, Bielefeld, Germany

    Robert Haschke & Helge Ritter

  4. Honda Research Institute Europe, Offenbach/Main, Germany

    Bernhard Sendhoff

Authors
  1. Benjamin Inden
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaochu Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert Haschke
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Helge Ritter
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bernhard Sendhoff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin Inden.

Additional information

Communicated by G. Acampora.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Inden, B., Jin, Y., Haschke, R. et al. An examination of different fitness and novelty based selection methods for the evolution of neural networks. Soft Comput 17, 753–767 (2013). https://doi.org/10.1007/s00500-012-0960-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-012-0960-z

Keywords

Search

Navigation