Evolutionary Intelligence

, Volume 7, Issue 2, pp 71–93 | Cite as

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

Special Issue

Abstract

Evolutionary robotics (ER) is often viewed as the application of a family of black-box optimization algorithms—evolutionary algorithms—to the design of robots, or parts of robots. When considering ER as black-box optimization, the selective pressure is mainly driven by a user-defined, black-box fitness function, and a domain-independent selection procedure. However, most ER experiments face similar challenges in similar setups: the selective pressure, and, in particular, the fitness function, is not a pure user-defined black box. The present review shows that, because ER experiments share common features, selective pressures for ER are a subject of research on their own. The literature has been split into two categories: goal refiners, aimed at changing the definition of a good solution, and process helpers, designed to help the search process. Two sub-categories are further considered: task-specific approaches, which require knowledge on how to solve the task and task-agnostic ones, which do not need it. Besides highlighting the diversity of the approaches and their respective goals, the present review shows that many task-agnostic process helpers have been proposed during the last years, thus bringing us closer to the goal of a fully automated robot behavior design process.

Keywords

Evolutionary robotics Selective pressures Goal refiner Process helper Task-specific Task-agnostic 

References

  1. 1.
    Alpaydin E (2004) Introduction to machine learning. The MIT PressGoogle Scholar
  2. 2.
    Angeline PJ (2000) Competitive fitness evaluation. In: Back T, Fogel DB, Michalewicz Z (eds) Evolutionary computation, vol 2. Taylor & Francis, London, pp 12–14Google Scholar
  3. 3.
    Auerbach JE, Bongard JC (2009) How robot morphology and training order affect the learning of multiple behaviors. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC’2009), pp 39–46Google Scholar
  4. 4.
    Auerbach JE, Bongard JC (2012) On the relationship between environmental and mechanical complexity in evolved robots. In: Proceedings of artificial life conference (ALife XIII), pp 309–316Google Scholar
  5. 5.
    Auerbach JE, Bongard JC (2012) On the relationship between environmental and morphological complexity in evolved robots. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’12). ACM Press, New York, NY, USA, pp 521–528Google Scholar
  6. 6.
    Bajaj D, Ang M (2000) An incremental approach in evolving robot behavior. In: Proceedings of the international conference on control, automation, robotics and vision (ICARCV’2000)Google Scholar
  7. 7.
    Barate R, Manzanera A (2009) Evolution of visual controllers for obstacle avoidance in mobile robotics. Evoluti Intell 2(3):85–102CrossRefGoogle Scholar
  8. 8.
    Barlow GJ, Oh CK, Grant E (2004) Incremental evolution of autonomous controllers for unmanned aerial vehicles using multi-objective genetic programming. In: Proceedings of IEEE conference on cybernetics and intelligent systems (CIS’2004), vol 2, pp 689–694Google Scholar
  9. 9.
    Berlanga A, Sanchis A, Isasi P, Molina JM (2000) A general learning co-evolution method to generalize autonomous robot navigation behavior. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC’2000), pp 769–776Google Scholar
  10. 10.
    Berlanga A, Sanchis A, Isasi P, Molina JM (2002) Neural network controller against environment: a coevolutive approach to generalize robot navigation behavior. J Intell Robot Syst 33(2):139–166CrossRefMATHGoogle Scholar
  11. 11.
    Blanchard P, Devaney RL, Hall GR (2006) Differential equations. Thompson, LondonGoogle Scholar
  12. 12.
    Boeing A, Braunl T (2012) Leveraging multiple simulators for crossing the reality gap. In: Proceedings of international conference on control, automation, robotics and vision (ICARV’2012), pp 1113–1119Google Scholar
  13. 13.
    Bongard JC (2007) Action-selection and crossover strategies for self-modeling machines. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’07). ACM Press, pp 198–205Google Scholar
  14. 14.
    Bongard JC (2008) Behavior chaining: incremental behavior integration for evolutionary robotics. In: Proceedings of artificial life conference (ALife XI), pp 64–71Google Scholar
  15. 15.
    Bongard JC (2009) Accelerating self-modeling in cooperative robot teams. IEEE Trans Evol Comput 13(2):321–332CrossRefGoogle Scholar
  16. 16.
    Bongard JC (2010) The utility of evolving simulated robot morphology increases with task complexity for object manipulation. Artif Life 16(3):201–23CrossRefGoogle Scholar
  17. 17.
    Bongard JC (2011) Innocent until proven guilty: reducing robot shaping from polynomial to linear time. IEEE Trans Evol Comput 15(4):571–585  Google Scholar
  18. 18.
    Bongard JC (2011) Morphological and environmental scaffolding synergize when evolving robot controllers. In: Proceedings of the international conference on genetic and evolutionary computation conference (GECCO’11), pp 179–186Google Scholar
  19. 19.
    Bongard JC (2011) Morphological change in machines accelerates the evolution of robust behavior. Proc Natl Acad Sci 108(4):1234–1239Google Scholar
  20. 20.
    Bongard JC (2013) Evolutionary robotics. Commun ACM 56(08):74–83CrossRefGoogle Scholar
  21. 21.
    Bongard JC, Hornby GS (2010) Guarding against premature convergence while accelerating evolutionary search. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’10), pp 111–118. ACMGoogle Scholar
  22. 22.
    Bongard JC, Hornby GS (2013) Combining fitness-based search and user modeling in evolutionary robotics. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’13). ACM, pp 159–166Google Scholar
  23. 23.
    Bongard JC, Lipson H (2004) Automated damage diagnosis and recovery for remote robotics. In: Proceedings of the international conference of robotics and automation (ICRA’2004), vol 4:, pp 545–3550Google Scholar
  24. 24.
    Bongard JC, Lipson H (2004) Automated robot function recovery after unanticipated failure or environmental change using a minimum of hardware trials. In: Proceedings of evolvable hardware, pp 169–176Google Scholar
  25. 25.
    Bongard JC, Lipson H (2004) Once more unto the breach: co-evolving a robot and its simulator. In: Proceedings of the international conference on the simulation and synthesis of living systems (ALIFE9), pp 57–62Google Scholar
  26. 26.
    Bongard JC, Zykov V, Lipson H (2006) Resilient machines through continuous self-modeling. Science 314(5802):1118–1121CrossRefGoogle Scholar
  27. 27.
    Bredeche N, Montanier JM (2010) Environment-driven embodied evolution in a population of autonomous agents. In: Parallel problem solving from nature (PPSN XI). PPSN, vol 6239, pp 290–299Google Scholar
  28. 28.
    Buason G, Bergfeldt N, Ziemke T (2005) Brains, bodies, and beyond: competitive co-evolution of robot controllers, morphologies and environments. Genet Program Evol Mach 6(1):25–51CrossRefGoogle Scholar
  29. 29.
    Buason G, Ziemke T (2003) Competitive co-evolution of predator and prey sensory-motor systems. In: Applications of evolutionary computing, pp 605–615Google Scholar
  30. 30.
    Carroll L (1866) Alice’s adventures in wonderland and through the looking glass. MacMillan, New YorkGoogle Scholar
  31. 31.
    Celis S, Hornby GS, Bongard JC (2013) Avoiding local optima with user demonstrations and low-level control. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC’2013), pp 3403–3410Google Scholar
  32. 32.
    Cliff D, Miller GF (1995) Tracking the red queen: measurements of adaptive progress in co-evolutionary simulations. In: Proceedings of the Third European Conference on Artificial Life. LCNS vol 929, pp 200–218Google Scholar
  33. 33.
    Cliff D, Miller GF (1996) Co-evolution of pursuit and evasion II: simulation methods and results. In: Proceedings of the international conference on simulation of adaptive behavior (SAB’96)Google Scholar
  34. 34.
    Clune J, Lipson H (2011) Evolving three-dimensional objects with a generative encoding inspired by developmental biology. In: Proceedings of the European conference on artificial life (ECAL’11)Google Scholar
  35. 35.
    Clune J, Stanley KO, Pennock RT, Ofria C (2011) On the performance of indirect encoding across the continuum of regularity. IEEE Trans Evol Comput 15(3):346–367Google Scholar
  36. 36.
    Cuccu G, Gomez F (2011) When novelty is not enough. In: Applications of evolutionary computation, pp 234–243Google Scholar
  37. 37.
    Cully A, Mouret J-B (2013) Behavioral repertoire learning in robotics. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’13), pp 175–182Google Scholar
  38. 38.
    Dawkins R (1976) The selfish gene. Oxford University Press, OxfordGoogle Scholar
  39. 39.
    Dawkins R, Krebs JR (1979) Arms races between and within species. Proc R Soc B Biol Sci 205(1161):489–511CrossRefGoogle Scholar
  40. 40.
    De Garis H (1990) Building nanobrains with genetically programmed neural networks modules. In: Proceedings of the international joint conference on neural networks (IJCNN’1990), pp 511–516Google Scholar
  41. 41.
    De Jong ED, Pollack JB (2004) Ideal evaluation from coevolution. Evol Comput 12(2):159–192Google Scholar
  42. 42.
    De Jong ED, Watson RA, Pollack JB (2001) Reducing bloat and promoting diversity using multi-objective methods. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’01), pp 11–18. ACMGoogle Scholar
  43. 43.
    De Jong KA (2006) Evolutionary computation: a unified approach, vol 262041944. MIT Press, CambridgeGoogle Scholar
  44. 44.
    de Nardi R, Holland OE (2008) Coevolutionary modelling of a miniature rotorcraft. In: Proceedings of the international conference on intelligent autonomous systems (IAS10)Google Scholar
  45. 45.
    de Nardi R, Togelius J, Holland OE, Lucas SM (2006) Evolution of neural networks for helicopter control: why modularity matters. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC’2006), pp 1799–1806. IEEEGoogle Scholar
  46. 46.
    Deb K (2001) Multi-objectives optimization using evolutionnary algorithms. Wiley, LondonGoogle Scholar
  47. 47.
    Delarboulas P, Schoenauer M, Sebag M (2010) Open-ended evolutionary robotics: an information theoretic approach. In: Proceedings of parallel problem solving from nature (PPSN XI), vol 216342, pp 334–343Google Scholar
  48. 48.
    Di Mario E, Navarro I, Martinoli A (2013) The effect of the environment in the synthesis of robotic controllers: a case study in multi-robot obstacle avoidance using distributed particle swarm optimization. In: Advances in artificial life, ECAL 2013, Sept 2013, pp 561–568Google Scholar
  49. 49.
    Doncieux S (2013) Transfer learning for direct policy search: a reward shaping approach. In: Proceedings of the IEEE conference on development and learning and epigenetic robotics (ICDL-EpiRob 2013)Google Scholar
  50. 50.
    Doncieux S, Meyer J-A (2004) Evolving modular neural networks to solve challenging control problems. In: Proceedings of the fourth international ICSC symposium on engineering of intelligent systems (EIS 2004)Google Scholar
  51. 51.
    Doncieux S, Mouret J-B (2010) Behavioral diversity measures for evolutionary robotics. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2010), pp 1303–1310Google Scholar
  52. 52.
    Doncieux S, Mouret J-B (2013) Behavioral diversity with multiple behavioral distances. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2013), June 2013, pp 1427–1434. IEEEGoogle Scholar
  53. 53.
    Doncieux S, Mouret J-B, Bredeche N, Padois V (2011) Evolutionary robotics: exploring new horizons. Springer, Berlin, pp 3–25Google Scholar
  54. 54.
    Dozier G (2001) Evolving robot behavior via interactive evolutionary computation: from real-world to simulation. In: Proceedings of the ACM symposium on applied computing (SAC’2001), pp 340–344. ACMGoogle Scholar
  55. 55.
    Duarte M, Oliveira S, Christensen AL (2012) Hierarchical evolution of robotic controllers for complex tasks. In: Proceedings of the IEEE conference on development and learning and epigenetic robotics (ICDL-EpiRob 2012)Google Scholar
  56. 56.
    Eiben AE, Smith JE (2008) Introduction to evolutionary computing (natural computing series). Springer, BerlinGoogle Scholar
  57. 57.
    Farchy A, Barrett S, MacAlpine P, Stone P (2013) Humanoid robots learning to walk faster: from the real world to simulation and back. In: Proceedings of the international conference on autonomous agents and multi-agent systems (AAMAS’2013), pp 39–46Google Scholar
  58. 58.
    Filliat D, Kodjabachian J, Meyer J-A (1999) Evolution of neural controllers for locomotion and obstacle-avoidance in a 6-legged robot. Connect Sci 11:223–240CrossRefGoogle Scholar
  59. 59.
    Floreano D, Dürr P, Mattiussi C (2008) Neuroevolution: from architectures to learning. Evol Intell 1(1):47–62CrossRefGoogle Scholar
  60. 60.
    Floreano D, Mattiussi C (2008) Bio-inspired artificial intelligence: theories, methods, and technologies. Intelligent robotics and autonomous agents. MIT Press, CambridgeGoogle Scholar
  61. 61.
    Floreano D, Mondada F (1998) Evolutionary neurocontrollers for autonomous mobile robots. Neural Networks 11(7–8):1461–1478CrossRefGoogle Scholar
  62. 62.
    Floreano D, Nolfi S (1997) Adaptive behavior in competing co-evolving species. In: Proceedings of the European conference on artificial life (ECAL’97), pp 378–387Google Scholar
  63. 63.
    Floreano D, Nolfi S (1997) God save the red queen! Competition in co-evolutionary robotics. In: Proceedings of the 2nd conference on genetic programming, vol 5Google Scholar
  64. 64.
    Floreano D, Nolfi S, Mondada F (1998) Competitive co-evolutionary robotics: from theory to practice. In: Proceedings of the international conference on simulation of adaptive behavior (SAB98), pp 515–524Google Scholar
  65. 65.
    Floreano D, Nolfi S, Mondada F (2001) Co-evolution and ontogenetic change in competing robots. In: Advances in the evolutionary synthesis of intelligent agents, pp 273–306Google Scholar
  66. 66.
    Fogel LJ, Owens AJ, Walsh MJ (1966) Artificial intelligence through simulated evolution. Wiley, New YorkMATHGoogle Scholar
  67. 67.
    Friedrich T, Oliveto PS, Sudholt D, Witt C (2008) Theoretical analysis of diversity mechanisms for global exploration. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’08), pp 945–952. ACMGoogle Scholar
  68. 68.
    Goldberg DE (1987) Simple genetic algorithms and the minimal, deceptive problem. In: Davis L (eds) Genetic algorithms and simulated annealing. Morgan Kaufman, San Mato, pp 74–88Google Scholar
  69. 69.
    Gomes J, Christensen AL (2013) Generic behaviour similarity measures for evolutionary swarm robotics. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO13), pp 199–206Google Scholar
  70. 70.
    Gomes J, Urbano P, Christensen AL (2012) Introducing novelty search in evolutionary swarm robotics. In: Proceedings of the international conference on swarm intelligence (ANTS’2012), pp 85–96Google Scholar
  71. 71.
    Gomes J, Urbano P, Christensen AL (2012) Progressive minimal criteria novelty search. In: Advances in artificial intelligence (IBERAMIA), pp 281–290Google Scholar
  72. 72.
    Gomes J, Urbano P, Christensen AL (2013) Evolution of swarm robotics systems with novelty search. Swarm Intell 7(2–3):115–144Google Scholar
  73. 73.
    Gomez FJ (2009) Sustaining diversity using behavioral information distance. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO09), pp 113–120. ACMGoogle Scholar
  74. 74.
    Gomez FJ, Miikkulainen R (1997) Incremental evolution of complex general behavior. Adapt Behav 5(3–4):317–342CrossRefGoogle Scholar
  75. 75.
    Gomez FJ, Miikkulainen R (2004) Transfer of neuroevolved controllers in unstable domains. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO04), pp 957–968Google Scholar
  76. 76.
    Gould SJ, Vrba ES (1982) Exaptation—a missing term in the science of form. Paleobiology 8(1):4–15Google Scholar
  77. 77.
    Grefenstette J, Daley R (1996) Methods for competitive and cooperative co-evolution. In: Adaptation, coevolution and learning in multiagent systems: papers from the 1996 AAAI Spring SymposiumGoogle Scholar
  78. 78.
    Gruau F, Quatramaran K (1997) Cellular encoding for interactive evolutionary robotics. In: Proceedings of European conference on artificial life (ECAL’97), pp 368–377Google Scholar
  79. 79.
    Haasdijk E, Weel B, Eiben A (2013) Right on the monee. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO13), pp 207–214Google Scholar
  80. 80.
    Hartland C, Bredeche N, Sebag M (2009) Memory-enhanced evolutionary robotics: the echo state network approach. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC’2009), pp 2788–2795Google Scholar
  81. 81.
    Harvey I, Husbands P, Cliff D (1994) Seeing the light: artificial evolution; real vision. In: Cliff D, Husbands P, Meyer J-A, Wilson S (eds) Proceedings of the international conference on simulation of adaptive behavior (SAB94). MIT Press/Bradford Books, Cambridge, pp 392–401Google Scholar
  82. 82.
    Hillis WD (1990) Co-evolving parasites improve simulated evolution as an optimization procedure. Phys D Nonlinear Phenom 42(1):228–234CrossRefGoogle Scholar
  83. 83.
    Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann ArborGoogle Scholar
  84. 84.
    Hornby GS (2009) Steady-state ALPS for real-valued problems. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’09), pp 795–802, New York, NY, USA. ACM PressGoogle Scholar
  85. 85.
    Hornby GS, Pollack JB (2002) Creating high-level components with a generative representation for body–brain evolution. Artif Life 8(3):223–246CrossRefGoogle Scholar
  86. 86.
    Hornby GS (2006) ALPS: the age-layered population structure for reducing the problem of premature convergence. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’06), pp 815–822Google Scholar
  87. 87.
    Hsu WH, Gustafson SM (2002) Genetic programming and multi-agent layered learning by reinforcements. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO02), pp 764–771Google Scholar
  88. 88.
    Jakobi N (1997) Evolutionary robotics and the radical envelope of noise hypothesis. Adapt Behav 6(1):131–174CrossRefGoogle Scholar
  89. 89.
    Jakobi N, Husbands P, Harvey I (1995) Noise and the reality gap: the use of simulation in evolutionary robotics. In: Lecture notes in computer science, vol 929, pp 704–720CrossRefGoogle Scholar
  90. 90.
    Jensen MT (2004) Helper-objectives: using multi-objective evolutionary algorithms for single-objective optimisation. J Math Model Algorithms 3(4):323–347CrossRefMATHMathSciNetGoogle Scholar
  91. 91.
    Klyubin AS, Polani D, Nehaniv CL (2005) Empowerment: a universal agent-centric measure of control. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp 128–135Google Scholar
  92. 92.
    Knowles J, Watson Richard A, Corne D (2001) Reducing local optima in single-objective problems by multi-objectivization. In Evolutionary multi-criterion optimization, pp 269–283. SpringerGoogle Scholar
  93. 93.
    Knowles JD, Watson RA, Corne DW (2001) Reducing local optima in single-objective problems by multi-objectivization. In: Proceedings of first international conference on evolutionary multi-criterion optimization 1993, pp 268–282Google Scholar
  94. 94.
    Chavas J, Corne C, Horvai P, Kodjabachian J, Meyer JA (1998) Incremental evolution of neural controllers for robust obstacle-avoidance in Khepera. In: Husbands P, Meyer JA (eds) Proceedings of the first European workshop on evolutionary robotics - EvoRobot'98. LCNS vol 1468. Springer, pp 227–247Google Scholar
  95. 95.
    Kodjabachian J, Meyer J-A (1997) Evolution and development of neural networks controlling locomotion, gradient-following, and obstacle-avoidance in artificial insects. IEEE Trans Neural Netw 9:796–812CrossRefGoogle Scholar
  96. 96.
    Koos S, Mouret J-B, Doncieux S (2009) Automatic system identification based on coevolution of models and tests. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2009), pp 560–567Google Scholar
  97. 97.
    Koos S, Mouret J-B, Doncieux S (2010) Crossing the reality gap in evolutionary robotics by promoting transferable controllers. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO10), pp 119–126Google Scholar
  98. 98.
    Koos S, Mouret J-B, Doncieux S (2013) The transferability spproach: crossing the reality gap in evolutionary robotics. IEEE Trans Evol Comput 17(1):122–145CrossRefGoogle Scholar
  99. 99.
    Koza JR (1993) Genetic programming: on the programming of computers by means of natural selection. MIT Press, LondonGoogle Scholar
  100. 100.
    Krcah P (2010) Solving deceptive tasks in robot body–brain co-evolution by searching for behavioral novelty. In: Proceedings of the international conference on intelligent systems design and applications (ISDA’2010), pp 284–289Google Scholar
  101. 101.
    Kuhn TS (1962) The structure of scientific revolutions. University of Chicago Press, ChicagoGoogle Scholar
  102. 102.
    Lee W (1999) Evolving complex robot behaviors. Inf Sci 121(1–2):1–25CrossRefGoogle Scholar
  103. 103.
    Lehman J, Risi S, Ambrosio DD, Stanley KO (2013) Encouraging reactivity to create robust machines. Adapt Behav 21:484–500Google Scholar
  104. 104.
    Lehman J, Stanley KO (2008) Exploiting open-endedness to solve problems through the search for novelty. In: Proceedings of artificial life conference (ALife XI), pp 329–336Google Scholar
  105. 105.
    Lehman J, Stanley KO (2010) Revising the evolutionary computation abstraction: minimal criteria novelty search. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO10), pp 103–110Google Scholar
  106. 106.
    Lehman J, Stanley KO (2011) Abandoning objectives: evolution through the search for novelty alone. Evol Comput 19(2):189–223CrossRefGoogle Scholar
  107. 107.
    Lehman J, Stanley KO (2011) Evolving a diversity of creatures through novelty search and local competition. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO11), pp 211–218Google Scholar
  108. 108.
    Lehman J, Stanley KO (2011) Novelty search and the problem with objectives. Genet Program Theory Pract IX, pp 37–56Google Scholar
  109. 109.
    Lehman J, Stanley KO (2013) Evolvability is inevitable: increasing evolvability without the pressure to adapt. PloS One 8(4):e62186CrossRefGoogle Scholar
  110. 110.
    Lehman J, Stanley KO, Miikkulainen R (2013) Effective diversity maintenance in deceptive domains. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO13). ACM Press, New York, NY, USA, pp 215–222Google Scholar
  111. 111.
    Lewis MA, Fagg AH, Solidum A (1992) Genetic programming approach to the construction of a neural network for control of a walking robot. In: Proceedings of the IEEE international conference on robotics and automation (ICRA’1992), pp 2618–2623Google Scholar
  112. 112.
    Liapis A, Yannakakis GN, Togelius Julian (2013) Enhancements to constrained novelty search: two-population novelty search for generating game content. In: Proceedings of of the international conference on genetic and evolutionary computation (GECCO’13), pp 343–350Google Scholar
  113. 113.
    Lipson H (2005) Evolutionary robotics and open-ended design automation. Biomimetics 17(9):129–155CrossRefGoogle Scholar
  114. 114.
    Lipson H, Pollack JB (2000) Automatic design and manufacture of robotic lifeforms. Nature 406:974–978CrossRefGoogle Scholar
  115. 115.
    Lund HH, Miglino O (1998) Evolving and breeding robots. Evol Robot, LCNS vol 1468. Springer, pp 192–210Google Scholar
  116. 116.
    Lund HH, Miglino O, Pagliarini L, Billard A, Ijspeert A (1998) Evolutionary robotics—a children’s game. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC1998), pp 154–158. IEEEGoogle Scholar
  117. 117.
    Mahfoud SW (1997) Niching methods. In: Bäck T, Fogel DB, Michalewicz Z (eds) Handbook of evolutionary computation. Taylor & Francis, LondonGoogle Scholar
  118. 118.
    Meyer J-A, Guillot A (2008) Biologically-inspired robots. In: Siciliano O, Khatib B (eds) Handbook of robotics. Springer, Berlin, pp 1–38Google Scholar
  119. 119.
    Meyer J-A, Guillot A, Girard B, Khamassi M, Pirim P, Berthoz A (2005) The Psikharpax project: towards building an artificial rat. Robot Auton Syst 50(4):211–223CrossRefGoogle Scholar
  120. 120.
    Meyer J-A, Wilson S (1991) Simulation of adaptive behavior in animats: review and prospect. In: Proceedings of the international conference on simulation of adaptive behavior (SAB91), pp 2–14Google Scholar
  121. 121.
    Miglino O, Lund HH, Nolfi S (1995) Evolving mobile robots in simulated and real environments. Artif Life 2(4):417–434CrossRefGoogle Scholar
  122. 122.
    Miller GF, Cliff D (1994) Protean behavior in dynamic games: arguments for the co-evolution of pursuit-evasion tactics. In: Proceedings of the international conference on simulation of adaptive behavior (SAB94), pp 411–420. MIT PressGoogle Scholar
  123. 123.
    Moriarty DE, Miikkulainen R (1997) Forming neural networks through efficient and adaptive coevolution. Evol Comput 5(4):373–399CrossRefGoogle Scholar
  124. 124.
    Moriguchi H, Honiden S (2010) Sustaining behavioral diversity in NEAT. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO10), pp 611–618. ACMGoogle Scholar
  125. 125.
    Moshaiov A, Ashram-Wittenberg A (2009) Multi-objective evolution of robot neuro-controllers. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2009), pp 1093–1100Google Scholar
  126. 126.
    Mouret J-B (2011) Novelty-based multiobjectivization. In New horizons in evolutionary robotics: extended contributions of the 2009 EvoDeRob workshop, pp 139–154. SpringerGoogle Scholar
  127. 127.
    Mouret J-B, Doncieux S (2008) Incremental evolution of animats’ behaviors as a multi-objective optimization. In: Proceedings of the international conference on simulation of adaptive behavior (SAB08), vol 5040, pp 210–219. SpringerGoogle Scholar
  128. 128.
    Mouret J-B, Doncieux S (2008) MENNAG: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars. Evol Intell 1:187–207CrossRefGoogle Scholar
  129. 129.
    Mouret J-B, Doncieux S (2009) Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2009), pp 1161–1168Google Scholar
  130. 130.
    Mouret J-B, Doncieux S (2009) Using behavioral exploration objectives to solve deceptive problems in neuro-evolution. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO09), pp 627–634. ACMGoogle Scholar
  131. 131.
    Mouret J-B, Doncieux S (2012) Encouraging behavioral diversity in evolutionary robotics: an empirical study. Evol Comput 20(1):91–133CrossRefGoogle Scholar
  132. 132.
    Mouret J-B, Doncieux S, Meyer J-A (2006) Incremental evolution of target-following neuro-controllers for flapping-wing animats. In: Proceedings of the international conference on simulation of adaptive behavior (SAB06), pp 606–618Google Scholar
  133. 133.
    Mouret J-B, Koos S, Doncieux S (2012) Crossing the reality gap: a short introduction to the transferability approach. In: Proceedings of the ALIFE workshop “evolution in physical systems”Google Scholar
  134. 134.
    Nelson AL, Barlow GJ, Doitsidis L (2009) Fitness functions in evolutionary robotics: a survey and analysis. Robot Auton Syst 57(4):345–370CrossRefGoogle Scholar
  135. 135.
    Nelson AL, Grant E, Henderson TC (2004) Evolution of neural controllers for competitive game playing with teams of mobile robots. Robot Auton Syst 46(3):135–150CrossRefGoogle Scholar
  136. 136.
    Nitschke G (2003) Co-evolution of cooperation in a pursuit evasion game. In: Procedings of IEEE/RSJ international conference on intelligent robots and systems (IROS 2003) 2:2037–2042Google Scholar
  137. 137.
    Nojima Y, Kojima F, Kubota N (2003) Trajectory generation for human-friendly behavior of partner robot using fuzzy evaluating interactive genetic algorithm. In: Proceedings of the IEEE international symposium on computational intelligence in robotics and automation. Computational intelligence in robotics and automation for the new millennium, vol 1, pp 306–311. IEEEGoogle Scholar
  138. 138.
    Nolfi S (1997) Evolving non-trivial behaviors on real robots: a garbage collecting robot. Robot Auton Syst 22(3–4):187–198CrossRefGoogle Scholar
  139. 139.
    Nolfi S (2011) Co-evolving predator and prey robots. Adapt Behav 20(1):10–15CrossRefGoogle Scholar
  140. 140.
    Nolfi S, Floreano D (1998) How co-evolution can enhance the adaptive power of artificial evolution: implications for evolutionary robotics. In: Proceedings of the first European workshop on evolutionary robotics (EvoRobot98), pp 22–38Google Scholar
  141. 141.
    Nolfi S, Floreano D (2001) Evolutionary robotics: the biology, intelligence, and technology of self-organizing machines. Bradford Book, CambridgeGoogle Scholar
  142. 142.
    Oliveira MAC, Doncieux S, Mouret J-B, Peixoto dos Santos CM (2013) Optimization of humanoid walking controller: crossing the reality gap. In: Proceedings of the IEEE-RAS international conference on humanoid robots (Humanoids’2013)Google Scholar
  143. 143.
    Oliveira MAC, Santos CP (2011) Multi-objective parameter CPG optimization for gait generation of a quadruped robot considering behavioral diversity. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS’2011), Sept 2011, pp 2286–2291. IEEEGoogle Scholar
  144. 144.
    Ollion C (2013) Emergence of internal representations in evolutionary robotics: influence of multiple selective pressures. PhD thesis, Pierre and Marie Curie UniversityGoogle Scholar
  145. 145.
    Ollion C, Doncieux S (2011) Why and how to measure exploration in behavioral space. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO11), pp 267–274Google Scholar
  146. 146.
    Ollion C, Doncieux S (2012) Towards behavioral consistency in neuroevolution. In: Proceedings of the international conference on simulation of adaptive behavior (SAB12), pp 177–186Google Scholar
  147. 147.
    Ollion C, Pinville T, Doncieux S (2012) With a little help from selection pressures: evolution of memory in robot controllers. In: Proceedings of artificial life conference (ALife XIII), pp 407–414Google Scholar
  148. 148.
    Ostergaard EH, Lund HH (2003) Co-evolving complex robot behavior. From biology to hardware. In: Evolvable systems, pp 308–319Google Scholar
  149. 149.
    Oudeyer P-Y, Kaplan F, Hafner VV (2007) Intrinsic motivation systems for autonomous mental development. IEEE Trans Evol Comput 11(2):265–286CrossRefGoogle Scholar
  150. 150.
    Paredis J (2000) Coevolutionary algorithms. In: Evolutionary computation, vol 2. Taylor & Francis, London, pp 224–238Google Scholar
  151. 151.
    Parker GB (2001) The incremental evolution of gaits for hexapod robots. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO01), pp 1114–1121Google Scholar
  152. 152.
    Pfeifer R, Bongard JC (2006) How the body shapes the way we think. MIT Press, LondonGoogle Scholar
  153. 153.
    Pfeifer R, Lungarella M, Iida F (2007) Self-organization, embodiment, and biologically inspired robotics. Science 318(5853):1088–93CrossRefGoogle Scholar
  154. 154.
    Pinville T, Koos S, Mouret J-B, Doncieux S (2011) How to promote generalisation in evolutionary robotics: the ProGAb approach formalising the generalisation ability. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO11), pp 259–266Google Scholar
  155. 155.
    Prokopenko M, Gerasimov V, Tanev I (2006) Evolving spatiotemporal coordination in a modular robotic system. In: Proceedings of the international conference on simulation of adaptive behavior (SAB06), pp 558–569Google Scholar
  156. 156.
    Risi S, Vanderbleek SD, Hughes CE, Stanley KO (2009) How novelty search escapes the deceptive trap of learning to learn. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO09), pp 153–160. ACMGoogle Scholar
  157. 157.
    Roberts RM (1989) Serendipity: accidental discoveries in science. Wiley Science Editions, LondonGoogle Scholar
  158. 158.
    Sakamoto K, Zhao Q (2006) A study on generating good environment patterns for evolving robot navigators. In: Proceedings of the IEEE international conference on systems, man and cybernetics, vol 4, pp 3280–3285, Oct 2006. IEEEGoogle Scholar
  159. 159.
    Sareni B, Krahenbuhl L (1998) Fitness sharing and niching methods revisited. IEEE Trans Evol Comput 2(3):97–106CrossRefGoogle Scholar
  160. 160.
    Schaul T, Sun Y, Wierstra D, Gomez F, Schmidhuber J (2011) Curiosity-driven optimization. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2011), pp 1343–1349Google Scholar
  161. 161.
    Schmidt M, Lipson H (2010) Age-fitness pareto optimization. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO10), pp 543–544Google Scholar
  162. 162.
    Schwefel H-P (1977) Numerische Optimierung von Computer Modellen mittels der Evolutionsstrategie. Birkhäuser, BaselGoogle Scholar
  163. 163.
    Secretan J, Beato N, D’Ambrosio DB, Rodriguez A, Campbell A, Folsom-Kovarik JT, Stanley KO (2011) Picbreeder: a case study in collaborative evolutionary exploration of design space. Evol Comput 19(3):373–403Google Scholar
  164. 164.
    Siciliano O, Khatib B (2008) Handbook of robotics. Springer, BerlinCrossRefMATHGoogle Scholar
  165. 165.
    Sims K (1994) Evolving virtual creatures. In: Proceedings of SIGGRAPH ’94, pp 15–22, New York, NY, USA. ACM PressGoogle Scholar
  166. 166.
    Smith T, Husbands P, O’Shea M (2001) Neutral networks in an evolutionary robotics search space. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2001), vol 1, pp 136–143Google Scholar
  167. 167.
    Sperati V, Trianni V, Nolfi S (2008) Evolving coordinated group behaviours through maximisation of mean mutual information. Swarm Intell 2(2–4):73–95CrossRefGoogle Scholar
  168. 168.
    Sporns O, Lungarella M (2006) Evolving coordinated behavior by maximizing information structure. In: Proceedings of the Artificial Life Conference (ALIFE X), pp 323–329Google Scholar
  169. 169.
    Stanley KO, D’Ambrosio DB, Gauci J (2009) A hypercube-based indirect encoding for evolving large-scale neural networks. Artif Life 15(2):185–212CrossRefGoogle Scholar
  170. 170.
    Stanley KO, Miikkulainen R (2002) Continual coevolution through complexification. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO02), pp 113–120Google Scholar
  171. 171.
    Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10(2):99–127CrossRefGoogle Scholar
  172. 172.
    Stanley KO, Miikkulainen R (2004) Competitive coevolution through evolutionary complexification. J Artif Intell Res 21:63–100Google Scholar
  173. 173.
    Takagi H (2001) Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation. Proc IEEE 89(9):1275–1296CrossRefGoogle Scholar
  174. 174.
    Toffolo A, Benini E (2003) Genetic diversity as an objective in multi-objective evolutionary algorithms. Evoluti Comput 11(2):151–167CrossRefGoogle Scholar
  175. 175.
    Trujillo L, Olague G, Lutton E, De Vega FF (2008) Behavior-based speciation for evolutionary robotics. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO08), pp 297–298, New York, NY, USA. ACMGoogle Scholar
  176. 176.
    Trujillo L, Olague G, Lutton E, De Vega FF (2008) Discovering several robot behaviors through speciation. In: Application of evolutionary computing: 4th European workshop on bio-inspired heuristics for design automation, pp 165–174. SpringerGoogle Scholar
  177. 177.
    Trujillo L, Olague G, Lutton E, Dozal L, Clemente E (2011) Speciation in behavioral space for evolutionary robotics. J Intell ZZ Robot Syst 64(3):323–351CrossRefGoogle Scholar
  178. 178.
    Uchibe E, Nakamura M, Asada M (1999) Cooperative and competitive behavior acquisition for mobile robots through co-evolution. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO99), vol 1, pp 425–430Google Scholar
  179. 179.
    Urzelai J, Floreano D (1999) Incremental evolution with minimal resources. In: Proceedings of IKW99, pp 796–803Google Scholar
  180. 180.
    Urzelai J, Floreano D, Dorigo M, Colombetti M (1998) Incremental robot shaping. Connect Sci 10(3):341–360CrossRefGoogle Scholar
  181. 181.
    Van Valen L (1973) Body size and numbers of plants and animals. Evolution 27(1):27–35CrossRefGoogle Scholar
  182. 182.
    Watson RA, Ficici SG, Pollack JB (2002) Embodied evolution: distributing an evolutionary algorithm in a population of robots. Robot Auton Syst 39:1–18CrossRefGoogle Scholar
  183. 183.
    Whiteson S, Kohl N, Miikkulainen R, Stone P (2005) Evolving soccer keepaway players through task decomposition. Mach Learn 59(1):5–30Google Scholar
  184. 184.
    Winkeler JF, Manjunath BS (1998) Incremental evolution in genetic programming. In: Proceedings of the third annual conference on genetic programming, pp 403–411Google Scholar
  185. 185.
    Woolley BG, Stanley KO (2011) On the deleterious effects of a priori objectives on evolution and representation. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO11). ACM Press, New York, NY, USA, pp 957–964Google Scholar
  186. 186.
    Woolley BG, Stanley KO (2014) A novel human–computer collaboration: combining novelty search with interactive evolution. In: Proceedings of GECCO’2014, pp 1–8Google Scholar
  187. 187.
    Zagal JC, Delpiano J, Ruiz-del Solar J (2009) Self-modeling in humanoid soccer robots. Robot Auton Syst 57(8):819–827CrossRefGoogle Scholar
  188. 188.
    Zagal JC, Ruiz-del Solar J (2007) Combining simulation and reality in evolutionary robotics. J Intell Robot Syst 50(1):19–39CrossRefGoogle Scholar
  189. 189.
    Zagal JC, Ruiz-del Solar J, Vallejos P (2004) Back to reality: crossing the reality gap in evolutionary robotics. In: Proceedings of IAVGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Sorbonne UniversitésUPMC Univ Paris 06, UMR 7222, ISIRParisFrance
  2. 2.CNRSUMR 7222, ISIRParisFrance

Personalised recommendations