Skip to main content
SpringerLink
Log in

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

  • Special Issue
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

Evolutionary robotics (ER) is often viewed as the application of a family of black-box optimization algorithms—evolutionary algorithms—to the design of robots, or parts of robots. When considering ER as black-box optimization, the selective pressure is mainly driven by a user-defined, black-box fitness function, and a domain-independent selection procedure. However, most ER experiments face similar challenges in similar setups: the selective pressure, and, in particular, the fitness function, is not a pure user-defined black box. The present review shows that, because ER experiments share common features, selective pressures for ER are a subject of research on their own. The literature has been split into two categories: goal refiners, aimed at changing the definition of a good solution, and process helpers, designed to help the search process. Two sub-categories are further considered: task-specific approaches, which require knowledge on how to solve the task and task-agnostic ones, which do not need it. Besides highlighting the diversity of the approaches and their respective goals, the present review shows that many task-agnostic process helpers have been proposed during the last years, thus bringing us closer to the goal of a fully automated robot behavior design process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. At this modeling level, it can be hypothesized that the morphology can be included in \(u\).

  2. http://www.ode.org/.

  3. http://bulletphysics.org/wordpress/.

  4. Some process helpers may have side effects and change the optimum of the fitness function, whereas it was not the intent of its authors. They are here considered to be helper processes as long as such optimum modifications are not straightforward and have not been clearly identified.

  5. In these studies, a model of the robot is learned before launching the EA. It was put in this category as, after the initial training—independent from the EA—the simulation model was not updated.

  6. The term “red queen effect” is a reference to a statement made by the Red Queen to Alice In Lewis Carrol’s Through the Looking-Glass [30]: “Now, here, you see, it takes all the running you can do, to keep in the same place.”

References

  1. Alpaydin E (2004) Introduction to machine learning. The MIT Press

  2. Angeline PJ (2000) Competitive fitness evaluation. In: Back T, Fogel DB, Michalewicz Z (eds) Evolutionary computation, vol 2. Taylor & Francis, London, pp 12–14

  3. Auerbach JE, Bongard JC (2009) How robot morphology and training order affect the learning of multiple behaviors. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC’2009), pp 39–46

  4. Auerbach JE, Bongard JC (2012) On the relationship between environmental and mechanical complexity in evolved robots. In: Proceedings of artificial life conference (ALife XIII), pp 309–316

  5. Auerbach JE, Bongard JC (2012) On the relationship between environmental and morphological complexity in evolved robots. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’12). ACM Press, New York, NY, USA, pp 521–528

  6. Bajaj D, Ang M (2000) An incremental approach in evolving robot behavior. In: Proceedings of the international conference on control, automation, robotics and vision (ICARCV’2000)

  7. Barate R, Manzanera A (2009) Evolution of visual controllers for obstacle avoidance in mobile robotics. Evoluti Intell 2(3):85–102

    Article  Google Scholar 

  8. Barlow GJ, Oh CK, Grant E (2004) Incremental evolution of autonomous controllers for unmanned aerial vehicles using multi-objective genetic programming. In: Proceedings of IEEE conference on cybernetics and intelligent systems (CIS’2004), vol 2, pp 689–694

  9. Berlanga A, Sanchis A, Isasi P, Molina JM (2000) A general learning co-evolution method to generalize autonomous robot navigation behavior. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC’2000), pp 769–776

  10. Berlanga A, Sanchis A, Isasi P, Molina JM (2002) Neural network controller against environment: a coevolutive approach to generalize robot navigation behavior. J Intell Robot Syst 33(2):139–166

    Article  MATH  Google Scholar 

  11. Blanchard P, Devaney RL, Hall GR (2006) Differential equations. Thompson, London

    Google Scholar 

  12. Boeing A, Braunl T (2012) Leveraging multiple simulators for crossing the reality gap. In: Proceedings of international conference on control, automation, robotics and vision (ICARV’2012), pp 1113–1119

  13. Bongard JC (2007) Action-selection and crossover strategies for self-modeling machines. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’07). ACM Press, pp 198–205

  14. Bongard JC (2008) Behavior chaining: incremental behavior integration for evolutionary robotics. In: Proceedings of artificial life conference (ALife XI), pp 64–71

  15. Bongard JC (2009) Accelerating self-modeling in cooperative robot teams. IEEE Trans Evol Comput 13(2):321–332

    Article  Google Scholar 

  16. Bongard JC (2010) The utility of evolving simulated robot morphology increases with task complexity for object manipulation. Artif Life 16(3):201–23

    Article  Google Scholar 

  17. Bongard JC (2011) Innocent until proven guilty: reducing robot shaping from polynomial to linear time. IEEE Trans Evol Comput 15(4):571–585  

  18. Bongard JC (2011) Morphological and environmental scaffolding synergize when evolving robot controllers. In: Proceedings of the international conference on genetic and evolutionary computation conference (GECCO’11), pp 179–186

  19. Bongard JC (2011) Morphological change in machines accelerates the evolution of robust behavior. Proc Natl Acad Sci 108(4):1234–1239

  20. Bongard JC (2013) Evolutionary robotics. Commun ACM 56(08):74–83

    Article  Google Scholar 

  21. Bongard JC, Hornby GS (2010) Guarding against premature convergence while accelerating evolutionary search. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’10), pp 111–118. ACM

  22. Bongard JC, Hornby GS (2013) Combining fitness-based search and user modeling in evolutionary robotics. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’13). ACM, pp 159–166

  23. Bongard JC, Lipson H (2004) Automated damage diagnosis and recovery for remote robotics. In: Proceedings of the international conference of robotics and automation (ICRA’2004), vol 4:, pp 545–3550

  24. Bongard JC, Lipson H (2004) Automated robot function recovery after unanticipated failure or environmental change using a minimum of hardware trials. In: Proceedings of evolvable hardware, pp 169–176

  25. Bongard JC, Lipson H (2004) Once more unto the breach: co-evolving a robot and its simulator. In: Proceedings of the international conference on the simulation and synthesis of living systems (ALIFE9), pp 57–62

  26. Bongard JC, Zykov V, Lipson H (2006) Resilient machines through continuous self-modeling. Science 314(5802):1118–1121

    Article  Google Scholar 

  27. Bredeche N, Montanier JM (2010) Environment-driven embodied evolution in a population of autonomous agents. In: Parallel problem solving from nature (PPSN XI). PPSN, vol 6239, pp 290–299

    Google Scholar 

  28. Buason G, Bergfeldt N, Ziemke T (2005) Brains, bodies, and beyond: competitive co-evolution of robot controllers, morphologies and environments. Genet Program Evol Mach 6(1):25–51

    Article  Google Scholar 

  29. Buason G, Ziemke T (2003) Competitive co-evolution of predator and prey sensory-motor systems. In: Applications of evolutionary computing, pp 605–615

  30. Carroll L (1866) Alice’s adventures in wonderland and through the looking glass. MacMillan, New York

  31. Celis S, Hornby GS, Bongard JC (2013) Avoiding local optima with user demonstrations and low-level control. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC’2013), pp 3403–3410

  32. Cliff D, Miller GF (1995) Tracking the red queen: measurements of adaptive progress in co-evolutionary simulations. In: Proceedings of the Third European Conference on Artificial Life. LCNS vol 929, pp 200–218

  33. Cliff D, Miller GF (1996) Co-evolution of pursuit and evasion II: simulation methods and results. In: Proceedings of the international conference on simulation of adaptive behavior (SAB’96)

  34. Clune J, Lipson H (2011) Evolving three-dimensional objects with a generative encoding inspired by developmental biology. In: Proceedings of the European conference on artificial life (ECAL’11)

  35. Clune J, Stanley KO, Pennock RT, Ofria C (2011) On the performance of indirect encoding across the continuum of regularity. IEEE Trans Evol Comput 15(3):346–367

  36. Cuccu G, Gomez F (2011) When novelty is not enough. In: Applications of evolutionary computation, pp 234–243

  37. Cully A, Mouret J-B (2013) Behavioral repertoire learning in robotics. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’13), pp 175–182

  38. Dawkins R (1976) The selfish gene. Oxford University Press, Oxford

  39. Dawkins R, Krebs JR (1979) Arms races between and within species. Proc R Soc B Biol Sci 205(1161):489–511

    Article  Google Scholar 

  40. De Garis H (1990) Building nanobrains with genetically programmed neural networks modules. In: Proceedings of the international joint conference on neural networks (IJCNN’1990), pp 511–516

  41. De Jong ED, Pollack JB (2004) Ideal evaluation from coevolution. Evol Comput 12(2):159–192

  42. De Jong ED, Watson RA, Pollack JB (2001) Reducing bloat and promoting diversity using multi-objective methods. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’01), pp 11–18. ACM

  43. De Jong KA (2006) Evolutionary computation: a unified approach, vol 262041944. MIT Press, Cambridge

  44. de Nardi R, Holland OE (2008) Coevolutionary modelling of a miniature rotorcraft. In: Proceedings of the international conference on intelligent autonomous systems (IAS10)

  45. de Nardi R, Togelius J, Holland OE, Lucas SM (2006) Evolution of neural networks for helicopter control: why modularity matters. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC’2006), pp 1799–1806. IEEE

  46. Deb K (2001) Multi-objectives optimization using evolutionnary algorithms. Wiley, London

    Google Scholar 

  47. Delarboulas P, Schoenauer M, Sebag M (2010) Open-ended evolutionary robotics: an information theoretic approach. In: Proceedings of parallel problem solving from nature (PPSN XI), vol 216342, pp 334–343

  48. Di Mario E, Navarro I, Martinoli A (2013) The effect of the environment in the synthesis of robotic controllers: a case study in multi-robot obstacle avoidance using distributed particle swarm optimization. In: Advances in artificial life, ECAL 2013, Sept 2013, pp 561–568

    Google Scholar 

  49. Doncieux S (2013) Transfer learning for direct policy search: a reward shaping approach. In: Proceedings of the IEEE conference on development and learning and epigenetic robotics (ICDL-EpiRob 2013)

  50. Doncieux S, Meyer J-A (2004) Evolving modular neural networks to solve challenging control problems. In: Proceedings of the fourth international ICSC symposium on engineering of intelligent systems (EIS 2004)

  51. Doncieux S, Mouret J-B (2010) Behavioral diversity measures for evolutionary robotics. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2010), pp 1303–1310

  52. Doncieux S, Mouret J-B (2013) Behavioral diversity with multiple behavioral distances. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2013), June 2013, pp 1427–1434. IEEE

  53. Doncieux S, Mouret J-B, Bredeche N, Padois V (2011) Evolutionary robotics: exploring new horizons. Springer, Berlin, pp 3–25

  54. Dozier G (2001) Evolving robot behavior via interactive evolutionary computation: from real-world to simulation. In: Proceedings of the ACM symposium on applied computing (SAC’2001), pp 340–344. ACM

  55. Duarte M, Oliveira S, Christensen AL (2012) Hierarchical evolution of robotic controllers for complex tasks. In: Proceedings of the IEEE conference on development and learning and epigenetic robotics (ICDL-EpiRob 2012)

  56. Eiben AE, Smith JE (2008) Introduction to evolutionary computing (natural computing series). Springer, Berlin

    Google Scholar 

  57. Farchy A, Barrett S, MacAlpine P, Stone P (2013) Humanoid robots learning to walk faster: from the real world to simulation and back. In: Proceedings of the international conference on autonomous agents and multi-agent systems (AAMAS’2013), pp 39–46

  58. Filliat D, Kodjabachian J, Meyer J-A (1999) Evolution of neural controllers for locomotion and obstacle-avoidance in a 6-legged robot. Connect Sci 11:223–240

    Article  Google Scholar 

  59. Floreano D, Dürr P, Mattiussi C (2008) Neuroevolution: from architectures to learning. Evol Intell 1(1):47–62

    Article  Google Scholar 

  60. Floreano D, Mattiussi C (2008) Bio-inspired artificial intelligence: theories, methods, and technologies. Intelligent robotics and autonomous agents. MIT Press, Cambridge

  61. Floreano D, Mondada F (1998) Evolutionary neurocontrollers for autonomous mobile robots. Neural Networks 11(7–8):1461–1478

    Article  Google Scholar 

  62. Floreano D, Nolfi S (1997) Adaptive behavior in competing co-evolving species. In: Proceedings of the European conference on artificial life (ECAL’97), pp 378–387

  63. Floreano D, Nolfi S (1997) God save the red queen! Competition in co-evolutionary robotics. In: Proceedings of the 2nd conference on genetic programming, vol 5

  64. Floreano D, Nolfi S, Mondada F (1998) Competitive co-evolutionary robotics: from theory to practice. In: Proceedings of the international conference on simulation of adaptive behavior (SAB98), pp 515–524

  65. Floreano D, Nolfi S, Mondada F (2001) Co-evolution and ontogenetic change in competing robots. In: Advances in the evolutionary synthesis of intelligent agents, pp 273–306

  66. Fogel LJ, Owens AJ, Walsh MJ (1966) Artificial intelligence through simulated evolution. Wiley, New York

    MATH  Google Scholar 

  67. Friedrich T, Oliveto PS, Sudholt D, Witt C (2008) Theoretical analysis of diversity mechanisms for global exploration. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’08), pp 945–952. ACM

  68. Goldberg DE (1987) Simple genetic algorithms and the minimal, deceptive problem. In: Davis L (eds) Genetic algorithms and simulated annealing. Morgan Kaufman, San Mato, pp 74–88

  69. Gomes J, Christensen AL (2013) Generic behaviour similarity measures for evolutionary swarm robotics. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO13), pp 199–206

  70. Gomes J, Urbano P, Christensen AL (2012) Introducing novelty search in evolutionary swarm robotics. In: Proceedings of the international conference on swarm intelligence (ANTS’2012), pp 85–96

  71. Gomes J, Urbano P, Christensen AL (2012) Progressive minimal criteria novelty search. In: Advances in artificial intelligence (IBERAMIA), pp 281–290

  72. Gomes J, Urbano P, Christensen AL (2013) Evolution of swarm robotics systems with novelty search. Swarm Intell 7(2–3):115–144

  73. Gomez FJ (2009) Sustaining diversity using behavioral information distance. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO09), pp 113–120. ACM

  74. Gomez FJ, Miikkulainen R (1997) Incremental evolution of complex general behavior. Adapt Behav 5(3–4):317–342

    Article  Google Scholar 

  75. Gomez FJ, Miikkulainen R (2004) Transfer of neuroevolved controllers in unstable domains. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO04), pp 957–968

  76. Gould SJ, Vrba ES (1982) Exaptation—a missing term in the science of form. Paleobiology 8(1):4–15

    Google Scholar 

  77. Grefenstette J, Daley R (1996) Methods for competitive and cooperative co-evolution. In: Adaptation, coevolution and learning in multiagent systems: papers from the 1996 AAAI Spring Symposium

  78. Gruau F, Quatramaran K (1997) Cellular encoding for interactive evolutionary robotics. In: Proceedings of European conference on artificial life (ECAL’97), pp 368–377

  79. Haasdijk E, Weel B, Eiben A (2013) Right on the monee. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO13), pp 207–214

  80. Hartland C, Bredeche N, Sebag M (2009) Memory-enhanced evolutionary robotics: the echo state network approach. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC’2009), pp 2788–2795

  81. Harvey I, Husbands P, Cliff D (1994) Seeing the light: artificial evolution; real vision. In: Cliff D, Husbands P, Meyer J-A, Wilson S (eds) Proceedings of the international conference on simulation of adaptive behavior (SAB94). MIT Press/Bradford Books, Cambridge, pp 392–401

  82. Hillis WD (1990) Co-evolving parasites improve simulated evolution as an optimization procedure. Phys D Nonlinear Phenom 42(1):228–234

    Article  Google Scholar 

  83. Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor

    Google Scholar 

  84. Hornby GS (2009) Steady-state ALPS for real-valued problems. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’09), pp 795–802, New York, NY, USA. ACM Press

  85. Hornby GS, Pollack JB (2002) Creating high-level components with a generative representation for body–brain evolution. Artif Life 8(3):223–246

    Article  Google Scholar 

  86. Hornby GS (2006) ALPS: the age-layered population structure for reducing the problem of premature convergence. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO’06), pp 815–822

  87. Hsu WH, Gustafson SM (2002) Genetic programming and multi-agent layered learning by reinforcements. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO02), pp 764–771

  88. Jakobi N (1997) Evolutionary robotics and the radical envelope of noise hypothesis. Adapt Behav 6(1):131–174

    Article  Google Scholar 

  89. Jakobi N, Husbands P, Harvey I (1995) Noise and the reality gap: the use of simulation in evolutionary robotics. In: Lecture notes in computer science, vol 929, pp 704–720

    Article  Google Scholar 

  90. Jensen MT (2004) Helper-objectives: using multi-objective evolutionary algorithms for single-objective optimisation. J Math Model Algorithms 3(4):323–347

    Article  MATH  MathSciNet  Google Scholar 

  91. Klyubin AS, Polani D, Nehaniv CL (2005) Empowerment: a universal agent-centric measure of control. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp 128–135

  92. Knowles J, Watson Richard A, Corne D (2001) Reducing local optima in single-objective problems by multi-objectivization. In Evolutionary multi-criterion optimization, pp 269–283. Springer

  93. Knowles JD, Watson RA, Corne DW (2001) Reducing local optima in single-objective problems by multi-objectivization. In: Proceedings of first international conference on evolutionary multi-criterion optimization 1993, pp 268–282

  94. Chavas J, Corne C, Horvai P, Kodjabachian J, Meyer JA (1998) Incremental evolution of neural controllers for robust obstacle-avoidance in Khepera. In: Husbands P, Meyer JA (eds) Proceedings of the first European workshop on evolutionary robotics - EvoRobot'98. LCNS vol 1468. Springer, pp 227–247

  95. Kodjabachian J, Meyer J-A (1997) Evolution and development of neural networks controlling locomotion, gradient-following, and obstacle-avoidance in artificial insects. IEEE Trans Neural Netw 9:796–812

    Article  Google Scholar 

  96. Koos S, Mouret J-B, Doncieux S (2009) Automatic system identification based on coevolution of models and tests. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2009), pp 560–567

  97. Koos S, Mouret J-B, Doncieux S (2010) Crossing the reality gap in evolutionary robotics by promoting transferable controllers. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO10), pp 119–126

  98. Koos S, Mouret J-B, Doncieux S (2013) The transferability spproach: crossing the reality gap in evolutionary robotics. IEEE Trans Evol Comput 17(1):122–145

    Article  Google Scholar 

  99. Koza JR (1993) Genetic programming: on the programming of computers by means of natural selection. MIT Press, London

    Google Scholar 

  100. Krcah P (2010) Solving deceptive tasks in robot body–brain co-evolution by searching for behavioral novelty. In: Proceedings of the international conference on intelligent systems design and applications (ISDA’2010), pp 284–289

  101. Kuhn TS (1962) The structure of scientific revolutions. University of Chicago Press, Chicago

    Google Scholar 

  102. Lee W (1999) Evolving complex robot behaviors. Inf Sci 121(1–2):1–25

    Article  Google Scholar 

  103. Lehman J, Risi S, Ambrosio DD, Stanley KO (2013) Encouraging reactivity to create robust machines. Adapt Behav 21:484–500

  104. Lehman J, Stanley KO (2008) Exploiting open-endedness to solve problems through the search for novelty. In: Proceedings of artificial life conference (ALife XI), pp 329–336

  105. Lehman J, Stanley KO (2010) Revising the evolutionary computation abstraction: minimal criteria novelty search. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO10), pp 103–110

  106. Lehman J, Stanley KO (2011) Abandoning objectives: evolution through the search for novelty alone. Evol Comput 19(2):189–223

    Article  Google Scholar 

  107. Lehman J, Stanley KO (2011) Evolving a diversity of creatures through novelty search and local competition. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO11), pp 211–218

  108. Lehman J, Stanley KO (2011) Novelty search and the problem with objectives. Genet Program Theory Pract IX, pp 37–56

  109. Lehman J, Stanley KO (2013) Evolvability is inevitable: increasing evolvability without the pressure to adapt. PloS One 8(4):e62186

    Article  Google Scholar 

  110. Lehman J, Stanley KO, Miikkulainen R (2013) Effective diversity maintenance in deceptive domains. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO13). ACM Press, New York, NY, USA, pp 215–222

  111. Lewis MA, Fagg AH, Solidum A (1992) Genetic programming approach to the construction of a neural network for control of a walking robot. In: Proceedings of the IEEE international conference on robotics and automation (ICRA’1992), pp 2618–2623

  112. Liapis A, Yannakakis GN, Togelius Julian (2013) Enhancements to constrained novelty search: two-population novelty search for generating game content. In: Proceedings of of the international conference on genetic and evolutionary computation (GECCO’13), pp 343–350

  113. Lipson H (2005) Evolutionary robotics and open-ended design automation. Biomimetics 17(9):129–155

    Article  Google Scholar 

  114. Lipson H, Pollack JB (2000) Automatic design and manufacture of robotic lifeforms. Nature 406:974–978

    Article  Google Scholar 

  115. Lund HH, Miglino O (1998) Evolving and breeding robots. Evol Robot, LCNS vol 1468. Springer, pp 192–210

  116. Lund HH, Miglino O, Pagliarini L, Billard A, Ijspeert A (1998) Evolutionary robotics—a children’s game. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC1998), pp 154–158. IEEE

  117. Mahfoud SW (1997) Niching methods. In: Bäck T, Fogel DB, Michalewicz Z (eds) Handbook of evolutionary computation. Taylor & Francis, London

    Google Scholar 

  118. Meyer J-A, Guillot A (2008) Biologically-inspired robots. In: Siciliano O, Khatib B (eds) Handbook of robotics. Springer, Berlin, pp 1–38

  119. Meyer J-A, Guillot A, Girard B, Khamassi M, Pirim P, Berthoz A (2005) The Psikharpax project: towards building an artificial rat. Robot Auton Syst 50(4):211–223

    Article  Google Scholar 

  120. Meyer J-A, Wilson S (1991) Simulation of adaptive behavior in animats: review and prospect. In: Proceedings of the international conference on simulation of adaptive behavior (SAB91), pp 2–14

  121. Miglino O, Lund HH, Nolfi S (1995) Evolving mobile robots in simulated and real environments. Artif Life 2(4):417–434

    Article  Google Scholar 

  122. Miller GF, Cliff D (1994) Protean behavior in dynamic games: arguments for the co-evolution of pursuit-evasion tactics. In: Proceedings of the international conference on simulation of adaptive behavior (SAB94), pp 411–420. MIT Press

  123. Moriarty DE, Miikkulainen R (1997) Forming neural networks through efficient and adaptive coevolution. Evol Comput 5(4):373–399

    Article  Google Scholar 

  124. Moriguchi H, Honiden S (2010) Sustaining behavioral diversity in NEAT. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO10), pp 611–618. ACM

  125. Moshaiov A, Ashram-Wittenberg A (2009) Multi-objective evolution of robot neuro-controllers. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2009), pp 1093–1100

  126. Mouret J-B (2011) Novelty-based multiobjectivization. In New horizons in evolutionary robotics: extended contributions of the 2009 EvoDeRob workshop, pp 139–154. Springer

  127. Mouret J-B, Doncieux S (2008) Incremental evolution of animats’ behaviors as a multi-objective optimization. In: Proceedings of the international conference on simulation of adaptive behavior (SAB08), vol 5040, pp 210–219. Springer

  128. Mouret J-B, Doncieux S (2008) MENNAG: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars. Evol Intell 1:187–207

    Article  Google Scholar 

  129. Mouret J-B, Doncieux S (2009) Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2009), pp 1161–1168

  130. Mouret J-B, Doncieux S (2009) Using behavioral exploration objectives to solve deceptive problems in neuro-evolution. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO09), pp 627–634. ACM

  131. Mouret J-B, Doncieux S (2012) Encouraging behavioral diversity in evolutionary robotics: an empirical study. Evol Comput 20(1):91–133

    Article  Google Scholar 

  132. Mouret J-B, Doncieux S, Meyer J-A (2006) Incremental evolution of target-following neuro-controllers for flapping-wing animats. In: Proceedings of the international conference on simulation of adaptive behavior (SAB06), pp 606–618

  133. Mouret J-B, Koos S, Doncieux S (2012) Crossing the reality gap: a short introduction to the transferability approach. In: Proceedings of the ALIFE workshop “evolution in physical systems”

  134. Nelson AL, Barlow GJ, Doitsidis L (2009) Fitness functions in evolutionary robotics: a survey and analysis. Robot Auton Syst 57(4):345–370

    Article  Google Scholar 

  135. Nelson AL, Grant E, Henderson TC (2004) Evolution of neural controllers for competitive game playing with teams of mobile robots. Robot Auton Syst 46(3):135–150

    Article  Google Scholar 

  136. Nitschke G (2003) Co-evolution of cooperation in a pursuit evasion game. In: Procedings of IEEE/RSJ international conference on intelligent robots and systems (IROS 2003) 2:2037–2042

  137. Nojima Y, Kojima F, Kubota N (2003) Trajectory generation for human-friendly behavior of partner robot using fuzzy evaluating interactive genetic algorithm. In: Proceedings of the IEEE international symposium on computational intelligence in robotics and automation. Computational intelligence in robotics and automation for the new millennium, vol 1, pp 306–311. IEEE

  138. Nolfi S (1997) Evolving non-trivial behaviors on real robots: a garbage collecting robot. Robot Auton Syst 22(3–4):187–198

    Article  Google Scholar 

  139. Nolfi S (2011) Co-evolving predator and prey robots. Adapt Behav 20(1):10–15

    Article  Google Scholar 

  140. Nolfi S, Floreano D (1998) How co-evolution can enhance the adaptive power of artificial evolution: implications for evolutionary robotics. In: Proceedings of the first European workshop on evolutionary robotics (EvoRobot98), pp 22–38

  141. Nolfi S, Floreano D (2001) Evolutionary robotics: the biology, intelligence, and technology of self-organizing machines. Bradford Book, Cambridge

  142. Oliveira MAC, Doncieux S, Mouret J-B, Peixoto dos Santos CM (2013) Optimization of humanoid walking controller: crossing the reality gap. In: Proceedings of the IEEE-RAS international conference on humanoid robots (Humanoids’2013)

  143. Oliveira MAC, Santos CP (2011) Multi-objective parameter CPG optimization for gait generation of a quadruped robot considering behavioral diversity. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS’2011), Sept 2011, pp 2286–2291. IEEE

  144. Ollion C (2013) Emergence of internal representations in evolutionary robotics: influence of multiple selective pressures. PhD thesis, Pierre and Marie Curie University

  145. Ollion C, Doncieux S (2011) Why and how to measure exploration in behavioral space. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO11), pp 267–274

  146. Ollion C, Doncieux S (2012) Towards behavioral consistency in neuroevolution. In: Proceedings of the international conference on simulation of adaptive behavior (SAB12), pp 177–186

  147. Ollion C, Pinville T, Doncieux S (2012) With a little help from selection pressures: evolution of memory in robot controllers. In: Proceedings of artificial life conference (ALife XIII), pp 407–414

  148. Ostergaard EH, Lund HH (2003) Co-evolving complex robot behavior. From biology to hardware. In: Evolvable systems, pp 308–319

  149. Oudeyer P-Y, Kaplan F, Hafner VV (2007) Intrinsic motivation systems for autonomous mental development. IEEE Trans Evol Comput 11(2):265–286

    Article  Google Scholar 

  150. Paredis J (2000) Coevolutionary algorithms. In: Evolutionary computation, vol 2. Taylor & Francis, London, pp 224–238

  151. Parker GB (2001) The incremental evolution of gaits for hexapod robots. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO01), pp 1114–1121

  152. Pfeifer R, Bongard JC (2006) How the body shapes the way we think. MIT Press, London

    Google Scholar 

  153. Pfeifer R, Lungarella M, Iida F (2007) Self-organization, embodiment, and biologically inspired robotics. Science 318(5853):1088–93

    Article  Google Scholar 

  154. Pinville T, Koos S, Mouret J-B, Doncieux S (2011) How to promote generalisation in evolutionary robotics: the ProGAb approach formalising the generalisation ability. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO11), pp 259–266

  155. Prokopenko M, Gerasimov V, Tanev I (2006) Evolving spatiotemporal coordination in a modular robotic system. In: Proceedings of the international conference on simulation of adaptive behavior (SAB06), pp 558–569

  156. Risi S, Vanderbleek SD, Hughes CE, Stanley KO (2009) How novelty search escapes the deceptive trap of learning to learn. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO09), pp 153–160. ACM

  157. Roberts RM (1989) Serendipity: accidental discoveries in science. Wiley Science Editions, London

    Google Scholar 

  158. Sakamoto K, Zhao Q (2006) A study on generating good environment patterns for evolving robot navigators. In: Proceedings of the IEEE international conference on systems, man and cybernetics, vol 4, pp 3280–3285, Oct 2006. IEEE

  159. Sareni B, Krahenbuhl L (1998) Fitness sharing and niching methods revisited. IEEE Trans Evol Comput 2(3):97–106

    Article  Google Scholar 

  160. Schaul T, Sun Y, Wierstra D, Gomez F, Schmidhuber J (2011) Curiosity-driven optimization. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2011), pp 1343–1349

  161. Schmidt M, Lipson H (2010) Age-fitness pareto optimization. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO10), pp 543–544

  162. Schwefel H-P (1977) Numerische Optimierung von Computer Modellen mittels der Evolutionsstrategie. Birkhäuser, Basel

  163. Secretan J, Beato N, D’Ambrosio DB, Rodriguez A, Campbell A, Folsom-Kovarik JT, Stanley KO (2011) Picbreeder: a case study in collaborative evolutionary exploration of design space. Evol Comput 19(3):373–403

  164. Siciliano O, Khatib B (2008) Handbook of robotics. Springer, Berlin

    Book  MATH  Google Scholar 

  165. Sims K (1994) Evolving virtual creatures. In: Proceedings of SIGGRAPH ’94, pp 15–22, New York, NY, USA. ACM Press

  166. Smith T, Husbands P, O’Shea M (2001) Neutral networks in an evolutionary robotics search space. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC2001), vol 1, pp 136–143

  167. Sperati V, Trianni V, Nolfi S (2008) Evolving coordinated group behaviours through maximisation of mean mutual information. Swarm Intell 2(2–4):73–95

    Article  Google Scholar 

  168. Sporns O, Lungarella M (2006) Evolving coordinated behavior by maximizing information structure. In: Proceedings of the Artificial Life Conference (ALIFE X), pp 323–329

  169. Stanley KO, D’Ambrosio DB, Gauci J (2009) A hypercube-based indirect encoding for evolving large-scale neural networks. Artif Life 15(2):185–212

    Article  Google Scholar 

  170. Stanley KO, Miikkulainen R (2002) Continual coevolution through complexification. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO02), pp 113–120

  171. Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10(2):99–127

    Article  Google Scholar 

  172. Stanley KO, Miikkulainen R (2004) Competitive coevolution through evolutionary complexification. J Artif Intell Res 21:63–100

    Google Scholar 

  173. Takagi H (2001) Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation. Proc IEEE 89(9):1275–1296

    Article  Google Scholar 

  174. Toffolo A, Benini E (2003) Genetic diversity as an objective in multi-objective evolutionary algorithms. Evoluti Comput 11(2):151–167

    Article  Google Scholar 

  175. Trujillo L, Olague G, Lutton E, De Vega FF (2008) Behavior-based speciation for evolutionary robotics. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO08), pp 297–298, New York, NY, USA. ACM

  176. Trujillo L, Olague G, Lutton E, De Vega FF (2008) Discovering several robot behaviors through speciation. In: Application of evolutionary computing: 4th European workshop on bio-inspired heuristics for design automation, pp 165–174. Springer

  177. Trujillo L, Olague G, Lutton E, Dozal L, Clemente E (2011) Speciation in behavioral space for evolutionary robotics. J Intell ZZ Robot Syst 64(3):323–351

    Article  Google Scholar 

  178. Uchibe E, Nakamura M, Asada M (1999) Cooperative and competitive behavior acquisition for mobile robots through co-evolution. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO99), vol 1, pp 425–430

  179. Urzelai J, Floreano D (1999) Incremental evolution with minimal resources. In: Proceedings of IKW99, pp 796–803

  180. Urzelai J, Floreano D, Dorigo M, Colombetti M (1998) Incremental robot shaping. Connect Sci 10(3):341–360

    Article  Google Scholar 

  181. Van Valen L (1973) Body size and numbers of plants and animals. Evolution 27(1):27–35

    Article  Google Scholar 

  182. Watson RA, Ficici SG, Pollack JB (2002) Embodied evolution: distributing an evolutionary algorithm in a population of robots. Robot Auton Syst 39:1–18

    Article  Google Scholar 

  183. Whiteson S, Kohl N, Miikkulainen R, Stone P (2005) Evolving soccer keepaway players through task decomposition. Mach Learn 59(1):5–30

  184. Winkeler JF, Manjunath BS (1998) Incremental evolution in genetic programming. In: Proceedings of the third annual conference on genetic programming, pp 403–411

  185. Woolley BG, Stanley KO (2011) On the deleterious effects of a priori objectives on evolution and representation. In: Proceedings of the international conference on genetic and evolutionary computation (GECCO11). ACM Press, New York, NY, USA, pp 957–964

  186. Woolley BG, Stanley KO (2014) A novel human–computer collaboration: combining novelty search with interactive evolution. In: Proceedings of GECCO’2014, pp 1–8

  187. Zagal JC, Delpiano J, Ruiz-del Solar J (2009) Self-modeling in humanoid soccer robots. Robot Auton Syst 57(8):819–827

    Article  Google Scholar 

  188. Zagal JC, Ruiz-del Solar J (2007) Combining simulation and reality in evolutionary robotics. J Intell Robot Syst 50(1):19–39

    Article  Google Scholar 

  189. Zagal JC, Ruiz-del Solar J, Vallejos P (2004) Back to reality: crossing the reality gap in evolutionary robotics. In: Proceedings of IAV

Download references

Acknowledgments

This work has been funded by the ANR Creadapt project (ANR-12-JS03-0009).

Author information

Authors and Affiliations

  1. Sorbonne Universités, UPMC Univ Paris 06, UMR 7222, ISIR, 75005, Paris, France

    Stephane Doncieux & Jean-Baptiste Mouret

  2. CNRS, UMR 7222, ISIR, 75005, Paris, France

    Stephane Doncieux & Jean-Baptiste Mouret

Authors
  1. Stephane Doncieux
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jean-Baptiste Mouret
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephane Doncieux.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doncieux, S., Mouret, JB. Beyond black-box optimization: a review of selective pressures for evolutionary robotics. Evol. Intel. 7, 71–93 (2014). https://doi.org/10.1007/s12065-014-0110-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-014-0110-x

Keywords

Search

Navigation