Evolution of Hybrid Robotic Controllers for Complex Tasks

  • Miguel Duarte
  • Sancho Moura Oliveira
  • Anders Lyhne Christensen


We propose an approach to the synthesis of hierarchical control systems comprising both evolved and manually programmed control for autonomous robots. We recursively divide the goal task into sub-tasks until a solution can be evolved or until a solution can easily be programmed by hand. Hierarchical composition of behavior allows us to overcome the fundamental challenges that typically prevent evolutionary robotics from being applied to complex tasks: bootstrapping the evolutionary process, avoiding deception, and successfully transferring control evolved in simulation to real robotic hardware. We demonstrate the proposed approach by synthesizing control systems for two tasks whose complexity is beyond state of the art in evolutionary robotics. The first task is a rescue task in which all behaviors are evolved. The second task is a cleaning task in which evolved behaviors are combined with a manually programmed behavior that enables the robot to open doors in the environment. We demonstrate incremental transfer of evolved control from simulation to real robotic hardware, and we show how our approach allows for the reuse of behaviors in different tasks.


Evolutionary robotics Hierarchical control Artificial evolution 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Becerra, J.A., Bellas, F., Reyes, J.S., Duro, R.J.: Complex behaviours through modulation in autonomous robot control. In Proceedings of the International Work-Conference on Artificial Neural Networks (IWANN), volume 3512 of Lecture Notes in Computer Science, pp 717–724. Springer, Berlin (2005)Google Scholar
  2. 2.
    Beer, R.D., Gallagher, J.C.: Evolving dynamical neural networks for adaptive behavior. Adapt. Behav. 1 (1), 91–122 (1992)CrossRefGoogle Scholar
  3. 3.
    Blynel, J., Floreano, D.: Exploring the T-Maze: Evolving learning-like robot behaviors using CTRNNs. In Applications of Evolutionary Computing, pp 593–604. Springer, Berlin (2003)Google Scholar
  4. 4.
    Bradski, G.: The OpenCV Library. Dr. Dobb’s J. Softw. Tools 25, 11 (2000)Google Scholar
  5. 5.
    Brooks, R.: A robust layered control system for a mobile robot. IEEE J. Robot. Autom. 2 (1), 14–23 (1986)CrossRefGoogle Scholar
  6. 6.
    Celis, S., Hornby, G.S, Bongard, J.: Avoiding local optima with user demonstrations and low-level control. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp 3403–3410. IEEE Press, Piscataway (2013)Google Scholar
  7. 7.
    Cheney, N., MacCurdy, R., Clune, J., Lipson, H.: Unshackling evolution: Evolving soft robots with multiple materials and a powerful generative encoding. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), pp. 167–174. ACM Press, New York (2013)Google Scholar
  8. 8.
    Christensen, A.L., Dorigo, M.: Incremental evolution of robot controllers for a highly integrated task. In From Animals to Animats 9: Proceedings of the International Conference on the Simulation of Adaptive Behavior (SAB), pp 473–484. Springer, Berlin (2006a)Google Scholar
  9. 9.
    Christensen, A.L., Dorigo, M.: Evolving an integrated phototaxis and hole avoidance behavior for a swarm-bot. In Proceedings of the International Conference on the Simulation & Synthesis of Living Systems (ALIFE), pp 248–254. MIT Press, Cambridge (2006b)Google Scholar
  10. 10.
    Christensen, A.L., Oliveira, S., Duartem, M.: JBotEvolver (2014).
  11. 11.
    de Nardi, R., Togelius, J., Holland, O.E., Lucas, S.M.: Evolution of neural networks for helicopter control, Why modularity matters. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp 1799–1806. IEEE Press, Piscataway (2006)Google Scholar
  12. 12.
    Duarte, M., Oliveira, S., Christensen, A.L.: Towards artificial evolution of complex behavior observed in insect colonies. In Proceedings of the Portuguese Conference on Artificial Intelligence (EPIA), pp 153–167. Springer, Berlin (2011)Google Scholar
  13. 13.
    Duarte, M., Oliveira, S., Christensen, A.L.: Hierarchical evolution of robotic controllers for complex tasks. In IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL), pp 1–6. IEEE Press, Piscataway (2012)Google Scholar
  14. 14.
    Duarte, M., Oliveira, S., Christensen, A.L.: Automatic synthesis of controllers for real robots based on preprogrammed behaviors. In From Animals to Animats 12: Proceedings of the International Conference on Adaptive Behaviour (SAB), pp 249–258. Springer, Berlin (2012)Google Scholar
  15. 15.
    Duarte, M., Oliveira, S., Christensen, A.L.: Videos and source code from real robot experiments (2014).
  16. 16.
    Duarte, M., Silva, F., Rodrigues, T., Oliveira, S.M., Christensen, A.L.: JBotEvolver: A versatile simulation platform for evolutionary robotics. In: Proceedings of the International Conference on the Simulation & Synthesis of Living Systems (ALIFE), pp 210–211. MIT Press, Cambridge (2014)Google Scholar
  17. 17.
    Er, M.J., Kee, B.H., Tan, C.C.: Design and development of an intelligent controller for a pole-balancing robot. Microprocess. Microsyst. 26 (9-10), 433–448 (2002)CrossRefGoogle Scholar
  18. 18.
    Floreano, D., Keller, L.: Evolution of adaptive behaviour in robots by means of Darwinian selection. PLoS Biol. 8 (1), 1–8 (2010)CrossRefGoogle Scholar
  19. 19.
    Floreano, D., Mondada, F.: Automatic creation of an autonomous agent, Genetic evolution of a neural-network driven robot. In From Animals to Animats 3: Proceedings of the International Conference on Simulation of Adaptive Behavior (SAB), pp 421–430. MIT Press, Cambridge (1994). USAGoogle Scholar
  20. 20.
    Floreano, D., Mondada, F.: Evolution of homing navigation in a real mobile robot. IEEE Trans. Syst. Man Cybern.–Part B: Cybern. 26 (3), 396–407 (1996)CrossRefGoogle Scholar
  21. 21.
    Floreano, D., Urzelai, J.: Evolutionary robots with on-line self-organization and behavioral fitness. Neural Netw. 13 (4–5), 431–443 (2000)CrossRefGoogle Scholar
  22. 22.
    Fogel, D.B., Fogel, L.J., Porto, V.W.: Evolving neural networks. Biol. Cybern. 63 (6), 487–493 (1990)CrossRefGoogle Scholar
  23. 23.
    Goldberg, D.E.: Genetic algorithms in search, optimization and machine learning. Addison-Wesley, MA (1989)zbMATHGoogle Scholar
  24. 24.
    Gomes, J., Urbano, P., Christensen, A.L.: Evolution of swarm robotics systems with novelty search. Swarm Intell. 7 (2-3), 115–144 (2013)CrossRefGoogle Scholar
  25. 25.
    Gomes, J., Urbano, P., Christensen, A.L.: PMCNS: using a progressively stricter fitness criterion to guide novelty search. In International Journal of Natural Computing Research, in press (2014)Google Scholar
  26. 26.
    Gomez, F., Miikkulainen, R.: Incremental evolution of complex general behavior. Adapt. Behav. 3-4 (5), 317–342 (1997)CrossRefGoogle Scholar
  27. 27.
    Groß, R., Bonani, M., Mondada, F., Dorigo, M.: Autonomous self-assembly in swarmbots. IEEE Trans. Robot. 22 (6), 1115–1130 (2006)CrossRefGoogle Scholar
  28. 28.
    Gutiérrez, A., Campo, A., Dorigo, M., Amor, D., Magdalena, L., Monasterio-Huelin, F.: An open localization and local communication embodied sensor. Sensors 8 (11), 7545–7563 (2008)CrossRefGoogle Scholar
  29. 29.
    Hamann, H., Schmickl, T., Crailsheim, K.: A hormone-based controller for evaluation-minimal evolution in decentrally controlled systems. Artif. Life 18 (2), 165–198 (2012)CrossRefMathSciNetGoogle Scholar
  30. 30.
    Hansen, L.K., Salamon, P.: Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12 (10), 993–1001 (1990)CrossRefGoogle Scholar
  31. 31.
    Harvey, I., Husbands, P., Cliff, D.: Seeing the light: artificial evolution, real vision. In From Animals to Animats 3: Proceedings of the International Conference on Simulation of Adaptive Behavior (SAB), pp 392–401. MIT Press, Cambridge (1994)Google Scholar
  32. 32.
    Heaton, J.: Encog machine learning framework (2008).
  33. 33.
    Hehn, M., D’Andrea, R.: A flying inverted pendulum. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp 763–770. IEEE Press, Piscataway (2011)Google Scholar
  34. 34.
    Hiller, J., Lipson, H.: Automatic design and manufacture of soft robots. IEEE Trans. Robot. 28 (2), 457–466 (2012)CrossRefGoogle Scholar
  35. 35.
    Hornby, G.S., Takamura, S., Yamamoto, T., Fujita, M.: Autonomous evolution of dynamic gaits with two quadruped robots. IEEE Trans. Robot. 21 (3), 402–410 (2005)CrossRefGoogle Scholar
  36. 36.
    Husbands, P.: Evolving robot behaviours with diffusing gas networks. In Proceedigs of the European Workshop Evolutionary Robotics (EvoRobot), pp 71–86. Springer, Berlin (1998)Google Scholar
  37. 37.
    Jakobi, N.: Evolutionary robotics and the radical envelope-of-noise hypothesis. Adapt. Behav. 6 (2), 325–368 (1997)CrossRefGoogle Scholar
  38. 38.
    Koos, S., Mouret, J.-B., Doncieux, S.: The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Trans. Evol. Comput. 17 (1), 122–145 (2013)CrossRefGoogle Scholar
  39. 39.
    Larsen, T., Hansen, S.T.: Evolving composite robot behaviour - a modular architecture. In Proceedings of the International Workshop on Robot Motion and Control (RoMoCo), pp 271–276. IEEE Press, Piscataway (2005)Google Scholar
  40. 40.
    Lee, W.-P.: Evolving complex robot behaviors. Inf. Sci. 121 (1-2), 1–25 (1999)CrossRefGoogle Scholar
  41. 41.
    Lehman, J., Stanley, K.O.: Abandoning objectives: Evolution through the search for novelty alone. Evol. Comput. 19 (2), 189–223 (2011)CrossRefGoogle Scholar
  42. 42.
    Meyer, J.-A., Husbands, P., Harvey, I.: Evolutionary robotics: a survey of applications and problems. In Proceedings of the European Workshop on Evolutionary Robotics (EvoRobot), pp 1–21. Springer, Berlin (1998)CrossRefGoogle Scholar
  43. 43.
    Miglino, O., Lund, H.H., Nolfi, S.: Evolving mobile robots in simulated and real environments. Artif. Life 2 (4), 417–434 (1996)CrossRefGoogle Scholar
  44. 44.
    Moioli, R.C., Vargas, P.A., Von Zuben, F.J., Husbands, P.: Towards the evolution of an artificial homeostatic system. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), pp 4023–4030. IEEE Press, Piscataway (2008)Google Scholar
  45. 45.
    Mondada, F., Bonani, M., Raemy, X., Pugh, J., Cianci, C., Klaptocz, A., Magnenat, S., Zufferey, J.-C., Floreano, D., Martinoli, A.: The e-puck, a robot designed for education in engineering. In Proceedings of the Conference on Autonomous Robot Systems and Competitions (ROBOTICA), pp 59–65 (2009). Instituto Politecnico de Castelo Branco, Castelo Branco, PortugalGoogle Scholar
  46. 46.
    Nakamura, H., Ishiguro, A., Uchilkawa, Y.: Evolutionary construction of behavior arbitration mechanisms based on dynamically-rearranging neural networks. In Proceedings of Congress on Evolutionary Computation (CEC), pp 158–165. IEEE Press, Piscataway (2000)Google Scholar
  47. 47.
    Nelson, A.L., Barlow, G.J., Doitsidis, L.: Fitness functions in evolutionary robotics: a survey and analysis. Robot. Auton. Syst. 57 (4), 345–370 (2009)CrossRefGoogle Scholar
  48. 48.
    Nolfi, S., Floreano, D.: Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines. MIT press, Cambridge (2000)Google Scholar
  49. 49.
    Nolfi, S., Parisi, D.: Evolving non-trivial behaviors on real robots: an autonomous robot that picks up objects. In Proceedings of the Congress of the Italian Association for Artificial Intelligence (AI*IA), pp 187–198. Springer, Berlin (1995)Google Scholar
  50. 50.
    Nolfi, S., Floreano, D., Miglino, O., Mondada, F.: How to evolve autonomous robots: Different approaches in evolutionary robotics. In Proceedings of the International Workshop on Artificial Life (ALIFE), pp 190–197. MIT Press, Cambridge (1994)Google Scholar
  51. 51.
    O’Grady, R., Christensen, A.L., Dorigo, M.: SWARMORPH: multirobot morphogenesis using directional self-assembly. IEEE Trans. Robot. 25 (3), 738–743 (2009)CrossRefGoogle Scholar
  52. 52.
    Okamura, A.M., Smaby, N., Cutkosky, M.R.: An overview of dexterous manipulation. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), vol. 1, pp. 255–263. IEEE Press, Piscataway (2000)Google Scholar
  53. 53.
    Reynolds, C.W.: Evolution of corridor following behavior in a noisy world. In From Animals to Animats 3: Proceedings of the International Conference on Simulation of Adaptive Behavior (SAB), pp 402–410. MIT Press, Cambridge (1994)Google Scholar
  54. 54.
    Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evolutionary Computation 10 (2), 99–127 (2002)CrossRefGoogle Scholar
  55. 55.
    Stanley, K.O., Miikkulainen, R.P.: Efficient evolution of neural networks through complexification. Computer Science Department, University of Texas at Austin (2004)Google Scholar
  56. 56.
    Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 15 (2), 185–212 (2009)CrossRefGoogle Scholar
  57. 57.
    Tolman, E.C., Honzik, C.H.: Introduction and removal of reward, and maze performance in rats. Univ. Calif. Publ. Psychol. 4, 257–275 (1930)Google Scholar
  58. 58.
    Torta, A.B. L., Kramer, M.A., Thorn, C., Gibson, D.J., Kubota, Y., Graybiel, A.M., Kopell, N.J.: Dynamic cross-frequency couplings of local field potential oscillations in rat striatum and hippocampus during performance of a T-maze task. Proc. Natl. Acad. Sci. 105 (51), 20517–20522 (2008)CrossRefGoogle Scholar
  59. 59.
    Tuci, E., Trianni, V., M. Dorigo: ‘Feeling’ the flow of time through sensorimotor co-ordination. Connect. Sci. 16 (4), 301–324 (2004)CrossRefGoogle Scholar
  60. 60.
    Tunstel, E.: Mobile robot autonomy via hierarchical fuzzy behavior control. In: Proceedings of the International Symposium on Robotics and Manufacturing (WAC), pp 837–842. ASME Press, New York (1996)Google Scholar
  61. 61.
    Watson, R.A., Ficici, S.G., Pollack, J.B.: Embodied evolution: Distributing an evolutionary algorithm in a population of robots. Robot. Auton. Syst. 39 (1), 1–18 (2002)CrossRefGoogle Scholar
  62. 62.
    Whitley, L.D.: Fundamental principles of deception in genetic search. In Foundations of Genetic Algorithms, pp. 221–241. Morgan Kaufmann (1991)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  • Miguel Duarte
    • 1
  • Sancho Moura Oliveira
    • 1
  • Anders Lyhne Christensen
    • 1
  1. 1.Instituto Universitário de Lisboa (ISCTE-IUL) and Instituto de TelecomunicaçõesLisboaPortugal

Personalised recommendations