Evolutionary Intelligence

, Volume 5, Issue 1, pp 45–56 | Cite as

Constructing controllers for physical multilegged robots using the ENSO neuroevolution approach

  • Vinod K. Valsalam
  • Jonathan Hiller
  • Robert MacCurdy
  • Hod Lipson
  • Risto Miikkulainen
Short Note


Evolving controllers for multilegged robots in simulation is convenient and flexible, making it possible to prototype ideas rapidly. However, transferring the resulting controllers to physical robots is challenging because it is difficult to simulate real-world complexities with sufficient accuracy. This paper bridges this gap by utilizing the Evolution of Network Symmetry and mOdularity (ENSO) approach to evolve modular neural network controllers that are robust to discrepancies between simulation and reality. This approach was evaluated by building a physical quadruped robot and by evolving controllers for it in simulation. An approximate model of the robot and its environment was built in a physical simulation and uncertainties in the real world were modeled as noise. The resulting controllers produced well-synchronized trot gaits when they were transferred to the physical robot, even on different walking surfaces. In contrast to a hand-designed PID controller, the evolved controllers also generalized well to changes in experimental conditions such as loss of voltage and were more robust against faults such as loss of a leg, making them strong candidates for real-world applications.


Modular controller Symmetry Neuroevolution Physical simulation Quadruped robot Gait 



Thanks to Ricardo Garcia, Franz Nigl, and Michael Tolley for helping to build the physical robot. This research was supported in part by NSF under grants IIS-0915038, IIS-0757479, and EIA-0303609, and the Texas Higher Education Coordinating Board under grant 003658-0036-2007.


  1. 1.
    Beer RD, Chiel HJ, Sterling LS (1989) Heterogeneous neural networks for adaptive behavior in dynamic environments. Adv Neural Inf Proc Syst 1:577–585Google Scholar
  2. 2.
    Beer RD, Gallagher JC (1992) Evolving dynamical neural networks for adaptive behavior. Adapt Behav 1(1):91–122CrossRefGoogle Scholar
  3. 3.
  4. 4.
    Billard A, Ijspeert AJ (2000) Biologically inspired neural controllers for motor control in a quadruped robot. In: Proceedings of IJCNN, pp 637–641Google Scholar
  5. 5.
    Bongard JC, Lipson H (2004) Once more unto the breach: co-evolving a robot and its simulator. In: Proceedings of ALIFE9, pp 57–62Google Scholar
  6. 6.
    Brooks RA (1989) A robot that walks; emergent behaviors from a carefully evolved network. Technical report AIM-1091, Massachusetts Institute of TechnologyGoogle Scholar
  7. 7.
    Brooks RA (1992) Artificial life and real robots. In Proceedings of the First European Conference on artificial life, pp 3–10Google Scholar
  8. 8.
    Clark JE (2004) Design, simulation, and stability of a hexapedal running robot. PhD thesis, Department of Mechanical Engineering, Stanford UniversityGoogle Scholar
  9. 9.
    Collins JJ, Stewart IN (1993) Coupled nonlinear oscillators and the symmetries of animal gaits. J Nonlinear Sci 3(1):349–392MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Ferrell C (1994) Failure recognition and fault tolerance of an autonomous robot. Adapt Behav 2(4):375–398CrossRefGoogle Scholar
  11. 11.
    Floreano D, Mondada F (1998) Evolutionary neurocontrollers for autonomous mobile robots. Neural Netw 11:1461–1478CrossRefGoogle Scholar
  12. 12.
    Gao L, Liu S, Dougal RA (2002) Dynamic lithium-ion battery model for system simulation. IEEE Trans Comp Pack Technol 25(3):495–505CrossRefGoogle Scholar
  13. 13.
    Gomez F, Miikkulainen R (2004) Transfer of neuroevolved controllers in unstable domains. In: Proceedings of GECCOGoogle Scholar
  14. 14.
    Gruau F (1994) Automatic definition of modular neural networks. Adapt Behav 3(2):151–183CrossRefGoogle Scholar
  15. 15.
    Holmes P, Full RJ, Koditschek D, Guckenheimer J (2006) The dynamics of legged locomotion: models, analyses, and challenges. SIAM Rev 48(2):207–304MathSciNetMATHCrossRefGoogle Scholar
  16. 16.
    Hornby GS, Takamura S, Yokono J, Hanagata O, Yamamoto T, Fujita M (2000) Evolving robust gaits with AIBO. In: Proceedings of ICRA, vol 3, pp 3040–3045Google Scholar
  17. 17.
    Jakobi N (1998) Minimal simulations for evolutionary robotics. PhD thesis, School of Cognitive and Computing Sciences, University of SussexGoogle Scholar
  18. 18.
    Jakobi N, Husbands P, Harvey I (1995) Noise and the reality gap: The use of simulation in evolutionary robotics. In Moran F, Moreno A, Merelo J, Chacon P (eds), Advances in artificial life, vol 929. Lecture notes in computer science, pp 704–720Google Scholar
  19. 19.
    Koditschek DE, Full RJ, Buehler M (2004) Mechanical aspects of legged locomotion control. Arthropod Struct Dev 33(3):251–272CrossRefGoogle Scholar
  20. 20.
    Kohl N, Stone P (2004) Policy gradient reinforcement learning for fast quadrupedal locomotion. In: Proceedings of ICRA, vol 3, pp 2619–2624Google Scholar
  21. 21.
    Koos S, Mouret J-B, Doncieux S (2010) Crossing the reality gap in evolutionary robotics by promoting transferable controllers. In: Proceedings of GECCO, pp 119–126Google Scholar
  22. 22.
    Lipson H, Bongard J, Zykov V, Malone E (2006) Evolutionary robotics for legged machines: from simulation to physical reality. In: Proceedings of the 9th international conference on intelligent autonomous systems, pp 11–18Google Scholar
  23. 23.
    Mataric M, Cliff D (1996) Challenges in evolving controllers for physical robots. Robot Auton Syst 19(1):67–83CrossRefGoogle Scholar
  24. 24.
    Miglino O, Lund HH, Nolfi S (1995) Evolving mobile robots in simulated and real environments. Artificial Life 2:417–434CrossRefGoogle Scholar
  25. 25.
    Nolfi S, Floreano D, Miglino O, Mondada F (1994) How to evolve autonomous robots: different approaches in evolutionary robotics. In: Proceedings of artificial life, vol IV, pp 190–197Google Scholar
  26. 26.
  27. 27.
    ODE: Open dynamics engine (2007) http://www.ode.org/
  28. 28.
    OPAL: Open physics abstraction layer (2007) http://opal.sourceforge.net/
  29. 29.
    Raibert MH (1986) Legged robots. Commun ACM 29(6):499–514MATHCrossRefGoogle Scholar
  30. 30.
    Robotis (2010) http://www.robotis.com/
  31. 31.
    SolidWorks (2010) http://www.solidworks.com/
  32. 32.
    Stanley KO, Miikkulainen R (2004) Competitive coevolution through evolutionary complexification. J Artif Intell Res 21:63–100Google Scholar
  33. 33.
    Valsalam VK (2010) Utilizing symmetry in evolutionary design. PhD thesis, Department of Computer Sciences, The University of Texas at Austin, technical report AI-10-04Google Scholar
  34. 34.
    Valsalam VK, Miikkulainen R (2008) Modular neuroevolution for multilegged locomotion. In: Proceedings of GECCO, pp 265–272Google Scholar
  35. 35.
    Valsalam VK, Miikkulainen R (2009) Evolving symmetric and modular neural networks for distributed control. In: Proceedings of GECCO, pp 731–738Google Scholar
  36. 36.
    Valsalam VK, Miikkulainen R (2011) Evolving symmetry for modular system design. IEEE Trans Evol Comput 15(3):368–386CrossRefGoogle Scholar
  37. 37.
    Watson RA, Ficici SG, Pollack JB (2002) Embodied evolution: distributing an evolutionary algorithm in a population of robots. Robot Auton Syst 39(1):1–18CrossRefGoogle Scholar
  38. 38.
    Zagal JC, Ruiz-Del-Solar J (2007) Combining simulation and reality in evolutionary robotics. J Intell Robot Syst 50(1):19–39CrossRefGoogle Scholar
  39. 39.
    Zhang SS, Xu K, Jow TR (2003) The low temperature performance of li-ion batteries. J Power Sourc 115(1):137–140CrossRefGoogle Scholar
  40. 40.
    Zykov V, Bongard J, Lipson H (2004) Evolving dynamic gaits on a physical robot. In: Proceedings of GECCO, late-breaking papersGoogle Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  • Vinod K. Valsalam
    • 1
  • Jonathan Hiller
    • 2
  • Robert MacCurdy
    • 2
  • Hod Lipson
    • 2
  • Risto Miikkulainen
    • 1
  1. 1.Department of Computer SciencesThe University of Texas at AustinAustinUSA
  2. 2.Sibley School of Mechanical and Aerospace EngineeringCornell UniversityIthacaUSA

Personalised recommendations