Journal of Intelligent and Robotic Systems

, Volume 50, Issue 1, pp 19–39 | Cite as

Combining Simulation and Reality in Evolutionary Robotics

  • Juan Cristóbal ZagalEmail author
  • Javier Ruiz-del-Solar


Evolutionary Robotics (ER) is a promising methodology, intended for the autonomous development of robots, in which their behaviors are obtained as a consequence of the structural coupling between robot and environment. It is essential that there be a great amount of interaction to generate complex behaviors. Thus, nowadays, it is common to use simulation to speed up the learning process; however simulations are achieved from arbitrary off-line designs, rather than from the result of embodied cognitive processes. According to the reality gap problem, controllers evolved in simulation usually do not allow the same behavior to arise once transferred to the real robot. Some preliminary approaches for combining simulation and reality exist in the ER literature; nonetheless, there is no satisfactory solution available. In this work we discuss recent advances in neuroscience as a motivation for the use of environmentally adapted simulations, which can be achieved through the co-evolution of robot behavior and simulator. We present an algorithm in which only the differences between the behavior fitness obtained in reality versus that obtained in simulations are used as feedback for adapting a simulation. The proposed algorithm is experimentally validated by showing the successful development and continuous transference to reality of two complex low-level behaviors with Sony AIBO1 robots: gait optimization and ball-kicking behavior.


AIBO robots Evolutionary robotics Mobile robotics Organismically inspired robotics RoboCup Simulation of solid dynamics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bedau, M.A.: Can unrealistic computer models illuminate theoretical biology? In: Wu, A.S. (ed.) Proceedings of the 1999 Genetic and Evolutionary Computation Conference Workshop Program, pp. 20–23. Morgan Kaufmann, San Francisco, CA (1999)Google Scholar
  2. 2.
    Bongard, J.C., Lipson, H.: Once more unto the breach: co-evolving a robot and its simulator. In: Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems (ALIFE9), pp. 57–62 (2004)Google Scholar
  3. 3.
    Brooks, R.: Artificial life and real robots. In: F.J. Varela and P. Bourgine. (ed.) Toward a Practice in Autonomous Systems. Proceedings of the First European Conference on Artificial Life, pp. 3–10. MIT Press, Cambridge, MA (1992)Google Scholar
  4. 4.
    Di Paolo, E.A.: Homeostatic adaptation to inversion of the visual field and other sensorimotor disruptions. In: J-A Meyer, A. Berthoz, D. Floreano, H.L. Roitblat, and S.W. Wilson (eds.) From Animals to Animats, Vol. 6. Proceedings of the VI International Conference on Simulation of Adaptive Behavior, pp. 440–449. MIT Press, Cambridge, MA (2000)Google Scholar
  5. 5.
    Ficici, S.: Solution concepts in coevolutionary algorithms. PhD thesis, Brandeis University (2004)Google Scholar
  6. 6.
    Franklin, M.S, Zyphur, M.J.: The role of dreams in the evolution of the human mind. Evolutionary Psychology 3, 59–78 (2005)Google Scholar
  7. 7.
    Golubovic, D., Hu, H.: A hybrid evolutionary algorithm for gait generation of Sony legged robots. In: Proceedings of the 28th Annual Conference of the IEEE Industrial Electronics Society, Seville, Spain, 5–8 November 2002Google Scholar
  8. 8.
    Grefenstette, J.J., Ramsey C.L.: An approach to anytime learning. In: Proceedings of the Ninth International Machine Learning Workshop, Morgan Kaufmann. San Mateo, CA (1992)Google Scholar
  9. 9.
    Hardt, M., Stryk, O.: The role of motion dynamics in the design, control and stability of bipedal and quadrupedal robots. In: Kaminka, G.A., Lima, P.U., Rojas, R. (eds.) Lecture Notes in Computer Science, vol. 2752, pp. 206–223. Springer, Berlin Heidelberg New York (2002)Google Scholar
  10. 10.
    Hobson, J.A., Pace-Schott, E.F.: The cognitive neuroscience of sleep: neuronal systems, consciousness and learning. Nat. Rev., Neurosci. 3(9), 679–93 (2002)CrossRefGoogle Scholar
  11. 11.
    Hornby, G.S., Fujita, M., Takamura, S., Yamamoto, T., Hanagata, O.: Autonomous evolution of gaits with the Sony quadruped robot. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1297–1304. Morgan Kaufmann, San Mateo, CA (1999)Google Scholar
  12. 12.
    Husbands, P., Harvey, I., Cliff, D.: An evolutionary approach to situated AI. In: A. Sloman et al. (ed.) Proceedings of the 9th Conference of the Society for the Study of Artificial Intelligence and the Simulation of Behavior (AISB), pp. 61–70. IOS Press, Amsterdam, The Netherlands (1993)Google Scholar
  13. 13.
    Jakobi, N.: Minimal simulations for evolutionary robotics. PhD thesis, University of Sussex (1998)Google Scholar
  14. 14.
    Kim, M., Uther, W.: Automatic gait optimisation for quadruped robots. In: Roberts, J., Wyeth, G. (eds.) Australasian Conference on Robotics and Automation, Brisbane (December 2003)Google Scholar
  15. 15.
    Kitano, H., Hamahashi, S., Kitazawa, J., Takao, K., Imai, S.: The virtual biology laboratories: a new approach of computational biology. In: Husbands, P., Harvey, I. (eds.) Proceedings of the Fourth European Conference on Artificial Life, pp. 274–283. MIT Press, Cambridge, MA (1997)Google Scholar
  16. 16.
    Kohl, N. and Stone, P.: Policy gradient reinforcement learning for fast quadrupedal locomotion. In: Proceedings of IEEE International Conference on Robotics and Automation (2004)Google Scholar
  17. 17.
    Nolfi, S., Floreano, D., Miglio, O., Mondada, F.: How to evolve autonomous robots: different approaches in evolutionary robotics. In: Brooks, R., Maes, P. (eds.) Artificial Life IV, pp. 190–197. MIT Press, Cambridge, MA (1994)Google Scholar
  18. 18.
    Nolfi, S., Floreano, D.: Evolutionary robotics – the biology, intelligence, and technology of self-organizing machines. In: Intelligent Robotics and Automation Agents. MIT Press, Cambridge, MA (2000)Google Scholar
  19. 19.
    Quilan, M.J., Chalup, S.K., Middleton, R.H.: Techniques for improving vision and locomotion on the Sony AIBO robot. In: Australasian Conference on Robotics and Automation (ACRA 2003) (2003)Google Scholar
  20. 20.
    Revonsuo, A.: The reinterpretation of dreams: an evolutionary hypothesis of the function of dreaming. Behav. Brain Sci. 23(66), 877–903 (2000)CrossRefGoogle Scholar
  21. 21.
    Röfer, T.: Evolutionary gait-optimization using a fitness function based on proprioception. In: Proceedings of RoboCup 2004: Robot Soccer World Cup VIII. Lecture Notes of Artificial Intelligence, pp. 310–322. Springer, Berlin Heidelberg New York (2004)Google Scholar
  22. 22.
    Ruiz-del-Solar, J., Zagal, J.C., Guerrero, P., Vallejos, P., Middleton, C., Olivares, X.: UChile1 team description. In: Proceedings of RoboCup 2003: Robot Soccer World Cup VII. Lecture Notes of Artificial Intelligence. Springer, Berlin Heidelberg New York (2003)Google Scholar
  23. 23.
    Stickgold, R, Hobson, J.A., Fosse, R. Fosse, M.: Sleep, learning and dreams: off-line memory reprocessing. Science 294 (5544), 1052–1057 (2001)CrossRefGoogle Scholar
  24. 24.
    Varela, F.J.: Principles of Biological Autonomy, New York: Elsevier (1979)Google Scholar
  25. 25.
    Zagal J.C., Ruiz-del-Solar J.: UCHILSIM: A dynamically and visually realistic simulator for the RoboCup four legged league. In: Proceedings of RoboCup 2004: Robot Soccer World Cup VIII. Lecture Notes of Artificial Intelligence, pp. 34–45. Springer, Berlin Heidelberg New York (2004)Google Scholar
  26. 26.
    Zagal, J.C., Ruiz-del-Solar, J.: Learning to kick the ball using back to reality. In: Proceedings of RoboCup 2004: Robot Soccer World Cup VIII. Lecture Notes of Artificial Intelligence, pp. 335–346. Springer, Berlin Heidelberg New York (2004)Google Scholar
  27. 27.
    Zagal, J.C., Ruiz-del-Solar, J., Vallejos, P.: Back to reality: crossing the reality gap in evolutionary robotics. In: Proceedings of the 5th IFAC Symposium on Intelligent Autonomous Vehicles, IAV 2004. Elsevier, Amsterdam, The Netherlands (2005)Google Scholar
  28. 28.
    Zagal, J.C., Sarmiento, I., Ruiz-del-Solar, J.: An application interface for UCHILSIM and the arrival of new challenges. In: Proceedings of RoboCup 2005: Robot Soccer World Cup IX. Lecture Notes of Artificial Intelligence, pp. 464–471. Springer, Berlin Heidelberg New York (2005)Google Scholar

Copyright information

© Springer Science+Business Media, Inc. 2007

Authors and Affiliations

  1. 1.Department of Electrical EngineeringUniversidad de ChileSantiagoChile

Personalised recommendations