Distributed Online Learning of Central Pattern Generators in Modular Robots

  • David Johan Christensen
  • Alexander Spröwitz
  • Auke Jan Ijspeert
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6226)


In this paper we study distributed online learning of locomotion gaits for modular robots. The learning is based on a stochastic approximation method, SPSA, which optimizes the parameters of coupled oscillators used to generate periodic actuation patterns. The strategy is implemented in a distributed fashion, based on a globally shared reward signal, but otherwise utilizing local communication only. In a physics-based simulation of modular Roombots robots we experiment with online learning of gaits and study the effects of: module failures, different robot morphologies, and rough terrains. The experiments demonstrate fast online learning, typically 5-30 min. for convergence to high performing gaits (≈ 30 cm/sec), despite high numbers of open parameters (45-54). We conclude that the proposed approach is efficient, effective and a promising candidate for online learning on many other robotic platforms.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bongard, J., Zykov, V., Lipson, H.: Resilient machines through continuous self-modeling. Science 314(5802), 1118–1121 (2006)CrossRefGoogle Scholar
  2. 2.
    Christensen, D.J., Bordignon, M., Schultz, U.P., Shaikh, D., Stoy, K.: Morphology independent learning in modular robots. In: Proceedings of International Symposium on Distributed Autonomous Robotic Systems 8 (DARS 2008), pp. 379–391 (2008)Google Scholar
  3. 3.
    Christensen, D.J., Schultz, U.P., Stoy, K.: A distributed strategy for gait adaptation in modular robots. In: Proceedings of the IEEE Int. Conference on Robotics and Automation, ICRA (2010)Google Scholar
  4. 4.
    Ijspeert, A.J.: Central pattern generators for locomotion control in animals and robots: a review. Neural Networks 21(4), 642–653 (2008)CrossRefGoogle Scholar
  5. 5.
    Kamimura, A., Kurokawa, H., Yoshida, E., Murata, S., Tomita, K., Kokaji, S.: Automatic locomotion design and experiments for a modular robotic system. IEEE/ASME Transactions on Mechatronics 10(3), 314–325 (2005)CrossRefGoogle Scholar
  6. 6.
    Lipson, H., Pollack, J.B.: Automatic design and manufacture of robotic lifeforms. Nature 406, 974–978 (2000)CrossRefGoogle Scholar
  7. 7.
    Maes, P., Brooks, R.A.: Learning to coordinate behaviors. In: National Conference on Artificial Intelligence, pp. 796–802 (1990)Google Scholar
  8. 8.
    Marbach, D., Ijspeert, A.J.: Co-evolution of configuration and control for homogenous modular robots. In: Proc. 8th Int. Conf. on Intelligent Autonomous Systems, Amsterdam, Holland, pp. 712–719 (2004)Google Scholar
  9. 9.
    Marbach, D., Ijspeert, A.J.: Online Optimization of Modular Robot Locomotion. In: Proceedings of the IEEE Int. Conference on Mechatronics and Automation (ICMA 2005), pp. 248–253 (2005)Google Scholar
  10. 10.
    Sims, K.: Evolving 3d morphology and behavior by competition. In: Brooks, R., Maes, P. (eds.) Proc. Artificial Life IV, pp. 28–39. MIT Press, Cambridge (1994)Google Scholar
  11. 11.
    Smith, R.: Open dynamics engine (2005), http://www.ode.org
  12. 12.
    Spall, J.C.: Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control 37(3), 332–341 (1992)MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Sproewitz, A., Billard, A., Dillenbourg, P., Ijspeert, A.J.: Roombots-mechanical design of self-reconfiguring modular robots for adaptive furniture. In: International Conference on Robotics and Automation (ICRA 2009), Kobe, Japan (May 2009)Google Scholar
  14. 14.
    Sproewitz, A., Moeckel, R., Maye, J., Ijspeert, A.J.: Learning to move in modular robots using central pattern generators and online optimization. Int. J. Rob. Res. 27(3-4), 423–443 (2008)CrossRefGoogle Scholar
  15. 15.
    van den Kieboom, J.: Biped locomotion and stability a practical approach. Master’s thesis, University of Groningen, The Netherlands (2009)Google Scholar
  16. 16.
    Webots. Commercial Mobile Robot Simulation Software, http://www.cyberbotics.com

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • David Johan Christensen
    • 1
  • Alexander Spröwitz
    • 2
  • Auke Jan Ijspeert
    • 2
  1. 1.The Maersk Mc-Kinney Moller InstituteUniversity of Southern Denmark 
  2. 2.Biorobotics LaboratoryEcole Polytechnique Fédérale de LausanneSwitzerland

Personalised recommendations