Abstract
In this paper we study distributed online learning of locomotion gaits for modular robots. The learning is based on a stochastic approximation method, SPSA, which optimizes the parameters of coupled oscillators used to generate periodic actuation patterns. The strategy is implemented in a distributed fashion, based on a globally shared reward signal, but otherwise utilizing local communication only. In a physics-based simulation of modular Roombots robots we experiment with online learning of gaits and study the effects of: module failures, different robot morphologies, and rough terrains. The experiments demonstrate fast online learning, typically 5-30 min. for convergence to high performing gaits (≈ 30 cm/sec), despite high numbers of open parameters (45-54). We conclude that the proposed approach is efficient, effective and a promising candidate for online learning on many other robotic platforms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bongard, J., Zykov, V., Lipson, H.: Resilient machines through continuous self-modeling. Science 314(5802), 1118–1121 (2006)
Christensen, D.J., Bordignon, M., Schultz, U.P., Shaikh, D., Stoy, K.: Morphology independent learning in modular robots. In: Proceedings of International Symposium on Distributed Autonomous Robotic Systems 8 (DARS 2008), pp. 379–391 (2008)
Christensen, D.J., Schultz, U.P., Stoy, K.: A distributed strategy for gait adaptation in modular robots. In: Proceedings of the IEEE Int. Conference on Robotics and Automation, ICRA (2010)
Ijspeert, A.J.: Central pattern generators for locomotion control in animals and robots: a review. Neural Networks 21(4), 642–653 (2008)
Kamimura, A., Kurokawa, H., Yoshida, E., Murata, S., Tomita, K., Kokaji, S.: Automatic locomotion design and experiments for a modular robotic system. IEEE/ASME Transactions on Mechatronics 10(3), 314–325 (2005)
Lipson, H., Pollack, J.B.: Automatic design and manufacture of robotic lifeforms. Nature 406, 974–978 (2000)
Maes, P., Brooks, R.A.: Learning to coordinate behaviors. In: National Conference on Artificial Intelligence, pp. 796–802 (1990)
Marbach, D., Ijspeert, A.J.: Co-evolution of configuration and control for homogenous modular robots. In: Proc. 8th Int. Conf. on Intelligent Autonomous Systems, Amsterdam, Holland, pp. 712–719 (2004)
Marbach, D., Ijspeert, A.J.: Online Optimization of Modular Robot Locomotion. In: Proceedings of the IEEE Int. Conference on Mechatronics and Automation (ICMA 2005), pp. 248–253 (2005)
Sims, K.: Evolving 3d morphology and behavior by competition. In: Brooks, R., Maes, P. (eds.) Proc. Artificial Life IV, pp. 28–39. MIT Press, Cambridge (1994)
Smith, R.: Open dynamics engine (2005), http://www.ode.org
Spall, J.C.: Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control 37(3), 332–341 (1992)
Sproewitz, A., Billard, A., Dillenbourg, P., Ijspeert, A.J.: Roombots-mechanical design of self-reconfiguring modular robots for adaptive furniture. In: International Conference on Robotics and Automation (ICRA 2009), Kobe, Japan (May 2009)
Sproewitz, A., Moeckel, R., Maye, J., Ijspeert, A.J.: Learning to move in modular robots using central pattern generators and online optimization. Int. J. Rob. Res. 27(3-4), 423–443 (2008)
van den Kieboom, J.: Biped locomotion and stability a practical approach. Master’s thesis, University of Groningen, The Netherlands (2009)
Webots. Commercial Mobile Robot Simulation Software, http://www.cyberbotics.com
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Christensen, D.J., Spröwitz, A., Ijspeert, A.J. (2010). Distributed Online Learning of Central Pattern Generators in Modular Robots. In: Doncieux, S., Girard, B., Guillot, A., Hallam, J., Meyer, JA., Mouret, JB. (eds) From Animals to Animats 11. SAB 2010. Lecture Notes in Computer Science(), vol 6226. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15193-4_38
Download citation
DOI: https://doi.org/10.1007/978-3-642-15193-4_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15192-7
Online ISBN: 978-3-642-15193-4
eBook Packages: Computer ScienceComputer Science (R0)