Theory in Biosciences

, Volume 131, Issue 3, pp 129–137 | Cite as

Variants of guided self-organization for robot control

  • Georg Martius
  • J. Michael Herrmann
Original Paper


Autonomous robots can generate exploratory behavior by self-organization of the sensorimotor loop. We show that the behavioral manifold that is covered in this way can be modified in a goal-dependent way without reducing the self-induced activity of the robot. We present three strategies for guided self-organization, namely by using external rewards, a problem-specific error function, or assumptions about the symmetries of the desired behavior. The strategies are analyzed for two different robots in a physically realistic simulation.


Guided self-organization Autonomous robots Homeokinesis Machine learning 



Both authors are grateful to Ralf Der for fruitful discussion. The project was supported by grants #01GQ0811 and #01GQ0432 within the National Bernstein Network Computational Neuroscience.


  1. Abu-Mostafa YS (1995) Hints. Neural Comput 7(4):639–671PubMedCrossRefGoogle Scholar
  2. Amari S (1998) Natural gradients work efficiently in learning. Neural Comput 10(2):251–276CrossRefGoogle Scholar
  3. Cannon WB (1939) In: The wisdom of the body. Norton, New York Google Scholar
  4. Choi J, Wehrspohn RB, Gösele U (2005) Mechanism of guided self-organization producing quasi-monodomain porous alumina. Electrochimica Acta 50(13):2591–2595CrossRefGoogle Scholar
  5. Der R (2001) Self-organized acquisition of situated behavior. Theory Biosci 120:179–187Google Scholar
  6. Der R, Herrmann M, Liebscher R (2002) Homeokinetic approach to autonomous learning in mobile robots. In: Dillman R, Schraft RD, Wörn H (eds) Robotik 2002, no.1679 in VDI-Berichte. VDI, Berichte, pp 301–306Google Scholar
  7. Der R, Hesse F, Martius G (2006) Rocking stamper and jumping snake from a dynamical system approach to artificial life. Adapt Behav 14(2):105–115CrossRefGoogle Scholar
  8. Der R, Liebscher R (2002) True autonomy from self-organized adaptivity. In: Proceedings of EPSRC/BBSRC International workshop on biologically inspired robotics. HP Labs, Bristol Google Scholar
  9. Der R, Martius G, Hesse F, Güttler F (2009) Videos of self-organized behavior in autonomous robots.
  10. Di Paolo E (2003) Organismically-inspired robotics: homeostatic adaptation and natural teleology beyond the closed sensorimotor loop. In: Murase K, Asakura T (eds) Dynamical systems approach to embodiment and sociality, pp 19–42 Google Scholar
  11. Dongyong Y, Jingping J, Yuzo Y (2000) Distal supervised learning control and its application to CSTRsystems. In: SICE 2000. Proceedings of the 39th SICE annual conference, Iizuka, pp 209–214 Google Scholar
  12. Herrmann JM (2001) Dynamical systems for predictive control of autonomous robots. Theory Biosci 120:241–252Google Scholar
  13. Ijspeert AJ, Hallam J, Willshaw D (1999) Evolving swimming controllers for a simulated lamprey with inspiration from neurobiology. Adapt Behav 7(2):151–172CrossRefGoogle Scholar
  14. Jordan MI, Rumelhart DE (1992) Forward models: Supervised learning with a distal teacher. Cognit Sci 16(3):307–354CrossRefGoogle Scholar
  15. Kelso JAS (1995) Dynamic patterns: the selforganization of brain and behavior. The MIT Press, CambridgeGoogle Scholar
  16. de Margerie E, Mouret JB, Doncieux S, Meyer JA (2007) Artificial evolution of the morphology and kinematics in a flapping-wing mini UAV. Bioinspiration Biomim 2:65–82CrossRefGoogle Scholar
  17. Martius G, Herrmann JM (2011) Tipping the scales: guidance and intrinsically motivated behavior. In: Proceedings of advances in artificial life, 11th European Conference (ECAL 2011). MIT Press, pp 506–513Google Scholar
  18. Martius G, Herrmann JM, Der R (2007) Guided self-organisation for autonomous robot development. In: Costa e FA (ed) Proceedings of advances in artificial life, 9th European Conference (ECAL 2007), LNCS, vol. 4648. Springer, San Francisco, pp 766–775Google Scholar
  19. Mazzapioda M, Cangelosi A, Nolfi S (2009) Evolving morphology and control: a distributed approach. In: IEEE congress on evolutionary computation, New Orleans, pp. 2217–2224 Google Scholar
  20. Nolfi S, Floreano D (2001) Evolutionary robotics. The biology, intelligence, and technology of self-organizing machines. MIT Press, CambridgeGoogle Scholar
  21. Peters J, Schaal S (2008) Natural actor-critic. Neurocomputing 71(7-9):1180–1190CrossRefGoogle Scholar
  22. Peters J, Vijayakumar S, Schaal S (2005) Natural actor-critic. In: Proceeidngs of the 16th European conference on machine learning (ECML 2005). Springer, Porto, pp. 280–291. Google Scholar
  23. Prokopenko M (2008) Design vs self-organization. In: Prokopenko M (ed) Advances in applied self-organizing systems. Springer, London, pp. 3–17. Google Scholar
  24. Prokopenko M (2009) Guided self-organization. HFSP J 3(5):287–289PubMedCrossRefGoogle Scholar
  25. Prokopenko M, Gerasimov V, Tanev I (2006) Evolving spatiotemporal coordination in a modular robotic system. In: Nolfi S, Baldassarre G, Calabretta R, Hallam JCT, Marocco D, Meyer JA, Miglino O, Parisi D (eds) SAB, LNCS, vol. 4095. Springer, Heidelberg, pp 558–569. Google Scholar
  26. Rodriguez A (2007) Guided self-organizing particle systems for basic problem solving. Ph.D. thesis, University of Maryland, College Park Google Scholar
  27. Stitt S, Zheng YF (1994) Distal learning applied to biped robots. In: Proceedings of the IEEE International Conference on robotics and automation. IEEE Computer Society, San Diego, pp 137–142Google Scholar
  28. Sutton RS, Barton AG (1998) Reinforcement learning: past, present and future. SEAL, Florham Park, pp 195–197Google Scholar
  29. Verschure PFMJ, Kröse BJA, Pfeifer R (1992) Distributed adaptive control: the self-organization of structured behavior. Robot Auton Sys 9(3):181–196CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.Bernstein Center for Computational Neuroscience and Max Planck Institute for Dynamics and Self-OrganizationGöttingenGermany
  2. 2.Institute for Perception, Action and BehaviourSchool of Informatics, University of EdinburghEdinburghUK

Personalised recommendations