Robot Learning by Guided Self-Organization

Part of the Emergence, Complexity and Computation book series (ECC, volume 9)


Self-organizing processes are not only crucial for the development of living beings, but can also spur new developments in robotics, e. g. to increase fault tolerance and enhance flexibility, provided that the prescribed goals can be realized at the same time. This combination of an externally specified objective and autonomous exploratory behavior is very interesting for practical applications of robot learning. In this chapter, we will present several forms of guided self-organization in robots based on homeokinesis.


Motor Neuron Reinforcement Learning Forward Model Vision Sensor Locomotion Behavior 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Amari, S.: Natural gradients work efficiently in learning. Neural Computation 10 (1998)Google Scholar
  2. Bongard, J.C., Zykov, V., Lipson, H.: Resilient machines through continuous self-modeling. Science 314, 1118–1121 (2006)CrossRefGoogle Scholar
  3. Butera, F.M.: Urban development as a guided self-organisation process. In: The City and Its Sciences, pp. 225–242. Springer (1998)Google Scholar
  4. Cannon, W.B.: The wisdom of the body. Norton, New York (1939)Google Scholar
  5. Chemova, S., Veloso, M.: An evolutionary approach to gait learning for four-legged robots. In: Proc. IEEE IROS 2004, vol. 3, pp. 2562–2567 (2004)Google Scholar
  6. Choi, J., Wehrspohn, R.B., Gösele, U.: Mechanism of guided self-organization producing quasi-monodomain porous alumina. Electrochimica Acta 50(13), 2591–2595 (2005)CrossRefGoogle Scholar
  7. Cruse, H., Dürr, V., Schmitz, J., Schneider, A.: Control of hexapod walking in biological systems. In: Adaptive Motion of Animals and Machines, pp. 17–29. Springer (2006)Google Scholar
  8. de Margerie, E., Mouret, J.-B., Doncieux, S., Meyer, J.-A.: Artificial evolution of the morphology and kinematics in a flapping-wing mini UAV. Bioinspiration and Biomimetics 2, 65–82 (2007)CrossRefGoogle Scholar
  9. Der, R.: Self-organized acquisition of situated behaviors. Theory Biosci. 120, 179–187 (2001)Google Scholar
  10. Der, R., Liebscher, R.: True autonomy from self-organized adaptivity. In: Proc. Workshop Biologically Inspired Robotics, Bristol (2002)Google Scholar
  11. Der, R., Martius, G.: The Playful Machine - Theoretical Foundation and Practical Realization of Self-Organizing Robots. Springer (2012)Google Scholar
  12. Der, R., Martius, G.: Behavior as broken symmetry in embodied self-organizing robots. In: Advances in Artificial Life, ECAL 2013 (accepted, 2013)Google Scholar
  13. Dongyong, Y., Jingping, J., Yuzo, Y.: Distal supervised learning control and its application to CSTRsystems. In: SICE 2000. Proc. of the 39th SICE Annual Conference, pp. 209–214 (2000)Google Scholar
  14. Ijspeert, A.J., Hallam, J., Willshaw, D.: Evolving Swimming Controllers for a Simulated Lamprey with Inspiration from Neurobiology. Adaptive Behavior 7(2), 151–172 (1999)CrossRefGoogle Scholar
  15. Jordan, M.I., Rumelhart, D.E.: Forward models: Supervised learning with a distal teacher. Cognitive Science 16(3), 307–354 (1992)CrossRefGoogle Scholar
  16. Klyubin, A.S., Polani, D., Nehaniv, C.L.: Empowerment: a universal agent-centric measure of control. In: IEEE Congress on Evolutionary Computation, pp. 128–135. IEEE (2005)Google Scholar
  17. Martius, G.: Robustness of guided self-organization against sensorimotor disruptions. Advances in Complex Systems 16(02n03), 1350001 (2013)CrossRefGoogle Scholar
  18. Martius, G., Der, R., Ay, N.: Information driven self-organization of complex robotic behaviors. PLoS ONE 8(5), e63400 (2013)Google Scholar
  19. Martius, G., Herrmann, J.M.: Taming the beast: Guided self-organization of behavior in autonomous robots. In: Doncieux, S., Girard, B., Guillot, A., Hallam, J., Meyer, J.-A., Mouret, J.-B. (eds.) SAB 2010. LNCS, vol. 6226, pp. 50–61. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  20. Martius, G., Herrmann, J.M.: Tipping the scales: Guidance and intrinsically motivated behavior. In: Advances in Artificial Life, ECAL 2011, pp. 506–513. MIT Press (2011)Google Scholar
  21. Martius, G., Herrmann, J.M.: Variants of guided self-organization for robot control. Theory in Biosci. 131(3), 129–137 (2012)Google Scholar
  22. Martius, G., Herrmann, J.M., Der, R.: Guided self-organisation for autonomous robot development. In: Almeida e Costa, F., Rocha, L.M., Costa, E., Harvey, I., Coutinho, A. (eds.) ECAL 2007. LNCS (LNAI), vol. 4648, pp. 766–775. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  23. Martius, G., Hesse, F., Güttler, F., Der, R.: LpzRobots: A free and powerful robot simulator (2012),
  24. Mazzapioda, M., Nolfi, S.: Synchronization and gait adaptation in evolving hexapod robots. In: Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J.C.T., Marocco, D., Meyer, J.-A., Miglino, O., Parisi, D. (eds.) SAB 2006. LNCS (LNAI), vol. 4095, pp. 113–125. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  25. Nolfi, S., Floreano, D.: Evolutionary Robotics. The Biology, Intelligence, and Technology of Self-organizing Machines. MIT Press, Cambridge (2000) (1st print) (2001) (2nd print)Google Scholar
  26. Ott, E., Grebogi, C., Yorke, J.: Controlling chaos. Phys. Rev. Lett. 64, 1196–1199 (1990)zbMATHMathSciNetCrossRefGoogle Scholar
  27. Pearson, K., Gordon, J.: Spinal reflexes. In: Kandel, E., Schwartz, J.H., Jessell, T.M. (eds.) Principles of Neural Science, 4th edn., pp. 713–736. McGraw-Hill, New York (2000)Google Scholar
  28. Peters, J., Schaal, S.: Natural Actor-Critic. Neurocomputing 71(7-9), 1180–1190 (2008)CrossRefGoogle Scholar
  29. Popp, J.: Spherical robots (2004),
  30. Prokopenko, M.: Design vs self-organization. In: Prokopenko, M. (ed.) Advances in Applied Self-organizing Systems, pp. 3–17. Springer (2008)Google Scholar
  31. Prokopenko, M.: Guided self-organization. HFSP Journal 3(5), 287–289 (2009)CrossRefGoogle Scholar
  32. Rodriguez, A.: Guided Self-Organizing Particle Systems for Basic Problem Solving. PhD thesis, University of Maryland (College Park, Md., USA) (2007)Google Scholar
  33. Santello, M., Soechting, J.F.: Force synergies for multifingered grasping. Experimental Brain Research 133(4), 457–467 (2000)CrossRefGoogle Scholar
  34. Schaal, S., Ijspeert, A., Billard, A.: Computational approaches to motor learning by imitation, vol. 1431, pp. 199–218. Oxford University Press (2004)Google Scholar
  35. Smith, S.C., Herrmann, J.M.: Homeokinetic reinforcement learning. In: Schwenker, F., Trentin, E. (eds.) PSL 2011. LNCS, vol. 7081, pp. 82–91. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  36. Stitt, S., Zheng, Y.F.: Distal learning applied to biped robots. In: Proc. of the IEEE Intl. Conf. on Robotics and Automation, pp. 137–142. IEEE Computer Society (1994)Google Scholar
  37. Sutton, R.S.: Reinforcement learning: Past, present and future. In: McKay, B., Yao, X., Newton, C.S., Kim, J.-H., Furuhashi, T. (eds.) SEAL 1998. LNCS (LNAI), vol. 1585, p. 195. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  38. Wikipedia (2013). Homeostasis — wikipedia, the free encyclopedia (Online accessed July 23, 2013)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Max Planck Institute for Mathematics in the SciencesLeipzigGermany
  2. 2.Bernstein Center for Computational NeuroscienceGöttingenGermany
  3. 3.Institute for Perception, Action and Behaviour, School of InformaticsUniversity of EdinburghEdinburghScotland, U.K.

Personalised recommendations