Self-exploration of the Stumpy Robot with Predictive Information Maximization

  • Georg Martius
  • Luisa Jahn
  • Helmut Hauser
  • Verena V. Hafner
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8575)


One of the long-term goals of artificial life research is to create autonomous, self-motivated, and intelligent animats. We study an intrinsic motivation system for behavioral self-exploration based on the maximization of the predictive information using the Stumpy robot, which is the first evaluation of the algorithm on a real robot. The control is organized in a closed-loop fashion with a reactive controller that is subject to fast synaptic dynamics. Even though the available sensors of the robot produce very noisy and peaky signals, the self-exploration algorithm was successful and various emerging behaviors were observed.


Self-exploration intrinsic motivation robot control information theory dynamical systems learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Artificial Intelligence Laboratory, Zurich (2013),
  2. 2.
    Bialek, W., Nemenman, I., Tishby, N.: Predictability, complexity and learning. Neural Computation 13(11), 2409–2463 (2001)CrossRefzbMATHGoogle Scholar
  3. 3.
    Bongard, J.C., Zykov, V., Lipson, H.: Resilient machines through continuous self-modeling. Science 314, 1118–1121 (2006)CrossRefGoogle Scholar
  4. 4.
    Der, R., Martius, G.: The Playful Machine - Theoretical Foundation and Practical Realization of Self-Organizing Robots. Springer (2012)Google Scholar
  5. 5.
    Smith, D.: Cwiid: Linux Nintendo Wiimote interface library (2014),
  6. 6.
    Friston, K., Thornton, C., Clark, A.: Free-energy minimization and the dark room problem. Frontiers in Psychology 3(130) (2012)Google Scholar
  7. 7.
    Iida, F., Dravid, R., Paul, C.: Design and control of a pendulum driven hopping robot. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 3, pp. 2141–2146 (2002)Google Scholar
  8. 8.
    Klyubin, A.S., Polani, D., Nehaniv, C.L.: Empowerment: A universal agent-centric measure of control. In: Evolutionary Computation. pp. 128–135 (2005)Google Scholar
  9. 9.
    Lehman, J., Stanley, K.O.: Exploiting open-endedness to solve problems through the search for novelty. In: Proc. Intl. Conf. on Artificial Life (ALIFE XI), p. 329. MIT Press, Cambridge (2008)Google Scholar
  10. 10.
    Luciw, M., Kompella, V., Kazerounian, S., Schmidhuber, J.: An intrinsic value system for developing multiple invariant representations with incremental slowness learning. Frontiers in Neurorobotics 7(9) (2013)Google Scholar
  11. 11.
    Lungarella, M., Metta, G., Pfeifer, R., Sandini, G.: Developmental robotics: A survey. Connection Science 15(4), 151–190 (2003)CrossRefGoogle Scholar
  12. 12.
    Martius, G., Der, R., Ay, N.: Information driven self-organization of complex robotic behaviors. PLoS ONE 8(5), e63400 (2013)Google Scholar
  13. 13.
    Martius, G., Der, R., Herrmann, J.M.: Robot learning by guided self-organization. In: Prokopenko, M. (ed.) Guided Self-Organization: Inception. Springer (2014)Google Scholar
  14. 14.
    Martius, G., Fiedler, K., Herrmann, J.M.: Structure from Behavior in Autonomous Agents. In: Proc. IEEE IROS 2008, pp. 858–862 (2008)Google Scholar
  15. 15.
    Martius, G., Herrmann, J.M.: Tipping the scales: Guidance and intrinsically motivated behavior. In: Advances in Artificial Life, pp. 506–513. MIT Press (2011)Google Scholar
  16. 16.
    Martius, G., Jahn, L., Hauser, H., Hafner, V.V.: Supplementary materials (2014),
  17. 17.
    Nintendo: Wii official website (released 2006),
  18. 18.
    Oudeyer, P.Y., Kaplan, F., Hafner, V.V.: Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation 11(2), 265–286 (2007)CrossRefGoogle Scholar
  19. 19.
    Pfeifer, R., Bongard, J.C.: How the Body Shapes the Way We Think: A New View of Intelligence (Bradford Books). The MIT Press (2006)Google Scholar
  20. 20.
    Schillaci, G., Hafner, V.V., Lara, B.: Coupled inverse-forward models for action execution leading to tool-use in a humanoid robot. In: Proc. of 7th Intl. Conf. on Human-Robot Interaction (HRI 2012), pp. 231–232. ACM (2012)Google Scholar
  21. 21.
    Schmidhuber, J.: Curious model-building control systems. In: Proc. Intl. Joint Conf. on Neural Networks, Singapore, pp. 1458–1463. IEEE (1991)Google Scholar
  22. 22.
    Wolpert, D.M., Miall, R.C., Kawato, M.: Internal models in the cerebellum. Trends in Cognitive Sciences 2, 338–347 (1998)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Georg Martius
    • 1
  • Luisa Jahn
    • 2
    • 3
  • Helmut Hauser
    • 3
  • Verena V. Hafner
    • 2
  1. 1.Max Planck Institute for Mathematics in the SciencesLeipzigGermany
  2. 2.Institut für InformatikHumboldt-Universität zu BerlinBerlinGermany
  3. 3.Artificial Intelligence LabUniversity of ZurichZurichSwitzerland

Personalised recommendations