Autonomous Robots

, Volume 32, Issue 2, pp 97–114 | Cite as

Interactive imitation learning of object movement skills

  • Manuel MühligEmail author
  • Michael Gienger
  • Jochen J. Steil


In this paper we present a new robot control and learning system that allows a humanoid robot to extend its movement repertoire by learning from a human tutor. The focus is learning and imitating motor skills to move and position objects. We concentrate on two major aspects. First, the presented teaching and imitation scenario is fully interactive. A human tutor can teach the robot which is in turn able to integrate newly learned skills into different movement sequences online. Second, we combine a number of novel concepts to enhance the flexibility and generalization capabilities of the system. Generalization to new tasks is obtained by decoupling the learned movements from the robot’s embodiment using a task space representation. It is chosen automatically from a commonly used task space pool. The movement descriptions are further decoupled from specific object instances by formulating them with respect to so-called linked objects. They act as references and can interactively be bound to real objects. When executing a learned task, a flexible kinematic description allows to change the robot’s body schema online and thereby apply the learned movement relative to different body parts or new objects. An efficient optimization scheme adapts movements to such situations performing online obstacle and self-collision avoidance. Finally, all described processes are combined within a comprehensive architecture. To demonstrate the generalization capabilities we show experiments where the robot performs a movement bimanually in different environments, although the task was demonstrated by the tutor only one-handed.


Imitation learning Human-robot interaction Robot control Kinematics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

(MPG 32.2 MB)


  1. Acosta-Calderon, C. A., & Hu, H. (2005). Robot imitation: body schema and body percept. Applied Bionics and Biomechanics, 2, 131–148. CrossRefGoogle Scholar
  2. Asfour, T., Gyarfas, F., Azad, P., & Dillmann, R. (2006). Imitation learning of dual-arm manipulation tasks in humanoid robots. In International conference on humanoid robots, 2006 6th IEEE-RAS (pp. 40–47). CrossRefGoogle Scholar
  3. Azad, P., Asfour, T., & Dillmann, R. (2007). Toward an unified representation for imitation of human motion on humanoids. In Proceedings 2007 IEEE international conference on robotics and automation, IEEE, April (pp. 2558–2563). CrossRefGoogle Scholar
  4. Beetz, M., Jain, D., Mösenlechner, L., & Tenorth, M. (2010). Towards performing everyday manipulation activities. Robotics and Autonomous Systems, 58(9), 1085–1095. CrossRefGoogle Scholar
  5. Bohg, J., Barck-Holst, C., Huebner, K., Ralph, M., Rasolzadeh, B., Song, D., & Kragic, D. (2009). Towards grasp-oriented visual perception for humanoid robots. International Journal of Humanoid Robotics, 6(3), 387–434. CrossRefGoogle Scholar
  6. Bolder, B., Dunn, M., Gienger, M., Janssen, H., Sugiura, H., & Goerick, C. (2007). Visually guided whole body interaction. In IEEE international conference on robotics and automation (ICRA) (pp. 3054–3061). Google Scholar
  7. Bundesen, C. (1990). A theory of visual attention. Psychological Review, 97(4), 523–547. CrossRefGoogle Scholar
  8. Burghart, C., Mikut, R., Stiefelhagen, R., Asfour, T., Holzapfel, H., Steinhaus, P., & Dillmann, R. (2005). A cognitive architecture for a humanoid robot: a first approach. In Proceedings of 2005 5th IEEE-RAS international conference on humanoid robots (pp. 357–362). CrossRefGoogle Scholar
  9. Calinon, S. (2009). Robot programming by demonstration: a probabilistic approach. Lausanne: EPFL Press. Google Scholar
  10. Calinon, S., & Billard, A. (2008). A framework integrating statistical and social cues to teach a humanoid robot new skills. In Proc. IEEE intl conf. on robotics and automation (ICRA), workshop on social interaction with intelligent indoor robots. Google Scholar
  11. Calinon, S., Guenter, F., & Billard, A. G. (2007). On learning, representing, and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man and Cybernetics. Part B. Cybernetics: A Publication of the IEEE Systems, Man, and Cybernetics Society, 37(2), 286–298. CrossRefGoogle Scholar
  12. Einecke, N., Mühlig, M., Schmüdderich, J., & Gienger, M. (2011). “Bring it to me”—Generation of behavior-relevant scene elements for interactive robot scenarios. In 2011 IEEE international conference on robotics and automation (ICRA). Google Scholar
  13. Eppner, C., Sturm, J., Bennewitz, M., Stachniss, C., & Burgard, W. (2009). Imitation learning with generalized task descriptions. In Proceedings of the IEEE/RSJ international conference on robotics and automation (ICRA), Kobe, Japan. Google Scholar
  14. Gienger, M., Janssen, H., & Goerick, C. (2005). Task-oriented whole body motion for humanoid robots. In 2005 5th IEEE-RAS international conference on humanoid robots (pp. 238–244). CrossRefGoogle Scholar
  15. Gienger, M., Mühlig, M., & Steil, J. J. (2010a). Imitating object movement skills with robots—a task-level approach exploiting generalization and invariance. In 2010 IEEE/RSJ international conference on intelligent robots and systems (IROS 2010). Google Scholar
  16. Gienger, M., Toussaint, M., & Christian, G. (2010b). Motion planning for humanoid robots. In K. Harada, E. Yoshida, & K. Yokoi (Eds.), Whole-body motion planning—building blocks for intelligent systems. Berlin: Springer. Chap. 3. Google Scholar
  17. Hecht, F., Azad, P., & Dillmann, R. (2009). Markerless human motion tracking with a flexible model and appearance learning. In IEEE international conference on robotics and automation (ICRA). Google Scholar
  18. Heracles, M., Bolder, B., & Goerick, C. (2009). Fast detection of arbitrary planar surfaces from unreliable 3D data. In International conference on intelligent robots and systems (IROS), IEEE/RSJ. Google Scholar
  19. Hersch, M., Sauser, E., & Billard, A. (2008). Online learning of the body schema. International Journal of Humanoid Robotics, 5(2), 161–181. CrossRefGoogle Scholar
  20. Igel, C., & Hüsken, M. (2003). Empirical evaluation of the improved Rprop learning algorithm. Neurocomputing, 50(C), 105–123. zbMATHCrossRefGoogle Scholar
  21. Ijspeert, A., Nakanishi, J., & Schaal, S. (2003). Learning attractor landscapes for learning motor primitives. In Advances in neural information processing systems (pp. 1547–1554). Google Scholar
  22. Inamura, T., Toshima, I., Tanie, H., & Nakamura, Y. (2004). Embodied symbol emergence based on mimesis theory. The International Journal of Robotics Research, 23, 363–377. CrossRefGoogle Scholar
  23. Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254–1259. CrossRefGoogle Scholar
  24. Khansari-Zadeh, S., & Billard, A. G. (2010). Imitation learning of globally stable non-linear point-to-point robot motions using nonlinear programming. In Proceeding of the 2010 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2676–2683). Google Scholar
  25. Kober, J., & Peters, J. (2009). Learning motor primitives for robotics. In 2009 IEEE international conference on robotics and automation, IEEE (pp. 2112–2118). CrossRefGoogle Scholar
  26. Kormushev, P., Calinon, S., & Caldwell, D. G. (2010). Robot motor skill coordination with EM-based reinforcement learning. In 2010 IEEE/RSJ international conference on intelligent robots and systems (pp. 3232–3237). CrossRefGoogle Scholar
  27. Liégeois, A. (1977). Automatic supervisory control of configuration and behavior of multibody mechanisms. IEEE Transactions on Systems, Man, and Cybernetics, 7(12), 861–871. Google Scholar
  28. Lopes, M., & Santos-Victor, J. (2005). Visual learning by imitation with motor representations. IEEE Transactions on Systems, Man, and Cybernetics. Part B, 35(3), 438–449. CrossRefGoogle Scholar
  29. Lopes, M., Melo, F. S., & Montesano, L. (2007). Affordance-based imitation learning in robots. In Proceedings of the 2007 IEEE/RSJ international conference on intelligent robots and systems. Google Scholar
  30. Matarić, M. J., & Pomplun, M. (1998). Fixation behavior in observation and imitation of human movement. Cognitive Brain Research, 7, 191–202. CrossRefGoogle Scholar
  31. McGuire, P., Fritsch, J., Steil, J. J., Röthling, F., Fink, G. A., Wachsmuth, S., Sagerer, G., & Ritter, H. (2002). Multi-modal human-machine communication for instructing robot grasping tasks. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 1082–1088). Google Scholar
  32. Mühlig, M., Gienger, M., Hellbach, S., Steil, J. J., & Goerick, C. (2009a). Task-level imitation learning using variance-based movement optimization. In IEEE international conference on robotics and automation (ICRA). Google Scholar
  33. Mühlig, M., Gienger, M., Steil, J. J., & Goerick, C. (2009b). Automatic selection of task spaces for imitation learning. In 2009 IEEE/RSJ international conference on intelligent robots and systems. Google Scholar
  34. Mühlig, M., Gienger, M., & Steil, J. J. (2010). Human-robot interaction for learning and adaptation of object movements. In 2010 IEEE/RSJ international conference on intelligent robots and systems (IROS 2010). Google Scholar
  35. Nabeshima, C., Kuniyoshi, Y., & Lungarella, M. (2006). Adaptive body schema for robotic tool-use. Advanced Robotics, 20(10), 1105–1126. CrossRefGoogle Scholar
  36. Nagai, Y., Muhl, C., & Rohlfing, K. J. (2008). Toward designing a robot that learns actions from parental demonstrations. In 2008 IEEE international conference on robotics and automation, Pasadena, CA, USA (pp. 3545–3550). CrossRefGoogle Scholar
  37. Nakamura, Y. (1991). Advanced robotics: redundancy and optimization. Reading: Addison-Wesley. Google Scholar
  38. Nicolescu, M., & Matarić, M. J. (2003). Natural methods for robot task learning: instructive demonstrations, generalization and practice. In Proceedings of the second international joint conference on autonomous agents and multi-agent systems (pp. 241–248). New York: ACM. CrossRefGoogle Scholar
  39. Nicolescu, M., & Matarić, M. J. (2006). Task learning through imitation and human-robot interaction. In Models and mechanisms of imitation and social learning in robots, humans and animals: behavioural, social and communicative dimensions (pp. 407–424). Google Scholar
  40. Orabona, F., Metta, G., & Sandini, G. (2007). A proto-object based visual attention model. In Attention in cognitive systems theories and systems from an interdisciplinary viewpoint (pp. 198–215). Google Scholar
  41. Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009). Learning and generalization of motor skills by learning from demonstration. In IEEE international conference on robotics and automation. Google Scholar
  42. Sakoe, H. (1978). Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing, 26, 43–49. zbMATHCrossRefGoogle Scholar
  43. Schaal, S., Ijspeert, A., & Billard, A. G. (2003). Computational approaches to motor learning by imitation. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 358(1431), 537–547. CrossRefGoogle Scholar
  44. Schmuedderich, J. (2010). Multimodal learning of grounded concepts in embodied systems. Berichte aus der Robotik. Aachen: Shaker. Google Scholar
  45. Scholl, B. J. (2001). Objects and attention: the state of the art. Cognition, 80(1–2), 1–46. CrossRefGoogle Scholar
  46. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461–464. MathSciNetzbMATHCrossRefGoogle Scholar
  47. Steil, J. J., Roethling, F., Haschke, R., & Ritter, H. (2004). Situated robot learning for multi-modal instruction and imitation of grasping. Robotics and Autonomous Systems Special Issue, 47, 129–141. CrossRefGoogle Scholar
  48. Stoytchev, A. (2003). Computational model for an extendable robot body schema. Tech. Rep. GIT-CC-03-44, Georgia Institute of Technology, College of Computing. Google Scholar
  49. Sugiura, K., Iwahashi, N., Kashioka, H., & Nakamura, S. (2010). Statistical imitation learning in sequential object manipulation tasks. In Advances in robot manipulators (pp. 589–606). Google Scholar
  50. Toussaint, M., Gienger, M., & Goerick, C. (2007). Optimization of sequential attractor-based movement for compact behaviour generation. In 7th IEEE-RAS international conference on humanoid robots (Humanoids 2007). Google Scholar
  51. Toussaint, M., Plath, N., Lang, T., & Jetchev, N. (2010). Integrated motor control, planning, grasping and high-level reasoning in a blocks world using probabilistic inference. In Robotics and automation (ICRA), 2010 IEEE international conference on, IEEE (pp. 385–391). CrossRefGoogle Scholar
  52. Walther, D., & Koch, C. (2006). Modeling attention to salient proto-objects. Neural Networks, 19(9), 1395–1407. zbMATHCrossRefGoogle Scholar
  53. Wischnewski, M., Belardinelli, A., Schneider, W. X., & Steil, J. J. (2010). Where to look next? Combining static and dynamic proto-objects in a TVA-based model of visual attention. In Cognitive computation (pp. 1–18). Google Scholar
  54. Yamashita, Y., & Tani, J. (2008). Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment. PLoS computational biology, 4(11). Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Manuel Mühlig
    • 1
    Email author
  • Michael Gienger
    • 1
  • Jochen J. Steil
    • 2
  1. 1.Honda Research Institute EuropeOffenbach/MainGermany
  2. 2.Research Institute for Cognition and Robotics (CoR-Lab)Bielefeld UniversityBielefeldGermany

Personalised recommendations