Incremental kinesthetic teaching of motion primitives using the motion refinement tube

Abstract

We present an approach for kinesthetic teaching of motion primitives for a humanoid robot. The proposed teaching method starts with observational learning and applies iterative kinesthetic motion refinement using a forgetting factor. Kinesthetic teaching is supported by introducing the motion refinement tube, which represents an area of allowed motion refinement around the nominal trajectory. On the realtime control level, the kinesthetic teaching is handled by a customized impedance controller, which combines tracking performance with compliant physical interaction and allows to implement soft boundaries for the motion refinement. A novel method for continuous generation of motions from a hidden Markov model (HMM) representation of motion primitives is proposed, which incorporates time information for each state. The proposed methods were implemented and tested using DLR’s humanoid upper-body robot Justin.

This is a preview of subscription content, access via your institution.

References

  1. Albu-Schäffer, A., Ott, C., Frese, U., & Hirzinger, G. (2003). Cartesian impedance control of redundant robots: recent results with the dlr-light-weight-arms. In IEEE int. conf. on robotics and automation (pp. 3704–3709).

    Google Scholar 

  2. Alissandrakis, A., Nehaniv, C. L., & Dautenhahn, K. (2007). Correspondence mapping induced state and action metrics for robotic imitation. IEEE Transactions on Systems, Man and Cybernetics. Part B. Cybernetics, 37(2), 299–307. Special issue on robot learning by observation, demonstration and imitation.

    Article  Google Scholar 

  3. Asfour, T., Gyarfas, F., Azad, P., & Dillmann, R. (2006). Imitation learning of dual-arm manipulation tasks in humanoid robots. In IEEE-RAS int. conf. on humanoid robots (pp. 40–47).

    Google Scholar 

  4. Billard, A., Calinon, S., & Guenter, F. (2006). Discriminative and adaptive imitation in uni-manual and bi-manual tasks. Robotics and Autonomous Systems, 54, 370–384.

    Article  Google Scholar 

  5. Billard, A., Calinon, S., Dillmann, R., & Schaal, S. (2008). Robot programming by demonstration. In B. Siciliano & O. Khatib (Eds.), Handbook of robotics. Berlin: Springer.

    Google Scholar 

  6. Blimes, J. A. (1997). A gentle tutorial of the em algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models (Tech. Rep. ICSI-TR-97-021). University of Berkeley.

  7. Calinon, S., & Billard, A. (2007a). Active teaching in robot programming by demonstration. In IEEE international conference on robot and human interactive communication (pp. 702–707).

    Google Scholar 

  8. Calinon, S., & Billard, A. (2007b). Incremental learning of gestures by imitation in a humanoid robot. In ACM/IEEE international conference on human-robot interaction (pp. 255–262).

    Google Scholar 

  9. Calinon, S., Guenter, F., & Billard, A. (2007). On learning, representing and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man and Cybernetics. Part B, 37(2), 286–298. Special issue on robot learning by observation, demonstration and imitation.

    Article  Google Scholar 

  10. Calinon, S., D’halluin, F., Sauser, E., Caldwell, D., & Billard, A. G. (2010). Learning and reproduction of gestures by imitation: an approach based on hidden Markov model and Gaussian mixture regression. IEEE Robotics & Automation Magazine, 17(2), 44–54.

    Article  Google Scholar 

  11. Cohn, D. A., Ghahramani, Z., & Jordan, M. I. (1996). Active learning with statistical models. The Journal of Artificial Intelligence Research, 4, 129–145.

    MATH  Google Scholar 

  12. Dariush, B., Gienger, M., Jian, B., Goerick, C., & Fujimura, K. (2008). Whole body humanoid control from human motion descriptors. In IEEE int. conf. on robotics and automation (pp. 2677–2684).

    Google Scholar 

  13. De Luca, A., Albu-Schäffer, A., Haddadin, S., & Hirzinger, G. (2006). Collision detection and safe reaction with the dlr-iii lightweight manipulator arm. In IEEE/RSJ int. conference on intelligent robots and systems (pp. 1623–1630).

    Google Scholar 

  14. Dillmann, R. (2004). Teaching and learning of robot tasks via observation of human performance. Robotics and Autonomous Systems, 47, 109–116.

    Article  Google Scholar 

  15. Dixon, K. R., Dolan, J. M., & Khosla, P. K. (2004). Predictive robot programming: theoretical and experimental analysis. The International Journal of Robotics Research, 23, 955–973.

    Article  Google Scholar 

  16. Hogan, N. (1985). Impedance control: an approach to manipulation, part I—theory. ASME Journal of Dynamic Systems, Measurement and Control, 107, 1–7.

    MATH  Article  Google Scholar 

  17. Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2002). Movement imitation with nonlinear dynamical systems in humanoid robots. In IEEE int. conf. on robotics and automation (pp. 1398–1403).

    Google Scholar 

  18. Inamura, T., Nakamura, Y., & Toshima, I. (2004). Embodied symbol emergence based on Mimesis theory. The International Journal of Robotics Research, 23(4), 363–377.

    Article  Google Scholar 

  19. Inamura, T., Kojo, N., & Inaba, M. (2006). Situation recognition and behavior induction based on geometric symbol representation of multimodal sensorimotor patterns. In IEEE/RSJ int. conf. on intelligent robots and systems (pp. 5147–5152).

    Google Scholar 

  20. Ito, M., Noda, K., Hoshino, Y., & Tani, J. (2006). Dynamic and interactive generation of object handling behaviors by a small humanoid robot using a dynamic neural network model. Neural Networks, 19(3), 323–337.

    MATH  Article  Google Scholar 

  21. Khalil, H. K. (2002). Nonlinear systems (3rd edn.). New York: Prentice Hall.

    Google Scholar 

  22. Kulić, D., Takano, W., & Nakamura, Y. (2007a). Incremental on-line hierarchical clustering of whole body motion patterns. In IEEE international symposium on robot and human interactive communication.

    Google Scholar 

  23. Kulić, D., Takano, W., & Nakamura, Y. (2007b). Representability of human motions by factorial hidden Markov models. In IEEE/RSJ int. conf. on intelligent robots and systems.

    Google Scholar 

  24. Kulić, D., Takano, W., & Nakamura, Y. (2008). Combining automated on-line segmentation and incremental clustering for whole body motions. In IEEE int. conf. on robotics and automation (pp. 2591–2598).

    Google Scholar 

  25. Kulić, D., Takano, W., & Nakamura, Y. (2009). On-line segmentation and clustering from continuous observation of whole body motions. IEEE Transactions on Robotics, 25(5), 1158–1166.

    Article  Google Scholar 

  26. Lee, D., & Nakamura, Y. (2005). Mimesis from partial observations. In IEEE/RSJ int. conf. on intelligent robots and systems (pp. 1911–1916).

    Google Scholar 

  27. Lee, D., & Nakamura, Y. (2010). Mimesis model from partial observations for a humanoid robot. The International Journal of Robotics Research, 29(1), 60–80.

    Article  Google Scholar 

  28. Lee, D., & Ott, C. (2010). Incremental motion primitive learning by physical coaching using impedance control. In IEEE/RSJ int. conf. on intelligent robots and systems (pp. 4133–4140).

    Google Scholar 

  29. Lee, D., Kulić, D., & Nakamura, Y. (2008). Missing motion data recovery using factorial hidden Markov models. In IEEE int. conf. on robotics and automation (pp. 1722–1728).

    Google Scholar 

  30. Lee, D., Ott, C., & Nakamura, Y. (2010). Mimetic communication model with compliant physical contact in human-humanoid interaction. The International Journal of Robotics Research, 29(13), 1684–1704.

    Article  Google Scholar 

  31. Nakaoka, S., Nakazawa, A., Kanahiro, F., Kaneko, K., Morisawa, M., & Ikeuchi, K. (2005). Task model of lower body motion for a biped humanoid robot to imitate human dances. In IEEE/RSJ int. conf. on intelligent robots and systems (pp. 2769–2774).

    Google Scholar 

  32. Okada, M., Tatani, K., & Nakamura, Y. (2002). Polynomial design of the nonlinear dynamics for the brain-like information processing of whole body motion. In IEEE int. conf. on robotics and automation (pp. 1410–1415).

    Google Scholar 

  33. Ott, C., Albu-Schäffer, A., & Hirzinger, G. (2002). Comparison of adaptive and nonadaptive tracking control laws for a flexible joint manipulator. In IEEE/RSJ int. conference on intelligent robots and systems (pp. 2018–2024).

    Google Scholar 

  34. Ott, C., Eiberger, O., Friedl, W., Bäuml, B., Hillenbrand, U., Borst, C., Albu-Schäffer, A., Brunner, B., Hirschmüller, H., Kielhöfer, S., Konietschke, R., Suppa, M., Wimböck, T., Zacharias, F., & Hirzinger, G. (2006). A humanoid two-arm system for dexterous manipulation. In IEEE-RAS int. conf. on humanoid robots (pp. 276–283).

    Google Scholar 

  35. Ott, C., Lee, D., & Nakamura, Y. (2008a). Motion capture based human motion recognition and imitation by direct marker control. In IEEE-RAS international conference on humanoid robots.

    Google Scholar 

  36. Ott, C., Lee, D., & Nakamura, Y. (2008b). Motion capture based human motion recognition and imitation by direct marker control. In IEEE-RAS int. conf. on humanoid robots (pp. 399–405).

    Google Scholar 

  37. Paden, B., & Panja, R. (1988). Globally asymptotically stable ‘pd+’ controller for robot manipulators. International Journal of Control, 47(6), 1697–1712.

    MATH  Article  Google Scholar 

  38. Peters, J., Vijayakumar, S., & Schaal, S. (2003). Reinforcement learning for humanoid robotics (pp. 1–20).

  39. Platt, R., Abdallah, M., & Wampler, C. (2010). Multi-priority Cartesian impedance control. In Proceedings of robotics: science and systems.

    Google Scholar 

  40. Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286.

    Article  Google Scholar 

  41. Saunders, J., Nehaniv, C., Dautenhahn, K., & Alissandrakis, A. (2007). Self-imitation and environmental scaffolding for robot teaching. International Journal of Advanced Robotics Systems, 4(1), 109–124.

    Google Scholar 

  42. Schaal, S., & Atkeson, C. (1998). Constructive incremental learning from only local information. Neural Computation, 10(8), 2047–2087.

    Article  Google Scholar 

  43. Schaal, S., Ijspeert, A., & Billard, A. (2003). Computational approaches to motor learning by imitation. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 358, 537–547.

    Article  Google Scholar 

  44. Sentis, L., & Khatib, O. (2005). Synthesis of whole-body behaviors through hierarchical control of behavioral primitives. International Journal of Humanoid Robotics, 2(4), 505–518.

    Article  Google Scholar 

  45. Siciliano, B., Sciavicco, L., Villani, L., & Oriolo, G. (2009). Robotics: modelling, planning and control. Berlin: Springer.

    Google Scholar 

  46. Spong, M. (1987). Modeling and control of elastic joint robots. Transactions of the ASME, Journal of Dynamic Systems, Measurement, and Control, 109, 310–319.

    MATH  Article  Google Scholar 

  47. Tani, J., & Ito, M. (2003). Self-organization of behavioral primitives as multiple attractor dynamics: a robot experiment. IEEE Trans on Systems, Man, and Cybernetics Part A: Systems and Humans, 33(4), 481–488.

    Article  Google Scholar 

  48. Vijayakumar, S., D’souza, A., & Schaal, S. (2005). Incremental online learning in high dimensions. Neural Computation, 17(12), 2602–2634.

    MathSciNet  Article  Google Scholar 

  49. Yamane, K., & Hodgins, J. (2009). Simultaneous tracking and balancing of humanoid robots for imitating human motion capture data. In IEEE/RSJ int. conf. on intelligent robots and systems (pp. 2510–2517).

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Dongheui Lee.

Additional information

An earlier version of this work was presented at the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 2010 (Lee and Ott 2010).

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(WMV 18.00 MB)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lee, D., Ott, C. Incremental kinesthetic teaching of motion primitives using the motion refinement tube. Auton Robot 31, 115–131 (2011). https://doi.org/10.1007/s10514-011-9234-3

Download citation

Keywords

  • Programming by demonstration
  • Imitation learning
  • Physical coaching
  • Incremental learning
  • Motion refinement tube
  • Impedance control