Advertisement

Towards Motor Skill Learning for Robotics

  • Jan Peters
  • Katharina Mülling
  • Jens Kober
  • Duy Nguyen-Tuong
  • Oliver Krömer
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 70)

Abstract

Learning robots that can acquire new motor skills and refine existing one has been a long standing vision of robotics, artificial intelligence, and the cognitive sciences. Early steps towards this goal in the 1980s made clear that reasoning and human insights will not suffice. Instead, new hope has been offered by the rise of modern machine learning approaches. However, to date, it becomes increasingly clear that off-the-shelf machine learning approaches will not suffice for motor skill learning as these methods often do not scale into the high-dimensional domains of manipulator and humanoid robotics nor do they fulfill the real-time requirement of our domain. As an alternative, we propose to break the generic skill learning problem into parts that we can understand well from a robotics point of view. After designing appropriate learning approaches for these basic components, these will serve as the ingredients of a general approach to motor skill learning. In this paper, we discuss our recent and current progress in this direction. For doing so, we present our work on learning to control, on learning elementary movements as well as our steps towards learning of complex tasks. We show several evaluations both using real robots as well as physically realistic simulations.

Keywords

Motor Skill Reinforcement Learning Real Robot Rigid Body Dynamic Gaussian Process Regression 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Sciavicco, L., Siciliano, B.: Modeling and control of robot manipulators. MacGraw-Hill, Heidelberg (2007)Google Scholar
  2. 2.
    Schaal, S., Atkeson, C.G., Vijayakumar, S.: Scalable techniques from nonparameteric statistics for real-time robot learning. Applied Intelligence, 49–60 (2002)Google Scholar
  3. 3.
    Schaal, S., Ijspeert, A.J., Billard, A.: Computational approaches to motor learning by imitation. In: Frith, C.D., Wolpert, D. (eds.) The Neuroscience of Social Interaction, pp. 199–218. Oxford University Press, Oxford (2004)Google Scholar
  4. 4.
    Ijspeert, A.J., Nakanishi, J., Schaal, S.: Learning attractor landscapes for learning motor primitives. In: Advances in Neural Information Processing Systems, vol. 15, pp. 1547–1554. MIT Press, Cambridge (2003)Google Scholar
  5. 5.
    Nakanishi, J., Farrell, J.A., Schaal, S.: Composite adaptive control with locally weighted statistical learning. Neural Networks 18(1), 71–90 (2005)CrossRefzbMATHGoogle Scholar
  6. 6.
    Nguyen-Tuong, D., Seeger, M., Peters, J.: Local Gaussian Process Regression for Real Time Online Model Learning and Control. In: Advances in Neural Information Processing Systems (NIPS 2008), vol. 21. MIT Press, Cambridge (2009)Google Scholar
  7. 7.
    Nguyen-Tuong, D., Seeger, M., Peters, J.: Computed torque control with nonparametric regression models. In: Proceedings of the 2008 American Control Conference (ACC 2008) (2008)Google Scholar
  8. 8.
    Nguyen-Tuong, D., Peters, J.: Semi-parametric regression in learning inverse dynamics. In: International Conference on Robotics & Automation (ICRA) (2010) (Submitted)Google Scholar
  9. 9.
    Chiappa, S., Peters, J.: Motion segmentation by detecting in continuous time-series. In: Advances in Neural Information Processing Systems (NIPS 2009), vol. 22. MIT Press, Cambridge (2010) (Submitted)Google Scholar
  10. 10.
    Chiappa, S., Kober, J., Peters, J.: Using Bayesian Dynamical Systems for Motion Template Libraries. In: Advances in Neural Information Processing Systems (NIPS 2008), vol. 21. MIT Press, Cambridge (2009)Google Scholar
  11. 11.
    Dayan, P., Hinton, G.E.: Using expectation-maximization for reinforcement learning. Neural Computation 9(2), 271–278 (1997)CrossRefzbMATHGoogle Scholar
  12. 12.
    Peters, J., Schaal, S.: Reinforcement learning of motor skills with policy gradients. Neural Networks 21(4), 682–697 (2008)CrossRefGoogle Scholar
  13. 13.
    Kober, J., Peters, J.: Policy Search for Motor Primitives in Robotics. In: Advances in Neural Information Processing Systems (NIPS 2008), vol. 21. MIT Press, Cambridge (2009)Google Scholar
  14. 14.
    Kober, J., Mohler, B., Peters, J.: Learning Perceptual Coupling for Motor Primitives. In: Proceedings of the IEEE International Conference on Intelligent RObots and Systems (IROS) (2008)Google Scholar
  15. 15.
    Kober, J., Peters, J.: Learning Motor Primitives for Robotics. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2009)Google Scholar
  16. 16.
    Jordan, M., Jacobs, R.: Hierarchical mixture of experts and the EM algorithm. Neural Computation 6, 181–214 (1994)CrossRefGoogle Scholar
  17. 17.
    Muelling, K.: Learning (2009)Google Scholar
  18. 18.
    Peters, J., Mistry, M., Udwadia, F.E., Nakanishi, J., Schaal, S.: A unifying methodology for robot control with redundant DOFs. Autonomous Robots 24(1), 1–12 (2008)CrossRefGoogle Scholar
  19. 19.
    Peters, J., Schaal, S.: Reinforcement learning by reward-weighted regression for operational space control. In: Proceedings of the International Conference on Machine Learning (ICML 2007), Oregon, USA, pp. 745–750 (2007)Google Scholar
  20. 20.
    Peters, J., Schaal, S.: Learning to Control in Operational Space. The International Journal of Robotics Research 27(2), 197–212 (2008)CrossRefGoogle Scholar
  21. 21.
    Pastor, P., Hoffmann, H., Asfour, T., Schaal, S.: Learning and generalization of motor skills by learning from demonstration. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2009)Google Scholar
  22. 22.
    Peters, J., Nguyen, D.: Real-Time Learning of Resolved Velocity Control on a Mitsubishi PA-10. In: International Conference on Robotics and Automation (ICRA) (2008)Google Scholar
  23. 23.
    Sentis, L., Khatib, O.: Synthesis of whole-body behaviors through hierarchical control of behavioral primitives. International Journal of Humanoid Robotics 2(4), 505–518 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Jan Peters
    • 1
  • Katharina Mülling
    • 1
  • Jens Kober
    • 1
  • Duy Nguyen-Tuong
    • 1
  • Oliver Krömer
    • 1
  1. 1.Department of Empirical Inference & Machine LearningMax Planck Institute for Biological CyberneticsTübingenGermany

Personalised recommendations