Advertisement

Towards Machine Learning of Motor Skills

  • Jan Peters
  • Stefan Schaal
  • Bernhard Schölkopf
Part of the Informatik aktuell book series (INFORMAT)

Abstract

Autonomous robots that can adapt to novel situations has been a long standing vision of robotics, artificial intelligence, and cognitive sciences. Early approaches to this goal during the heydays of artificial intelligence research in the late 1980s, however, made it clear that an approach purely based on reasoning or human insights would not be able to model all the perceptuomotor tasks that a robot should fulfill. Instead, new hope was put in the growing wake of machine learning that promised fully adaptive control algorithms which learn both by observation and trial-and-error. However, to date, learning techniques have yet to fulfill this promise as only few methods manage to scale into the high-dimensional domains of manipulator robotics, or even the new upcoming trend of humanoid robotics, and usually scaling was only achieved in precisely pre-structured domains. In this paper, we investigate the ingredients for a general approach to motor skill learning in order to get one step closer towards human-like performance. For doing so, we study two major components for such an approach, i.e., firstly, a theoretically well-founded general approach to representing the required control structures for task representation and execution and, secondly, appropriate learning algorithms which can be applied in this setting.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    J.J. Craig. Introduction to Robotics: Mechanics and Control. Pearson Prentice Hall, Upper Saddle River, NJ, 2005.Google Scholar
  2. 2.
    L. Sciavicco and B. Siciliano. Modeling and control of robot manipulators. MacGraw-Hill, Heidelberg, Germany, 2007.Google Scholar
  3. 3.
    S. Schaal, A. Ijspeert, and A. Billard. Computational approaches to motor learning by imitation. In C. D. Frith and D. Wolpert, editors, The Neuroscience of Social Interaction, pages 199–218. Oxford University Press, Oxford, UK, 2004.Google Scholar
  4. 4.
    J. A. Ijspeert, J. Nakanishi, and S. Schaal. Movement imitation with nonlinear dynamical systems in humanoid robots. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Washinton, DC, May 11–15 2002.Google Scholar
  5. 5.
    A. Ijspeert, J. Nakanishi, and S. Schaal. Learning attractor landscapes for learning motor primitives. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems, volume 15, pages 1547–1554, Cambridge, MA, 2003. MIT Press.Google Scholar
  6. 6.
    Douglas Aberdeen. POMDPs and policy gradients. In Proceedings of the Machine Learning Summer School (MLSS), Canberra, Australia, 2006.Google Scholar
  7. 7.
    J. Peters and S. Schaal. Learning operational space control. In Proceedings of Robotics: Science and Systems (RSS), Philadelphia, PA, 2006.Google Scholar
  8. 8.
    J. Peters and S. Schaal. Reinforcement learning by reward-weighted regression for operational space control. In Proceedings of the International Conference on Machine Learning (ICML), 2007.Google Scholar
  9. 9.
    J. Peters and S. Schaal. Reinforcement learning for operational space. In Proceedings of the International Conference on Robotics and Automation (ICRA), Rome, Italy, 2007.Google Scholar
  10. 10.
    J. Peters, S. Vijayakumar, and S. Schaal. Reinforcement learning for humanoid robotics. In Proceedings of the IEEE-RAS International Conference on Humanoid Robots (HUMANOIDS), Karlsruhe, Germany, September 2003.Google Scholar
  11. 11.
    J. Peters, S. Vijayakumar, and S. Schaal. Natural actor-critic. In Proceedings of the European Conference on Machine Learning (ECML), pages 280–291. springer, 2005.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Jan Peters
    • 1
    • 2
  • Stefan Schaal
    • 2
  • Bernhard Schölkopf
    • 1
  1. 1.Max-Planck Institute for Biological CyberneticsTübingen
  2. 2.University of Southern CaliforniaLos Angeles

Personalised recommendations