Towards Machine Learning of Motor Skills
Autonomous robots that can adapt to novel situations has been a long standing vision of robotics, artificial intelligence, and cognitive sciences. Early approaches to this goal during the heydays of artificial intelligence research in the late 1980s, however, made it clear that an approach purely based on reasoning or human insights would not be able to model all the perceptuomotor tasks that a robot should fulfill. Instead, new hope was put in the growing wake of machine learning that promised fully adaptive control algorithms which learn both by observation and trial-and-error. However, to date, learning techniques have yet to fulfill this promise as only few methods manage to scale into the high-dimensional domains of manipulator robotics, or even the new upcoming trend of humanoid robotics, and usually scaling was only achieved in precisely pre-structured domains. In this paper, we investigate the ingredients for a general approach to motor skill learning in order to get one step closer towards human-like performance. For doing so, we study two major components for such an approach, i.e., firstly, a theoretically well-founded general approach to representing the required control structures for task representation and execution and, secondly, appropriate learning algorithms which can be applied in this setting.
Unable to display preview. Download preview PDF.
- 1.J.J. Craig. Introduction to Robotics: Mechanics and Control. Pearson Prentice Hall, Upper Saddle River, NJ, 2005.Google Scholar
- 2.L. Sciavicco and B. Siciliano. Modeling and control of robot manipulators. MacGraw-Hill, Heidelberg, Germany, 2007.Google Scholar
- 3.S. Schaal, A. Ijspeert, and A. Billard. Computational approaches to motor learning by imitation. In C. D. Frith and D. Wolpert, editors, The Neuroscience of Social Interaction, pages 199–218. Oxford University Press, Oxford, UK, 2004.Google Scholar
- 4.J. A. Ijspeert, J. Nakanishi, and S. Schaal. Movement imitation with nonlinear dynamical systems in humanoid robots. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Washinton, DC, May 11–15 2002.Google Scholar
- 5.A. Ijspeert, J. Nakanishi, and S. Schaal. Learning attractor landscapes for learning motor primitives. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems, volume 15, pages 1547–1554, Cambridge, MA, 2003. MIT Press.Google Scholar
- 6.Douglas Aberdeen. POMDPs and policy gradients. In Proceedings of the Machine Learning Summer School (MLSS), Canberra, Australia, 2006.Google Scholar
- 7.J. Peters and S. Schaal. Learning operational space control. In Proceedings of Robotics: Science and Systems (RSS), Philadelphia, PA, 2006.Google Scholar
- 8.J. Peters and S. Schaal. Reinforcement learning by reward-weighted regression for operational space control. In Proceedings of the International Conference on Machine Learning (ICML), 2007.Google Scholar
- 9.J. Peters and S. Schaal. Reinforcement learning for operational space. In Proceedings of the International Conference on Robotics and Automation (ICRA), Rome, Italy, 2007.Google Scholar
- 10.J. Peters, S. Vijayakumar, and S. Schaal. Reinforcement learning for humanoid robotics. In Proceedings of the IEEE-RAS International Conference on Humanoid Robots (HUMANOIDS), Karlsruhe, Germany, September 2003.Google Scholar
- 11.J. Peters, S. Vijayakumar, and S. Schaal. Natural actor-critic. In Proceedings of the European Conference on Machine Learning (ECML), pages 280–291. springer, 2005.Google Scholar