From Motor Learning to Interaction Learning in Robots

  • Olivier Sigaud
  • Jan Peters

Abstract

The number of advanced robot systems has been increasing in recent years yielding a large variety of versatile designs with many degrees of freedom. These robots have the potential of being applicable in uncertain tasks outside wellstructured industrial settings. However, the complexity of both systems and tasks is often beyond the reach of classical robot programming methods. As a result, a more autonomous solution for robot task acquisition is needed where robots adaptively adjust their behaviour to the encountered situations and required tasks.

Learning approaches pose one of the most appealing ways to achieve this goal. However, while learning approaches are of high importance for robotics, we cannot simply use off-the-shelf methods from the machine learning community as these usually do not scale into the domains of robotics due to excessive computational cost as well as a lack of scalability. Instead, domain appropriate approaches are needed. In this book, we focus on several core domains of robot learning. For accurate task execution, we need motor learning capabilities. For fast learning of the motor tasks, imitation learning offers the most promising approach. Self improvement requires reinforcement learning approaches that scale into the domain of complex robots. Finally, for efficient interaction of humans with robot systems, we will need a form of interaction learning. This chapter provides a general introduction to these issues and briefly presents the contributions of the subsequent chapters to the corresponding research topics.

Keywords

motor learning interaction learning imitation learning reinforcement learning robotics 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abbeel, P., Ng, A.Y.: Apprenticeship learning via inverse reinforcement learning. In: Proceedings of the International Conference on Machine Learning (2004)Google Scholar
  2. Argall, B.D.: Mobile robot motion control from demonstration and corrective feedback. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 431–450. Springer, Heidelberg (2010)Google Scholar
  3. Atkeson, C.G., Schaal, S.: Robot learning from demonstration. In: Proceedings of the International Conferenec on Machine Learning, pp. 12–20 (1997)Google Scholar
  4. Calinon, S., Guenter, F., Billard, A.: On Learning, Representing and Generalizing a Task in a Humanoid Robot. IEEE Transactions on Systems, Man and Cybernetics, Part B 37(2), 286–298 (2007)CrossRefGoogle Scholar
  5. Chalodhorn, R., Rao, R.P.N.: Learning to imitate human actions through eigenposes. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 357–381. Springer, Heidelberg (2010)Google Scholar
  6. Coates, A., Abbeel, P., Ng, A.Y.: Learning for control from multiple demonstrations. In: Proceedings of the International Conference on Machine Learning (2008)Google Scholar
  7. Demiris, Y., Khadhouri, B.: Hierarchical attentive multiple models for execution and recognition of actions. Robotics and Autonomous Systems 54, 361–369 (2006)CrossRefGoogle Scholar
  8. Detry, R., Baseski, E., Popovi, M., Touati, Y., Krüger, N., Kroemer, O., Peters, J., Piater, J.: Learning continuous grasp affordances from sensorimotor interaction. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 451–465. Springer, Heidelberg (2010)Google Scholar
  9. Doya, K.: What are the computations of the cerebellum, the basal ganglia, and the cerebral cortex? Neural Networks 12, 961–974 (1999)CrossRefGoogle Scholar
  10. Duff, A., César, R., Costa, R., Marcos, E., Luvizotto, A.L., Giovannucci, A., Sanchez Fibla, M., Bernardet, U., Verschure, P.F.M.J.: Distributed adaptive control: A proposal on the neuronal organization of adaptive goal oriented behavior. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 15–41. Springer, Heidelberg (2010)Google Scholar
  11. Fumagalli, M., Gijsberts, A., Ivaldi, S., Jamone, L., Metta, G., Natale, L., Nori, F., Sandini, G.: Learning how to exploit proximal force sensing: a comparison approach. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 149–167. Springer, Heidelberg (2010)Google Scholar
  12. Grollman, D.H., Jenkins, O.C.: Can we learn finite state machine robot controllers from interactive demonstration? In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 407–430. Springer, Heidelberg (2010)Google Scholar
  13. Herbort, O., Butz, M.V., Pedersen, G.: The sure reach model for motor learning and control of a redundant arm: from modeling human behavior to applications in robots. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 85–106. Springer, Heidelberg (2010)Google Scholar
  14. Hörnstein, J., Gustavsson, L., Santos-Victor, J., Lacerda, F.: Multimodal language acquisition based on motor learning and interaction. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 467–489. Springer, Heidelberg (2010)Google Scholar
  15. Howard, M., Klanke, S., Gienger, M., Goerick, C., Vijayakumar, S.: Methods for learning control policies from variable-constraint demonstrations. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 253–291. Springer, Heidelberg (2010)Google Scholar
  16. Ijspeert, A.J., Nakanishi, J., Schaal, S.: Learning attractor landscapes for learning motor primitives. In: Advances in Neural Information Processing Systems (NIPS), pp. 1523–1530 (2003)Google Scholar
  17. Kober, J., Mohler, B., Peters, J.: Imitation and reinforcement learning for motor primitives with perceptual coupling. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 209–225. Springer, Heidelberg (2010)Google Scholar
  18. Kulić, D., Nakamura, Y.: Incremental learning of full body motion primitives. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 383–406. Springer, Heidelberg (2010)Google Scholar
  19. Lagarde, M., Andry, P., Gaussier, P., Boucenna, S., Hafemeister, L.: Proprioception and imitation: on the road to agent individuation. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 43–63. Springer, Heidelberg (2010)Google Scholar
  20. Lallee, S., Yoshida, E., Mallet, A., Nori, F., Natale, L., Metta, G., Warneken, F., Dominey, P.F.: Human-robot cooperation based on interaction learning. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 491–536. Springer, Heidelberg (2010)Google Scholar
  21. Lopes, M., Melo, F., Montesano, L., Santos-Victor, J.: Abstraction levels for robotic imitation: Overview and computational approaches. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 313–355. Springer, Heidelberg (2010)Google Scholar
  22. Lungarella, M., Metta, G., Pfeifer, R., Sandini, G.: Developmental robotics: a survey. Connection Science 0, 1–40 (2004)Google Scholar
  23. Mitrovic, D., Klanke, S., Vijayakumar, S.: Adaptive optimal feedback control with learned internal dynamics models. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 65–84. Springer, Heidelberg (2010)Google Scholar
  24. Nguyen-Tuong, D., Seeger, M., Peters, J.: Real-time local gp model learning. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 193–207. Springer, Heidelberg (2010)Google Scholar
  25. Oudeyer, P.Y., Baranes, A., Kaplan, F.: Intrinsically motivated exploration and active sensorimotor learning. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 107–146. Springer, Heidelberg (2010)Google Scholar
  26. Ratliff, N.D., Silver, D., Bagnell, J.A.: Learning to search: Functional gradient techniques for imitation learning. Autonomous Robots 27(1), 25–53 (2009)CrossRefGoogle Scholar
  27. Roberts, J.W., Moret, L., Zhang, J., Tedrake, R.: Motor Learning at Intermediate Reynolds Number: Experiments with Policy Gradient on the Flapping Flight of a RigidWing. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 293–309. Springer, Heidelberg (2010)Google Scholar
  28. Salaün, C., Padois, V., Sigaud, O.: Learning forward models for the operational space control of redundant robots. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 169–192. Springer, Heidelberg (2010)Google Scholar
  29. Schaal, S.: Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences 6, 233–242 (1999)CrossRefGoogle Scholar
  30. Todorov, E.: Optimality principles in sensorimotor control. Nature Neurosciences 7(9), 907–915 (2004)CrossRefGoogle Scholar
  31. Todorov, E., Jordan, M.I.: Optimal feedback control as a theory of motor coordination. Nature Neurosciences 5(11), 1226–1235 (2002)CrossRefGoogle Scholar
  32. Todorov, E., Li, W.: A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems. In: Proceedings of the American Control Conference, pp. 300–306 (2005)Google Scholar
  33. Toussaint, M., Goerick, C.: A bayesian view on motor control and planning. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 227–252. Springer, Heidelberg (2010)Google Scholar
  34. Wolpert, D.M., Doya, K., Kawato, M.: A unifying computational framework for motor control and social interaction. Philosophical Transactions of the Royal Society 358, 593–602 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Olivier Sigaud
    • 1
  • Jan Peters
    • 2
  1. 1.Institut des Systèmes Intelligents et de Robotique - CNRS UMR 7222Université Pierre et Marie Curie, Pyramide Tour 55Paris CEDEX 5France
  2. 2.Dept. SchölkopfMax Planck Institute for Biological CyberneticsTübingenGermany

Personalised recommendations