Learning from Humans

  • Aude G. Billard
  • Sylvain Calinon
  • Rüdiger Dillmann

Abstract

This chapter surveys the main approaches developed to date to endow robots with the ability to learn from human guidance. The field is best known as robot programming by demonstration, robot learning from/by demonstration, apprenticeship learning and imitation learning. We start with a brief historical overview of the field. We then summarize the various approaches taken to solve four main questions: when, what, who and when to imitate. We emphasize the importance of choosing well the interface and the channels used to convey the demonstrations, with an eye on interfaces providing force control and force feedback. We then review algorithmic approaches to model skills individually and as a compound and algorithms that combine learning from human guidance with reinforcement learning. We close with a look on the use of language to guide teaching and a list of open issues.

2-D

two-dimensional

ANN

artificial neural network

EO

elementary operator

HMM

hidden Markov model

HRI

human–robot interaction

IRL

inverse reinforcement learning

LfD

learning from demonstration

learning from human demonstration

ML

machine learning

PbD

programming by demonstration

POMDP

partially observable Markov decision process

RBF

radial basis function network

RL

reinforcement learning

References

  1. 74.1
    T. Lozano-Perez: Robot programming, Proceedings IEEE 71(7), 821–841 (1983)CrossRefGoogle Scholar
  2. 74.2
    B. Dufay, J.-C. Latombe: An approach to automatic robot programming based on inductive learning, Int. J. Robotics Res. 3(4), 3–20 (1984)CrossRefGoogle Scholar
  3. 74.3
    A. Levas, M. Selfridge: A user-friendly high-level robot teaching system, IEEE Int. Conf. Robotics, Altanta (1984) pp. 413–416Google Scholar
  4. 74.4
    A.B. Segre, G. DeJong: Explanation-based manipulator learning: Acquisition of planning ability through observation, IEEE Conf. Robotics Autom. St. Louis (1985) pp. 555–560Google Scholar
  5. 74.5
    A.M. Segre: Machine Learning of Robot Assembly Plans (Kluwer, Boston 1988)CrossRefGoogle Scholar
  6. 74.6
    S. Muench, J. Kreuziger, M. Kaiser, R. Dillmann: Robot programming by demonstration (RPD) - Using machine learning and user interaction methods for the development of easy and comfortable robot programming systems, Proc. Int. Symp. Indus. Robots (ISIR) (1994) pp. 685–693Google Scholar
  7. 74.7
    A. Billard: Imitation: A review. In: The Handbook of Brain Theory and Neural Network, 2nd edn., ed. by M.A. Arbib (MIT Press, Cambridge 2002) pp. 566–569Google Scholar
  8. 74.8
    E. Oztop, M. Kawato, M.A. Arbib: Mirror neurons and imitation: A computationally guided review, Neural Netw. 19(3), 254–271 (2006)MATHCrossRefGoogle Scholar
  9. 74.9
    J. Demiris, G. Hayes: Imitation as a dual-route process featuring predictive and learning components: A biologically-plausible computational model. In: Imitation in Animals and Artifacs, ed. by C. Nehaniv, K. Dautenhahn (MIT Press, Cambridge 2002)Google Scholar
  10. 74.10
    J. Nadel, A. Revel, P. Andry, P. Gaussier: Toward communication: First imitations in infants, low-functioning children with autism and robots, Interact. Stud. 5(1), 45–74 (2004)CrossRefGoogle Scholar
  11. 74.11
    F. Kaplan, P.-Y. Oudeyer: The progress-drive hypothesis: An interpretation of early imitation. In: Models and Mechanisms of Imitation and Social Learning: Behavioural, Social and Communication Dimensions, ed. by K. Dautenhahn, C. Nehaniv (Cambridge Univ. Press, Cambridge 2007) pp. 361–377Google Scholar
  12. 74.12
    B.D. Argall, M. Veloso, B. Browning: Teacher feedback to scaffold and refine demonstrated motion primitives on a mobile robot, Robotics Auton. Syst. 59(3/4), 243–255 (2011)CrossRefGoogle Scholar
  13. 74.13
    B. Robins, K. Dautenhahn, C.L. Nehaniv, N.A. Mirza, D. Francois, L. Olsson: Sustaining interaction dynamics and engagement in dyadic child-robot interaction kinesics: Lessons learnt from an exploratory study, IEEE Int. Workshop Robot Human Int. Commun. (ROMAN) (2005) pp. 716–722Google Scholar
  14. 74.14
    M. Cakmak, A.L. Thomaz: Designing robot learners that ask good questions, IEEE-ACML Int. Conf. Human-Robot Int. (HRI) (2012)Google Scholar
  15. 74.15
    C. Breazeal, A. Brooks, J. Gray, G. Hoffman, C. Kidd, H. Lee, J. Lieberman, A. Lockerd, D. Chilongo: Tutelage and collaboration for humanoid robots, Human. Robots 1(2), 315–348 (2004)CrossRefGoogle Scholar
  16. 74.16
    Y. Kuniyoshi, M. Inaba, H. Inoue: Teaching by showing: Generating robot programs by visual observation of human performance, Proc. Int. Symp. Ind. Robots, Tokyo (1989) pp. 119–126Google Scholar
  17. 74.17
    Y. Kuniyoshi, M. Inaba, H. Inoue: Learning by watching: Extracting reusable task knowledge from visual observation of human performance, IEEE Trans. Robotics Autom. 10(6), 799–822 (1994)CrossRefGoogle Scholar
  18. 74.18
    M. Ehrenmann, O. Rogalla, R. Zöllner, R. Dillmann: Teaching service robots complex tasks: Programming by demonstation for workshop and household environments, Proc. IEEE Int. Conf. Field Serv. Robotics (FRS) (2001)Google Scholar
  19. 74.19
    C.P. Tung, A.C. Kak: Automatic learning of assembly task using a dataglove system, IEEE/RSJ Int. Conf. Intell. Robots Syst., Pittsburgh (1995) pp. 1–8Google Scholar
  20. 74.20
    K. Ikeuchi, T. Suchiro: Towards an assembly plan from observation, Part I: Assembly task recognition using face-contact relations (polyhedral objects), Proc. IEEE Int. Conf. Robot. Autom. (ICRA), Vol. 3 (1992) pp. 2171–2177Google Scholar
  21. 74.21
    S. Calinon, F. Guenter, A. Billard: On learning, representing and generalizing a task in a humanoid robot, IEEE Trans. Syst. Man Cybern. B 37(2), 286–298 (2007)CrossRefGoogle Scholar
  22. 74.22
    M. Ito, K. Noda, Y. Hoshino, J. Tani: Dynamic and interactive generation of object handling behaviors by a small humanoid robot using a dynamic neural network model, Neural Netw. 19(3), 323–337 (2006)MATHCrossRefGoogle Scholar
  23. 74.23
    T. Inamura, N. Kojo, M. Inaba: Situation recognition and behavior induction based on geometric symbol representation of multimodal sensorimotor patterns, IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS) (2006) pp. 5147–5152Google Scholar
  24. 74.24
    S. Liu, H. Asada: Teaching and learning of deburring robots using neural networks, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) (1993) pp. 339–345CrossRefGoogle Scholar
  25. 74.25
    A. Billard: Learning motor skills by imitation: A biologically inspired robotic model, J. Cybern. Syst. 32(1/2), 155–193 (2001)MATHCrossRefGoogle Scholar
  26. 74.26
    M. Kaiser, R. Dillmann: Building elementary robot skills from human demonstration, Proc. IEEE Int. Conf. Robotics Autom. (ICRA), Vol. 3 (1996) pp. 2700–2705CrossRefGoogle Scholar
  27. 74.27
    R. Dillmann, M. Kaiser, A. Ude: Acquisition of elementary robot skills from human demonstration, Int. Symp. Intell. Robotics Syst. (SIRS) (1995) pp. 1–38Google Scholar
  28. 74.28
    W. Yang: Hidden Markov model approach to skill learning and its application in telerobotics, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) (1993) pp. 396–402CrossRefGoogle Scholar
  29. 74.29
    P.K. Pook, D.H. Ballard: Recognizing teleoperated manipulations, Proc. IEEE Int. Conf. Robotics Autom., Atlanta (1993) pp. 578–585CrossRefGoogle Scholar
  30. 74.30
    G.E. Hovland, P. Sikka, B.J. McCarragher: Skill acquisition from human demonstration using a hidden Markov model, Proc. IEEE Int. Conf. Robotics Autom., Minneapolis (1996) pp. 2706–2711CrossRefGoogle Scholar
  31. 74.31
    S.K. Tso, K.P. Liu: Hidden Markov model for intelligent extraction of robot trajectory command from demonstrated trajectories, Proc. IEEE Int. Conf. Ind. Technol. (ICIT) (1996) pp. 294–298CrossRefGoogle Scholar
  32. 74.32
    C. Lee, Y. Xu: Online, interactive learning of gestures for human/robot interfaces, Proc. IEEE Int. Conf. Robotics Autom. (ICRA), Vol. 4 (1996) pp. 2982–2987CrossRefGoogle Scholar
  33. 74.33
    D. Kulic, W. Takano, Y. Nakamura: Incremental learning, clustering and hierarchy formation of whole body motion patterns using adaptive hidden Markov chains, Int. J. Robotics Res. 27(7), 761–784 (2008)CrossRefGoogle Scholar
  34. 74.34
    D. Nguyen-Tuong, M. Seeger, J. Peters: Local Gaussian process regression for real time online model learning and control, Adv. Neural Inf. Process. Syst. 21, 1193–1200 (2009)Google Scholar
  35. 74.35
    S.M. Khansari Zadeh, A. Billard: Learning stable non-linear dynamical systems with Gaussian mixture models, IEEE Trans. Robotics 27(5), 943–957 (2011)CrossRefGoogle Scholar
  36. 74.36
    C. Nehaniv, K. Dautenhahn: Of hummingbirds and helicopters: An algebraic framework for interdisciplinary studies of imitation and its applications. In: Interdisciplinary Approaches to Robot Learning, Vol. 24, ed. by J. Demiris, A. Birk (World Scientific, Singapore 2000) pp. 136–161CrossRefGoogle Scholar
  37. 74.37
    C.L. Nehaniv: Nine billion correspondence problems and some methods for solving them, Proc. Int. Symp. Imit. Anim. Artifacts (2003) pp. 93–95Google Scholar
  38. 74.38
    P. Bakker, Y. Kuniyoshi: Robot see, robot do: An overview of robot imitation, AISB Workshop Learn. Robot. Anim., Brighton (1996)Google Scholar
  39. 74.39
    M. Skubic, R.A. Volz: Acquiring robust, force-based assembly skills from human demonstration, IEEE Trans. Robotics Autom. 16(6), 772–781 (2000)CrossRefGoogle Scholar
  40. 74.40
    M. Yeasin, S. Chaudhuri: Toward automatic robot programming: Learning human skill from visual data, IEEE Trans. Syst. Man Cybern. B 30(1), 180–185 (2000)CrossRefGoogle Scholar
  41. 74.41
    J. Zhang, B. Rössler: Self-valuing learning and generalization with application in visually guided grasping of complex objects, Robotics Auton. Syst. 47(2/3), 117–127 (2004)CrossRefGoogle Scholar
  42. 74.42
    M. Frank, M. Plaue, H. Rapp, U. Koethe, B. Jaehne, F.A. Hamprecht: Theoretical and experimental error analysis of continuous-wave time-of-flight range cameras, Opt. Eng. 48(1), 013602 (2009)CrossRefGoogle Scholar
  43. 74.43
    M. Freese, S. Singh, F. Ozaki, N. Matsuhira: Virtual robot experimentation platform v-rep: A versatile 3d robot simulator, Proc. Int. Conf. Simul. Model. Progr. Auton. Robots (SIMPAR) (2010) pp. 51–62Google Scholar
  44. 74.44
    S. Hak, N. Mansard, O. Ramos, L. Saab, O. Stasse: Capture, recognition and imitation of anthropomorphic motion, IEEE-RAS Int. Conf. Robotics Autom. (2012) pp. 3539–3540Google Scholar
  45. 74.45
    G. Gioioso, G. Salvietti, M. Malvezzi, D. Prattichizzo: An object-based approach to map human hand synergies onto robotic hands with dissimilar kinematics. In: Robotics - Science and Systems VIII, ed. by N. Roy, P. Newman, S. Srinivasa (MIT Press, Cambridge 2012) pp. 97–105Google Scholar
  46. 74.46
    A. Shon, K. Grochow, A. Hertzmann, R. Rao: Learning shared latent structure for image synthesis and robotic imitation, Adv. Neural Inf. Process. Syst. (NIPS) 18, 1233–1240 (2006)Google Scholar
  47. 74.47
    A. Ude, C.G. Atkeson, M. Riley: Programming full-body movements for humanoid robots by observation, Robotics Auton. Syst. 47, 93–108 (2004)CrossRefGoogle Scholar
  48. 74.48
    S. Kim, C. Kim, B. You, S. Oh: Stable whole-body motion generation for humanoid robots to imitate human motions, Proc. IEEE/RSJ Int. Conf. Intell. Robotics Syst. (IROS) (2009)Google Scholar
  49. 74.49
    S. Nakaoka, A. Nakazawa, F. Kanehiro, K. Kaneko, M. Morisawa, H. Hirukawa, K. Ikeuchi: Learning from observation paradigm: Leg task models for enabling a biped humanoid robot to imitate human dances, Int. J. Robotics Res. 26(8), 829–844 (2007)CrossRefGoogle Scholar
  50. 74.50
    E.L. Sauser, B.D. Argall, G. Metta, A.G. Billard: Iterative learning of grasp adaptation through human corrections, Robotics Auton. Syst. 60(1), 55–71 (2012)CrossRefGoogle Scholar
  51. 74.51
    A. Coates, P. Abbeel, A.Y. Ng: Learning for control from multiple demonstrations, Proc. 25th Int. Conf. Mach. Learn. (2008)Google Scholar
  52. 74.52
    D. Grollman, O.C. Jenkins: Incremental learning of subtasks from unsegmented demonstration, Int. Conf. Intell. Robots Syst. (2010)Google Scholar
  53. 74.53
    L. Peternel, J. Babic: Humanoid robot posture-control learning in real-time based on human sensorimotor learning ability, IEEE Int. Conf. Robotics Autom. (ICRA) Karlsruhe (2013)Google Scholar
  54. 74.54
    A. Ude: Robust estimation of human body kinematics from video, Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS) (1999) pp. 1489–1494Google Scholar
  55. 74.55
    B. Akgun, M. Cakmak, K. Jiang, A.L. Thomaz: Keyframe-based learning from demonstration, Int. J. Soc. Robotics 4, 343–355 (2012)CrossRefGoogle Scholar
  56. 74.56
    P. Evrard, E. Gribovskaya, S. Calinon, A. Billard, A. Kheddar: Teaching physical collaborative tasks: Object-lifting case study with a humanoid, Proc. IEEE-RAS Int. Conf. Humanoid Robots (Humanoids), Paris (2009) pp. 399–404Google Scholar
  57. 74.57
    C. Chao, M. Cakmak, A.L. Thomaz: Designing interactions for robot active learners, IEEE Trans. Auton. Mental Dev. 2(2), 108–118 (2010)CrossRefGoogle Scholar
  58. 74.58
    S. Calinon, A. Billard: PDA interface for humanoid robots, Proc. IEEE Int. Conf. Humanoid Robots (Humanoids) (2003)Google Scholar
  59. 74.59
    A. Shon, K. Grochow, R. Rao: Robotic imitation from human motion capture using Gaussian processes, Proc. IEEE/RAS Int. Conf. Humanoid Robots (Humanoids) (2005)Google Scholar
  60. 74.60
    Y. Wu, Y. Demiris: Towards one shot learning by imitation for humanoid robots, IEEE-RAS Int. Conf. Robotics Autom. (ICRA) (2010)Google Scholar
  61. 74.61
    J. Nakanishi, J. Morimoto, G. Endo, G. Cheng, S. Schaal, M. Kawato: Learning from demonstration and adaptation of biped locomotion, Robotics Auton. Syst. 47(2/3), 79–91 (2004)CrossRefGoogle Scholar
  62. 74.62
    D. Lee, C. Ott: Incremental kinesthetic teaching of motion primitives using the motion refinement tube, Auton. Robot. 31(2), 115–131 (2011)CrossRefGoogle Scholar
  63. 74.63
    A. Ude: Trajectory generation from noisy positions of object features for teaching robot paths, Robotics Auton. Syst. 11(2), 113–127 (1993)CrossRefGoogle Scholar
  64. 74.64
    J. Yang, Y. Xu, C.S. Chen: Human action learning via hidden Markov model, IEEE Trans. Syst. Man Cybern. A 27(1), 34–44 (1997)CrossRefGoogle Scholar
  65. 74.65
    K. Yamane, Y. Nakamura: Dynamics filter - concept and implementation of online motion generator for human figures, IEEE Trans. Robotics Autom. 19(3), 421–432 (2003)CrossRefGoogle Scholar
  66. 74.66
    S. Vijayakumar, S. Schaal: Locally weighted projection regression: An O(n) algorithm for incremental real time learning in high dimensional spaces, Proc. Int. Conf. Mach. Learn. (ICML) (2000) pp. 288–293Google Scholar
  67. 74.67
    S. Vijayakumar, A. D'souza, S. Schaal: Incremental online learning in high dimensions, Neural Comput. 17(12), 2602–2634 (2005)MathSciNetCrossRefGoogle Scholar
  68. 74.68
    N. Kambhatla: Local Models and Gaussian Mixture Models for Statistical Data Processing, PhD Thesis (Oregon Graduate Institute of Science and Technology, Portland 1996)Google Scholar
  69. 74.69
    K. Grochow, S.L. Martin, A. Hertzmann, Z. Popovic: Style-based inverse kinematics, Proc. ACM Int. Conf. Comput. Gr. Interact. Tech. (SIGGRAPH) (2004) pp. 522–531Google Scholar
  70. 74.70
    K.F. MacDorman, R. Chalodhorn, M. Asada: Periodic nonlinear principal component neural networks for humanoid motion segmentation, generalization, and generation, Proc. Int. Conf. Pattern Recogn. (ICPR) (2004) pp. 537–540Google Scholar
  71. 74.71
    M. Mühlig, M. Gienger, J.J. Steil, C. Goerick: Automatic selection of task spaces for imitation learning, IEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS) (2009) pp. 4996–5002Google Scholar
  72. 74.72
    R. Jäkel, P. Meißner, S. Schmidt-Rohr, R. Dillmann: Distributed generalization of learned planning models in robot programming by demonstration, IEEE/RSJ Int. Conf. Intell. Robot. Syst. (2011)Google Scholar
  73. 74.73
    A. Gams, M. Do, A. Ude, T. Asfour, R. Dillmann: On-line periodic movement and force-profile learning for adaptation to new surfaces, Proc. IEEE-RAS Int. Conf. Human. Robot. (2010) pp. 560–565Google Scholar
  74. 74.74
    P. Kormushev, S. Calinon, D. Caldwell: Imitation learning of positional and force skills demonstrated via kinesthetic teaching and haptic input, Adv. Robotics 25(5), 581–603 (2011)CrossRefGoogle Scholar
  75. 74.75
    L. Rozo, S. Calinon, D.G. Caldwell, P. Jimenez, C. Torras: Learning collaborative impedance-based robot behaviors, Proc. AAAI Conf. Artif. Intell., Bellevue (2013) pp. 1422–1428Google Scholar
  76. 74.76
    L. Peternel, T. Petric, E. Oztop, J. Babic: Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach, Auton. Robots 36(1/2), 123–136 (2014)CrossRefGoogle Scholar
  77. 74.77
    C. Daniel, G. Neumann, J. Peters: Learning concurrent motor skills in versatile solution spaces, Proc. IEEE Int. Conf. Robotics Intell. Syst. (IROS'2012) (2012) pp. 3591–3597Google Scholar
  78. 74.78
    O. Mangin, P.-Y. Oudeyer: Unsupervised learning of simultaneous motor primitives through imitation, IEEE Int. Conf. Dev. Learn. (2011)Google Scholar
  79. 74.79
    R. Dillmann: Teaching and learning of robot tasks via observation of human performance, Robotics Auton. Syst. 47(2/3), 109–116 (2004)CrossRefGoogle Scholar
  80. 74.80
    A. Skoglund, B. Iliev, B. Kadmiry, R. Palm: Programming by demonstration of pick-and-place tasks for industrial manipulators using task primitives, Int. Symp. Comput. Intell. Robotics Autom. (2007)Google Scholar
  81. 74.81
    K. Muelling, J. Kober, O. Kroemer, J. Peters: Learning to select and generalize striking movements in robot table tennis, Int. J. Robotics Res. 32(3), 280–298 (2013)CrossRefGoogle Scholar
  82. 74.82
    D. Kulic, C. Ott, C. Lee, J. Ishikawa, Y. Nakamura: Incremental learning of full body motion primitives and their sequencing through human motion observation, Int. J. Robotics Res. 31(3), 330–345 (2012)CrossRefGoogle Scholar
  83. 74.83
    S. Niekum, G. Osentoski, A.G. Konidaris, A. Barto: Learning and generalization of complex tasks from unstructured demonstrations, IEEE Int. Conf. Intell. Robotics Syst. (2012) pp. 5239–5246Google Scholar
  84. 74.84
    P. Gaussier, S. Moga, J.P. Banquet, M. Quoy: From perception-action loop to imitation processes: A bottom-up approach of learning by imitation, Appl. Artif. Intell. 7(1), 701–729 (1998)CrossRefGoogle Scholar
  85. 74.85
    M.N. Nicolescu, M.J. Mataric: Natural methods for robot task learning: Instructive demonstrations, generalization and practice, Proc. Int. Jt. Conf. Auton. Agents Multiagent Syst. (AAMAS) (2003) pp. 241–248CrossRefGoogle Scholar
  86. 74.86
    J. Tani, M. Ito: Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment, IEEE Trans. Syst. Man Cybern. A 33(4), 481–488 (2003)CrossRefGoogle Scholar
  87. 74.87
    H. Friedrich, S. Muench, R. Dillmann, S. Bocionek, M. Sassin: Robot programming by demonstration (RPD): Supporting the induction by human interaction, Mach. Learn. 23(2), 163–189 (1996)Google Scholar
  88. 74.88
    M. Pardowitz, R. Zoellner, S. Knoop, R. Dillmann: Incremental learning of tasks from user demonstrations, past experiences and vocal comments, IEEE Trans. Syst. Man Cybern. B 37(2), 322–332 (2007)CrossRefGoogle Scholar
  89. 74.89
    S. Ekvall, D. Kragic: Learning task models from multiple human demonstrations, Proc. IEEE Int. Symp. Robot Human Int. Commun. (RO-MAN) (2006) pp. 358–363Google Scholar
  90. 74.90
    J. Saunders, C.L. Nehaniv, K. Dautenhahn: Teaching robots by moulding behavior and scaffolding the environment, Proc. ACM SIGCHI/SIGART Conf. Human-Robot Interaction (HRI) (2006) pp. 118–125CrossRefGoogle Scholar
  91. 74.91
    A. Alissandrakis, C.L. Nehaniv, K. Dautenhahn: Correspondence mapping induced state and action metrics for robotic imitation, IEEE Trans. Syst. Man Cybern. B 37(2), 299–307 (2007)CrossRefGoogle Scholar
  92. 74.92
    J. Rainer, R. Sven, S. Schmidt-Rohr, W. Rühl, K. Alexander, X. Zhixing, R. Dillmann: Learning of planning models for dexterous manipulation based on human demonstrations, Int. J. Soc. Robotics 4(4), 437–448 (2012)CrossRefGoogle Scholar
  93. 74.93
    S.R. Schmidt-Rohr, M. Lösch, R. Jäkel, R. Dillmann: Programming by demonstration of probabilistic decision making on a multi-modal service robot, Proc. 2010 IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS) (2010)Google Scholar
  94. 74.94
    C. Breazeal, M. Berlin, A. Brooks, J. Gray, A.L. Thomaz: Using perspective taking to learn from ambiguous demonstrations, Robotics Auton. Syst. 54, 385–393 (2006)CrossRefGoogle Scholar
  95. 74.95
    Y. Sato, K. Bernardin, H. Kimura, K. Ikeuchi: Task analysis based on observing hands and objects by vision, IEEE/RSJ Int. Conf. Intell. Robots Syst. Lausanne (2002) pp. 1208–1213CrossRefGoogle Scholar
  96. 74.96
    R. Zoellner, M. Pardowitz, S. Knoop, R. Dillmann: Towards cognitive robots: Building hierarchical task representations of manipulations from human demonstration, Int. Conf. Robotics Autom. (ICRA) Barcelona (2005)Google Scholar
  97. 74.97
    M. Pardowitz, R. Zöllner, R. Dillmann: Incremental learning of task sequences with information-theoretic metrics, Proc. Eur. Robotics Symp. (EUROS06) (2005)Google Scholar
  98. 74.98
    M. Pardowitz, R. Zöllner, R. Dillmann: Learning sequential constraints of tasks from user demonstrations, Proc. IEEE-RAS Int. Conf. Humanoid Robots (HUMANOIDS05) (2005) pp. 424–429CrossRefGoogle Scholar
  99. 74.99
    S. Calinon, P. Kormushev, D.G. Caldwell: Compliant skills acquisition and multi-optima policy search with EM-based reinforcement learning, Robotics Auton. Syst. 61(4), 369–379 (2013)CrossRefGoogle Scholar
  100. 74.100
    C.G. Atkeson, A.W. Moore, S. Schaal: Locally weighted learning for control, Artif. Intell. Rev. 11(1–5), 75–113 (1997)CrossRefGoogle Scholar
  101. 74.101
    J. Peters, S. Vijayakumar, S. Schaal: Reinforcement learning for humanoid robotics, Proc. IEEE Int. Conf. Humanoid Robots (Humanoids) (2003)Google Scholar
  102. 74.102
    T. Yoshikai, N. Otake, I. Mizuuchi, M. Inaba, H. Inoue: Development of an imitation behavior in humanoid kenta with reinforcement learning algorithm based on the attention during imitation, Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS) (2004) pp. 1192–1197Google Scholar
  103. 74.103
    D.C. Bentivegna, C.G. Atkeson, G. Cheng: Learning tasks from observation and practice, Robotics Auton. Syst. 47(2/3), 163–169 (2004)CrossRefGoogle Scholar
  104. 74.104
    P. Kormushev, S. Calinon, R. Saegusa, G. Metta: Learning the skill of archery by a humanoid robot iCub, Proc. IEEE Int. Conf. Human. Robots Nashville (2010)Google Scholar
  105. 74.105
    P. Pastor, M. Kalakrishnan, S. Chitta, E. Theodorou, S. Schaal: Skill learning and task outcome prediction for manipulation, IEEE Int. Conf. Robotics Autom. (2011)Google Scholar
  106. 74.106
    J. Kober, J. Peters: Policy search for motor primitives in robotics, Mach. Learn. 84(1/2), 171–203 (2011)MathSciNetMATHCrossRefGoogle Scholar
  107. 74.107
    P. Kormushev, S. Calinon, D.G. Caldwell: Robot motor skill coordination with EM-based reinforcement learning, Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS) Taipei (2010) pp. 3232–3237Google Scholar
  108. 74.108
    N. Jetchev, M. Toussaint: Fast motion planning from experience: Trajectory prediction for speeding up movement generation, Auton. Robots 34(1/2), 111–127 (2013)CrossRefGoogle Scholar
  109. 74.109
    F. Guenter, M. Hersch, S. Calinon, A. Billard: Reinforcement learning for imitating constrained reaching movements, RSJ Adv. Robotics 21(13), 1521–1544 (2007)Google Scholar
  110. 74.110
    B.D. Ziebart, A. Mass, A. Bagnell, A.K. Dey: Maximum entropy inverse reinforcement learning, Proc. AAAI Conf. Artif. Intell. (2008)Google Scholar
  111. 74.111
    P. Abbeel, A. Coates, A. Ng: Autonomous helicopter aerobatics through apprenticeship learning, Int. J. Robotics Res. 29(13), 1608–1639 (2010)CrossRefGoogle Scholar
  112. 74.112
    S. Ross, G. Gordon, J.A. Bagnell: A reduction of imitation learning and structured prediction to no-regret online learning, Proc. 14th Int. Conf. Artif. Intell. Stat. (AISTATS11) (2011)Google Scholar
  113. 74.113
    Y.K. Hwang, K.J. Choi, D.S. Hong: Self-learning control of cooperative motion for a humanoid robot, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) (2006) pp. 475–480Google Scholar
  114. 74.114
    B. Jansen, T. Belpaeme: A computational model of intention reading in imitation, Robotics Auton. Syst. 54(5), 394–402 (2006)CrossRefGoogle Scholar
  115. 74.115
    A. Billard, K. Dautenhahn: Grounding communication in autonomous robots: An experimental study, Robotics Auton. Syst. 24(1/2), 71–81 (1998)CrossRefGoogle Scholar
  116. 74.116
    P. Abbeel, A. Ng: Apprenticeship learning via inverse reinforcement learning, Int. Conf. Mach. Learn. (2004)Google Scholar
  117. 74.117
    N. Ratliff, A.J. Bagnell, M. Zinkevich: Maximum margin planning, Int. Conf. Mach. Learn. (2006)Google Scholar
  118. 74.118
    A. Billard, S. Calinon, F. Guenter: Discriminative and adaptive imitation in uni-manual and bi-manual tasks, Robotics Auton. Syst. 54, 370–384 (2006)CrossRefGoogle Scholar
  119. 74.119
    J. Choi, K. Kim: Nonparametric Bayesian inverse reinforcement learning for multiple reward functions, Adv. Neural Inf. Process. Syst. 25, 305–313 (2012)Google Scholar
  120. 74.120
    A.K. Tanwani, A. Billard: Transfer in inverse reinforcement learning for multiple strategies, IEEE/RSJ Int. Conf. Intell. Robots Syst. (2013)Google Scholar
  121. 74.121
    D.H. Grollman, A. Billard: Donut as i do: Learning from failed demonstrations, IEEE Int. Conf. Robotics Autom. (2011)Google Scholar
  122. 74.122
    A. Rai, G. de Chambrier, A. Billard: Learning from failed demonstrations in unreliable systems, IEEE-RAS Int. Conf. Humanoid Robots (2013)Google Scholar
  123. 74.123
    M. Goodrich, A. Schultz: Human-robot interaction: A survey, Found. Trend. Human-Comput. Int. 1(3), 203–275 (2007)MATHCrossRefGoogle Scholar
  124. 74.124
    T. Fong, I. Nourbakhsh, K. Dautenhahn: A survey of socially interactive robots, Robotics Auton. Syst. 42(3/4), 143–166 (2003)MATHCrossRefGoogle Scholar
  125. 74.125
    C. Breazeal, B. Scassellati: Robots that imitate humans, Trends Cogn. Sci. 6(11), 481–487 (2002)CrossRefGoogle Scholar
  126. 74.126
    B. Scassellati: Imitation and mechanisms of joint attention: A developmental structure for building social skills on a humanoid robot, Lect. Notes Comput. Sci. 1562, 176–195 (1999)CrossRefGoogle Scholar
  127. 74.127
    H. Kozima, H. Yano: A robot that learns to communicate with human caregivers, Int. Workshop Epigenet. Robotics (2001)Google Scholar
  128. 74.128
    H. Ishiguro, T. Ono, M. Imai, T. Kanda: Development of an interactive humanoid robot Robovie -- An interdisciplinary approach, Springer Tracts Adv. Robotics 6, 179–192 (2003)CrossRefGoogle Scholar
  129. 74.129
    K. Nickel, R. Stiefelhagen: Pointing gesture recognition based on 3d-tracking of face, hands and head orientation, Int. Conf. Multimodal Interfaces (ICMI) (2003) pp. 140–146Google Scholar
  130. 74.130
    M. Ito, J. Tani: Joint attention between a humanoid robot and users in imitation game, Int. Conf. Dev. Learn. (ICDL) (2004)Google Scholar
  131. 74.131
    V.V. Hafner, F. Kaplan: Learning to interpret pointing gestures: Experiments with four-legged autonomous robots, Lect. Notes Comput. Sci. 3575, 225–234 (2005)CrossRefGoogle Scholar
  132. 74.132
    C. Breazeal, D. Buchsbaum, J. Gray, D. Gatenby, B. Blumberg: Learning from and about others: Towards using imitation to bootstrap the social understanding of others by robots, Artif. Life 11(1/2), 31–62 (2005)CrossRefGoogle Scholar
  133. 74.133
    P.F. Dominey, M. Alvarez, B. Gao, M. Jeambrun, A. Cheylus, A. Weitzenfeld, A. Martinez, A. Medrano: Robot command, interrogation and teaching via social interaction, Proc. IEEE-RAS Int. Conf. Humanoid Robots (Humanoids) (2005) pp. 475–480Google Scholar
  134. 74.134
    A.L. Thomaz, M. Berlin, C. Breazeal: Robot science meets social science: An embodied computational model of social referencing, Workshop Toward Soc. Mech. Android Sci. (CogSci) (2005) pp. 7–17Google Scholar
  135. 74.135
    C. Breazeal, L. Aryananda: Recognition of affective communicative intent in robot-directed speech, Auton. Robots 12(1), 83–104 (2002)MATHCrossRefGoogle Scholar
  136. 74.136
    S. Calinon, A. Billard: Teaching a humanoid robot to recognize and reproduce social cues, Proc. IEEE Int. Symp. Robot Human Int. Commun. (RO-MAN) (2006) pp. 346–351Google Scholar
  137. 74.137
    Y. Yoshikawa, K. Shinozawa, H. Ishiguro, N. Hagita, T. Miyamoto: Responsive robot gaze to interaction partner, Proc. Robotics Sci. Syst. (RSS) Philadelphia (2006)Google Scholar
  138. 74.138
    S. Calinon, A. Billard: What is the teacher's role in robot programming by demonstration? - Toward benchmarks for improved learning, Int. Stud. Spec. Issue Psychol, Benchmarks Human-Robot Int. 8(3), 441–464 (2007)CrossRefGoogle Scholar
  139. 74.139
    S. Chernova, M. Veloso: Interactive policy learning through confidence-based autonomy, J. Artif. Intell. Res. 34, 1–25 (2009)MathSciNetMATHGoogle Scholar
  140. 74.140
    B.D. Argall, E.L. Sauser: Tactile guidance for policy adaptation, Found. Trend. Robotics 1(2), 79–133 (2010)CrossRefGoogle Scholar
  141. 74.141
    S. Calinon, A. Billard: Active teaching in robot programming by demonstration, Proc. IEEE Int. Symp. Robot Human Int. Commun. (RO-MAN), Jeju (2007) pp. 702–707Google Scholar
  142. 74.142
    D. Silver, A. Bagnell, A. Stentz: Active learning from demonstration for robust autonomous navigation, IEEE Conf. Robot. Autom. ICRA'12 (2012)Google Scholar
  143. 74.143
    M. Riley, A. Ude, C. Atkeson, G. Cheng: Coaching: An approach to efficiently and intuitively create humanoid robot behaviors, Proc. IEEE-RAS Int. Conf. Humanoid Robots (Humanoids) (2006) pp. 567–574Google Scholar
  144. 74.144
    H. Bekkering, A. Wohlschlaeger, M. Gattis: Imitation of gestures in children is goal-directed, Q. J. Exp. Psychol. 53A(1), 153–164 (2000)CrossRefGoogle Scholar
  145. 74.145
    M. Nicolescu, M.J. Mataric: Task learning through imitation and human-robot interaction. In: Models and Mechanisms of Imitation and Social Learning in Robots, Humans and Animals, (MIT Press, Cambridge 2006) pp. 407–424Google Scholar
  146. 74.146
    J. Demiris, G. Hayes: Imitative learning mechanisms in robots and humans, 5th Eur. Workshop Learn. Robots, ed. by V. Klingspor (1996) pp. 9–16Google Scholar
  147. 74.147
    J. Aleotti, S. Caselli: Robust trajectory learning and approximation for robot programming by demonstration, Robotics Auton. Syst. 54(5), 409–413 (2006)CrossRefGoogle Scholar
  148. 74.148
    M. Ogino, H. Toichi, Y. Yoshikawa, M. Asada: Interaction rule learning with a human partner based on an imitation faculty with a simple visuo-motor mapping, Robotics Auton. Syst. 54, 414–418 (2006)CrossRefGoogle Scholar
  149. 74.149
    A. Billard, M. Matarić: Learning human arm movements by imitation: Evaluation of a biologically-inspired connectionist architecture, Robotics Auton. Syst. 941, 1–16 (2001)MATHGoogle Scholar
  150. 74.150
    A.J. Ijspeert, J. Nakanishi, S. Schaal: Movement imitation with nonlinear dynamical systems in humanoid robots, IEEE Int. Conf. Robotics Autom. (ICRA2002) (2002) pp. 1398–1403Google Scholar
  151. 74.151
    R.H. Cuijpers, H.T. van Schie, M. Koppen, W. Erlhagen, H. Bekkering: Goals and means in action observation: A computational approach, Neural Netw. 19(3), 311–322 (2006)MATHCrossRefGoogle Scholar
  152. 74.152
    M.W. Hoffman, D.B. Grimes, A.P. Shon, R.P.N. Rao: A probabilistic model of gaze imitation and shared attention, Neural Netw. 19(3), 299–310 (2006)MATHCrossRefGoogle Scholar
  153. 74.153
    A. Billard: Drama, a connectionist architecture for on-line learning and control of autonomous robots: Experiments on learning of a synthetic proto-language with a doll robot, Ind. Robot 26(1), 59–66 (1999)CrossRefGoogle Scholar
  154. 74.154
    P. Gaussier, S. Moga, J.P. Banquet, J. Nadel: Learning and communication via imitation: An autonomous robot perspective systems, IEEE Trans. Man Cybern. A 31(5), 431–442 (2001)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Aude G. Billard
    • 1
  • Sylvain Calinon
    • 2
  • Rüdiger Dillmann
    • 3
  1. 1.School of EngineeringSwiss Federal Institute of Technology (EPFL)LausanneSwitzerland
  2. 2.Idiap Research InstituteMartignySwitzerland
  3. 3.Institute for Technical InformaticsKarlsruhe Institute of TechnologyKarlsruheGermany

Personalised recommendations