Skip to main content
SpringerLink
Log in

Learning from Humans

  • Chapter
  • First Online:
Springer Handbook of Robotics

Part of the book series: Springer Handbooks ((SHB))

Abstract

This chapter surveys the main approaches developed to date to endow robots with the ability to learn from human guidance. The field is best known as robot programming by demonstration, robot learning from/by demonstration, apprenticeship learning and imitation learning. We start with a brief historical overview of the field. We then summarize the various approaches taken to solve four main questions: when, what, who and when to imitate. We emphasize the importance of choosing well the interface and the channels used to convey the demonstrations, with an eye on interfaces providing force control and force feedback. We then review algorithmic approaches to model skills individually and as a compound and algorithms that combine learning from human guidance with reinforcement learning. We close with a look on the use of language to guide teaching and a list of open issues.

figure a

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 269.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 349.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abbreviations

2-D:

two-dimensional

ANN:

artificial neural network

EO:

elementary operator

HMM:

hidden Markov model

HRI:

human–robot interaction

IRL:

inverse reinforcement learning

LfD:

learning from demonstration

learning from human demonstration

ML:

machine learning

PbD:

programming by demonstration

POMDP:

partially observable Markov decision process

RBF:

radial basis function network

RL:

reinforcement learning

References

  1. T. Lozano-Perez: Robot programming, Proceedings IEEE 71(7), 821–841 (1983)

    Article  Google Scholar 

  2. B. Dufay, J.-C. Latombe: An approach to automatic robot programming based on inductive learning, Int. J. Robotics Res. 3(4), 3–20 (1984)

    Article  Google Scholar 

  3. A. Levas, M. Selfridge: A user-friendly high-level robot teaching system, IEEE Int. Conf. Robotics, Altanta (1984) pp. 413–416

    Google Scholar 

  4. A.B. Segre, G. DeJong: Explanation-based manipulator learning: Acquisition of planning ability through observation, IEEE Conf. Robotics Autom. St. Louis (1985) pp. 555–560

    Google Scholar 

  5. A.M. Segre: Machine Learning of Robot Assembly Plans (Kluwer, Boston 1988)

    Book  Google Scholar 

  6. S. Muench, J. Kreuziger, M. Kaiser, R. Dillmann: Robot programming by demonstration (RPD) - Using machine learning and user interaction methods for the development of easy and comfortable robot programming systems, Proc. Int. Symp. Indus. Robots (ISIR) (1994) pp. 685–693

    Google Scholar 

  7. A. Billard: Imitation: A review. In: The Handbook of Brain Theory and Neural Network, 2nd edn., ed. by M.A. Arbib (MIT Press, Cambridge 2002) pp. 566–569

    Google Scholar 

  8. E. Oztop, M. Kawato, M.A. Arbib: Mirror neurons and imitation: A computationally guided review, Neural Netw. 19(3), 254–271 (2006)

    Article  MATH  Google Scholar 

  9. J. Demiris, G. Hayes: Imitation as a dual-route process featuring predictive and learning components: A biologically-plausible computational model. In: Imitation in Animals and Artifacs, ed. by C. Nehaniv, K. Dautenhahn (MIT Press, Cambridge 2002)

    Google Scholar 

  10. J. Nadel, A. Revel, P. Andry, P. Gaussier: Toward communication: First imitations in infants, low-functioning children with autism and robots, Interact. Stud. 5(1), 45–74 (2004)

    Article  Google Scholar 

  11. F. Kaplan, P.-Y. Oudeyer: The progress-drive hypothesis: An interpretation of early imitation. In: Models and Mechanisms of Imitation and Social Learning: Behavioural, Social and Communication Dimensions, ed. by K. Dautenhahn, C. Nehaniv (Cambridge Univ. Press, Cambridge 2007) pp. 361–377

    Google Scholar 

  12. B.D. Argall, M. Veloso, B. Browning: Teacher feedback to scaffold and refine demonstrated motion primitives on a mobile robot, Robotics Auton. Syst. 59(3/4), 243–255 (2011)

    Article  Google Scholar 

  13. B. Robins, K. Dautenhahn, C.L. Nehaniv, N.A. Mirza, D. Francois, L. Olsson: Sustaining interaction dynamics and engagement in dyadic child-robot interaction kinesics: Lessons learnt from an exploratory study, IEEE Int. Workshop Robot Human Int. Commun. (ROMAN) (2005) pp. 716–722

    Google Scholar 

  14. M. Cakmak, A.L. Thomaz: Designing robot learners that ask good questions, IEEE-ACML Int. Conf. Human-Robot Int. (HRI) (2012)

    Google Scholar 

  15. C. Breazeal, A. Brooks, J. Gray, G. Hoffman, C. Kidd, H. Lee, J. Lieberman, A. Lockerd, D. Chilongo: Tutelage and collaboration for humanoid robots, Human. Robots 1(2), 315–348 (2004)

    Article  Google Scholar 

  16. Y. Kuniyoshi, M. Inaba, H. Inoue: Teaching by showing: Generating robot programs by visual observation of human performance, Proc. Int. Symp. Ind. Robots, Tokyo (1989) pp. 119–126

    Google Scholar 

  17. Y. Kuniyoshi, M. Inaba, H. Inoue: Learning by watching: Extracting reusable task knowledge from visual observation of human performance, IEEE Trans. Robotics Autom. 10(6), 799–822 (1994)

    Article  Google Scholar 

  18. M. Ehrenmann, O. Rogalla, R. Zöllner, R. Dillmann: Teaching service robots complex tasks: Programming by demonstation for workshop and household environments, Proc. IEEE Int. Conf. Field Serv. Robotics (FRS) (2001)

    Google Scholar 

  19. C.P. Tung, A.C. Kak: Automatic learning of assembly task using a dataglove system, IEEE/RSJ Int. Conf. Intell. Robots Syst., Pittsburgh (1995) pp. 1–8

    Google Scholar 

  20. K. Ikeuchi, T. Suchiro: Towards an assembly plan from observation, Part I: Assembly task recognition using face-contact relations (polyhedral objects), Proc. IEEE Int. Conf. Robot. Autom. (ICRA), Vol. 3 (1992) pp. 2171–2177

    Google Scholar 

  21. S. Calinon, F. Guenter, A. Billard: On learning, representing and generalizing a task in a humanoid robot, IEEE Trans. Syst. Man Cybern. B 37(2), 286–298 (2007)

    Article  Google Scholar 

  22. M. Ito, K. Noda, Y. Hoshino, J. Tani: Dynamic and interactive generation of object handling behaviors by a small humanoid robot using a dynamic neural network model, Neural Netw. 19(3), 323–337 (2006)

    Article  MATH  Google Scholar 

  23. T. Inamura, N. Kojo, M. Inaba: Situation recognition and behavior induction based on geometric symbol representation of multimodal sensorimotor patterns, IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS) (2006) pp. 5147–5152

    Google Scholar 

  24. S. Liu, H. Asada: Teaching and learning of deburring robots using neural networks, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) (1993) pp. 339–345

    Google Scholar 

  25. A. Billard: Learning motor skills by imitation: A biologically inspired robotic model, J. Cybern. Syst. 32(1/2), 155–193 (2001)

    Article  MATH  Google Scholar 

  26. M. Kaiser, R. Dillmann: Building elementary robot skills from human demonstration, Proc. IEEE Int. Conf. Robotics Autom. (ICRA), Vol. 3 (1996) pp. 2700–2705

    Chapter  Google Scholar 

  27. R. Dillmann, M. Kaiser, A. Ude: Acquisition of elementary robot skills from human demonstration, Int. Symp. Intell. Robotics Syst. (SIRS) (1995) pp. 1–38

    Google Scholar 

  28. W. Yang: Hidden Markov model approach to skill learning and its application in telerobotics, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) (1993) pp. 396–402

    Google Scholar 

  29. P.K. Pook, D.H. Ballard: Recognizing teleoperated manipulations, Proc. IEEE Int. Conf. Robotics Autom., Atlanta (1993) pp. 578–585

    Google Scholar 

  30. G.E. Hovland, P. Sikka, B.J. McCarragher: Skill acquisition from human demonstration using a hidden Markov model, Proc. IEEE Int. Conf. Robotics Autom., Minneapolis (1996) pp. 2706–2711

    Google Scholar 

  31. S.K. Tso, K.P. Liu: Hidden Markov model for intelligent extraction of robot trajectory command from demonstrated trajectories, Proc. IEEE Int. Conf. Ind. Technol. (ICIT) (1996) pp. 294–298

    Google Scholar 

  32. C. Lee, Y. Xu: Online, interactive learning of gestures for human/robot interfaces, Proc. IEEE Int. Conf. Robotics Autom. (ICRA), Vol. 4 (1996) pp. 2982–2987

    Chapter  Google Scholar 

  33. D. Kulic, W. Takano, Y. Nakamura: Incremental learning, clustering and hierarchy formation of whole body motion patterns using adaptive hidden Markov chains, Int. J. Robotics Res. 27(7), 761–784 (2008)

    Article  Google Scholar 

  34. D. Nguyen-Tuong, M. Seeger, J. Peters: Local Gaussian process regression for real time online model learning and control, Adv. Neural Inf. Process. Syst. 21, 1193–1200 (2009)

    Google Scholar 

  35. S.M. Khansari Zadeh, A. Billard: Learning stable non-linear dynamical systems with Gaussian mixture models, IEEE Trans. Robotics 27(5), 943–957 (2011)

    Article  Google Scholar 

  36. C. Nehaniv, K. Dautenhahn: Of hummingbirds and helicopters: An algebraic framework for interdisciplinary studies of imitation and its applications. In: Interdisciplinary Approaches to Robot Learning, Vol. 24, ed. by J. Demiris, A. Birk (World Scientific, Singapore 2000) pp. 136–161

    Chapter  Google Scholar 

  37. C.L. Nehaniv: Nine billion correspondence problems and some methods for solving them, Proc. Int. Symp. Imit. Anim. Artifacts (2003) pp. 93–95

    Google Scholar 

  38. P. Bakker, Y. Kuniyoshi: Robot see, robot do: An overview of robot imitation, AISB Workshop Learn. Robot. Anim., Brighton (1996)

    Google Scholar 

  39. M. Skubic, R.A. Volz: Acquiring robust, force-based assembly skills from human demonstration, IEEE Trans. Robotics Autom. 16(6), 772–781 (2000)

    Article  Google Scholar 

  40. M. Yeasin, S. Chaudhuri: Toward automatic robot programming: Learning human skill from visual data, IEEE Trans. Syst. Man Cybern. B 30(1), 180–185 (2000)

    Article  Google Scholar 

  41. J. Zhang, B. Rössler: Self-valuing learning and generalization with application in visually guided grasping of complex objects, Robotics Auton. Syst. 47(2/3), 117–127 (2004)

    Article  Google Scholar 

  42. M. Frank, M. Plaue, H. Rapp, U. Koethe, B. Jaehne, F.A. Hamprecht: Theoretical and experimental error analysis of continuous-wave time-of-flight range cameras, Opt. Eng. 48(1), 013602 (2009)

    Article  Google Scholar 

  43. M. Freese, S. Singh, F. Ozaki, N. Matsuhira: Virtual robot experimentation platform v-rep: A versatile 3d robot simulator, Proc. Int. Conf. Simul. Model. Progr. Auton. Robots (SIMPAR) (2010) pp. 51–62

    Google Scholar 

  44. S. Hak, N. Mansard, O. Ramos, L. Saab, O. Stasse: Capture, recognition and imitation of anthropomorphic motion, IEEE-RAS Int. Conf. Robotics Autom. (2012) pp. 3539–3540

    Google Scholar 

  45. G. Gioioso, G. Salvietti, M. Malvezzi, D. Prattichizzo: An object-based approach to map human hand synergies onto robotic hands with dissimilar kinematics. In: Robotics - Science and Systems VIII, ed. by N. Roy, P. Newman, S. Srinivasa (MIT Press, Cambridge 2012) pp. 97–105

    Google Scholar 

  46. A. Shon, K. Grochow, A. Hertzmann, R. Rao: Learning shared latent structure for image synthesis and robotic imitation, Adv. Neural Inf. Process. Syst. (NIPS) 18, 1233–1240 (2006)

    Google Scholar 

  47. A. Ude, C.G. Atkeson, M. Riley: Programming full-body movements for humanoid robots by observation, Robotics Auton. Syst. 47, 93–108 (2004)

    Article  Google Scholar 

  48. S. Kim, C. Kim, B. You, S. Oh: Stable whole-body motion generation for humanoid robots to imitate human motions, Proc. IEEE/RSJ Int. Conf. Intell. Robotics Syst. (IROS) (2009)

    Google Scholar 

  49. S. Nakaoka, A. Nakazawa, F. Kanehiro, K. Kaneko, M. Morisawa, H. Hirukawa, K. Ikeuchi: Learning from observation paradigm: Leg task models for enabling a biped humanoid robot to imitate human dances, Int. J. Robotics Res. 26(8), 829–844 (2007)

    Article  Google Scholar 

  50. E.L. Sauser, B.D. Argall, G. Metta, A.G. Billard: Iterative learning of grasp adaptation through human corrections, Robotics Auton. Syst. 60(1), 55–71 (2012)

    Article  Google Scholar 

  51. A. Coates, P. Abbeel, A.Y. Ng: Learning for control from multiple demonstrations, Proc. 25th Int. Conf. Mach. Learn. (2008)

    Google Scholar 

  52. D. Grollman, O.C. Jenkins: Incremental learning of subtasks from unsegmented demonstration, Int. Conf. Intell. Robots Syst. (2010)

    Google Scholar 

  53. L. Peternel, J. Babic: Humanoid robot posture-control learning in real-time based on human sensorimotor learning ability, IEEE Int. Conf. Robotics Autom. (ICRA) Karlsruhe (2013)

    Google Scholar 

  54. A. Ude: Robust estimation of human body kinematics from video, Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS) (1999) pp. 1489–1494

    Google Scholar 

  55. B. Akgun, M. Cakmak, K. Jiang, A.L. Thomaz: Keyframe-based learning from demonstration, Int. J. Soc. Robotics 4, 343–355 (2012)

    Article  Google Scholar 

  56. P. Evrard, E. Gribovskaya, S. Calinon, A. Billard, A. Kheddar: Teaching physical collaborative tasks: Object-lifting case study with a humanoid, Proc. IEEE-RAS Int. Conf. Humanoid Robots (Humanoids), Paris (2009) pp. 399–404

    Google Scholar 

  57. C. Chao, M. Cakmak, A.L. Thomaz: Designing interactions for robot active learners, IEEE Trans. Auton. Mental Dev. 2(2), 108–118 (2010)

    Article  Google Scholar 

  58. S. Calinon, A. Billard: PDA interface for humanoid robots, Proc. IEEE Int. Conf. Humanoid Robots (Humanoids) (2003)

    Google Scholar 

  59. A. Shon, K. Grochow, R. Rao: Robotic imitation from human motion capture using Gaussian processes, Proc. IEEE/RAS Int. Conf. Humanoid Robots (Humanoids) (2005)

    Google Scholar 

  60. Y. Wu, Y. Demiris: Towards one shot learning by imitation for humanoid robots, IEEE-RAS Int. Conf. Robotics Autom. (ICRA) (2010)

    Google Scholar 

  61. J. Nakanishi, J. Morimoto, G. Endo, G. Cheng, S. Schaal, M. Kawato: Learning from demonstration and adaptation of biped locomotion, Robotics Auton. Syst. 47(2/3), 79–91 (2004)

    Article  Google Scholar 

  62. D. Lee, C. Ott: Incremental kinesthetic teaching of motion primitives using the motion refinement tube, Auton. Robot. 31(2), 115–131 (2011)

    Article  Google Scholar 

  63. A. Ude: Trajectory generation from noisy positions of object features for teaching robot paths, Robotics Auton. Syst. 11(2), 113–127 (1993)

    Article  Google Scholar 

  64. J. Yang, Y. Xu, C.S. Chen: Human action learning via hidden Markov model, IEEE Trans. Syst. Man Cybern. A 27(1), 34–44 (1997)

    Article  Google Scholar 

  65. K. Yamane, Y. Nakamura: Dynamics filter - concept and implementation of online motion generator for human figures, IEEE Trans. Robotics Autom. 19(3), 421–432 (2003)

    Article  Google Scholar 

  66. S. Vijayakumar, S. Schaal: Locally weighted projection regression: An O(n) algorithm for incremental real time learning in high dimensional spaces, Proc. Int. Conf. Mach. Learn. (ICML) (2000) pp. 288–293

    Google Scholar 

  67. S. Vijayakumar, A. D'souza, S. Schaal: Incremental online learning in high dimensions, Neural Comput. 17(12), 2602–2634 (2005)

    Article  MathSciNet  Google Scholar 

  68. N. Kambhatla: Local Models and Gaussian Mixture Models for Statistical Data Processing, PhD Thesis (Oregon Graduate Institute of Science and Technology, Portland 1996)

    Google Scholar 

  69. K. Grochow, S.L. Martin, A. Hertzmann, Z. Popovic: Style-based inverse kinematics, Proc. ACM Int. Conf. Comput. Gr. Interact. Tech. (SIGGRAPH) (2004) pp. 522–531

    Google Scholar 

  70. K.F. MacDorman, R. Chalodhorn, M. Asada: Periodic nonlinear principal component neural networks for humanoid motion segmentation, generalization, and generation, Proc. Int. Conf. Pattern Recogn. (ICPR) (2004) pp. 537–540

    Google Scholar 

  71. M. Mühlig, M. Gienger, J.J. Steil, C. Goerick: Automatic selection of task spaces for imitation learning, IEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS) (2009) pp. 4996–5002

    Google Scholar 

  72. R. Jäkel, P. Meißner, S. Schmidt-Rohr, R. Dillmann: Distributed generalization of learned planning models in robot programming by demonstration, IEEE/RSJ Int. Conf. Intell. Robot. Syst. (2011)

    Google Scholar 

  73. A. Gams, M. Do, A. Ude, T. Asfour, R. Dillmann: On-line periodic movement and force-profile learning for adaptation to new surfaces, Proc. IEEE-RAS Int. Conf. Human. Robot. (2010) pp. 560–565

    Google Scholar 

  74. P. Kormushev, S. Calinon, D. Caldwell: Imitation learning of positional and force skills demonstrated via kinesthetic teaching and haptic input, Adv. Robotics 25(5), 581–603 (2011)

    Article  Google Scholar 

  75. L. Rozo, S. Calinon, D.G. Caldwell, P. Jimenez, C. Torras: Learning collaborative impedance-based robot behaviors, Proc. AAAI Conf. Artif. Intell., Bellevue (2013) pp. 1422–1428

    Google Scholar 

  76. L. Peternel, T. Petric, E. Oztop, J. Babic: Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach, Auton. Robots 36(1/2), 123–136 (2014)

    Article  Google Scholar 

  77. C. Daniel, G. Neumann, J. Peters: Learning concurrent motor skills in versatile solution spaces, Proc. IEEE Int. Conf. Robotics Intell. Syst. (IROS'2012) (2012) pp. 3591–3597

    Google Scholar 

  78. O. Mangin, P.-Y. Oudeyer: Unsupervised learning of simultaneous motor primitives through imitation, IEEE Int. Conf. Dev. Learn. (2011)

    Google Scholar 

  79. R. Dillmann: Teaching and learning of robot tasks via observation of human performance, Robotics Auton. Syst. 47(2/3), 109–116 (2004)

    Article  Google Scholar 

  80. A. Skoglund, B. Iliev, B. Kadmiry, R. Palm: Programming by demonstration of pick-and-place tasks for industrial manipulators using task primitives, Int. Symp. Comput. Intell. Robotics Autom. (2007)

    Google Scholar 

  81. K. Muelling, J. Kober, O. Kroemer, J. Peters: Learning to select and generalize striking movements in robot table tennis, Int. J. Robotics Res. 32(3), 280–298 (2013)

    Article  Google Scholar 

  82. D. Kulic, C. Ott, C. Lee, J. Ishikawa, Y. Nakamura: Incremental learning of full body motion primitives and their sequencing through human motion observation, Int. J. Robotics Res. 31(3), 330–345 (2012)

    Article  Google Scholar 

  83. S. Niekum, G. Osentoski, A.G. Konidaris, A. Barto: Learning and generalization of complex tasks from unstructured demonstrations, IEEE Int. Conf. Intell. Robotics Syst. (2012) pp. 5239–5246

    Google Scholar 

  84. P. Gaussier, S. Moga, J.P. Banquet, M. Quoy: From perception-action loop to imitation processes: A bottom-up approach of learning by imitation, Appl. Artif. Intell. 7(1), 701–729 (1998)

    Article  Google Scholar 

  85. M.N. Nicolescu, M.J. Mataric: Natural methods for robot task learning: Instructive demonstrations, generalization and practice, Proc. Int. Jt. Conf. Auton. Agents Multiagent Syst. (AAMAS) (2003) pp. 241–248

    Chapter  Google Scholar 

  86. J. Tani, M. Ito: Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment, IEEE Trans. Syst. Man Cybern. A 33(4), 481–488 (2003)

    Article  Google Scholar 

  87. H. Friedrich, S. Muench, R. Dillmann, S. Bocionek, M. Sassin: Robot programming by demonstration (RPD): Supporting the induction by human interaction, Mach. Learn. 23(2), 163–189 (1996)

    Google Scholar 

  88. M. Pardowitz, R. Zoellner, S. Knoop, R. Dillmann: Incremental learning of tasks from user demonstrations, past experiences and vocal comments, IEEE Trans. Syst. Man Cybern. B 37(2), 322–332 (2007)

    Article  Google Scholar 

  89. S. Ekvall, D. Kragic: Learning task models from multiple human demonstrations, Proc. IEEE Int. Symp. Robot Human Int. Commun. (RO-MAN) (2006) pp. 358–363

    Google Scholar 

  90. J. Saunders, C.L. Nehaniv, K. Dautenhahn: Teaching robots by moulding behavior and scaffolding the environment, Proc. ACM SIGCHI/SIGART Conf. Human-Robot Interaction (HRI) (2006) pp. 118–125

    Chapter  Google Scholar 

  91. A. Alissandrakis, C.L. Nehaniv, K. Dautenhahn: Correspondence mapping induced state and action metrics for robotic imitation, IEEE Trans. Syst. Man Cybern. B 37(2), 299–307 (2007)

    Article  Google Scholar 

  92. J. Rainer, R. Sven, S. Schmidt-Rohr, W. Rühl, K. Alexander, X. Zhixing, R. Dillmann: Learning of planning models for dexterous manipulation based on human demonstrations, Int. J. Soc. Robotics 4(4), 437–448 (2012)

    Article  Google Scholar 

  93. S.R. Schmidt-Rohr, M. Lösch, R. Jäkel, R. Dillmann: Programming by demonstration of probabilistic decision making on a multi-modal service robot, Proc. 2010 IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS) (2010)

    Google Scholar 

  94. C. Breazeal, M. Berlin, A. Brooks, J. Gray, A.L. Thomaz: Using perspective taking to learn from ambiguous demonstrations, Robotics Auton. Syst. 54, 385–393 (2006)

    Article  Google Scholar 

  95. Y. Sato, K. Bernardin, H. Kimura, K. Ikeuchi: Task analysis based on observing hands and objects by vision, IEEE/RSJ Int. Conf. Intell. Robots Syst. Lausanne (2002) pp. 1208–1213

    Google Scholar 

  96. R. Zoellner, M. Pardowitz, S. Knoop, R. Dillmann: Towards cognitive robots: Building hierarchical task representations of manipulations from human demonstration, Int. Conf. Robotics Autom. (ICRA) Barcelona (2005)

    Google Scholar 

  97. M. Pardowitz, R. Zöllner, R. Dillmann: Incremental learning of task sequences with information-theoretic metrics, Proc. Eur. Robotics Symp. (EUROS06) (2005)

    Google Scholar 

  98. M. Pardowitz, R. Zöllner, R. Dillmann: Learning sequential constraints of tasks from user demonstrations, Proc. IEEE-RAS Int. Conf. Humanoid Robots (HUMANOIDS05) (2005) pp. 424–429

    Chapter  Google Scholar 

  99. S. Calinon, P. Kormushev, D.G. Caldwell: Compliant skills acquisition and multi-optima policy search with EM-based reinforcement learning, Robotics Auton. Syst. 61(4), 369–379 (2013)

    Article  Google Scholar 

  100. C.G. Atkeson, A.W. Moore, S. Schaal: Locally weighted learning for control, Artif. Intell. Rev. 11(1–5), 75–113 (1997)

    Article  Google Scholar 

  101. J. Peters, S. Vijayakumar, S. Schaal: Reinforcement learning for humanoid robotics, Proc. IEEE Int. Conf. Humanoid Robots (Humanoids) (2003)

    Google Scholar 

  102. T. Yoshikai, N. Otake, I. Mizuuchi, M. Inaba, H. Inoue: Development of an imitation behavior in humanoid kenta with reinforcement learning algorithm based on the attention during imitation, Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS) (2004) pp. 1192–1197

    Google Scholar 

  103. D.C. Bentivegna, C.G. Atkeson, G. Cheng: Learning tasks from observation and practice, Robotics Auton. Syst. 47(2/3), 163–169 (2004)

    Article  Google Scholar 

  104. P. Kormushev, S. Calinon, R. Saegusa, G. Metta: Learning the skill of archery by a humanoid robot iCub, Proc. IEEE Int. Conf. Human. Robots Nashville (2010)

    Google Scholar 

  105. P. Pastor, M. Kalakrishnan, S. Chitta, E. Theodorou, S. Schaal: Skill learning and task outcome prediction for manipulation, IEEE Int. Conf. Robotics Autom. (2011)

    Google Scholar 

  106. J. Kober, J. Peters: Policy search for motor primitives in robotics, Mach. Learn. 84(1/2), 171–203 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  107. P. Kormushev, S. Calinon, D.G. Caldwell: Robot motor skill coordination with EM-based reinforcement learning, Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS) Taipei (2010) pp. 3232–3237

    Google Scholar 

  108. N. Jetchev, M. Toussaint: Fast motion planning from experience: Trajectory prediction for speeding up movement generation, Auton. Robots 34(1/2), 111–127 (2013)

    Article  Google Scholar 

  109. F. Guenter, M. Hersch, S. Calinon, A. Billard: Reinforcement learning for imitating constrained reaching movements, RSJ Adv. Robotics 21(13), 1521–1544 (2007)

    Article  Google Scholar 

  110. B.D. Ziebart, A. Mass, A. Bagnell, A.K. Dey: Maximum entropy inverse reinforcement learning, Proc. AAAI Conf. Artif. Intell. (2008)

    Google Scholar 

  111. P. Abbeel, A. Coates, A. Ng: Autonomous helicopter aerobatics through apprenticeship learning, Int. J. Robotics Res. 29(13), 1608–1639 (2010)

    Article  Google Scholar 

  112. S. Ross, G. Gordon, J.A. Bagnell: A reduction of imitation learning and structured prediction to no-regret online learning, Proc. 14th Int. Conf. Artif. Intell. Stat. (AISTATS11) (2011)

    Google Scholar 

  113. Y.K. Hwang, K.J. Choi, D.S. Hong: Self-learning control of cooperative motion for a humanoid robot, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) (2006) pp. 475–480

    Google Scholar 

  114. B. Jansen, T. Belpaeme: A computational model of intention reading in imitation, Robotics Auton. Syst. 54(5), 394–402 (2006)

    Article  Google Scholar 

  115. A. Billard, K. Dautenhahn: Grounding communication in autonomous robots: An experimental study, Robotics Auton. Syst. 24(1/2), 71–81 (1998)

    Article  Google Scholar 

  116. P. Abbeel, A. Ng: Apprenticeship learning via inverse reinforcement learning, Int. Conf. Mach. Learn. (2004)

    Google Scholar 

  117. N. Ratliff, A.J. Bagnell, M. Zinkevich: Maximum margin planning, Int. Conf. Mach. Learn. (2006)

    Google Scholar 

  118. A. Billard, S. Calinon, F. Guenter: Discriminative and adaptive imitation in uni-manual and bi-manual tasks, Robotics Auton. Syst. 54, 370–384 (2006)

    Article  Google Scholar 

  119. J. Choi, K. Kim: Nonparametric Bayesian inverse reinforcement learning for multiple reward functions, Adv. Neural Inf. Process. Syst. 25, 305–313 (2012)

    Google Scholar 

  120. A.K. Tanwani, A. Billard: Transfer in inverse reinforcement learning for multiple strategies, IEEE/RSJ Int. Conf. Intell. Robots Syst. (2013)

    Google Scholar 

  121. D.H. Grollman, A. Billard: Donut as i do: Learning from failed demonstrations, IEEE Int. Conf. Robotics Autom. (2011)

    Google Scholar 

  122. A. Rai, G. de Chambrier, A. Billard: Learning from failed demonstrations in unreliable systems, IEEE-RAS Int. Conf. Humanoid Robots (2013)

    Google Scholar 

  123. M. Goodrich, A. Schultz: Human-robot interaction: A survey, Found. Trend. Human-Comput. Int. 1(3), 203–275 (2007)

    Article  MATH  Google Scholar 

  124. T. Fong, I. Nourbakhsh, K. Dautenhahn: A survey of socially interactive robots, Robotics Auton. Syst. 42(3/4), 143–166 (2003)

    Article  MATH  Google Scholar 

  125. C. Breazeal, B. Scassellati: Robots that imitate humans, Trends Cogn. Sci. 6(11), 481–487 (2002)

    Article  Google Scholar 

  126. B. Scassellati: Imitation and mechanisms of joint attention: A developmental structure for building social skills on a humanoid robot, Lect. Notes Comput. Sci. 1562, 176–195 (1999)

    Article  Google Scholar 

  127. H. Kozima, H. Yano: A robot that learns to communicate with human caregivers, Int. Workshop Epigenet. Robotics (2001)

    Google Scholar 

  128. H. Ishiguro, T. Ono, M. Imai, T. Kanda: Development of an interactive humanoid robot Robovie -- An interdisciplinary approach, Springer Tracts Adv. Robotics 6, 179–192 (2003)

    Article  Google Scholar 

  129. K. Nickel, R. Stiefelhagen: Pointing gesture recognition based on 3d-tracking of face, hands and head orientation, Int. Conf. Multimodal Interfaces (ICMI) (2003) pp. 140–146

    Chapter  Google Scholar 

  130. M. Ito, J. Tani: Joint attention between a humanoid robot and users in imitation game, Int. Conf. Dev. Learn. (ICDL) (2004)

    Google Scholar 

  131. V.V. Hafner, F. Kaplan: Learning to interpret pointing gestures: Experiments with four-legged autonomous robots, Lect. Notes Comput. Sci. 3575, 225–234 (2005)

    Article  Google Scholar 

  132. C. Breazeal, D. Buchsbaum, J. Gray, D. Gatenby, B. Blumberg: Learning from and about others: Towards using imitation to bootstrap the social understanding of others by robots, Artif. Life 11(1/2), 31–62 (2005)

    Article  Google Scholar 

  133. P.F. Dominey, M. Alvarez, B. Gao, M. Jeambrun, A. Cheylus, A. Weitzenfeld, A. Martinez, A. Medrano: Robot command, interrogation and teaching via social interaction, Proc. IEEE-RAS Int. Conf. Humanoid Robots (Humanoids) (2005) pp. 475–480

    Google Scholar 

  134. A.L. Thomaz, M. Berlin, C. Breazeal: Robot science meets social science: An embodied computational model of social referencing, Workshop Toward Soc. Mech. Android Sci. (CogSci) (2005) pp. 7–17

    Google Scholar 

  135. C. Breazeal, L. Aryananda: Recognition of affective communicative intent in robot-directed speech, Auton. Robots 12(1), 83–104 (2002)

    Article  MATH  Google Scholar 

  136. S. Calinon, A. Billard: Teaching a humanoid robot to recognize and reproduce social cues, Proc. IEEE Int. Symp. Robot Human Int. Commun. (RO-MAN) (2006) pp. 346–351

    Google Scholar 

  137. Y. Yoshikawa, K. Shinozawa, H. Ishiguro, N. Hagita, T. Miyamoto: Responsive robot gaze to interaction partner, Proc. Robotics Sci. Syst. (RSS) Philadelphia (2006)

    Google Scholar 

  138. S. Calinon, A. Billard: What is the teacher's role in robot programming by demonstration? - Toward benchmarks for improved learning, Int. Stud. Spec. Issue Psychol, Benchmarks Human-Robot Int. 8(3), 441–464 (2007)

    Article  Google Scholar 

  139. S. Chernova, M. Veloso: Interactive policy learning through confidence-based autonomy, J. Artif. Intell. Res. 34, 1–25 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  140. B.D. Argall, E.L. Sauser: Tactile guidance for policy adaptation, Found. Trend. Robotics 1(2), 79–133 (2010)

    Article  Google Scholar 

  141. S. Calinon, A. Billard: Active teaching in robot programming by demonstration, Proc. IEEE Int. Symp. Robot Human Int. Commun. (RO-MAN), Jeju (2007) pp. 702–707

    Google Scholar 

  142. D. Silver, A. Bagnell, A. Stentz: Active learning from demonstration for robust autonomous navigation, IEEE Conf. Robot. Autom. ICRA'12 (2012)

    Google Scholar 

  143. M. Riley, A. Ude, C. Atkeson, G. Cheng: Coaching: An approach to efficiently and intuitively create humanoid robot behaviors, Proc. IEEE-RAS Int. Conf. Humanoid Robots (Humanoids) (2006) pp. 567–574

    Google Scholar 

  144. H. Bekkering, A. Wohlschlaeger, M. Gattis: Imitation of gestures in children is goal-directed, Q. J. Exp. Psychol. 53A(1), 153–164 (2000)

    Article  Google Scholar 

  145. M. Nicolescu, M.J. Mataric: Task learning through imitation and human-robot interaction. In: Models and Mechanisms of Imitation and Social Learning in Robots, Humans and Animals, (MIT Press, Cambridge 2006) pp. 407–424

    Google Scholar 

  146. J. Demiris, G. Hayes: Imitative learning mechanisms in robots and humans, 5th Eur. Workshop Learn. Robots, ed. by V. Klingspor (1996) pp. 9–16

    Google Scholar 

  147. J. Aleotti, S. Caselli: Robust trajectory learning and approximation for robot programming by demonstration, Robotics Auton. Syst. 54(5), 409–413 (2006)

    Article  Google Scholar 

  148. M. Ogino, H. Toichi, Y. Yoshikawa, M. Asada: Interaction rule learning with a human partner based on an imitation faculty with a simple visuo-motor mapping, Robotics Auton. Syst. 54, 414–418 (2006)

    Article  Google Scholar 

  149. A. Billard, M. Matarić: Learning human arm movements by imitation: Evaluation of a biologically-inspired connectionist architecture, Robotics Auton. Syst. 941, 1–16 (2001)

    MATH  Google Scholar 

  150. A.J. Ijspeert, J. Nakanishi, S. Schaal: Movement imitation with nonlinear dynamical systems in humanoid robots, IEEE Int. Conf. Robotics Autom. (ICRA2002) (2002) pp. 1398–1403

    Google Scholar 

  151. R.H. Cuijpers, H.T. van Schie, M. Koppen, W. Erlhagen, H. Bekkering: Goals and means in action observation: A computational approach, Neural Netw. 19(3), 311–322 (2006)

    Article  MATH  Google Scholar 

  152. M.W. Hoffman, D.B. Grimes, A.P. Shon, R.P.N. Rao: A probabilistic model of gaze imitation and shared attention, Neural Netw. 19(3), 299–310 (2006)

    Article  MATH  Google Scholar 

  153. A. Billard: Drama, a connectionist architecture for on-line learning and control of autonomous robots: Experiments on learning of a synthetic proto-language with a doll robot, Ind. Robot 26(1), 59–66 (1999)

    Article  Google Scholar 

  154. P. Gaussier, S. Moga, J.P. Banquet, J. Nadel: Learning and communication via imitation: An autonomous robot perspective systems, IEEE Trans. Man Cybern. A 31(5), 431–442 (2001)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Engineering, Swiss Federal Institute of Technology (EPFL), EPFL-STI-I2S-LASA, Station 9, 1015, Lausanne, Switzerland

    Aude G. Billard

  2. Idiap Research Institute, Rue Marconi 19, 1920, Martigny, Switzerland

    Sylvain Calinon

  3. Institute for Technical Informatics, Karlsruhe Institute of Technology, Haid-und-Neu-Strasse 7, 76131, Karlsruhe, Germany

    Rüdiger Dillmann

Authors
  1. Aude G. Billard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sylvain Calinon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rüdiger Dillmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aude G. Billard .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering, University of Naples Federico II, Via Claudio 21, 80125, Naples, Italy

    Bruno Siciliano

  2. Department of Computer Sciences, Artificial Intelligence Laboratory, Stanford University, 450 Serra Mall, CA 94305, Stanford, USA

    Oussama Khatib

Video-References

Video-References

:

Demonstrations and reproduction of the task of juicing an orange available from http://handbookofrobotics.org/view-chapter/74/videodetails/29

:

Demonstrations and reproduction of moving a chessman available from http://handbookofrobotics.org/view-chapter/74/videodetails/97

:

Full-body motion transfer under kinematic/dynamic disparity available from http://handbookofrobotics.org/view-chapter/74/videodetails/98

:

Demonstration by visual tracking of gestures available from http://handbookofrobotics.org/view-chapter/74/videodetails/99

:

Demonstration by kinesthetic teaching available from http://handbookofrobotics.org/view-chapter/74/videodetails/100

:

Demonstration by teleoperation of humanoid HRP-2 available from http://handbookofrobotics.org/view-chapter/74/videodetails/101

:

Probabilistic encoding of motion in a subspace of reduced dimensionality available from http://handbookofrobotics.org/view-chapter/74/videodetails/102

:

Reproduction of dishwasher unloading task based on task precedence graph available from http://handbookofrobotics.org/view-chapter/74/videodetails/103

:

Incremental learning of finger manipulation with tactile capability available from http://handbookofrobotics.org/view-chapter/74/videodetails/104

:

Policy refinement after demonstration available from http://handbookofrobotics.org/view-chapter/74/videodetails/105

:

Exploitation of social cues to speed up learning available from http://handbookofrobotics.org/view-chapter/74/videodetails/106

:

Active teaching available from http://handbookofrobotics.org/view-chapter/74/videodetails/107

:

Learning from failure I available from http://handbookofrobotics.org/view-chapter/74/videodetails/476

:

Learning from failure II available from http://handbookofrobotics.org/view-chapter/74/videodetails/477

:

Learning compliant motion from human demonstration available from http://handbookofrobotics.org/view-chapter/74/videodetails/478

:

Learning compliant motion from human demonstration II available from http://handbookofrobotics.org/view-chapter/74/videodetails/479

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Billard, A.G., Calinon, S., Dillmann, R. (2016). Learning from Humans. In: Siciliano, B., Khatib, O. (eds) Springer Handbook of Robotics. Springer Handbooks. Springer, Cham. https://doi.org/10.1007/978-3-319-32552-1_74

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-32552-1_74

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-32550-7

  • Online ISBN: 978-3-319-32552-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation