Encyclopedia of Robotics

Living Edition
| Editors: Marcelo H Ang, Oussama Khatib, Bruno Siciliano

Learning from Demonstration (Programming by Demonstration)

  • Sylvain CalinonEmail author
Living reference work entry
DOI: https://doi.org/10.1007/978-3-642-41610-1_27-1

Synonyms

Definition

Learning from demonstration (LfD), also called programming by demonstration (PbD), refers to the process used to transfer new skills to a machine by relying on demonstrations from a user. It is inspired by the imitation capability developed by humans and animals to acquire new skills. LfD aims at making programming accessible to novice users by providing them with an intuitive interface they are familiar with, as humans already exchange knowledge in this way.

Overview

In robotics, LfD appeared as a way to reprogram a robot without having to rely on a computer language or a complex interface. It instead introduces more intuitive skill transfer interactions with the robot (Billard et al., 2016; Argall et al., 2009). The goal is to provide user-friendly interfaces that do not require knowledge in computer programming or robotics. LfD can be considered at various levels, from the transfer of low-level...

This is a preview of subscription content, log in to check access.

References

  1. Argall BD, Chernova S, Veloso M, Browning B (2009) A survey of robot learning from demonstration. Robot Auton Syst 57(5):469–483CrossRefGoogle Scholar
  2. Bennequin D, Fuchs R, Berthoz A, Flash T (2009) Movement timing and invariance arise from several geometries. PLoS Comput Biol 5(7):1–27MathSciNetCrossRefGoogle Scholar
  3. Billard AG, Calinon S, Dillmann R (2016) Learning from humans, chapter 74. In: Siciliano B, Khatib O (eds) Handbook of robotics, 2nd edn. Springer, Secaucus, pp 1995–2014CrossRefGoogle Scholar
  4. Bruno D, Calinon S, Caldwell DG (2017) Learning autonomous behaviours for the body of a flexible surgical robot. Auton Robot 41(2):333–347CrossRefGoogle Scholar
  5. Cakmak M, DePalma N, Arriaga RI, Thomaz AL (2010) Exploiting social partners in robot learning. Auton Robot 29(3–4):309–329CrossRefGoogle Scholar
  6. Calinon S (2016) A tutorial on task-parameterized movement learning and retrieval. Intell Serv Robot 9(1):1–29CrossRefGoogle Scholar
  7. Calinon S, Alizadeh T, Caldwell DG (2013) On improving the extrapolation capability of task-parameterized movement models. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), Tokyo, pp 610–616, Nov 2013Google Scholar
  8. Calinon S, D’halluin F, Sauser EL, Caldwell DG, Billard AG (2010) Learning and reproduction of gestures by imitation: an approach based on hidden Markov model and Gaussian mixture regression. IEEE Robot Autom Mag 17(2):44–54Google Scholar
  9. Calinon S, Lee D (2018, in press) Learning control. In: Vadakkepat P, Goswami A (eds) Humanoid robotics: a reference. Springer. https://doi.org/10.1007/978-94-007-7194-9_68-2
  10. Calinon S, Li Z, Alizadeh T, Tsagarakis NG, Caldwell DG (2012) Statistical dynamical systems for skills acquisition in humanoids. In: Proceedings of IEEE international conference on humanoid robots (Humanoids), Osaka, pp 323–329Google Scholar
  11. Canal G, Alenyà G, Torras C (2016) Personalization framework for adaptive robotic feeding assistance. In: Proceedings of international conference on social robotics (ICSR), Kansas City, pp 22–31, Nov 2016CrossRefGoogle Scholar
  12. Chen J, Lau HYK, Xu W, Ren H (2016) Towards transferring skills to flexible surgical robots with programming by demonstration and reinforcement learning. In: Proceedings of international conference on advanced computational intelligence, pp 378–384, Feb 2016Google Scholar
  13. Coates A, Abbeel P, Ng AY (2009) Apprenticeship learning for helicopter control. Commun ACM 52(7): 97–105CrossRefGoogle Scholar
  14. Evrard P, Gribovskaya E, Calinon S, Billard AG, Kheddar A (2009) Teaching physical collaborative tasks: object-lifting case study with a humanoid. In: Proceedings of IEEE international conference on humanoid robots (Humanoids), Paris, pp 399–404, Dec 2009Google Scholar
  15. Hamaya M, Matsubara T, Noda T, Teramae T, Morimoto J (2017) Learning assistive strategies for exoskeleton robots from user-robot physical interaction. Pattern Recogn Lett 99:67–76CrossRefGoogle Scholar
  16. Ijspeert A, Nakanishi J, Pastor P, Hoffmann H, Schaal S (2013) Dynamical movement primitives: learning attractor models for motor behaviors. Neural Comput 25(2):328–373MathSciNetCrossRefGoogle Scholar
  17. Kelso JAS (2009) Synergies: atoms of brain and behavior. In: Sternad D (ed) Progress in motor control. Advances in experimental medicine and biology, vol 629. Springer, New York/London, pp 83–91CrossRefGoogle Scholar
  18. Khansari-Zadeh SM, Billard A (2011) Learning stable non-linear dynamical systems with Gaussian mixture models. IEEE Trans Robot 27(5):943–957CrossRefGoogle Scholar
  19. Krishnan S, Garg A, Patil S, Lea C, Hager G, Abbeel P, Goldberg K (2015) Unsupervised surgical task segmentation with milestone learning. In: Proceedings of international symposium on robotics research (ISRR)Google Scholar
  20. Kulic D, Takano W, Nakamura Y (2008) Incremental learning, clustering and hierarchy formation of whole body motion patterns using adaptive hidden Markov chains. Int J Robot Res 27(7):761–784CrossRefGoogle Scholar
  21. Lee D, Ott C (2011) Incremental kinesthetic teaching of motion primitives using the motion refinement tube. Auton Robot 31(2):115–131CrossRefGoogle Scholar
  22. Lee D, Ott C, Nakamura Y (2010) Mimetic communication model with compliant physical contact in human-humanoid interaction. Int J Robot Res 29(13): 1684–1704CrossRefGoogle Scholar
  23. Lee SH, Suh IH, Calinon S, Johansson R (2012) Learning basis skills by autonomous segmentation of humanoid motion trajectories. In: Proceedings of IEEE international conference on humanoid robots (Humanoids), Osaka, pp 112–119Google Scholar
  24. Liu W, Dai B, Humayun A, Tay C, Yu C, Smith LB, Rehg JM, Song L (2017) Iterative machine teaching. In: Proceedings of international conference on machine learning (ICML), Sydney, Aug 2017Google Scholar
  25. Maeda GJ, Neumann G, Ewerton M, Lioutikov R, Kroemer O, Peters J (2017) Probabilistic movement primitives for coordination of multiple human-robot collaborative tasks. Auton Robot 41(3):593–612CrossRefGoogle Scholar
  26. Mühlig M, Gienger M, Steil J (2012) Interactive imitation learning of object movement skills. Auton Robot 32(2):97–114CrossRefGoogle Scholar
  27. Nakanishi J, Morimoto J, Endo G, Cheng G, Schaal S, Kawato M (2004) Learning from demonstration and adaptation of biped locomotion. Robot Auton Syst 47(2–3):79–91CrossRefGoogle Scholar
  28. Nehaniv CL, Dautenhahn K (2002) The correspondence problem. In: Dautenhahn K, Nehaniv CL (eds) Imitation in animals and artifacts. MIT Press, Cambridge, pp 41–61Google Scholar
  29. Nehaniv CL, Dautenhahn K (eds) (2007) Imitation and social learning in robots, humans, and animals: behavioural, social and communicative dimensions. Cambridge University Press, CambridgeGoogle Scholar
  30. Neumann K, Steil JJ (2015) Learning robot motions with stable dynamical systems under diffeomorphic transformations. Robot Auton Syst 70:1–15CrossRefGoogle Scholar
  31. Niekum S, Osentoski S, Konidaris G, Chitta S, Marthi B, Barto AG (2015) Learning grounded finite-state representations from unstructured demonstrations. Int J Robot Res 34(2):131–157CrossRefGoogle Scholar
  32. Padoy N, Hager GD (2011) Human-machine collaborative surgery using learned models. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 5285–5292, May 2011Google Scholar
  33. Paraschos A, Daniel C, Peters J, Neumann G (2013) Probabilistic movement primitives. In: Burges CJC, Bottou L, Welling M, Ghahramani Z, Weinberger KQ (eds) Advances in neural information processing systems (NIPS). Curran Associates, Inc., Red Hook, pp 2616–2624Google Scholar
  34. Perrin N, Schlehuber-Caissier P (2016) Fast diffeomorphic matching to learn globally asymptotically stable nonlinear dynamical systems. Syst Control Lett 96: 51–59MathSciNetCrossRefGoogle Scholar
  35. Pignat E, Calinon S (2017) Learning adaptive dressing assistance from human demonstration. Robot Auton Syst 93:61–75CrossRefGoogle Scholar
  36. Ratliff N, Ziebart BD, Peterson K, Bagnell JA, Hebert M, Dey A, Srinivasa S (2009) Inverse optimal heuristic control for imitation learning. In: International conference on artificial intelligence and statistics (AIStats), pp 424–431, Apr 2009Google Scholar
  37. Reiley CE, Plaku E, Hager GD (2010) Motion generation of robotic surgical tasks: learning from expert demonstrations. In: International conference on IEEE engineering in medicine and biology society (EMBC), pp 967–970Google Scholar
  38. Rozo L, Calinon S, Caldwell DG, Jimenez P, Torras C (2016) Learning physical collaborative robot behaviors from human demonstrations. IEEE Trans Robot 32(3):513–527CrossRefGoogle Scholar
  39. Rueckert E, Mundo J, Paraschos A, Peters J, Neumann G (2015) Extracting low-dimensional control variables for movement primitives. In: Proceedings of IEEE international conference on robotics and automation (ICRA), Seattle, pp 1511–1518Google Scholar
  40. Savarimuthu TR, Buch AG, Schlette C, Wantia N, Rossmann J, Martinez D, Alenya G, Torras C, Ude A, Nemec B, Kramberger A, Worgotter F, Aksoy EE, Papon J, Haller S, Piater J, Kruger N (2018) Teaching a robot the semantics of assembly tasks. IEEE Trans Syst Man Cybernet Syst 48(5):670–692CrossRefGoogle Scholar
  41. Soh H, Demiris Y (2015) Learning assistance by demonstration: smart mobility with shared control and paired haptic controllers. J Hum Robot Interaction 4(3): 76–100CrossRefGoogle Scholar
  42. Sternad D, Park S-W, Mueller H, Hogan N (2010) Coordinate dependence of variability analysis. PLoS Comput Biol 6(4):1–16CrossRefGoogle Scholar
  43. Todorov E, Jordan MI (2002) A minimal intervention principle for coordinated movement. In: Advances in neural information processing systems (NIPS), pp 27–34Google Scholar
  44. Ude A, Gams A, Asfour T, Morimoto J (2010) Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Trans Robot 26(5):800–815CrossRefGoogle Scholar
  45. Whiten A, McGuigan N, Marshall-Pescini S, Hopper LM (2009) Emulation, imitation, over-imitation and the scope of culture for child and chimpanzee. Philos Trans R Soc B 364(1528):2417–2428CrossRefGoogle Scholar
  46. Yang T, Chui CK, Liu J, Huang W, Su Y, Chang SKY (2014) Robotic learning of motion using demonstrations and statistical models for surgical simulation. Int J Comput Assist Radiol Surg 9(5):813–823CrossRefGoogle Scholar
  47. Zeestraten MJA, Calinon S, Caldwell DG (2016) Variable duration movement encoding with minimal intervention control. In: Proceedings of IEEE international conference on robotics and automation (ICRA), May 2016, Stockholm, pp 497–503Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Idiap Research InstituteMartignySwitzerland

Section editors and affiliations

  • Jee-Hwan Ryu
    • 1
  1. 1.School of Mechanical EngineeringKorea University of Technology & EducationCheon-AnRepublic of Korea