Skip to main content
SpringerLink
Log in

Programming by Demonstration: A Taxonomy of Current Relevant Methods to Teach and Describe New Skills to Robots

  • Chapter
ROBOT2013: First Iberian Robotics Conference

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 252))

  • 2949 Accesses

Abstract

Programming by Demonstration (PbD) covers methods by which a robot learns new skills through human guidance and imitation. PbD has been a key topic in robotics during the last decade that includes the development of robust algorithms for motor control, motor learning, gesture recognition and the visual-motor integration. Nowadays, PbD deals more with learning methods than traditional approaches, and frequently it is referred to as Imitation Learning or Behavioral Cloning. This work will review and analyse existing works in order to create a taxonomy of the elements that constitute the most relevant approaches in this field to date. We intend to establish the categories and types of algorithms involved so far in PbD and describing their advantages and disadvantages and potential developments.

This research has been partially supported by the Industrial Doctorate program of the Government of Catalonia.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Calinon, S., Guenter, F., Billard, A.: On learning, representing, and generalizing a task in a humanoid robot. IEEE Trans. on Systems, Man and Cybernetics 32, 286–298 (2007)

    Article  Google Scholar 

  2. Ijspeert, A., Nakanishi, J., Schaal, S.: Movement imitation with nonlinear dynamical systems in humanoid robots. In: IEEE Int. Conf. on Robotics and Automation, ICRA (2002)

    Google Scholar 

  3. Mulling, K., Kober, J., Kroemer, O., Peters, J.: Learning to Select and Generalize Striking Movements in Robot Table Tennis. The International Journal of Robotics Research 32(3), 263–279 (2013)

    Article  Google Scholar 

  4. Nehaniv, C.L., Dautenhahn, K.: Like me? Measures of correspondence and imitation. Cybernetics and Systems 32(1-2), 11–51 (2001)

    Article  MATH  Google Scholar 

  5. Breazeal, C., Scassellati, B.: Robots that imitate humans. Trends in Cognitive Sciences 6(11), 481–487 (2002)

    Article  Google Scholar 

  6. Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A Survey of Robot Learning from Demonstration. Robotics and Autonomous Systems 57(5), 469–483 (2009)

    Article  Google Scholar 

  7. Pook, P.K., Ballard, D.H.: Recognizing teleoperated manipulations. In: Proc. of the IEEE Int. Conf. on Robotics and Automation, ICRA 1993 (1993)

    Google Scholar 

  8. Smart, W.D.: Making Reinforcement Learning Work on Real Robots Ph.D. Thesis, Department of Computer Science, Brown University, Providence, RI (2002)

    Google Scholar 

  9. Abbeel, P., Ng, A.Y.: Apprenticeship learning via inverse reinforcement learning. In: Proc. of the 21st Int. Conf. on Machine Learning, ICML 2004 (2004)

    Google Scholar 

  10. Chernova, S., Veloso, M.: Multi-thresholded approach to demonstration selection for interactive robot learning. In: Proc. of the 3rd ACM/IEEE Int. Conf. on Human-Robot Interaction, HRI 2008 (2008)

    Google Scholar 

  11. Breazeal, C., Berlin, M., Brooks, A., Gray, J., Thomaz, A.L.: Using perspective taking to learn from ambiguous demonstrations The Social Mechanisms of Robot Programming by Demonstration. Robotics and Autonomous Systems 54(5), 385–393 (2006)

    Article  Google Scholar 

  12. Rybski, P.E., Yoon, K., Stolarz, J., Veloso, M.: Interactive robot task training through dialog and demonstration. In: Proc. of the 2nd ACM/IEEE Int. Conf. on Human-Robot Interactions, HRI 2007 (2007)

    Google Scholar 

  13. Argall, B., Browning, B., Veloso, M.: Learning from demonstration with the critique of a human teacher. In: Proc. of the 2nd ACM/IEEE Int. Conf. on Human-Robot Interactions, HRI 2007 (2007)

    Google Scholar 

  14. Grollman, D.H., Jenkins, O.C.: Dogged learning for robots. In: Proc. of the IEEE Int. Conf. on Robotics and Automation, ICRA 2007 (2007)

    Google Scholar 

  15. Nehmzow, U., Akanyeti, O., Weinrich, C., Kyriacou, T., Billings, S.: Robot programming by demonstration through system identification. In: Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS 2007 (2007)

    Google Scholar 

  16. Ogino, M., Toichi, H., Yoshikawa, Y., Asada, M.: Interaction rule learning with a human partner based on an imitation faculty with a simple visuo-motor mapping. The Social Mechanisms of Robot Programming by Demonstration, Robotics and Autonomous Systems 54(5), 414–418 (2006)

    Article  Google Scholar 

  17. Calinon, S., Billard, A.: Incremental learning of gestures by imitation in a humanoid robot. In: Proc. of the 2nd ACM/IEEE Int. Conf. on Human Robot Interactions, HRI 2007 (2007)

    Google Scholar 

  18. Aleotti, J., Caselli, S.: Robust trajectory learning and approximation for robot programming by demonstration. The Social Mechanisms of Robot Programming by Demonstration, Robotics and Autonomous Systems 54(5), 409–413 (2006)

    Article  Google Scholar 

  19. Billard, A., Mataric, M.: Learning human arm movements by imitation: Evaluation of biologically inspired connectionist architecture. Robotics and Autonomous Systems 37(2-3), 145–160 (2001)

    Article  MATH  Google Scholar 

  20. Ude, A., Atkeson, C.G., Riley, M.: Programming full-body movements for humanoid robots by observation. Robotics and Autonomous Systems 47, 93–108 (2004)

    Article  Google Scholar 

  21. Steil, J., Rthling, F., Haschke, R., Ritter, H.: Situated robot learning for multi-modal instruction and imitation of grasping. Robot Learning by Demonstration, Robotics and Autonomous Systems 2-3(47), 129–141 (2004)

    Article  Google Scholar 

  22. Ratliff, N., Bagnell, J.A., Zinkevich, M.A.: Maximum margin planning. In: Proc. of the 23rd Int. Conf. on Machine Learning, ICML 2006 (2006)

    Google Scholar 

  23. Chernova, S., Veloso, M.: Confidence-based learning from demonstration using Gaussian Mixture Models. In: Proc. of the Int. Conf. on Autonomous Agents and Multiagent Systems, AAMAS 2007 (2007)

    Google Scholar 

  24. Saunders, J., Nehaniv, C.L., Dautenhahn, K.: Teaching robots by moulding behavior and scaffolding the environment. In: Proc. of the 1st ACM/IEEE Int. Conf. on Human Robot Interactions, HRI 2006 (2006)

    Google Scholar 

  25. Kober, J., Peters, J.: Imitation and Reinforcement Learning. Practical Learning Algorithms for Motor Primitives in Robotics 17(2), 1–8 (2010)

    Google Scholar 

  26. Rybski, P.E., Voyles, R.M.: Interactive task training of a mobile robot through human gesture recognition. In: Proc. of the IEEE Int. Conf. on Robotics and Automation, ICRA 1999 (1999)

    Google Scholar 

  27. Lockerd, A., Breazeal, C.: Tutelage and socially guided robot learning. In: Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS 2004 (2004)

    Google Scholar 

  28. Calinon, S., Sauser, E.L., Billard, A.G., Caldwell, D.G.: Evaluation of a Probabilistic Approach to Learn and Reproduce Gestures by Imitation. In: IEEE Int. Conf. on Robotics and Automation (x), pp. 2671–2676 (May 2010)

    Google Scholar 

  29. Khansari-Zadeh, S.M., Billard, A.: Learning Stable Nonlinear Dynamical Systems with Gaussian Mixture Models. IEEE Trans. on Robotics 27(5), 943–957 (2011)

    Article  Google Scholar 

  30. Khansari-Zadeh, S.M., Billard, A.: BM: Aniterative algorithm to learn stable non-linear dynamical systems with Gaussian Mixture Models. In: Proc. Int. Conf. Robotics and Automation, pp. 2381–2388 (2010)

    Google Scholar 

  31. Rasmussen, C., Williams, C.: Gaussian Processes for Machine Learning. Springer, New York (2006)

    MATH  Google Scholar 

  32. Ijspeert, A.J., Nakanishi, J., Schaal, S.: Learning rhythmic movements by demonstration using nonlinear oscillators. In: Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS 2002 (2002)

    Google Scholar 

  33. Vijayakumar, S., Schaal, S.: Locally weighted projection regression: An o(n) algorithm for incremental real time learning in high dimensional space. In: Proc. of the 17th Int. Conf. on Machine Learning, ICML 2000 (2000)

    Google Scholar 

  34. Grollman, D.H., Jenkins, O.C.: Sparse incremental learning for interactive robot control policy estimation. In: Proc. of the IEEE Int. Conf. on Robotics and Automation, ICRA 2008 (2008)

    Google Scholar 

  35. Chersi, F.: Learning Through Imitation: a Biological Approach to Robotics. IEEE Trans. on Autonomous Mental Development 4(3), 204–214 (2012)

    Article  Google Scholar 

  36. Merrick, K.: Intrinsic Motivation and Introspection in Reinforcement Learning 4(4), 315–329 (2012)

    Google Scholar 

  37. Guenter, F., Billard, A.: Using reinforcement learning to adapt an imitation task. In: Proc. of the IEEE/RSJ Int. Conf.onIntelligent Robots and Systems, IROS 2007 (2007)

    Google Scholar 

  38. Abbeel, P., Coates, A., Quigley, M., Ng, A.Y.: An application of reinforcement learning to aerobatic helicopter flight. In: Proc. of the Advances in Neural Information Processing, NIPS 2007 (2007)

    Google Scholar 

  39. Nicolescu, M.N., Mataric, M.J.: Methods for robot task learning: Demonstrations, generalization and practice. In: Proc. of the Second International Joint Conference on Autonomous Agents and Multi-Agent Systems, AAMAS 2003 (2003)

    Google Scholar 

  40. Chernova, S., Veloso, M.: Learning equivalent action choices from demonstration. In: Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS 2008 (2008)

    Google Scholar 

  41. Calinon, S., Dhalluin, F., Sauser, E.L., Caldwell, D.G., Billard, A.G.: Learning and Reproduction of Gestures by Imitation. An Approach Based on Hidden Markov Model and Gaussian Mixture Regression 17(2), 44–54 (2010)

    Google Scholar 

  42. Ijspeert, A., Nakanishi, J., Schaal, S.: Trajectory formation for imitation with nonlinear dynamical systems. In: Proc. IEEE Intl Conf. on Intelligent Robots and Systems (IROS), pp. 752–757 (2001)

    Google Scholar 

  43. Vijayakumar, S., Dsouza, A., Schaal, S.: Incremental online learning in high dimensions. Neural Computation 17(12), 2602–2634 (2005)

    Article  MathSciNet  Google Scholar 

  44. Dempster, A., Rubin, N.L.D.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Statist. Soc. B 39(1), 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  45. Pastor, P., Hoffmann, H., Asfour, T., Schaal, S.: Learning and generalization of motor skills by learning from demonstration. In: Proc. Int. Conf. Robotics and Automation, pp. 1293–1298 (2009)

    Google Scholar 

  46. Coates, A., Abbeel, P., Ng, A.Y.: Learning for Control from Multiple Demonstrations. In: Proc. of the 25th Int. Conf. on Machine Learning (ICML 2008), pp. 144–151 (2008)

    Google Scholar 

  47. Karaoguz, C., Rodemann, T., Wrede, B., Goerick, C.: Learning Information Acquisition for Multitasking Scenarios in Dynamic Environments. IEEE Trans. on Autonomous Mental Development 5(1), 46–61 (2013)

    Article  Google Scholar 

  48. Cangelosi, A., Metta, G., Sagerer, G., Nolfi, S., Nehaniv, C., Fischer, K., Tani, J., et al.: Integration of Action and Language Knowledge: A Roadmap for Developmental Robotics. IEEE Trans. on Auton. Mental Develop. 2(3), 167–195 (2010)

    Article  Google Scholar 

  49. Nicolescu, M.N., Jenkins, O.C., Olenderski, F.: Learning behavior fusion from demonstration. Interaction Studies 9(2), 319–352 (2008)

    Article  Google Scholar 

  50. Grollman, D.H., Jenkins, O.C.: Incremental Learning of Subtasks from Unsegmented Demonstration. In: Int. Conf. on Intelligent Robots and Systems (2008)

    Google Scholar 

  51. Chernova, S., Veloso, M.: Interactive Policy Learning through Confidence-Based Autonomy. Journal of Artificial Intelligence Research 34, 1–25 (2009)

    MathSciNet  MATH  Google Scholar 

  52. Argall, B., Browning, B., Veloso, M.: Teacher feedback to scaffold and refine demonstrated motion primitives on a mobile robot. Robotics and Autonomous Systems 59(3-4), 243–255 (2011)

    Article  Google Scholar 

  53. Argall, B., Sauser, E.L., Billard, A.G.: Tactile Guidance for Policy Adaptation. Foundations and Trends in Robotics 1(2), 79–133 (2010)

    Article  Google Scholar 

  54. Grollman, D.H., Billard, A.G.: Donut as I do: Learning from failed demonstrations. In: Int. Conf. on Robotics and Automation (2010)

    Google Scholar 

  55. Ratliff, N., Ziebart, B., Peterson, K., Bagnell, J.B., Hebert, H.: Inverse Optimal Heuristic Control for Imitation Learning. In: Proc. of the 12th Int. Conf. on Artificial Intelligence and Statistics (2009)

    Google Scholar 

  56. Choi, J., Kim, K.: Nonparametric bayesian inverse reinforcement learning for multiple reward functions. In: Advances in Neural Info. Proc. Sys. 25, NIPS 2012 (2012)

    Google Scholar 

  57. Billard, A., Calinon, S.: Robot Programming by Demonstration. In: Handbook of Robotics, ch. 59 (2007)

    Google Scholar 

  58. Farkhadinov, I., Hwan, J.: A User Study of a Mobile Robot Teleoperation. In: Proc. of the 4th International Conference on Ubiquitous Robotics and Ambient Intelligence (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ATEKNEA Solutions, Víctor Pradera, 45, 08940, Cornellà de Llobregat, Spain

    Jordi Bautista-Ballester & Jaume Vergés-Llahí

  2. Department of Computer Engineering and Mathematics, Universitat Rovira i Virgili, Tarragona, 43007, Spain

    Jordi Bautista-Ballester & Domènec Puig

Authors
  1. Jordi Bautista-Ballester
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jaume Vergés-Llahí
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Domènec Puig
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jordi Bautista-Ballester .

Editor information

Editors and Affiliations

  1. CAR UPM-CSIC, State Agency National Research Council (CSIC), Arganda del Rey, Spain

    Manuel A. Armada

  2. University Technical of Catalonia (UPC), Barcelona, Spain

    Alberto Sanfeliu

  3. University Polytechnic of Madrid (UPM), Madrid, Spain

    Manuel Ferre

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Bautista-Ballester, J., Vergés-Llahí, J., Puig, D. (2014). Programming by Demonstration: A Taxonomy of Current Relevant Methods to Teach and Describe New Skills to Robots. In: Armada, M., Sanfeliu, A., Ferre, M. (eds) ROBOT2013: First Iberian Robotics Conference. Advances in Intelligent Systems and Computing, vol 252. Springer, Cham. https://doi.org/10.1007/978-3-319-03413-3_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-03413-3_21

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-03412-6

  • Online ISBN: 978-3-319-03413-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation