Towards Learning of Generic Skills for Robotic Manipulation

Abstract

Learning versatile, reusable skills is one of the key prerequisites for autonomous robots. Imitation and reinforcement learning are among the most prominent approaches for learning basic robotic skills. However, the learned skills are often very specific and cannot be reused in different but related tasks. In the project "Behaviors for Mobile Manipulation", we develop hierarchical and transfer learning methods which allow a robot to learn a repertoire of versatile skills that can be reused in different situations. The development of new methods is closely integrated with the analysis of complex human behavior.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Notes

  1. 1.

    See http://robotik.dfki-bremen.de/en/research/projects/besman.html for more details on the project.

References

  1. 1.

    Argall BD, Chernova S, Veloso M, Browning B (2009) A survey of robot learning from demonstration. Robot Auton Syst 57(5):469–483

    Article  Google Scholar 

  2. 2.

    Schaal S (1997) Learning from demonstration. In: Advances in neural information processing systems, vol 9. pp 12–20

  3. 3.

    Kober J, Bagnell JA, Peters J (2013) Reinforcement learning in robotics: a survey. Int J Robot Res 30:820–833

    Google Scholar 

  4. 4.

    Taylor M, Stone P (2009) Transfer learning for reinforcement learning domains: a survey. J Mach Learn Res 10:1633–1685

    MATH  MathSciNet  Google Scholar 

  5. 5.

    Barto AG, Mahadevan S (2003) Recent advances in hierarchical reinforcement learning. Discret Event Dyn Syst 13(4):341–379

    Article  MathSciNet  Google Scholar 

  6. 6.

    Stulp F, Schaal S (2011) Hierarchical reinforcement learning with motion primitives. In: 11th IEEE-RAS international conference on humanoid robots

  7. 7.

    Peters J, Mülling K, Kober J, Nguyen-Tuong D, Krömer O (2012) Robot skill learning. In: Proceedings of the European conference on artificial intelligence

  8. 8.

    Peters J, Schaal S (2008) Natural actor-critic. Neurocomputing 71(7–9):1180–1190

    Article  Google Scholar 

  9. 9.

    Peters J, Schaal S (2007) Reinforcement learning by reward-weighted regression for operational space control. In: Proceedings of the international conference on machine learning, pp 745–750

  10. 10.

    Kober J, Peters J (2010) Policy search for motor primitives in robotics. Mach Learn 84:171–203

    Article  MathSciNet  Google Scholar 

  11. 11.

    Peters J, Mülling K, Altun Y (2010) Relative entropy policy search. In: Proceedings of the 24th AAAI conference on artificial intelligence

  12. 12.

    Daniel C, Neumann G, Peters J (2012) Hierarchical relative entropy policy search. In: Proceedings of the 15th international conference on artificial intelligence and statistics, pp 273–281

  13. 13.

    Theodorou E, Buchli J, Schaal S (2010) A generalized path integral control approach to reinforcement learning. J Mach Learn Res 11:3137–3181

    MATH  MathSciNet  Google Scholar 

  14. 14.

    Hansen N, Ostermeier A (2001) Completely derandomized self-adaptation in evolution strategies. Evolut Comput 9:159–195

    Article  Google Scholar 

  15. 15.

    Heidrich-Meisner V, Igel C (2008) Evolution strategies for direct policy search. In: Parallel problem solving from nature PPSN X, pp 428–437

  16. 16.

    Ijspeert AJ, Nakanishi J, Hoffmann H, Pastor P, Schaal S (2013) Dynamical movement primitives: learning attractor models for motor behaviors. Neural Comput 25(2):328–373

    Article  MATH  MathSciNet  Google Scholar 

  17. 17.

    Kober J, Muelling K, Kroemer O, Lampert CH, Scholkopf B, Peters J (2010) Movement templates for learning of hitting and batting. In: IEEE international conference on robotics and automation

  18. 18.

    Muelling K, Kober J, Kroemer O, Peters J (2013) Learning to select and generalize striking movements in robot table tennis. International J Robot Res 32(3):263–279

    Article  Google Scholar 

  19. 19.

    Pastor P, Hoffmann H, Asfour T, Schaal S (2009) Learning and generalization of motor skills by learning from demonstration. In: Proceedings of the 2009 IEEE international conference on robotics and automation, pp 1293–1298

  20. 20.

    Khansari-Zadeh SM, Billard A (2013) Learning stable non-linear dynamical systems with gaussian mixture models. IEEE Trans Robot

  21. 21.

    Graybiel A (1998) The basal ganglia and chunking of action repertoires. Neurobiol Learn Mem 70(1–2):119–36

    Article  Google Scholar 

  22. 22.

    Abdenebaoui L, Kirchner EA, Kassahun Y, Kirchner F (2007) A connectionist architecture for learning to play a simulated BRIO labyrinth game. In: Proceedings of the 30th annual German conference on artificial intelligence (KI07), Springer, pp 427–430

  23. 23.

    Fearnhead P, Liu Z (2007) On-line inference for multiple change point models. J Royal Stat Soc Ser B (Stat Methodol) 69:589–605

    Article  MathSciNet  Google Scholar 

  24. 24.

    Fox E, Sudderth E, Jordan M, Willsky A (2010) Sharing features among dynamical systems with beta processes. In: Neural information processing systems 22. MIT Press

  25. 25.

    Kober J, Oztop E, Peters J (2011) Reinforcement learning to adjust robot movements to new situations. In: Proceedings of the international joint conference on artificial intelligence, pp 1–6

  26. 26.

    da Silva BC, Konidaris G, Barto AG (2012) Learning parameterized skills. In: Proceedings of the 29th international conference on machine learning. Edinburgh

  27. 27.

    Rasmussen C, Williams C (2006) Gaussian processes for machine learning. MIT Press, Cambridge

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jan Hendrik Metzen.

Additional information

This work was supported through two grants of the German Federal Ministry of Economics and Technology (BMWi, FKZ 50 RA 1216 and FKZ 50 RA 1217).

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Metzen, J.H., Fabisch, A., Senger, L. et al. Towards Learning of Generic Skills for Robotic Manipulation. Künstl Intell 28, 15–20 (2014). https://doi.org/10.1007/s13218-013-0280-1

Download citation

Keywords

  • Multi-task learning
  • Skill learning
  • Movement primitives
  • Transfer learning
  • Reinforcement learning