Advertisement

Autonomous Robots

, Volume 43, Issue 4, pp 897–912 | Cite as

Learning position and orientation dynamics from demonstrations via contraction analysis

  • Harish chaandar Ravichandar
  • Ashwin DaniEmail author
Article
  • 600 Downloads

Abstract

This paper presents a unified framework of model-learning algorithms, called contracting dynamical system primitives (CDSP), that can be used to learn pose (i.e., position and orientation) dynamics of point-to-point motions from demonstrations. The position and the orientation (represented using quaternions) trajectories are modeled as two separate autonomous nonlinear dynamical systems. The special constraints of the \({\mathbb {S}}^{3}\) manifold are enforced in the formulation of the system that models the orientation dynamics. To capture the variability in the demonstrations, the dynamical systems are estimated using Gaussian mixture models (GMMs). The parameters of the GMMs are learned subject to the constraints derived using partial contraction analysis. The learned models’ reproductions are shown to accurately reproduce the demonstrations and are guaranteed to converge to the desired goal location. Experimental results illustrate the CDSP algorithm’s ability to accurately learn position and orientation dynamics and the utility of the learned models in path generation for a Baxter robot arm. The CDSP algorithm is evaluated on a publicly available dataset and a synthetic dataset, and is shown to have the lowest and comparable average reproduction errors when compared to state-of-the-art imitation learning algorithms.

Keywords

Learning from demonstration Gaussian mixture models Contraction analysis Model learning 

Notes

Acknowledgements

The authors would like to acknowledge Klaus Neumann and Jochen Steil for providing the SEA means and standard deviations of the seven state-of-the-art algorithms used for comparison presented in Sect. 5.1. The authors would like to thank the anonymous reviewers for their valuable comments and suggestions to improve the quality of the paper.

Funding

Funding was provided by the UTC Institute for Advanced Systems Engineering (UTC-IASE) of the University of Connecticut and the United Technologies Corporation.

Supplementary material

10514_2018_9758_MOESM1_ESM.mp4 (32.3 mb)
Supplementary material 1 (mp4 33061 KB)

References

  1. Abbeel, P., & Ng, A. Y., (2004). Apprenticeship learning via inverse reinforcement learning. In International conference on machine learning (pp. 1–8). ACM.Google Scholar
  2. Ahmadzadeh, R., Rana, M. A., & Chernova, S. (2017). Generalized cylinders for learning, reproduction, generalization, and refinement of robot skills. In Robotics: Science and systems.Google Scholar
  3. Akgun, B., Cakmak, M., Jiang, K., & Thomaz, A. L. (2012). Keyframe-based learning from demonstration. International Journal of Social Robotics, 4(4), 343–355.CrossRefGoogle Scholar
  4. Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009). A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5), 469–483.CrossRefGoogle Scholar
  5. Billard, A., & Matarić, M. J. (2001). Learning human arm movements by imitation: Evaluation of a biologically inspired connectionist architecture. Robotics and Autonomous Systems, 37(2), 145–160.CrossRefzbMATHGoogle Scholar
  6. Bowen, C., & Alterovitz, R. (2014). Closed-loop global motion planning for reactive execution of learned tasks. In IEEE/RSJ international conference on intelligent robots and systems (pp. 1754–1760).Google Scholar
  7. Calinon, S., Bruno, D., Caldwell, D. G. (2014). A task-parameterized probabilistic model with minimal intervention control. In IEEE international conference on robotics and automation (ICRA) (pp. 3339–3344).Google Scholar
  8. Cohn, D. A., Ghahramani, Z., & Jordan, M. I. (1996). Active learning with statistical models. Journal of Artificial Intelligence Research, 4, 129–145.CrossRefzbMATHGoogle Scholar
  9. Corke, P. I. (2011). Robotics, vision and control: Fundamental algorithms in Matlab. Berlin: Springer.CrossRefzbMATHGoogle Scholar
  10. Dani, A. P., Chung, S. J., & Hutchinson, S. (2015). Observer design for stochastic nonlinear systems via contraction-based incremental stability. IEEE Transactions on Automatic Control, 60(3), 700–714.MathSciNetCrossRefzbMATHGoogle Scholar
  11. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the E–M algorithm. Journal of the Royal Statistical Society Series B (Methodological), 39(1), 1–38.MathSciNetCrossRefzbMATHGoogle Scholar
  12. Diankov, R. (2010). Automated construction of robotic manipulation programs. Ph.D. thesis, Carnegie Mellon University.Google Scholar
  13. Dragan, A. D., Muelling, K., Bagnell, J. A., & Srinivasa, S. S. (2015). Movement primitives via optimization. In IEEE international conference on robotics and automation (ICRA) (pp. 2339–2346).Google Scholar
  14. Gribovskaya, E., Khansari-Zadeh, S. M., & Billard, A. (2010). Learning non-linear multivariate dynamics of motion in robotic manipulators. The International Journal of Robotics Research, 30(1), 80–117.CrossRefGoogle Scholar
  15. Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2002). Learning rhythmic movements by demonstration using nonlinear oscillators. In IEEE/RSJ international conference on intelligent robots and systems (pp. 958–963).Google Scholar
  16. Ijspeert, A. J., Nakanishi, J., Hoffmann, H., Pastor, P., & Schaal, S. (2013). Dynamical movement primitives: Learning attractor models for motor behaviors. Neural Computation, 25(2), 328–373.MathSciNetCrossRefzbMATHGoogle Scholar
  17. Jenkins, O. C., Mataric, M. J., & Weber, S., et al. (2000). Primitive-based movement classification for humanoid imitation. In IEEE-RAS international conference on humanoid robotics.Google Scholar
  18. Kalakrishnan, M., Righetti, L., Pastor, P., & Schaal, S. (2012). Learning force control policies for compliant robotic manipulation. In International conference on machine learning (pp. 49–50).Google Scholar
  19. Khansari-Zadeh, S. M., & Billard, A. (2010). Imitation learning of globally stable non-linear point-to-point robot motions using nonlinear programming. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2676–2683).Google Scholar
  20. Khansari-Zadeh, S. M., & Billard, A. (2011). Learning stable nonlinear dynamical systems with gaussian mixture models. IEEE Transactions on Robotics, 27(5), 943–957.CrossRefGoogle Scholar
  21. Khansari-Zadeh, S. M., & Billard, A. (2014). Learning control Lyapunov function to ensure stability of dynamical system-based robot reaching motions. Robotics and Autonomous Systems, 62(6), 752–765.CrossRefGoogle Scholar
  22. Kober, J., Bagnell, J. A., & Peters, J. (2013). Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 32(11), 1238–1274.CrossRefGoogle Scholar
  23. Koenig, N., & Matarić, M. J. (2016). Robot life-long task learning from human demonstrations: A bayesian approach. Autonomous Robots, 40(6), 1–16.Google Scholar
  24. Langsfeld, J. D., Kaipa, K. N., Gentili, R. J., Reggia, J. A., & Gupta, S. K. (2014). Incorporating failure-to-success transitions in imitation learning for a dynamic pouring task. In IEEE/RSJ international conference on intelligent robots and systems.Google Scholar
  25. Laumond, J. P., Mansard, N., & Lasserre, J. B. (2014). Optimality in robot motion: Optimal versus optimized motion. Communications of the ACM, 57(9), 82–89.CrossRefGoogle Scholar
  26. Laumond, J. P., Mansard, N., & Lasserre, J. B. (2015). Optimization as motion selection principle in robot action. Communications of the ACM, 58(5), 64–74.CrossRefGoogle Scholar
  27. Lemme, A., Neumann, K., Reinhart, R. F., & Steil, J. J. (2013). Neurally imprinted stable vector fields. In European symposium on artificial neural networks (pp. 327–332).Google Scholar
  28. Lohmiller, W., & Slotine, J. J. E. (1998). On contraction analysis for nonlinear systems. Automatica, 34(6), 683–696.MathSciNetCrossRefzbMATHGoogle Scholar
  29. Nemec, B., Tamo, M., Worgotter, F., & Ude, A. (2009). Task adaptation through exploration and action sequencing. In IEEE-RAS international conference on humanoid robots (pp. 610–616).Google Scholar
  30. Neumann, K., & Steil, J. J. (2015). Learning robot motions with stable dynamical systems under diffeomorphic transformations. Robotics and Autonomous Systems, 70, 1–15.CrossRefGoogle Scholar
  31. Paraschos, A., Daniel, C., Peters, J. R., & Neumann, G. (2013). Probabilistic movement primitives. In Advances in neural information processing systems (pp. 2616–2624).Google Scholar
  32. Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009). Learning and generalization of motor skills by learning from demonstration. In IEEE international conference on robotics and automation (pp. 763–768).Google Scholar
  33. Peters, J., & Schaal, S. (2008). Natural actor-critic. Neurocomputing, 71(7), 1180–1190.CrossRefGoogle Scholar
  34. Priess, M. C., Choi, J., & Radcliffe, C. (2014). The inverse problem of continuous-time linear quadratic gaussian control with application to biological systems analysis. In ASME 2014 dynamic systems and control conference.Google Scholar
  35. Rai, A., Meier, F., Ijspeert, A., & Schaal, S. (2014). Learning coupling terms for obstacle avoidance. In International conference on humanoid robotics (pp. 512–518).Google Scholar
  36. Ravichandar, H., & Dani, A. P. (2015). Learning contracting nonlinear dynamics from human demonstration for robot motion planning. In ASME dynamic systems and control conference (DSCC).Google Scholar
  37. Ravichandar, H., & Dani, A. P. (2016). Human modeling for bio-inspired robotics. In Intention Inference for human–robot collaboration in assistive robotics (pp. 217–249). Elsevier.Google Scholar
  38. Ravichandar, H., Salehi, I., & Dani, A. (2017). Learning partially contracting dynamical systems from demonstrations. In Proceedings of the 1st annual conference on robot learning (Vol. 78, pp. 369–378). PMLR.Google Scholar
  39. Ravichandar, H., Thota, P. K., & Dani, A. P. (2016). Learning periodic motions from human demonstrations using transverse contraction analysis. In American control conference (ACC) (pp. 4853–4858). IEEE.Google Scholar
  40. Rossano, G. F., Martinez, C., Hedelind, M., Murphy, S., & Fuhlbrigge, T. A. (2013). Easy robot programming concepts: An industrial perspective. In IEEE international conference on automation science and engineering (CASE) (pp. 1119–1126).Google Scholar
  41. Saunders, J., Nehaniv, C. L., & Dautenhahn, K. (2006). Teaching robots by moulding behavior and scaffolding the environment. In ACM SIGCHI/SIGART conference on human–robot interaction (pp. 118–125).Google Scholar
  42. Schaal, S. (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3(6), 233–242.CrossRefGoogle Scholar
  43. Schaal, S., Mohajerian, P., & Ijspeert, A. (2007). Dynamics systems vs. optimal control-a unifying view. Progress in Brain Research, 165, 425–445.CrossRefGoogle Scholar
  44. Silvério, J., Rozo, L., Calinon, S., & Caldwell, D. G. (2015). Learning bimanual end-effector poses from demonstrations using task-parameterized dynamical systems. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 464–470).Google Scholar
  45. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction (Vol. 1). Cambridge: MIT Press.zbMATHGoogle Scholar
  46. Todorov, E., & Jordan, M. I. (2002). Optimal feedback control as a theory of motor coordination. Nature Neuroscience, 5(11), 1226–1235.CrossRefGoogle Scholar
  47. Ude, A., Nemec, B., Petrić, T., & Morimoto, J. (2014). Orientation in cartesian space dynamic movement primitives. In IEEE international conference on robotics and automation (ICRA) (pp. 2997–3004).Google Scholar
  48. Ude, A. (1999). Filtering in a unit quaternion space for model-based object tracking. Robotics and Autonomous Systems, 28(2), 163–172.CrossRefGoogle Scholar
  49. Ude, A., Gams, A., Asfour, T., & Morimoto, J. (2010). Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Transactions on Robotics, 26(5), 800–815.CrossRefGoogle Scholar
  50. Wang, C., Zhao, Y., Lin, C. Y., & Tomizuka, M. (2014). Fast planning of well conditioned trajectories for model learning. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 1460–1465).Google Scholar
  51. Wang, W., & Slotine, J. J. E. (2005). On partial contraction analysis for coupled nonlinear oscillators. Biological Cybernetics, 92(1), 38–53.MathSciNetCrossRefzbMATHGoogle Scholar
  52. Zucker, M., Ratliff, N., Dragan, A. D., Pivtoraiko, M., Klingensmith, M., Dellin, C. M., et al. (2013). CHOMP: Covariant hamiltonian optimization for motion planning. The International Journal of Robotics Research, 32(9–10), 1164–1193.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringUniversity of ConnecticutStorrsUSA
  2. 2.Department of Electrical and Computer Engineering Management and Engineering for Manufacturing ProgramUniversity of ConnecticutStorrsUSA

Personalised recommendations