Learning position and orientation dynamics from demonstrations via contraction analysis

Abstract

This paper presents a unified framework of model-learning algorithms, called contracting dynamical system primitives (CDSP), that can be used to learn pose (i.e., position and orientation) dynamics of point-to-point motions from demonstrations. The position and the orientation (represented using quaternions) trajectories are modeled as two separate autonomous nonlinear dynamical systems. The special constraints of the \({\mathbb {S}}^{3}\) manifold are enforced in the formulation of the system that models the orientation dynamics. To capture the variability in the demonstrations, the dynamical systems are estimated using Gaussian mixture models (GMMs). The parameters of the GMMs are learned subject to the constraints derived using partial contraction analysis. The learned models’ reproductions are shown to accurately reproduce the demonstrations and are guaranteed to converge to the desired goal location. Experimental results illustrate the CDSP algorithm’s ability to accurately learn position and orientation dynamics and the utility of the learned models in path generation for a Baxter robot arm. The CDSP algorithm is evaluated on a publicly available dataset and a synthetic dataset, and is shown to have the lowest and comparable average reproduction errors when compared to state-of-the-art imitation learning algorithms.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Notes

  1. 1.

    Following the notation in partial contraction analysis literature (Wang and Slotine 2005), \(\varvec{x}\left( t\right) \) is written twice to represent the dependency of \(\varvec{x}\left( t\right) \) in multiple places in \(f\left( \cdot \right) \).

  2. 2.

    In \(f_{p}\left( \varvec{x}\left( t\right) ,\varvec{x}\left( t\right) \right) \), the first argument refers to the \(\varvec{x}\left( t\right) \) in \(h_{p}\left( \cdot \right) \) and the second argument refers to the \(\varvec{x}\left( t\right) \) in the affine part of \(f_{p}\left( \cdot \right) \).

  3. 3.

    In \(f_{o}\left( \varvec{\omega }\left( t\right) ,\varvec{\omega }\left( t\right) ,\varvec{q}\left( t\right) ,\varvec{q}_{g}\right) \), the first argument refers to the \(\varvec{\omega }\left( t\right) \) in \(h_{o}\left( \cdot \right) \) and the second argument refers to the \(\varvec{\omega }\left( t\right) \) in the affine part of \(f_{o}\left( \cdot \right) \).

References

  1. Abbeel, P., & Ng, A. Y., (2004). Apprenticeship learning via inverse reinforcement learning. In International conference on machine learning (pp. 1–8). ACM.

  2. Ahmadzadeh, R., Rana, M. A., & Chernova, S. (2017). Generalized cylinders for learning, reproduction, generalization, and refinement of robot skills. In Robotics: Science and systems.

  3. Akgun, B., Cakmak, M., Jiang, K., & Thomaz, A. L. (2012). Keyframe-based learning from demonstration. International Journal of Social Robotics, 4(4), 343–355.

    Article  Google Scholar 

  4. Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009). A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5), 469–483.

    Article  Google Scholar 

  5. Billard, A., & Matarić, M. J. (2001). Learning human arm movements by imitation: Evaluation of a biologically inspired connectionist architecture. Robotics and Autonomous Systems, 37(2), 145–160.

    Article  MATH  Google Scholar 

  6. Bowen, C., & Alterovitz, R. (2014). Closed-loop global motion planning for reactive execution of learned tasks. In IEEE/RSJ international conference on intelligent robots and systems (pp. 1754–1760).

  7. Calinon, S., Bruno, D., Caldwell, D. G. (2014). A task-parameterized probabilistic model with minimal intervention control. In IEEE international conference on robotics and automation (ICRA) (pp. 3339–3344).

  8. Cohn, D. A., Ghahramani, Z., & Jordan, M. I. (1996). Active learning with statistical models. Journal of Artificial Intelligence Research, 4, 129–145.

    Article  MATH  Google Scholar 

  9. Corke, P. I. (2011). Robotics, vision and control: Fundamental algorithms in Matlab. Berlin: Springer.

    Google Scholar 

  10. Dani, A. P., Chung, S. J., & Hutchinson, S. (2015). Observer design for stochastic nonlinear systems via contraction-based incremental stability. IEEE Transactions on Automatic Control, 60(3), 700–714.

    MathSciNet  Article  MATH  Google Scholar 

  11. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the E–M algorithm. Journal of the Royal Statistical Society Series B (Methodological), 39(1), 1–38.

    MathSciNet  Article  MATH  Google Scholar 

  12. Diankov, R. (2010). Automated construction of robotic manipulation programs. Ph.D. thesis, Carnegie Mellon University.

  13. Dragan, A. D., Muelling, K., Bagnell, J. A., & Srinivasa, S. S. (2015). Movement primitives via optimization. In IEEE international conference on robotics and automation (ICRA) (pp. 2339–2346).

  14. Gribovskaya, E., Khansari-Zadeh, S. M., & Billard, A. (2010). Learning non-linear multivariate dynamics of motion in robotic manipulators. The International Journal of Robotics Research, 30(1), 80–117.

    Article  Google Scholar 

  15. Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2002). Learning rhythmic movements by demonstration using nonlinear oscillators. In IEEE/RSJ international conference on intelligent robots and systems (pp. 958–963).

  16. Ijspeert, A. J., Nakanishi, J., Hoffmann, H., Pastor, P., & Schaal, S. (2013). Dynamical movement primitives: Learning attractor models for motor behaviors. Neural Computation, 25(2), 328–373.

    MathSciNet  Article  MATH  Google Scholar 

  17. Jenkins, O. C., Mataric, M. J., & Weber, S., et al. (2000). Primitive-based movement classification for humanoid imitation. In IEEE-RAS international conference on humanoid robotics.

  18. Kalakrishnan, M., Righetti, L., Pastor, P., & Schaal, S. (2012). Learning force control policies for compliant robotic manipulation. In International conference on machine learning (pp. 49–50).

  19. Khansari-Zadeh, S. M., & Billard, A. (2010). Imitation learning of globally stable non-linear point-to-point robot motions using nonlinear programming. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2676–2683).

  20. Khansari-Zadeh, S. M., & Billard, A. (2011). Learning stable nonlinear dynamical systems with gaussian mixture models. IEEE Transactions on Robotics, 27(5), 943–957.

    Article  Google Scholar 

  21. Khansari-Zadeh, S. M., & Billard, A. (2014). Learning control Lyapunov function to ensure stability of dynamical system-based robot reaching motions. Robotics and Autonomous Systems, 62(6), 752–765.

    Article  Google Scholar 

  22. Kober, J., Bagnell, J. A., & Peters, J. (2013). Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 32(11), 1238–1274.

    Article  Google Scholar 

  23. Koenig, N., & Matarić, M. J. (2016). Robot life-long task learning from human demonstrations: A bayesian approach. Autonomous Robots, 40(6), 1–16.

    Google Scholar 

  24. Langsfeld, J. D., Kaipa, K. N., Gentili, R. J., Reggia, J. A., & Gupta, S. K. (2014). Incorporating failure-to-success transitions in imitation learning for a dynamic pouring task. In IEEE/RSJ international conference on intelligent robots and systems.

  25. Laumond, J. P., Mansard, N., & Lasserre, J. B. (2014). Optimality in robot motion: Optimal versus optimized motion. Communications of the ACM, 57(9), 82–89.

    Article  Google Scholar 

  26. Laumond, J. P., Mansard, N., & Lasserre, J. B. (2015). Optimization as motion selection principle in robot action. Communications of the ACM, 58(5), 64–74.

    Article  Google Scholar 

  27. Lemme, A., Neumann, K., Reinhart, R. F., & Steil, J. J. (2013). Neurally imprinted stable vector fields. In European symposium on artificial neural networks (pp. 327–332).

  28. Lohmiller, W., & Slotine, J. J. E. (1998). On contraction analysis for nonlinear systems. Automatica, 34(6), 683–696.

    MathSciNet  Article  MATH  Google Scholar 

  29. Nemec, B., Tamo, M., Worgotter, F., & Ude, A. (2009). Task adaptation through exploration and action sequencing. In IEEE-RAS international conference on humanoid robots (pp. 610–616).

  30. Neumann, K., & Steil, J. J. (2015). Learning robot motions with stable dynamical systems under diffeomorphic transformations. Robotics and Autonomous Systems, 70, 1–15.

    Article  Google Scholar 

  31. Paraschos, A., Daniel, C., Peters, J. R., & Neumann, G. (2013). Probabilistic movement primitives. In Advances in neural information processing systems (pp. 2616–2624).

  32. Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009). Learning and generalization of motor skills by learning from demonstration. In IEEE international conference on robotics and automation (pp. 763–768).

  33. Peters, J., & Schaal, S. (2008). Natural actor-critic. Neurocomputing, 71(7), 1180–1190.

    Article  Google Scholar 

  34. Priess, M. C., Choi, J., & Radcliffe, C. (2014). The inverse problem of continuous-time linear quadratic gaussian control with application to biological systems analysis. In ASME 2014 dynamic systems and control conference.

  35. Rai, A., Meier, F., Ijspeert, A., & Schaal, S. (2014). Learning coupling terms for obstacle avoidance. In International conference on humanoid robotics (pp. 512–518).

  36. Ravichandar, H., & Dani, A. P. (2015). Learning contracting nonlinear dynamics from human demonstration for robot motion planning. In ASME dynamic systems and control conference (DSCC).

  37. Ravichandar, H., & Dani, A. P. (2016). Human modeling for bio-inspired robotics. In Intention Inference for human–robot collaboration in assistive robotics (pp. 217–249). Elsevier.

  38. Ravichandar, H., Salehi, I., & Dani, A. (2017). Learning partially contracting dynamical systems from demonstrations. In Proceedings of the 1st annual conference on robot learning (Vol. 78, pp. 369–378). PMLR.

  39. Ravichandar, H., Thota, P. K., & Dani, A. P. (2016). Learning periodic motions from human demonstrations using transverse contraction analysis. In American control conference (ACC) (pp. 4853–4858). IEEE.

  40. Rossano, G. F., Martinez, C., Hedelind, M., Murphy, S., & Fuhlbrigge, T. A. (2013). Easy robot programming concepts: An industrial perspective. In IEEE international conference on automation science and engineering (CASE) (pp. 1119–1126).

  41. Saunders, J., Nehaniv, C. L., & Dautenhahn, K. (2006). Teaching robots by moulding behavior and scaffolding the environment. In ACM SIGCHI/SIGART conference on human–robot interaction (pp. 118–125).

  42. Schaal, S. (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3(6), 233–242.

    Article  Google Scholar 

  43. Schaal, S., Mohajerian, P., & Ijspeert, A. (2007). Dynamics systems vs. optimal control-a unifying view. Progress in Brain Research, 165, 425–445.

    Article  Google Scholar 

  44. Silvério, J., Rozo, L., Calinon, S., & Caldwell, D. G. (2015). Learning bimanual end-effector poses from demonstrations using task-parameterized dynamical systems. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 464–470).

  45. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction (Vol. 1). Cambridge: MIT Press.

    Google Scholar 

  46. Todorov, E., & Jordan, M. I. (2002). Optimal feedback control as a theory of motor coordination. Nature Neuroscience, 5(11), 1226–1235.

    Article  Google Scholar 

  47. Ude, A., Nemec, B., Petrić, T., & Morimoto, J. (2014). Orientation in cartesian space dynamic movement primitives. In IEEE international conference on robotics and automation (ICRA) (pp. 2997–3004).

  48. Ude, A. (1999). Filtering in a unit quaternion space for model-based object tracking. Robotics and Autonomous Systems, 28(2), 163–172.

    Article  Google Scholar 

  49. Ude, A., Gams, A., Asfour, T., & Morimoto, J. (2010). Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Transactions on Robotics, 26(5), 800–815.

    Article  Google Scholar 

  50. Wang, C., Zhao, Y., Lin, C. Y., & Tomizuka, M. (2014). Fast planning of well conditioned trajectories for model learning. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 1460–1465).

  51. Wang, W., & Slotine, J. J. E. (2005). On partial contraction analysis for coupled nonlinear oscillators. Biological Cybernetics, 92(1), 38–53.

    MathSciNet  Article  MATH  Google Scholar 

  52. Zucker, M., Ratliff, N., Dragan, A. D., Pivtoraiko, M., Klingensmith, M., Dellin, C. M., et al. (2013). CHOMP: Covariant hamiltonian optimization for motion planning. The International Journal of Robotics Research, 32(9–10), 1164–1193.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge Klaus Neumann and Jochen Steil for providing the SEA means and standard deviations of the seven state-of-the-art algorithms used for comparison presented in Sect. 5.1. The authors would like to thank the anonymous reviewers for their valuable comments and suggestions to improve the quality of the paper.

Funding

Funding was provided by the UTC Institute for Advanced Systems Engineering (UTC-IASE) of the University of Connecticut and the United Technologies Corporation.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Ashwin Dani.

Electronic supplementary material

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ravichandar, H.c., Dani, A. Learning position and orientation dynamics from demonstrations via contraction analysis. Auton Robot 43, 897–912 (2019). https://doi.org/10.1007/s10514-018-9758-x

Download citation

Keywords

  • Learning from demonstration
  • Gaussian mixture models
  • Contraction analysis
  • Model learning