Autonomous Robots

, Volume 41, Issue 1, pp 45–69 | Cite as

Learning potential functions from human demonstrations with encapsulated dynamic and compliant behaviors

  • Seyed Mohammad Khansari-ZadehEmail author
  • Oussama Khatib


We consider the problem of devising a unified control policy capable of regulating both the robot motion and its physical interaction with the environment. We formulate this control policy by a non-parametric potential function and a dissipative field, which both can be learned from human demonstrations. We show that the robot motion and its stiffness behaviors can be encapsulated by the potential function’s gradient and curvature, respectively. The dissipative field can also be used to model desired damping behavior throughout the motion, hence generating motions that follows the same velocity profile as the demonstrations. The proposed controller can be realized as a unification approach between “realtime motion generation” and “variable impedance control”, with the advantages that it has guaranteed stability as well as does not rely on following a reference trajectory. Our approach, called unified motion and variable impedance control (UMIC), is completely time-invariant and can be learned from a few demonstrations via solving two (convex) constrained quadratic optimization problems. We validate UMIC on a library of 30 human handwriting motions and on a set of experiments on 7-DoF KUKA light weight robot.


Potential field Variable impedance control Compliant control Robot learning Physical interaction control Motion control Imitation learning Motion primitives 



Mohammad Khansari is supported by the Swiss National Science Foundation.

Supplementary material

Supplementary material 1 (mp4 22815 KB)


  1. Billard, A., Calinon, S., Dillmann, R., & Schaal, S. (2008). Robot programming by demonstration. In Handbook of Robotics (pp. 1371–1394). Berlin/Heidelberg: Springer.Google Scholar
  2. Brock, O., Kuffner, J., & Xiao, J. (2008). Handbook of robotics. In Motion for Manipulation Tasks (pp. 615–645). Berlin/Heidelberg: Springer. doi: 10.1007/978-3-540-30301-5_6.
  3. Buchli, J., Stulp, F., Theodorou, E., & Schaal, S. (2011). Learning variable impedance control. The International Journal of Robotics Research, 30(7), 820–833.CrossRefGoogle Scholar
  4. Calinon, S., D’halluin, F., Sauser, E. L., Caldwell, D. G., & Billard, A. G. (2010a). Learning and reproduction of gestures by imitation: An approach based on Hidden Markov Model and Gaussian Mixture Regression. IEEE Robotics and Automation Magazine, 17(2), 44–54.CrossRefGoogle Scholar
  5. Calinon, S., Sardellitti, I., & Caldwell, D. G. (2010b). Learning-based control strategy for safe human-robot interaction exploiting task and robot redundancies. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 249–254).Google Scholar
  6. Calinon, S., Pistillo, A., & Caldwell, D. G. (2011). Encoding the time and space constraints of a task in explicit-duration hidden Markov model. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 3413–3418).Google Scholar
  7. Cohen, M., & Flash, T. (1991). Learning impedance parameters for robot control using an associative search network. IEEE Transactions on Robotics and Automation, 7(3), 382–390. doi: 10.1109/70.88148.CrossRefGoogle Scholar
  8. Ferraguti, F., Secchi, C., & Fantuzzi, C. (2013). A tank-based approach to impedance control with variable stiffness. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (pp. 4948–4953).Google Scholar
  9. Ganesh, G., Jarrasse, N., Haddadin, S., Albu-Schaeffer, A., & Burdet, E. (2012). A versatile biomimetic controller for contact tooling and haptic exploration. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (pp. 3329–3334).Google Scholar
  10. Gomez, J. V., Alvarez, D., Garrido, S., & Moreno, L. (2012). Kinesthetic teaching via fast marching square. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1305–1310).Google Scholar
  11. Gribovskaya, E. (2010). Seyed Mohammad Khansari-Zadeh, and Aude Billard. Learning Nonlinear Multivariate Dynamics of Motion in Robotic Manipulators. The International Journal of Robotics Research, 30, 1–37.Google Scholar
  12. Haddadin, S., Albu-Schaffer, A., De Luca, A., & Hirzinger, G. (2008). Collision detection and reaction: A contribution to safe physical human-robot interaction. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008. IROS 2008 (pp. 3356–3363).Google Scholar
  13. Hogan, N. (1985). Impedance control: An approach to manipulation. ASME Journal of Dynamic Systems, Measurement, and Control, 107.Google Scholar
  14. Hogan, N., & Buerger, S. P. (2005). Robotics and Automation Handbook, Impedance and Interaction Control. Boca Raton, FL: CRC.Google Scholar
  15. Howard, M., Braun, D. J., & Vijayakumar, S. (2013). Transferring human impedance behavior to heterogeneous variable impedance actuators. IEEE Transactions on Robotics, 29(4), 847–862. doi: 10.1109/TRO.2013.2256311.CrossRefGoogle Scholar
  16. Howard, M., Klanke, S., Gienger, M., Goerick, C., & Vijayakumar, S. (2010). Methods for learning control policies from variable-constraint demonstrations. In From Motor Learning to Interaction Learning in Robots (vol. 264, pp. 253–291). Berlin/Heidelberg: Springer.Google Scholar
  17. Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2002). Movement imitation with nonlinear dynamical systems in humanoid robots. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (pp. 1398–1403).Google Scholar
  18. Kavraki, L. E., & LaValle, S. M. (2008). Handbook of Robotics, Incollection Motion Planning. Berlin/Heidelberg: Springer. doi: 10.1007/978-3-540-30301-5_6.Google Scholar
  19. Khansari-Zadeh, S. M. (2011). Lasa human handwriting library.
  20. Khansari-Zadeh, S. M. (2012). A Dynamical system-based approach to modeling stable robot control policies via imitation learning. Phd Thesis, cole Polytechnique Fdrale de Lausanne.
  21. Khansari-Zadeh, S. M., & Billard, A. (2011). Learning stable nonlinear dynamical systems with Gaussian mixture models. IEEE Transactions on Robotics, 27(5), 943–957. doi: 10.1109/TRO.2011.2159412. ISSN 1552-3098.CrossRefGoogle Scholar
  22. Khansari-Zadeh, S. M., & Billard, A. (2012). A dynamical system approach to realtime obstacle avoidance. Autonomous Robots, 32, 433–454. ISSN 0929-5593.CrossRefGoogle Scholar
  23. Khansari-Zadeh, S. M., & Billard, A. (2014). Learning control lyapunov function to ensure stability of dynamical system-based robot reaching motions. Robotics and Autonomous Systems, 62(6), 752–765.CrossRefGoogle Scholar
  24. Khansari-Zadeh, S. M., Lemme, A., Meirovitch, Y., Schrauwen, B., Giese, M. A., Steil, J., Ijspeert, A. J., & Billard, A. (2013). Benchmarking of state-of-the-art algorithms in generating human-like robot reaching motions. In Workshop at the IEEE-RAS International Conference on Humanoid Robots (Humanoids).
  25. Khansari-Zadeh, S. M., Kronander, K., & Billard, A. (2014). Modeling robot discrete movements with state-varying stiffness and damping: A framework for integrated motion generation and impedance control. In Proceedings of Robotics: Science and Systems X (RSS 2014). Berkeley, California.Google Scholar
  26. Khatib, O. (1986). Real-time obstacle avoidance for manipulators and mobile robots. International Journal of Robotics Research, 5, 90–98.CrossRefGoogle Scholar
  27. Khatib, O. (1987). A unified approach for motion and force control of robot manipulators: The operational space formulation. IEEE Journal of Robotics and Automation, 3, 43–53.CrossRefGoogle Scholar
  28. Khatib, O. (1995). Inertial properties in robotic manipulation: An object-level framework. The International Journal of Robotics Research, 14(1), 19–36.CrossRefGoogle Scholar
  29. Khatib, O., Sentis, L., & Park, J.-H. (2008). A unified framework for whole-body humanoid robot control with multiple constraints and contacts. In European Robotics Symposium 2008, volume 44 of Springer Tracts in Advanced Robotics (pp. 303–312). Berlin/Heidelberg: Springer.Google Scholar
  30. Kim, B., Park, J., Park, S., & Kang, S. (2010). Impedance learning for robotic contact tasks using natural actor-critic algorithm. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 40(2), 433–443. doi: 10.1109/TSMCB.2009.2026289.CrossRefGoogle Scholar
  31. Kim, J.-O., & Khosla, P. K. (1992). Real-time obstacle avoidance using harmonic potential functions. IEEE Transactions on Robotics and Automation, 8(3), 338–349.CrossRefGoogle Scholar
  32. Kishi, Y., Yamada, Y., & Yokoyama, K. (2012). The role of joint stiffness enhancing collision reaction performance of collaborative robot manipulators. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 376–381).Google Scholar
  33. Koditschek, D. (1989). Robot Planning and Control Via Potential Functions (pp. 349–367).Google Scholar
  34. Kormushev, P., Calinon, S., & Caldwell, D. G. (2010). Robot motor skill coordination with EM-based reinforcement learning. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 3232–3237). Taipei, Taiwan.Google Scholar
  35. Kronander, K., & Billard, A. (2013). Learning compliant manipulation through kinesthetic and tactile human–robot interaction. IEEE Transactions on Haptics, 7(3), 367–380.CrossRefGoogle Scholar
  36. Lee, K., & Buss, M. (2008). Force tracking impedance control with variable target stiffness. In Proceedings of the International Federation of Automatic Control World Congress (pp. 6751–6756).Google Scholar
  37. Li, M., Yin, H., Tahara, K., & Billard, A. (2014). Learning object-level impedance control for robust grasping and dexterous manipulation. In 2014 IEEE International Conference on Robotics and Automation (ICRA) (pp. 6784–6791).Google Scholar
  38. Mattingley, J., & Boyd, S. (2012). Cvxgen: A code generator for embedded convex optimization. Optimization and Engineering, 13(1), 1–27. ISSN 1389-4420.CrossRefzbMATHMathSciNetGoogle Scholar
  39. Mitrovic, D., Klanke, S., & Vijayakumar, S. (2011). Learning impedance control of antagonistic systems based on stochastic optimization principles. The International Journal of Robotics Research, 30(5), 556–573.CrossRefGoogle Scholar
  40. Muelling, K., Kober, J., Kroemer, O., & Peters, J. (2013). Learning to select and generalize striking movements in robot table tennis. International Journal of Robotics Research, 32, 263–279.CrossRefGoogle Scholar
  41. Ott, C. (2008). Cartesian Impedance Control of Redundant and Flexible-Joint Robots. Springer Tracts in Advanced Robotics.Google Scholar
  42. Pipe, A. G. (2000). An architecture for learning “potential field” cognitive maps with an application to mobile robotics. Adaptive Behavior, 8(2), 173–203. doi: 10.1177/105971230000800205.CrossRefGoogle Scholar
  43. Rimon, E., & Koditschek, D. E. (1992). Exact robot navigation using artificial potential functions. IEEE Transactions on Robotics and Automation, 8(5), 501–518. doi: 10.1109/70.163777.CrossRefGoogle Scholar
  44. Schaal, S. (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3(6), 233–242.CrossRefGoogle Scholar
  45. Siciliano, B., Sciavicco, L., Villani, L., & Oriolo, G. (2009). Robotics: Modelling, Planning and Control. Advanced Textbooks in Control and Signal Processing. Springer.Google Scholar
  46. Slotine, J. J. E., & Li, W. (1991). Applied Nonlinear Control. Englewood Cliffs: Prentice-Hall.zbMATHGoogle Scholar
  47. Stulp, F., Buchli, J., Ellmer, A., Mistry, M., Theodorou, E., & Schaal, S. (2012). Model-free reinforcement learning of impedance control in stochastic environments. IEEE Transactions on Autonomous Mental Development, 4(4), 330–341.CrossRefGoogle Scholar
  48. Ude, A., Gams, A., Asfour, T., & Morimoto, J. (2010). Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Transactions on Robotics, 26(5), 800–815.CrossRefGoogle Scholar
  49. Villani, L., & De Schutter, J. (2008). Handbook of Robotics, Force Control (pp. 161–185). Berlin/Heidelberg: Springer.Google Scholar
  50. Wolf, S., & Hirzinger, G. (2008). A new variable stiffness design: Matching requirements of the next robot generation. In IEEE International Conference on Robotics and Automation, 2008. ICRA 2008 (pp. 1741–1746). doi: 10.1109/ROBOT.2008.4543452.
  51. Zinn, M., Khatib, O., Roth, B., & Salisbury, J. K. (2004). Playing it safe (human-friendly robots). IEEE Robotics Automation Magazine, 11(2), 12–21. doi: 10.1109/MRA.2004.1310938. ISSN 1070-9932.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Seyed Mohammad Khansari-Zadeh
    • 1
    Email author
  • Oussama Khatib
    • 1
  1. 1.Computer Science DepartmentStanford UniversityStanfordUSA

Personalised recommendations