Autonomous Robots

, Volume 27, Issue 1, pp 3–23 | Cite as

On-line learning and modulation of periodic movements with nonlinear dynamical systems

  • Andrej GamsEmail author
  • Auke J. Ijspeert
  • Stefan Schaal
  • Jadran Lenarčič


The paper presents a two-layered system for (1) learning and encoding a periodic signal without any knowledge on its frequency and waveform, and (2) modulating the learned periodic trajectory in response to external events. The system is used to learn periodic tasks on a humanoid HOAP-2 robot. The first layer of the system is a dynamical system responsible for extracting the fundamental frequency of the input signal, based on adaptive frequency oscillators. The second layer is a dynamical system responsible for learning of the waveform based on a built-in learning algorithm. By combining the two dynamical systems into one system we can rapidly teach new trajectories to robots without any knowledge of the frequency of the demonstration signal. The system extracts and learns only one period of the demonstration signal. Furthermore, the trajectories are robust to perturbations and can be modulated to cope with a dynamic environment. The system is computationally inexpensive, works on-line for any periodic signal, requires no additional signal processing to determine the frequency of the input signal and can be applied in parallel to multiple dimensions. Additionally, it can adapt to changes in frequency and shape, e.g. to non-stationary signals, such as hand-generated signals and human demonstrations.


Learning by imitation Adaptive frequency oscillators Dynamic movement primitives Modulating learned trajectory 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Buchli, J., & Ijspeert, A. (2004). A simple, adaptive locomotion toy-system. In S. Schaal, A. Ijspeert, A. Billard, S. Vijayakumar, J. Hallam, & J. Meyer (Eds.), From animals to animats 8. Proceedings of the eighth international conference on the simulation of adaptive behavior (SAB’04) (pp. 153–162). Cambridge: MIT. Google Scholar
  2. Buchli, J., Righetti, L., & Ijspeert, A. (2006). Engineering entrainment and adaptation in limit cycle systems—from biological inspiration to applications in robotics. Biological Cybernetics, 95(6), 645–664. zbMATHCrossRefMathSciNetGoogle Scholar
  3. Bullock, D., & Grossberg, S. (1989). VITE and FLETE: neural modules for trajectory formation and postural control. In W. Hersberger (Ed.), Volitional control (pp. 253–297). Amsterdam: Elsevier. CrossRefGoogle Scholar
  4. Calinon, S., Guenter, F., & Billard, A. (2007). On learning, representing and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man and Cybernetics, Part B, Special issue on robot learning by observation, demonstration and imitation, 37(2), 286–298. Google Scholar
  5. Crespi, A., Badertscher, A., & Guignard, A. (2005). AmphiBot I: An amphibious snake-like robot. Robotics and Autonomous Systems, 50(4), 163–175. CrossRefGoogle Scholar
  6. Drumwright, E., Jenkins, O. C., & Mataric, M. (2004). Exemplar-based primitives for humanoid movement classification and control. In Proceedings of the 2004 IEEE international conference on robotics and automation (ICRA 2004), pp. 140–145. Google Scholar
  7. Fod, A., Mataric, M., & Jenkins, O. C. (2002). Automated derivation of primitives for movement classification. Autonomous Robots, 12(1), 39–54. zbMATHCrossRefGoogle Scholar
  8. Gams, A., Zlajpah, L., & Lenarcic, J. (2007). Imitating human acceleration of a gyroscopic device. Robotica, 25(4), 501–509. CrossRefGoogle Scholar
  9. Grimes, D., Rashid, D., & Rao, R. (2006). Learning nonparametric models for probabilistic imitation. In NIPS, pp. 521–528. Google Scholar
  10. Guenter, F., Hersch, M., Calinon, S., & Billard, A. (2007). Reinforcement learning for imitating constrained reaching movements. RSJ Advanced Robotics, Special Issue on Imitative Robots, 21(13), 1521–1544. Google Scholar
  11. Harris, C. M., & Wolpert, D. M. (1998). Signal-dependent noise determines motor planning. Nature, 394(6695), 780–784. doi: 10.1038/29528. CrossRefGoogle Scholar
  12. Hersch, M., Guenter, F., Calinon, S., & Billard, A. (2008). Dynamical system modulation for robot learning via kinesthetic demonstrations. IEEE Transactions on Robotics. Google Scholar
  13. Ijspeert, A., Hallam, J., & Willshaw, D. (1999). Evolving swimming controllers for a simulated lamprey with inspiration from neurobiology. Adaptive Behavior, 7(2), 151–172. CrossRefGoogle Scholar
  14. Ijspeert, A., Nakanishi, J. & Schaal, S. (2002a). Learning attractor landscapes for learning motor primitives. In Advances in neural information processing systems 15 (NIPS2002). Google Scholar
  15. Ijspeert, A., Nakanishi, J., & Schaal, S. (2002b). Learning rhythmic movements by demonstration using nonlinear oscillators. In Proceedings of the IEEE/RSJ Int. Conference on Intelligent Robots and Systems (IROS2002) (pp. 958–963). Google Scholar
  16. Ijspeert, A., Nakanishi, J., & Schaal, S. (2002c). Movement imitation with nonlinear dynamical systems in humanoid robots. In IEEE International Conference on Robotics and Automation (ICRA2002) (pp. 1398–1403). Google Scholar
  17. Ijspeert, A. J. (2008). Central pattern generators for locomotion control in animals and robots: a review. Neural Networks, 21(4), 642–653. CrossRefGoogle Scholar
  18. Inamura, T., Toshima, I., Tanie, H., & Nakamura, Y. (2004). Embodied symbol emergence based on mimesis theory. International Journal of Robotic Research, 23(4–5), 363–377. CrossRefGoogle Scholar
  19. Kawamura, S., & Fukao, N. (1994). Interpolation for input torque patterns obtained through learning control. In Proceedings of the third international conference on automation, robotics and computer vision (ICARCV’94). Google Scholar
  20. Kawato, M. (1996). Trajectory formation in arm movements: minimization principles and procedures. In H. Zelaznik (Ed.), Advance in motor learning and control (pp. 225–259). Champaign: Human Kinetics Publisher. Google Scholar
  21. Khatib, O. (1986). Real-time obstacle avoidance for manipulators and mobile robots. International Journal of Robotic Research, 5(1), 90–98. CrossRefMathSciNetGoogle Scholar
  22. Li, P., & Horowitz, R. (1999). Passive velocity field control of mechanical manipulators. IEEE Transactions on Robotics and Automation, 15(4), 751–763. CrossRefGoogle Scholar
  23. Ljung, L., & Söderström, T. (1986). Theory and Practice of Recursive Identification. Cambridge: MIT. Google Scholar
  24. Maass, W., Natschläger, T., & Markram, H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11), 2531–2560. zbMATHCrossRefGoogle Scholar
  25. Mataric, M. (1998). Behavior-based robotics as a tool for synthesis of artificial behavior and analysis of natural behavior. Trends in Cognitive Science, 2(3), 82–87. CrossRefGoogle Scholar
  26. Mezger, J. W. I., & Giese, M. (2005). Trajectory synthesis by hierarchical spatio-temporal correspondence: comparison of different methods. In APGV ’05: Proceedings of the 2nd symposium on Applied perception in graphics and visualization (pp. 25–32). New York: ACM. CrossRefGoogle Scholar
  27. Miyamoto, H., Schaal, S., Gandolfo, F., Koike, Y., Osu, R., Nakano, E., Wada, Y., & Kawato, M. (1996). A Kendama learning robot based on bi-directional theory. Neural Networks, 9, 1281–1302. CrossRefGoogle Scholar
  28. Mussa-Ivaldi, F. (1997). Nonlinear force fields: a distributed system of control primitives for representing and learning movements. In IEEE International Symposium on Computational Intelligence in Robotics and Automation (pp. 84–90). Los Alamitos: IEEE, Computer Society. Google Scholar
  29. Nishii, J. (1998). A learning model for oscillatory networks. Neural Networks, 11(2), 249–257. CrossRefMathSciNetGoogle Scholar
  30. Okada, M., Tatani, K., & Nakamura, Y. (2002). Polynomial design of the nonlinear dynamics for the brain-like information processing of whole body motion. In Proceedings of ICRA 2002 (Vol. 2, pp. 1410–1415). IEEE. Google Scholar
  31. Paine, R. W., & Tani, J. (2004). Motor primitive and sequence self-organization in a hierarchical recurrent neural network. Neural Networks, 17, 1291–1309. CrossRefGoogle Scholar
  32. Righetti, L., & Ijspeert, A. (2006). Programmable central pattern generators: an application to biped locomotion control. In Proceedings of the 2006 IEEE international conference on robotics and automation (pp. 1585–1590). Google Scholar
  33. Righetti, L., Buchli, J., & Ijspeert, A. (2006). Dynamic Hebbian learning in adaptive frequency oscillators. Physica D, 216(2), 269–281. zbMATHCrossRefMathSciNetGoogle Scholar
  34. Rohrer, B., & Hogan, N. (2003). Avoiding spurious submovement decompositions: a globally optimal algorithm. Biological Cybernetics, 89(3), 190–199. zbMATHCrossRefGoogle Scholar
  35. Schaal, S. (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3, 233–242. CrossRefGoogle Scholar
  36. Schaal, S., & Atkeson, C. (1998). Constructive incremental learning from only local information. Neural Computation, 10, 2047–2084. CrossRefGoogle Scholar
  37. Schaal, S., Mohajerian, P., & Ijspeert, A. (2007). Dynamics systems vs. optimal control a unifying view. Progress in Brain Research, 165, 425–445. CrossRefGoogle Scholar
  38. Simard, P., & LeCun, Y. (1991). Reverse TDNN: an architecture for trajectory generation. In NIPS (pp. 579–588). Google Scholar
  39. Tani, J., & Ito, M. (2003). Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment. IEEE Transactions on Systems, Man, and Cybernetics, 33(4), 481–488. CrossRefGoogle Scholar
  40. Tsuji, T., Tanaka, Y., Morasso, P., Sanguineti, V., & Kaneko, M. (2002). Bio-mimetic trajectory generation of robots via artificial potential field with time base generator. Systems, Man and Cybernetics, Part C, IEEE Transactions on, 32(4), 426–439. CrossRefGoogle Scholar
  41. Ude, A., Riley, M., Nemec, B., Kos, A., Asfour, T., & Cheng, G. (2007). Synthesizing goal-directed actions from a library of example movements. In IEEE-RAS/RSJ international conference on humanoid robots (Humanoids 2007). Google Scholar
  42. Zegers, P., & Sundareshan, M. (2003). Trajectory generation and modulation using dynamic neural networks. IEEE Transactions on Neural Networks, 14(3), 520–533. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Andrej Gams
    • 1
    Email author
  • Auke J. Ijspeert
    • 2
  • Stefan Schaal
    • 3
  • Jadran Lenarčič
    • 1
  1. 1.“Jožef Stefan” InstituteLjubljanaSlovenia
  2. 2.School of Computer and Communication SciencesEPFL—École Polytechnique Fédérale de LausanneLausanneSwitzerland
  3. 3.Computational Learning and Motor Control LaboratoryUniversity of Southern CaliforniaLos AngelesUSA

Personalised recommendations