Adaptive Optimal Control for Redundantly Actuated Arms

  • Djordje Mitrovic
  • Stefan Klanke
  • Sethu Vijayakumar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5040)


Optimal feedback control has been proposed as an attractive movement generation strategy in goal reaching tasks for anthropomorphic manipulator systems. Recent developments, such as the iterative Linear Quadratic Gaussian (iLQG) algorithm, have focused on the case of non-linear, but still analytically available, dynamics. For realistic control systems, however, the dynamics may often be unknown, difficult to estimate, or subject to frequent systematic changes. In this paper, we combine the iLQG framework with learning the forward dynamics for a simulated arm with two limbs and six antagonistic muscles, and we demonstrate how our approach can compensate for complex dynamic perturbations in an online fashion.


Adaptive optimal control learning dynamics redundant actuation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Stengel, R.F.: Optimal control and estimation. Dover Publications, New York (1994)MATHGoogle Scholar
  2. 2.
    Flash, T., Hogan, N.: The coordination of arm movements: an experimentally confirmed mathematical model. Journal of Neuroscience 5, 1688–1703 (1985)Google Scholar
  3. 3.
    Todorov, E., Jordan, M.: A minimal intervention principle for coordinated movement. In: Advances in Neural Information Processing Systems, vol. 15, pp. 27–34. MIT Press, Cambridge (2003)Google Scholar
  4. 4.
    Shadmehr, R., Wise, S.P.: The Computational Neurobiology of Reaching and Ponting. MIT Press, Cambridge (2005)Google Scholar
  5. 5.
    Li, W.: Optimal Control for Biological Movement Systems. PhD dissertation, University of California, San Diego (2006)Google Scholar
  6. 6.
    Scott, S.H.: Optimal feedback control and the neural basis of volitional motor control. Nature Reviews Neuroscience 5, 532–546 (2004)CrossRefGoogle Scholar
  7. 7.
    Dyer, P., McReynolds, S.: The Computational Theory of Optimal Control. Academic Press, New York (1970)Google Scholar
  8. 8.
    Jacobson, D.H., Mayne, D.Q.: Differential Dynamic Programming. Elsevier, New York (1970)MATHGoogle Scholar
  9. 9.
    Li, W., Todorov, E.: Iterative linear-quadratic regulator design for nonlinear biological movement systems. In: Proc. 1st Int. Conf. Informatics in Control, Automation and Robotics (2004)Google Scholar
  10. 10.
    Todorov, E., Li, W.: A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems. In: Proc. of the American Control Conference (2005)Google Scholar
  11. 11.
    Atkeson, C.G., Schaal, S.: Learning tasks from a single demonstration. In: Proc. Int. Conf. on Robotics and Automation (ICRA), Albuquerque, New Mexico, vol. 2, pp. 1706–1712 (1997)Google Scholar
  12. 12.
    Abbeel, P., Quigley, M., Ng, A.Y.: Using inaccurate models in reinforcement learning. In: Proc. Int. Conf. on Machine Learning, pp. 1–8 (2006)Google Scholar
  13. 13.
    Katayama, M., Kawato, M.: Virtual trajectory and stiffness ellipse during multijoint arm movement predicted by neural inverse model. Biol. Cybern. 69, 353–362 (1993)MATHGoogle Scholar
  14. 14.
    Corke, P.I.: A robotics toolbox for MATLAB. IEEE Robotics and Automation Magazine 3(1), 24–32 (1996)CrossRefGoogle Scholar
  15. 15.
    Özkaya, N., Nordin, M.: Fundamentals of biomechanics: equilibrium, motion, and deformation. Van Nostrand Reinhold, New York (1991)Google Scholar
  16. 16.
    Bertsekas, D.P.: Dynamic programming and optimal control. Athena Scientific, Belmont, Mass (1995)MATHGoogle Scholar
  17. 17.
    Thrun, S.: Monte carlo POMDPs. In: Advances in Neural Information Processing Systems 12, pp. 1064–1070. MIT Press, Cambridge (2000)Google Scholar
  18. 18.
    Atkeson, C.G.: Randomly sampling actions in dynamic programming. In: Proc. Int. Symp. on Approximate Dynamic Programming and Reinforcement Learning, pp. 185–192 (2007)Google Scholar
  19. 19.
    Vijayakumar, S., D’Souza, A., Schaal, S.: Incremental online learning in high dimensions. Neural Computation 17, 2602–2634 (2005)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Shadmehr, R., Mussa-Ivaldi, F.A.: Adaptive representation of dynamics during learning of a motor task. The Journal of Neurosciene 14(5), 3208–3224 (1994)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Djordje Mitrovic
    • 1
  • Stefan Klanke
    • 1
  • Sethu Vijayakumar
    • 1
  1. 1.Institute of Perception, Action & BehaviorUniversity of EdinburghEdinburghUnited Kingdom

Personalised recommendations