Creating Brain-Like Intelligence pp 151-191

Part of the Lecture Notes in Computer Science book series (LNCS, volume 5436) | Cite as

Planning and Moving in Dynamic Environments

A Statistical Machine Learning Approach
  • Sethu Vijayakumar
  • Marc Toussaint
  • Giorgios Petkos
  • Matthew Howard

Abstract

In this chapter, we develop a new view on problems of movement control and planning from a Machine Learning perspective. In this view, decision making, control, and planning are all considered as an inference or (alternately) an information processing problem, i.e., a problem of computing a posterior distribution over unknown variables conditioned on the available information (targets, goals, constraints). Further, problems of adaptation and learning are formulated as statistical learning problems to model the dependencies between variables. This approach naturally extends to cases when information is missing, e.g., when the context or load needs to be inferred from interaction; or to the case of apprentice learning where, crucially, latent properties of the observed behavior are learnt rather than the motion copied directly.

With this account, we hope to address the long-standing problem of designing adaptive control and planning systems that can flexibly be coupled to multiple sources of information (be they of purely sensory nature or higher-level modulations such as task and constraint information) and equally formulated on any level of abstraction (motor control variables or symbolic representations). Recent advances in Machine Learning provide a coherent framework for these problems.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Peters, J., Mistry, M., Udwadia, F.E., Cory, R., Nakanishi, J., Schaal, S.: A unifying framework for the control of robotics systems. In: IEEE Int. Conf. on Intelligent Robots and Systems (IROS 2005), pp. 1824–1831 (2005)Google Scholar
  2. 2.
    Nakamura, Y., Hanafusa, H.: Inverse kinematic solutions with singularity robustness for robot manipulator control. Journal of Dynamic Systems, Measurement and Control 108 (1986)Google Scholar
  3. 3.
    Baerlocher, P., Boulic, R.: An inverse kinematic architecture enforcing an arbitrary number of strict priority levels. The Visual Computer (2004)Google Scholar
  4. 4.
    Todorov, E.: Optimal control theory. In: Doya, K. (ed.) Bayesian Brain: Probabilistic Approaches to Neural Coding, pp. 269–298. MIT Press, Cambridge (2006)Google Scholar
  5. 5.
    Platt, R., Fagg, A., Grupen, R.: Nullspace composition of control laws for grasping. In: Proceedings of the IEEE-RSJ Int. Conf. on Intelligent Robots and Systems, Lausanne, Switzerland (2002)Google Scholar
  6. 6.
    Ijspeert, A.J., Nakanishi, J., Schaal, S.: Learning attractor landscapes for learning motor primitives. In: Advances in Neural Information Processing Systems, vol. 15, pp. 1523–1530. MIT Press, Cambridge (2003)Google Scholar
  7. 7.
    Schaal, S., Peters, J., Nakanishi, J., Ijspeert, A.: Control, planning, learning, and imitation with dynamic movement primitives. In: Workshop on Bilateral Paradigms on Humans and Humanoids, IEEE Int. Conf. on Intelligent Robots and Systems, Las Vegas, NV (2003)Google Scholar
  8. 8.
    Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., S., Schaal, K.M.: Learning from demonstration and adaptation of biped locomotion with dynamical movement primitives. In: Workshop on Robot Learning by Demonstration, IEEE Int. Conf. on Intelligent Robots and Systems (2003)Google Scholar
  9. 9.
    Vijayakumar, S., D’Souza, A., Schaal, S.: Incremental online learning in high dimensions. Neural Computation 17, 2602–2634 (2005)CrossRefPubMedGoogle Scholar
  10. 10.
    Klanke, S., Vijayakumar, S., Schaal, S.: A library for locally weighted projection regression. Journal of Machine Learning Research (2008)Google Scholar
  11. 11.
    Roweis, S., Ghahramani, Z.: 6. In: Haykin, S. (ed.) Learning Nonlinear Dynamical Systems using the EM Algorithm, pp. 175–220. Wiley, Chichester (2001)Google Scholar
  12. 12.
    Briegel, T., Tresp, V.: Fisher scoring and a mixture of modes approach for approximate inference and learning in nonlinear state space models (1999)Google Scholar
  13. 13.
    de Freitas, J., Niranjan, M., Gee, A.: Nonlinear state space estimation with neural networks and the em algorithm. Technical report (1999)Google Scholar
  14. 14.
    Sciavicco, L., Siciliano, B.: Modelling and Control of Robot Manipulators. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  15. 15.
    Craig, J.J.: Introduction to Robotics: Mechanics and Control. Pearson Prentice Hall, London (2005)Google Scholar
  16. 16.
    Liégeois, A.: Automatic supervisory control of the configuration and behavior of multibody mechanisms. IEEE Trans. Systems, Man, and Cybernetics SMC-7, 245–250 (1977)Google Scholar
  17. 17.
    Khatib, O.: A unified approach for motion and force control of robot manipulators: The operational space formulation. IEEE Journal of Robotics and Automation RA-3(1), 43–53 (1987)CrossRefGoogle Scholar
  18. 18.
    Peters, J., Mistry, M., Udwadia, F.E., Nakanishi, J., Schaal, S.: A unifying framework for robot control with redundant DOFs. Autonomous Robots Journal 24, 1–12 (2008)CrossRefGoogle Scholar
  19. 19.
    Howard, M., Gienger, M., Goerick, C., Vijayakumar, S.: Learning utility surfaces for movement selection. In: IEEE International Conference on Robotics and Biomimetics (ROBIO) (2006)Google Scholar
  20. 20.
    Park, J., Khatib, O.: Contact consistent control framework for humanoid robots. In: Proc. IEEE Int. Conf. on Robotics and Automation (ICRA) (May 2006)Google Scholar
  21. 21.
    Gienger, M., Janssen, H., Goerick, C.: Task-oriented whole body motion for humanoid robots. In: 5th IEEE-RAS International Conference on Humanoid Robots, 2005, December 5, 2005, pp. 238–244 (2005)Google Scholar
  22. 22.
    Howard, M., Vijayakumar, S.: Reconstructing null-space policies subject to dynamic task constraints in redundant manipulators. In: Workshop on Robotics and Mathematics (RoboMat) (September 2007)Google Scholar
  23. 23.
    Verbeek, J.J., Roweis, S.T., Vlassis, N.: Non-linear CCA and PCA by alignment of local models. In: Advances in Neural Information Processing Systems, vol. 16. MIT Press, Cambridge (2004)Google Scholar
  24. 24.
    Schaal, S., Ijspeert, A., Billard, A.: Computational approaches to motor learning by imitation. In: The Neuroscience of Social Interaction, pp. 199–218. Oxford University Press, Oxford (2004)Google Scholar
  25. 25.
    Ijspeert, A.J., Nakanishi, J., Schaal, S.: Learning attractor landscapes for learning motor primitives. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 1523–1530. MIT Press, Cambridge (2003)Google Scholar
  26. 26.
    Ijspeert, A.J., Nakanishi, J., Schaal, S.: Movement imitation with nonlinear dynamical systems in humanoid robots. In: Proc. IEEE International Conference on Robotics and Automation (ICRA), pp. 1398–1403 (2002)Google Scholar
  27. 27.
    Grimes, D.B., Chalodhorn, R., Rao, R.P.N.: Dynamic imitation in a humanoid robot through nonparametric probabilistic inference. In: Proceedings of Robotics: Science and Systems (RSS 2006). MIT Press, Cambridge (2006)Google Scholar
  28. 28.
    Grimes, D.B., Rashid, D.R., Rao, R.P.N.: Learning nonparametric models for probabilistic imitation. In: Advances in Neural Information Processing Systems (NIPS 2006), vol. 19. MIT Press, Cambridge (2007)Google Scholar
  29. 29.
    Antonelli, G., Arrichiello, F., Chiaverini, S.: The null-space-based behavioral control for soccer-playing mobile robots. Proceedings, pp. 1257–1262 (2005)Google Scholar
  30. 30.
    Nakamura, Y.: Advanced Robotics: Redundancy and Optimization. Addison Wesley, Reading (1991)Google Scholar
  31. 31.
    Howard, M., Klanke, S., VIjayakumar, S.: Learning nullspace potentials from constrained motion. In: Proc. IEEE International Conference on Intelligent Robots and Systems (IROS) (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Sethu Vijayakumar
    • 1
  • Marc Toussaint
    • 2
  • Giorgios Petkos
    • 1
  • Matthew Howard
    • 1
  1. 1.School of InformaticsUniversity of EdinburghEdinburghUK
  2. 2.Technical University of BerlinBerlinGermany

Personalised recommendations