Autonomous Robots

, Volume 41, Issue 4, pp 945–966 | Cite as

Towards a learnt neural body schema for dexterous coordination of action in humanoid and industrial robots

  • Ajaz Ahmad BhatEmail author
  • Sharath Chandra Akkaladevi
  • Vishwanathan Mohan
  • Christian Eitzinger
  • Pietro Morasso


During any goal oriented behavior the dual processes of generation of dexterous actions and anticipation of the consequences of potential actions must seamlessly alternate. This article presents a unified neural framework for generation and forward simulation of goal directed actions and validates the architecture through diverse experiments on humanoid and industrial robots. The basic idea is that actions are consequences of an simulation process that animates the internal model of the body (namely the body schema), in the context of intended goals/constraints. Specific focus is on (a) Learning: how the internal model of the body can be acquired by any robotic embodiment and extended to coordinated tools; (b) Configurability: how diverse forward/inverse models of action can be ‘composed’ at runtime by coupling/decoupling different body (body \(+\) tool) chains with task relevant goals and constraints represented as multi-referential force fields; and (c) Computational simplicity: how both the synthesis of motor commands to coordinate highly redundant systems and the ensuing forward simulations are realized through well-posed computations without kinematic inversions. The performance of the neural architecture is demonstrated through a range of motor tasks on a 53-DoFs robot iCub and two industrial robots performing real world assembly with emphasis on dexterity, accuracy, speed, obstacle avoidance, multiple task-specific constraints, task-based configurability. Putting into context other ideas in motor control like the Equilibrium Point Hypothesis, Optimal Control, Active Inference and emerging studies from neuroscience, the relevance of the proposed framework is also discussed.


Body schema Passive motion paradigm iCub Motor control Industrial assembly 



This work presented in this article is supported by Robotics, Brain and Cognitive Sciences Department IIT, the EU FP7 Project DARWIN (, Grant No. FP7-270138) and US Dept. of Defense Grant (W911QY-12-C0078).

Supplementary material

Supplementary material 1 (mp4 19871 KB)

Supplementary material 2 (mp4 26014 KB)

Supplementary material 3 (mp4 21926 KB)


  1. Arimoto, S., et al. (2005). Natural resolution of ill-posedness of inverse kinematics for redundant robots: A challenge to Bernstein’s degrees-of-freedom problem. Advanced Robotics, 19(4), 401–434.CrossRefGoogle Scholar
  2. Asatryan, D. G., & Feldman, A. G. (1965). Functional tuning of the nervous system with control of movements or maintenance of a steady posture. Biophysics, 10, 925–935.Google Scholar
  3. Baillieul, J., & Martin, D. P. (1990). Resolution of kinematic redundancy. Proceedings of Symposia in Applied Mathematics, 41, 49–89.MathSciNetCrossRefzbMATHGoogle Scholar
  4. Balestrino, A., De Maria, G., & Sciavicco, L. (1984). Robust control of robotic manipulators. In Proceedings of the 9th IFAC world congress (Vol. 5, pp. 2435–2440).Google Scholar
  5. Bekey, G., & Goldberg, K. Y. (Eds.). (2012). Neural networks in robotics (Vol. 202). Berlin: Springer.Google Scholar
  6. Bernstein, N. (1935). The problem of the interrelationships between coordination and localization. Retrieved November 13th, 2015 from
  7. Bernstein, N. (1967). The coordination and regulation of movements. Oxford: Pergamon Press.Google Scholar
  8. Bhat, A. A., & Mohan, V. (2015). How iCub learns to imitate use of a tool quickly by recycling the past knowledge learnt during drawing. In Biomimetic and biohybrid systems (pp. 339–347). Berlin: Springer.Google Scholar
  9. Bizzi, E., & Polit, A. (1978). Processes controlling arm movements in monkeys. Science, 201, 1235–1237.CrossRefGoogle Scholar
  10. Bryson, E. (1999). Dynamic optimization. Menlo Park, CA: Addison Wesley Longman.Google Scholar
  11. Buss, S. R., & Kim, J.-S. (2005). Selectively damped least squares for inverse kinematics. Journal of Graphics Tools, 10(3), 37–49.CrossRefGoogle Scholar
  12. Cai, H., Werner, T., & Matas, J. (2013). Fast detection of multiple textureless 3-D objects. In Computer vision systems (pp. 103–112). Berlin: Springer.Google Scholar
  13. DARWIN D9.4. (2014). Deliverable D9.4: Third year demonstrators and evaluation report. EC FP7 project DARWIN Grant No. 270138. Retrieved November 10th, 2015 from
  14. DARWIN D9.5. (2015). Deliverable D9.5: Industrial assembly demonstrator and final evaluation. EC FP7 project DARWIN Grant No. 270138. Retrieved November 10th, 2015 from
  15. De Luca, A., & Oriolo, G. (1991). Issues in acceleration resolution of robot redundancy. In Third IFAC symposium on robot control (pp. 93–98).Google Scholar
  16. De Luca, A., Oriolo, G., & Siciliano, B. (1992). Robot redundancy resolution at the acceleration level. Laboratory Robotics and Automation, 4, 97–106.Google Scholar
  17. Featherstone, R. (1987). Robot Dynamics Algorithms. Dordrecht: Kluwer.Google Scholar
  18. Featherstone, R., & Khatib, O. (1997). Load independence of the dynamically consistent inverse of the Jacobian matrix. International Journal of Robotics Research, 16(2), 168–170.CrossRefGoogle Scholar
  19. Flash, T., & Hogan, N. (1985). The coordination of arm movements: an experimentally confirmed mathematical model. Journal of Neuroscience, 5, 1688–1703.Google Scholar
  20. Frey, S. H., & Gerry, V. E. (2006). Modulation of neural activity during observational learning of actions and their sequential orders. Journal of Neuroscience, 26, 13194–13201.CrossRefGoogle Scholar
  21. Friston, K. (2010). The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11, 127–138.CrossRefGoogle Scholar
  22. Friston, K. (2011). What is optimal about motor control? Neuron, 72(3), 488–498.CrossRefGoogle Scholar
  23. Gallese, V., & Lakoff, G. (2005). The brain’s concepts: The role of the sensory-motor system in reason and language. Cognitive Neuropsychology, 22(3), 455–479.CrossRefGoogle Scholar
  24. Gallese, V., & Sinigaglia, C. (2011). What is so special about Embodied Simulation. Trends in Cognitive Sciences, 15(11), 512–519.CrossRefGoogle Scholar
  25. Grafton, S. T. (2009). Embodied cognition and the simulation of action to understand others. Annals of the New York Academy of Sciences, 1156, 97–117.CrossRefGoogle Scholar
  26. Graziano, M. S. A., & Botvinick, M. M. (2002). How the brain represents the body: Insights from neurophysiology and psychology. In W. Prinz & B. Hommel (Eds.), Common mechanisms in perception and action: Attention and performance (pp. 136–157). Oxford: Oxford University Press.Google Scholar
  27. Guigon, E. (2011). Models and architectures for motor control: Simple or complex? In F. Danion & M. L. Latash (Eds.), Motor control (pp. 478–502). Oxford: Oxford University Press.Google Scholar
  28. Haggard, P., & Wolpert, D. M. (2005). Disorders of body schema. In H. J. Freund, M. Jeannerod, M. Hallett, & R. Leiguarda (Eds.), Higher-order motor disorders: From neuroanatomy and neurobiology to clinical neurology (pp. 261–271). Oxford: Oxford University Press.Google Scholar
  29. Head, H., & Holmes, G. (1911). Sensory disturbances in cerebral lesions. Brain, 34, 102–254.CrossRefGoogle Scholar
  30. Hollerbach, J. M., & Suh, K. C. (1987). Redundancy resolution of manipulators through torque optimization. IEEE Journal of Robotics and Automation, 3(4), 308–316.CrossRefGoogle Scholar
  31. Hsu, P., Hauser, J., & Sastry, S. (1989). Dynamic control of redundant manipulators. Journal of Robotic Systems, 6(2), 133–148.CrossRefzbMATHGoogle Scholar
  32. Iriki, A., Tanaka, M., & Iwamura, Y. (1996). Coding of modified body schema during tool use by macaque postcentral neurones. Neuroreport, 7, 2325–2330.CrossRefGoogle Scholar
  33. Jordan, M. I. (1990). Motor learning and the degrees of freedom problem. In M. Jeannerod (Ed.), Attention and performance XIII. Hillsdale, NJ: Lawrence Erlbaum Associates Inc.Google Scholar
  34. Jordan, M. I., & Rumelhart, D. E. (1992). Forward models: Supervised learning with a distal teacher. Cognitive Science, 16(3), 307–354.CrossRefGoogle Scholar
  35. Khatib, O. (1987). A unified approach for motion and force control of robot manipulators: The operational space formulation. IEEE Journal of Robotics and Automation, 3(1), 43–53.CrossRefGoogle Scholar
  36. Khatib, O., et al. (2004). Human-centered robotics and interactive haptic simulation. International Journal of Robotics Research, 23(2), 167–478.CrossRefGoogle Scholar
  37. Kranczioch, C., Mathews, S., Dean, J. A., & Sterr, A. (2009). On the equivalence of executed and imagined movements. Human Brain Mapping, 30, 3275–3286.CrossRefGoogle Scholar
  38. Lashley, K. S. (1933). Integrative function of the cerebral cortex. Physiological Reviews, 13(1), 1–42.Google Scholar
  39. Lee, S., & Kil, R. M. (1990, June). Robot kinematic control based on bidirectional mapping neural network. In 1990 IJCNN international joint conference on neural networks, 1990 (pp. 327–335). New York: IEEE.Google Scholar
  40. Lewis, F. W., Jagannathan, S., & Yesildirak, A. (1998). Neural network control of robot manipulators and non-linear systems. Boca Raton: CRC Press.Google Scholar
  41. Li, S., Chen, S., Liu, B., Li, Y., & Liang, Y. (2012). Decentralized kinematic control of a class of collaborative redundant manipulators via recurrent neural networks. Neurocomputing, 91, 1–10.CrossRefGoogle Scholar
  42. Liégeois, A. (1977). Automatic supervisory control of the configuration and behavior of multibody mechanisms. IEEE Transactions on Systems, Man and Cybernetics, 7(12), 868–871.CrossRefzbMATHGoogle Scholar
  43. Lourakis, M., & Zabulis, X. (2013). Model-based pose estimation for rigid objects. In Computer vision systems (pp. 83–92). Berlin: Springer.Google Scholar
  44. Maravita, A., & Iriki, A. (2004). Tools for the body (schema). Trends in Cognitive Science, 8, 79–86.CrossRefGoogle Scholar
  45. Mel, B. W. (1988). MURPHY: A robot that learns by doing. In Neural information processing systems (pp. 544–553).Google Scholar
  46. Mohan, V., & Morasso, P. (2011). Passive motion paradigm: An alternative to optimal control. Frontiers in Neurorobotics, 5, 4.Google Scholar
  47. Mohan, V., Morasso, P., Metta, G., & Sandini, G. (2009). A biomimetic, force-field based computational model for motion planning and bimanual coordination in humanoid robots. Autonomous Robots, 27, 291–301.CrossRefGoogle Scholar
  48. Mohan, V., Morasso, P., Zenzeri, J., Metta, G., Chakravarthy, V. S., & Sandini, G. (2011). Teaching a humanoid robot to draw ‘Shapes’. Autonomous Robots, 31(1), 21–53.Google Scholar
  49. Mussa-Ivaldi, F. A., Morasso, P., & Zaccaria, R. (1988). Kinematic networks. A distributed model for representing and regularizing motor redundancy. Biological Cybernetics, 60, 1–16.Google Scholar
  50. Nakamura, Y., & Hanafusa, H. (1986). Inverse kinematics solutions with singularity robustness for robot manipulator control. Journal of Dynamic Systems, Measurement, and Control, 108, 163–171.CrossRefzbMATHGoogle Scholar
  51. Nakamura, Y., & Hanafusa, H. (1987). Optimal redundancy control of robot manipulators. International Journal of Robotics Research, 6(1), 32–42.CrossRefGoogle Scholar
  52. Nakanishi, J., Cory, R., Mistry, M., Peters, J., & Schaal, S. (2008). Operational space control: A theoretical and empirical comparison. The International Journal of Robotics Research, 27(6), 737–757.CrossRefGoogle Scholar
  53. Nguyen, L., Patel, R. V., & Khorasani, K. (1990, June). Neural network architectures for the forward kinematics problem in robotics. In 1990 IJCNN international joint conference on neural networks (pp. 393–399). New York: IEEE.Google Scholar
  54. Peters, J., & Schaal, S. (2008). Learning to control in operational space. The International Journal of Robotics Research, 27(2), 197–212.CrossRefGoogle Scholar
  55. Pickering, M. J., & Clark, A. (2014). Getting ahead: Forward models and their role in cognitive architecture. Trends in Cognitive Sciences, 18(9), 451–456.CrossRefGoogle Scholar
  56. Salaün, C., Padois, V., & Sigaud, O. (2009, October). Control of redundant robots using learned models: An operational space control approach. In IROS 2009 IEEE/RSJ international conference on intelligent robots and systems, 2009 (pp. 878–885). New York: IEEE.Google Scholar
  57. Scott, S. (2004). Optimal feedback control and the neural basis of volitional motor control. Nature Reviews Neuroscience, 5, 534–546.CrossRefGoogle Scholar
  58. Senda, K. (1999). Quasioptimal control of space redundant manipulators. AIAA Guidance, Navigation, and Control Conference, 3, 1877–1885.Google Scholar
  59. Sentis, L., & Khatib, O. (2005). Synthesis of wholebody behaviors through hierarchical control of behavioral primitives. International Journal of Humanoid Robotics, 2(4), 505–518.CrossRefGoogle Scholar
  60. Sevdalis, V., & Keller, P. E. (2011). Captured by motion: Dance, action understanding, and social cognition. Brain & Cognition, 77, 231–236.CrossRefGoogle Scholar
  61. Todorov, E. (2006). Optimal control theory. In K. Doya, et al. (Eds.), Bayesian brain: Probabilistic approaches to neural coding (pp. 269–298). Cambridge, MA: MIT Press.Google Scholar
  62. Umiltà, M. A., Escola, L., Intskirveli, I., Grammont, F., Rochat, M., Caruana, F., et al. (2008). When pliers become fingers in the monkey motor system. Proceedings of the National Academy of Sciences of the United States of America, 105(6), 2209–13.CrossRefGoogle Scholar
  63. Wampler, C. W. (1986). Manipulator inverse kinematic solutions based on vector formulations and damped least squares methods. IEEE Transaction on Systems, Man, and Cybernetics, 16, 93–101.CrossRefzbMATHGoogle Scholar
  64. Whitney, D. E. (1969). Resolved motion rate control of manipulators and human prostheses. IEEE Transactions on Man Machine Systems, 10(2), 47–53.CrossRefGoogle Scholar
  65. Wolovich, W. A., & Elliot, H. (1984). A computational technique for inverse kinematics. In Proceedings of the 23rd IEEE conference on decision and control (pp. 1359–1363).Google Scholar
  66. Zak, M. (1991). Terminal chaos for information processing in neurodynamics. Biological Cybernetics, 64, 343–351.CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Robotics, Brain and Cognitive Science DepartmentIstituto Italiano di TecnologiaGenoaItaly
  2. 2.Robotics and Assistive SystemsPROFACTOR GmbHSteyr-GleinkAustria

Personalised recommendations