Advertisement

Biological Cybernetics

, Volume 101, Issue 5–6, pp 361–377 | Cite as

MODEM: a multi-agent hierarchical structure to model the human motor control system

  • Mehran Emadi AndaniEmail author
  • Fariba Bahrami
  • Parviz Jabehdar Maralani
  • Auke Jan Ijspeert
Original Paper

Abstract

In this study, based on behavioral and neurophysiological facts, a new hierarchical multi-agent architecture is proposed to model the human motor control system. Performance of the proposed structure is investigated by simulating the control of sit to stand movement. To develop the model, concepts of mixture of experts, modular structure, and some aspects of equilibrium point hypothesis were brought together. We have called this architecture MODularized Experts Model (MODEM). Human motor system is modeled at the joint torque level and the role of the muscles has been embedded in the function of the joint compliance characteristics. The input to the motor system, i.e., the central command, is the reciprocal command. At the lower level, there are several experts to generate the central command to control the task according to the details of the movement. The number of experts depends on the task to be performed. At the higher level, a “gate selector” block selects the suitable subordinate expert considering the context of the task. Each expert consists of a main controller and a predictor as well as several auxiliary modules. The main controller of an expert learns to control the performance of a given task by generating appropriate central commands under given conditions and/or constraints. The auxiliary modules of this expert learn to scrutinize the generated central command by the main controller. Auxiliary modules increase their intervention to correct the central command if the movement error is increased due to an external disturbance. Each auxiliary module acts autonomously and can be interpreted as an agent. Each agent is responsible for one joint and, therefore, the number of the agents of each expert is equal to the number of joints. Our results indicate that this architecture is robust against external disturbances, signal-dependent noise in sensory information, and changes in the environment. We also discuss the neurophysiological and behavioral basis of the proposed model (MODEM).

Keywords

Human motor control Mixture of experts Modular structure Multi-agent Hierarchical structure Equilibrium point hypothesis Sit to stand 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alexander GE, DeLong MR, Strick PL (1986) Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Ann Rev Neurosci 9: 357–381CrossRefPubMedGoogle Scholar
  2. Alexandrov AV, Frolov AA, Massion J (2001) Biomechanical analysis of movement strategies in human forward trunk bending II. Exp Study Biol Cybern 84: 435–443CrossRefGoogle Scholar
  3. Alexandrov AV, Frolov AA, Horak FB, Carlson-Kuhta P, Park S (2005) Feedback equilibrium control during human standing. Biol Cybern 93: 309–322CrossRefPubMedGoogle Scholar
  4. Balasubramaniam R, Feldman AG (2004) Guiding movements without redundancy problems, In: Jirsa Vk, Kelso JAS (eds) Coordination dynamics. Springer, New York, pp 1–16Google Scholar
  5. Bhushan N, Shadmehr R (1999) Computational nature of human adaptive control during learning of reaching movements in force fields. Biol Cybern 81: 39–60Google Scholar
  6. Blakemore SJ, Wolpert DM, Frith CD (1998) Central cancellation of self produced tickle sensation. Nat Neurosci 1: 635–640CrossRefPubMedGoogle Scholar
  7. Cisek P (2005) Neural representations of motor plans, desired trajectories, and controlled objects. Cogn Process 6: 15–24CrossRefGoogle Scholar
  8. Craig JJ (1986) Introduction to robotics, mechanics & control. Addison-Wesley, Reading, MAGoogle Scholar
  9. Crutcher MD, DeLong MR (1984a) Single cell studies of the primate putamen I. Functional organization. Exp Brain Res 53: 233–243CrossRefPubMedGoogle Scholar
  10. Crutcher MD, DeLong MR (1984b) Single cell studies of the primate putamen. II. Relations to direction of movements and patterns of muscular activity. Exp Brain Res 53: 244–258CrossRefPubMedGoogle Scholar
  11. Darainy M, Malfait N, Gribble PL, Towhidkhoh F, Ostry DJ (2004) Learning to control arm stiffness under static conditions. J Neurophysiol 92: 3344–3350CrossRefPubMedGoogle Scholar
  12. Davidson PR, Wolpert DM (2004) Internal models underlying grasp can be additively ‘d. Exp Brain Res 155: 334–340CrossRefPubMedGoogle Scholar
  13. Domen K, Latash ML, Zatsiorsky VM (1999) Reconstruction of equilibrium trajectories during whole-body movements. Biol Cybernet 80: 195–204CrossRefGoogle Scholar
  14. Dove A, Pollmann S, Schubert T, Wiggins C, Cramon D (2000) Prefrontal cortex activation in task switching: an event-related fMRI study. Cogn Brain Res 9: 103–109CrossRefGoogle Scholar
  15. Doya K (2002) Multiple model-based reinforcement learning. Neural Comput 14: 1347–1369CrossRefPubMedGoogle Scholar
  16. Doya K, Yoshizawa S (1989) Adaptive neural oscillator using continuous-time back propagation learning. Neural Netw 2: 375–386CrossRefGoogle Scholar
  17. Doya K, Samejima K, Katagiri K, Kawato M (2002) Multiple model-based reinforcement learning. Neural Comput 14: 1347–1369CrossRefPubMedGoogle Scholar
  18. Emadi Andani M, Bahrami F, Jabedar Maralani P (2007) A biologically inspired modular structure to control the sit-to-stand transfer of a biped robot. 29th annual international conference of the IEEE EMBS, Cité internationale, Lyon, France, August 23–26, pp 3016–3019Google Scholar
  19. Emadi Andani M, Bahrami F, Jabehdar Maralani P (2009) AMA-MOSAICI: an automatic module assigning hierarchical structure to control human motion based on movement decomposition. Neurocomputing 72(10–12): 2310–2318CrossRefGoogle Scholar
  20. Farley CT, Houdijk HHP, van Strien C, Lourie M (1998) Mechanism of leg stiffness adjustment for hopping on surfaces of different stiffnesses. J Appl Physiol 85: 1044–1055PubMedGoogle Scholar
  21. Feldman A, Latash M (2005) Testing hypotheses and the advancement of science: recent attempts to falsify the equilibrium point hypothesis. Exp Brain Res 161: 91–103CrossRefPubMedGoogle Scholar
  22. Georgopoulos AP, DeLong MR, Crutcher MD (1983) Relation between parameters of step-tracking movements and single cell discharge in the globus pallidus and subthalamic nucleus of the behaving monkey. J Neurosci 3: 1586–1598PubMedGoogle Scholar
  23. Gomi H, Kawato M (1993) Recognition of manipulated objects by motor learning with modular architecture networks. Neural Netw 6: 485–497CrossRefGoogle Scholar
  24. Graybiel AM, Aosaki T, Flaherty AW, Kimura M (1994) The basal ganglia and adaptive motor control. Science 265: 1826–1831CrossRefPubMedGoogle Scholar
  25. Greene PR, McMahon TA (1979) Reflex stiffness of man’s antigravity muscles during knee bends while carrying extra weight. J Biomech 12: 881–891CrossRefPubMedGoogle Scholar
  26. Gu X, Ballard D (2006) An equilibrium point based model unifying movement control in humanoids. Robotics: Science and Systems, USAGoogle Scholar
  27. Guyton AC, Hall JE (2006) Textbook of medical physiology. Elsevier Inc., AmsterdamGoogle Scholar
  28. Harris CM, Wolpert DM (1998) Signal-dependent noise determines motor planning. Nature 394: 780–784CrossRefPubMedGoogle Scholar
  29. Haruno M, Wolpert DM, Kawato M (2001) MOSAIC model for sensorimotor learning and control. Neural Comput 13: 2201–2220CrossRefPubMedGoogle Scholar
  30. Haruno M, Wolpert DM, Kawato M (2003) Hierarchical MOSAIC for movement generation, vol 1250. International Congress Series, Elsevier Science, Amsterdam, pp 575–590Google Scholar
  31. Hemami H (1982) Some aspects of Euler–Newton equations of motions. Ingenieur-Archiv 52: 167–176CrossRefGoogle Scholar
  32. Horak FB, Macpherson JM (1996) Postural orientation and equilibrium. In: Rowell LB, Shepherd JT (eds) Handbook of physiology, sect. 12. Oxford University Press, Oxford, pp 255–292Google Scholar
  33. Horak FB, Nashner LM (1986) Central programming of postural movements: adaptation to altered support surface configurations. J Neurophysiol 55: 1369–1381PubMedGoogle Scholar
  34. Iacoboni M (2001) Playing tennis with the cerebellum. Nat Neurosci 4: 555–556CrossRefPubMedGoogle Scholar
  35. Imamizu H, Kuroda T, Yoshioka T, Kawato M (2004) Functional magnetic resonance imaging examination of two modular architectures for switching multiple internal models. J Neurosci 24(5): 1173–1181CrossRefPubMedGoogle Scholar
  36. Jordan MI, Jacobs RA (1994) Hierarchical mixture of experts and the EM algorithm. Neural Comput 6: 181–214CrossRefGoogle Scholar
  37. Kandel ER, Schwartz JH, Jessell TM (2000) Principles of neural science. McGraw-Hill CompaniesGoogle Scholar
  38. Kawato M (1999) Internal models for motor control and trajectory planning. Curr Opin Neurobiol 9: 718–727CrossRefPubMedGoogle Scholar
  39. Kermadi I, Joseph JP (1995) Activity in the caudate nucleus of monkey during spatial sequencing. J Neurophysiol 74: 911–933PubMedGoogle Scholar
  40. Kim W, Voloshin AS, Johnson SH (1994) Modeling of heel strike transients during running. J Human Movement Sci 13: 221–244CrossRefGoogle Scholar
  41. Latash ML, Gottlieb GL (1991) Reconstruction of shifting elbow joint compliant characteristics during fast and slow movements. Neuroscience 43((2/3): 697–712CrossRefPubMedGoogle Scholar
  42. Loria A, Lefeber E, Nijmeijer H (2000) Global asymptotic stability of robot manipulators with linear PID and PI2D control. SACTA 3(2): 138–149Google Scholar
  43. Miall RC, Reckess GZ, Imamizu H (2001) The cerebellum coordinates eye and hand tracking movements. Nat Neurosci 4: 638–644CrossRefPubMedGoogle Scholar
  44. Middleton FA, Strick PL (1998) The cerebellum: an overview. Trends in Neurosci 21: 367–369CrossRefGoogle Scholar
  45. Mizrahi J, Susak Z (1982) Elastic and damping response of the human leg to in vivo impact forces. ASME J Biomech Eng 104: 63–66CrossRefGoogle Scholar
  46. Moraes R, Bahrami F, Patla A (2002) Reprogramming sit-to-stand and sit-to-walk movement sequence under different temporal constraints. The IV world congress on biomechanics, Calgary, CanadaGoogle Scholar
  47. Mushiake H, Strick PL (1995) Pallidal neuron activity during sequential arm movements. J Neurophysiol 74: 2754–2758PubMedGoogle Scholar
  48. Nakano E, Imamizu H, Osu R, Uno Y, Gomi H, Yoshioka T, Kawato M (1999) Quantitive examination of internal representations for arm trajectory planning: minimum command torque change model. J Neuro-physiol 81: 2140–2155Google Scholar
  49. Nashner LM, McCollum G (1985) The organization of human postural movements: a formal basis è experimental synthesis. Behav Brain Sci 8: 135–172CrossRefGoogle Scholar
  50. Ozguven H, Berme N (1988) An experimental and analytical study of impact forces during human jumping. J Biomech 21: 1061–1066CrossRefPubMedGoogle Scholar
  51. Partridge LD (1967) Intrinsic feedback factors producing inertial compensation in muscle. Biophys J 7: 853–863CrossRefPubMedGoogle Scholar
  52. Pocock G, Richards CD (1999) Human physiology: the basis of medicine. Oxford University Press, OxfordGoogle Scholar
  53. Schaal S (2002) Arm and hand movement control. In: Arbib MA (eds) The handbook of brain theory and neural networks, 2nd edn. MIT Press, Cambridge, MA, pp 110–113Google Scholar
  54. Schaal S, Ijspeert A, Billard A (2003) Computational approaches to motor learning by imitation. Philos Trans R Soc Lond B 358: 537–547CrossRefGoogle Scholar
  55. Spagele T, Kistner A, Gollhofer A (1999) Modeling, simulation and optimization of a human vertical jump. ASME J Biomech Eng 32: 521–530Google Scholar
  56. Todorov E, Jordan M (2002) Optimal feedback control as a theory of motor coordination. Nat Neurosci 5(11): 1226–1235CrossRefPubMedGoogle Scholar
  57. Williams RJ, Zipser D (1989) A learning algorithm for continually running fully recurrent neural networks. Neural Comput 1: 270–280CrossRefGoogle Scholar
  58. Wolpert DM, Ghahramani Z (2000) Computational principles of movement neuroscience. Nat Neurosci (Suppl) 3: 1212–1217CrossRefGoogle Scholar
  59. Wolpert DM, Kawato M (1998) Multiple paired forward and inverse models for motor control. Neural Netw 11: 1317–1329CrossRefPubMedGoogle Scholar
  60. Wolpert DM, Miall RC, Kawato M (1998) Internal models in the cerebellum. Trends Cogn Sci 2: 338–347CrossRefGoogle Scholar
  61. Zatsiorsky VM, Seluyanov V (1983) The mass and inertia characteristics of the main segments of the human body. Biomechanics VIII-B: 1152–1159Google Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  • Mehran Emadi Andani
    • 1
    • 2
    • 3
    Email author
  • Fariba Bahrami
    • 2
  • Parviz Jabehdar Maralani
    • 2
  • Auke Jan Ijspeert
    • 3
  1. 1.Department of Biomedical Engineering, Faculty of EngineeringUniversity of IsfahanIsfahanIran
  2. 2.Control and Intelligent Processing Center of Excellence, CIPCEUniversity of TehranTehranIran
  3. 3.Biologically Inspired Robotics Group (BIRG)School of Computer and Communication Sciences, EPFLLausanneSwitzerland

Personalised recommendations