Abstract
Humans show admirable capabilities in movement planning and execution. They can perform complex tasks in various contexts, using the available sensory information very effectively. Body models and continuous body state estimations appear necessary to realize such capabilities. We introduce the Modular Modality Frame (MMF) model, which maintains a highly distributed, modularized body model continuously updating, modularized probabilistic body state estimations over time. Modularization is realized with respect to modality frames, that is, sensory modalities in particular frames of reference and with respect to particular body parts. We evaluate MMF performance on a simulated, nine degree of freedom arm in 3D space. The results show that MMF is able to maintain accurate body state estimations despite high sensor and motor noise. Moreover, by comparing the sensory information available in different modality frames, MMF can identify faulty sensory measurements on the fly. In the near future, applications to lightweight robot control should be pursued. Moreover, MMF may be enhanced with neural encodings by introducing neural population codes and learning techniques. Finally, more dexterous goal-directed behavior should be realized by exploiting the available redundant state representations.
Similar content being viewed by others
References
Andersen RA, Snyder LH, Bradley DC, Xing J (1997) Multimodal representation of space in the posterior parietal cortex and its use in planning movements. Ann Rev Neurosci 20:303–330
Beauchamp MS (2005) See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr Opin Neurobiol 15:145–153
Bernier PM, Gauthier GM, Blouin J (2007) Evidence for distinct, differentially adaptable sensorimotor transformations for reaches to visual and proprioceptive targets. J Neurophysiol 98:1815–1819
Botvinick M, Cohen J et al (1998) Rubber hands ‘feel’ touch that eyes see. Nature 391:756
Braitenberg V (1986) Vehicles: experiments in synthetic psychology. The MIT Press, Cambridge
Bronstein, IN, Semendjajew, KA, Musiol, G, Mühlig, H (eds) (2001) Taschenbuch der Mathematik, 5th edn. Harri Deutsch, Frankfurt
Brooks RA (1990) Elephants don’t play chess. Robot Auton Syst 6:3–15
Butz MV, Herbort O, Hoffmann J (2007) Exploiting redundancy for flexible behavior: unsupervised learning in a modular sensorimotor control architecture. Psychol Rev 114:1015–1046
Butz MV, Shirinov E, Reif KL (2010) Self-organizing sensorimotor maps plus internal motivations yield animal-like behavior. Adapt Behav 18:315–337
Calvert, GA, Spence, C, Stein, BE (eds) (2004) The handbook of multisensory processes. The MIT Press, Cambridge
Chinellato E, Antonelli M, Grzyb BJ, del Pobil AP (2011) Implicit sensorimotor mapping of the peripersonal space by gazing and reaching. IEEE Trans Auton Mental Dev 3:43–53
Chinellato E, Grzyb BJ, del Pobil AP (2012) Pose estimation through cue integration: a neuroscience-inspired approach. IEEE Trans Syst Man Cybern Part B 42:530–538
Christensen HI, Hager GD (2008) Sensing and estimation. In: Siciliano B, Khatib O (eds) Springer handbook of robotics. Springer, Berlin, chap 4, pp 87–107
Cruse H, Steinkühler U (1993) Solution of the direct and inverse kinematic problems by a common algorithm based on the mean of multiple computations. Biol Cybern 69:341–351
de Vignemont F, Majid A, Jola C, Haggard P (2009) Segmenting the body into parts: evidence from biases in tactile perception. Q J Exp Psychol 62:500–512
Denève S, Duhamel JR, Pouget A (2007) Optimal sensorimotor integration in recurrent cortical networks: a neural implementation of Kalman filters. J Neurosci 27:5744–5756
Doya K, Ishii S, Pouget A, Rao RPN (2007) Bayesian brain: probabilistic approaches to neural coding. The MIT Press, Cambridge
Durrant-Whyte H, Henderson TC (2008) Multisensor data fusion. In: Siciliano B, Khatib O (eds) Springer handbook of robotics. Springer, Berlin, chap 25, pp 585–610
Ehrenfeld S, Butz MV (2011) A modular, redundant, multi-frame of reference representation for kinematic chains. In: IEEE International Conference on Robotics and Automation, pp 141–147
Erlhagen W, Schöner G (2002) Dynamic field theory of movement preparation. Psychol Rev 109:545–572
Gadeyne K, Lefebvre T, Bruyninckx H (2005) Bayesian hybrid model-state estimation applied to simultaneous contact formation recognition and geometrical parameter estimation. Int J Robot Res 24:615–630
Gentner R, Classen J (2006) Modular organization of finger movements by the human central nervous system. Neuron 52:731–742
Gratal X, Romero J, Kragic D (2011) Virtual visual servoing for real-time robot pose estimation. In: Bittanti S, Cenedese A, Zampieri S (eds) World congress, vol 18, pp 9017–9022
Herbort O, Butz MV, Pedersen G (2010) The SURE_REACH model for motor learning and control of a redundant arm: from modeling human behavior to applications in robotics. In: Sigaud O, Peters J (eds) From motor learning to interaction learning in robots. Springer, Berlin, pp 85–106
Hoffmann M, Marques H, Arieta A, Sumioka H, Lungarella M, Pfeifer R (2010) Body schema in robotics: a review. IEEE Trans Auton Mental Dev 2:304–324
Holst E, von Mittelstaedt H (1950) Das Reafferenzprinzip (Wechselwirkungen zwischen Zentralnervensystem und Peripherie). Naturwissenschaften 37:464–476
Kohonen T (2001) Self-organizing maps. 3rd ed. Springer, Berlin
Körding KP, Wolpert DM (2004) Bayesian integration in sensorimotor learning. Nature 427:244–247
Latash ML (2008) Synergy. Oxford University Press, New York
Latash, ML, Turvey, MT (eds) (1996) Dexterity and its development. Lawrence Erlbaum Assoc Inc, Mahwah
Latash ML, Scholz JP, Schöner G (2007) Toward a new theory of motor synergies. Motor Control 11:276–308
Longo MR, Haggard P (2010) An implicit body representation underlying human position sense. PNAS 107:11727–11732
Makin TR, Holmes NP, Ehrsson HH (2008) On the other hand: dummy hands and peripersonal space. Behav Brain Res 191:1–10
Maravita A, Spence C, Driver J (2003) Multisensory integration and the body schema: close to hand and within reach. Curr Biol 13:531–539
McGuire LM, Sabes PN (2009) Sensory transformations and the use of multiple reference frames for reach planning. Nat Neurosci 12:1056–1061
Nguyen-Tuong D, Peters J (2011) Model learning for robot control: a survey. Cogn Process 12:319–340
Pouget A, Dayan P, Zemel R (2003) Inference and computation with population codes. Ann Rev Neurosci 26:381–410
Reimann H, Iossifidis I, Schöner G (2010) Integrating orientation constraints into the attractor dynamics approach for autonomous manipulation. In: International conference on humanoid robots, pp 294–301
Reimann H, Iossifidis I, Schoner G (2011) Generating collision free reaching movements for redundant manipulators using dynamical systems. In: 2010 IEEE/RSJ international conference on intelligent robots and systems (IROS 2011), pp 5372–5379
Ritter H, Martinetz T, Schulten K (1992) Neural computation and self-organizing maps—an introduction. Addison-Wesley, New York
Rosenbaum DA (2010) Human motor control. 2nd ed. Academic Press, San Diego
Schaal S, Atkeson CG (1998) Constructive incremental learning from only local information. Neural Comput 10:2047–2084
Scheinmann V, McCarthy JM (2008) Mechanisms and actuation. In: Siciliano B, Khatib O (eds) Springer handbook of robotics. Springer, Berlin, chap 3, pp 67–86
Schilling M (2011) Universally manipulable body models—dual quaternion representations in layered and dynamic MMCs. Auton Robots 30:399–425
Schmitz J, Schneider A, Schilling M, Cruse H (2008) No need for a body model: positive velocity feedback for the control of an 18-DOF robot walker. Appl Bionics Biomech 5:135–147
Shadmehr R, Krakauer JW (2008) A computational neuroanatomy for motor control. Exp Brain Res 185:359–381
Shadmehr R, Wise SP (2005) The computational neurobiology of reaching and pointing: a foundation for motor learning. MIT Press, Cambridge
Sigaud O, Salaun C, Padois V (2011) On-line regression algorithms for learning mechanical models of robots: a survey. Robotics Auton Syst 59:1115–1129
Stalph PO, Butz MV (2011) Learning local linear Jacobians for flexible and adaptive robot arm control. Genetic Programm Evolvable Mach 13:137–157
Streri A, Pownall TT, Kingerlee ST (1993) Seeing, reaching, touching: the relations between vision and touch in infancy. The MIT Press, Cambridge
Tononi G, Edelman GM, Sporns O (1998) Complexity and coherency: integrating information in the brain. Trends Cogn Sci 2:474–484
Toussaint M (2006) A sensorimotor map: modulating lateral interactions for anticipation and planning. Neural Comput 18:1132–1155
Vijayakumar S, D’Souza A, Schaal S (2005) Incremental online learning in high dimensions. Neural Comput 17:2602–2634
Wells JP, Hyler-Both DL, Danley TD, Wallace GH (2002) Biomechanics of growth and development in the healthy human infant: a pilot study. J Am Osteopath Assoc 102:313–319
Wolpert DM, Kawato M (1998) Multiple paired forward and inverse models for motor control. Neural Netw 11:1317–1329
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ehrenfeld, S., Butz, M.V. The modular modality frame model: continuous body state estimation and plausibility-weighted information fusion. Biol Cybern 107, 61–82 (2013). https://doi.org/10.1007/s00422-012-0526-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00422-012-0526-2