Cognitive Processing

, Volume 5, Issue 3, pp 159–166 | Cite as

Multisensory integration in multiple reference frames in the posterior parietal cortex

  • Marie Avillac
  • Etienne Olivier
  • Sophie Denève
  • Suliann Ben Hamed
  • Jean-René DuhamelEmail author
Research Report


Spatial information processing takes place in different brain regions that receive converging inputs from several sensory modalities. Because of our own movements—for example, changes in eye position, head rotations, and so forth—unimodal sensory representations move continuously relative to one another. It is generally assumed that for multisensory integration to be an orderly process, it should take place between stimuli at congruent spatial locations. In the monkey posterior parietal cortex, the ventral intraparietal (VIP) area is specialized for the analysis of movement information using visual, somatosensory, vestibular, and auditory signals. Focusing on the visual and tactile modalities, we found that in area VIP, like in the superior colliculus, multisensory signals interact at the single neuron level, suggesting that this area participates in multisensory integration. Curiously, VIP does not use a single, invariant coordinate system to encode locations within and across sensory modalities. Visual stimuli can be encoded with respect to the eye, the head, or halfway between the two reference frames, whereas tactile stimuli seem to be prevalently encoded relative to the body. Hence, while some multisensory neurons in VIP could encode spatially congruent tactile and visual stimuli independently of current posture, in other neurons this would not be the case. Future work will need to evaluate the implications of these observations for theories of optimal multisensory integration.


Multisensory integration Posterior parietal cortex Ventral intraparietal area Macaque 


  1. Andersen RA, Bracewell RM, Barash RM, Gnadt JW, Fogassi L (1990) Eye position effects on visual, memory, and saccade-related activity in areas LIP and 7a of the macaque. J Neurosci 66:1176–1196Google Scholar
  2. Bell AH, Corneil BD, Meredith MA, Munoz DP (2001) The influence of stimulus properties on multisensory processing in the awake primate superior colliculus. Can J Exp Psychol 55:125–134Google Scholar
  3. Bremmer F, Schlack A, Jon Shah N, Zafiris O, Kubischik M, Hoffmann K-P, Zilles K, Fink GR (2001) Polymodal motion processing in posterior parietal and premotor cortex: a human fMRI study strongly implies equivalencies between humans ans monkeys. Neuron 29:287–296CrossRefPubMedGoogle Scholar
  4. Bremmer F, Klam F, Duhamel J-R, Ben Hamed S, Graf W (2002) Visual-vestibular interactive responses in the macaque ventral intraparietal area (VIP). Eur J Neurosci 16:1569–1586CrossRefPubMedGoogle Scholar
  5. Bruce C, Desimone R, Gross CG (1981) Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J Neurophysiol 46:369–384PubMedGoogle Scholar
  6. Calvert GA, Bullmore ET, Brammer MJ, Campbell R, Williams SC, McGuire PK, Woodruff PW, Iversen SD, David AS (1997) Activation of auditory cortex during silent lipreading. Science 276:593–596CrossRefPubMedGoogle Scholar
  7. Calvert GA, Hansen PC, Iversen SD, Brammer MJ (2001) Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. NeuroImage 14:427–438CrossRefPubMedGoogle Scholar
  8. Cavada C (2001) The visual parietal areas in the macaque monkey: current structural knowledge and ignorance. NeuroImage 14:S21–S26CrossRefPubMedGoogle Scholar
  9. Colby CL, Duhamel J-R (1991) Heterogeneity of extrastriate visual areas and multiple parietal areas in the macaque monkey. Neuropsychologia 6:517–537CrossRefGoogle Scholar
  10. Colby CL, Duhamel J-R, Goldberg ME (1993) Ventral intraparietal area of the macaque: anatomic location and visual response properties. J Neurophysiol 69:902–914PubMedGoogle Scholar
  11. Cooke DF, Taylor CSR, Moore T, Graziano MSA (2003) Complex movements evoked by microstimulation of the ventral intraparietal area. Proc Natl Acad Sci U S A 100:6163–6168CrossRefPubMedGoogle Scholar
  12. Deneve S, Pouget A (2001) Efficient computation and cue integration with noisy population codes. Nat Neurosci 4:826–831CrossRefPubMedGoogle Scholar
  13. Duhamel J-R, Colby CL, Goldberg ME (1991) Congruent representations of visual and somatosensory space single neurons of monkey ventral intra-parietal cortex (area VIP). In: Paillard J (ed) Brain and space. Oxford University Press, Oxford, pp 223–236Google Scholar
  14. Duhamel J-R, Bremmer F, BenHamed S, Graf W (1997) Spatial invariance of visual receptive fields in parietal cortex neurons. Nature 389:845–848CrossRefPubMedGoogle Scholar
  15. Duhamel J-R, Colby CL, Goldberg ME (1998) Ventral intraparietal area of the macaque: congruent visual and somatic response properties. J Neurophysiol 79:126–136PubMedGoogle Scholar
  16. Fogassi L, Gallese V, Di Pellegrino G, Fadiga L, Gentilucci M, Luppino G, Pedotti A, Rizzolatti G (1992) Space coding by premotor cortex. Exp Brain Res 89:686–690PubMedGoogle Scholar
  17. Fogassi L, Gallese V, Fadiga L, Luppino G, Matelli M, Rizzolatti G (1996) Coding of peripersonal space in inferior premotor cortex (area F4). J Neurophysiol 76:141–157PubMedGoogle Scholar
  18. Foxe JJ, Wylie GR, Martinez A, Schroeder CE, Javitt DC, Guilfoyle D, Ritter W, Murray MM (2002) Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J Neurophysiol 88:540–543PubMedGoogle Scholar
  19. Galletti C, Battaglini PP, Fattori P (1993) Parietal neurons encoding spatial locations in craniotopic coordinates. Exp Brain Res 96:221–229PubMedGoogle Scholar
  20. Gentilucci M, Scandolara C, Pigarev IN, Rizzolatti G (1983) Visual responses in the postarcuate cortex (area 6) of the monkey that are independent of eye position. Exp Brain Res 50:464–468PubMedGoogle Scholar
  21. Graziano MSA, Gross CG (1994) The representation of extrapersonal space: a possible role for bimodal, visual-tactile neurons. In: Gazzaniga MS (ed) The cognitive neurosciences. MIT Press, Cambridge, Mass., pp 1021–1034Google Scholar
  22. Graziano MSA, Yap GS, Gross CG (1994) Coding of visual space by premotor neurons. Science 266:1054–1057PubMedGoogle Scholar
  23. Iriki A, Tanaka M, Iwamura Y (1996) Coding of modified body schema during tool use by macaque postcentral neurones. Neuroreport 7:2325–2330PubMedGoogle Scholar
  24. Jay MF, Sparks DL (1987) Sensorimotor integration in the primate superior colliculus. II. Coordinates of auditory signals. J Neurophysiol 57:35–55PubMedGoogle Scholar
  25. Jousmaki V, Hari R (1998) Parchment-skin illusion: sound-biased touch. Curr Biol 8:R190PubMedGoogle Scholar
  26. Lewis JW, Van Essen DC (2000) Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey. J Comp Neurol 428:112–137CrossRefPubMedGoogle Scholar
  27. Luppino G, Murata A, Govoni P, Matelli M (1999) Largely segregated parietofrontal connections linking rostral intraparietal cortex (areas AIP and VIP) and the ventral premotor cortex (areas F5 ans F4). Exp Brain Res 128:181–187CrossRefPubMedGoogle Scholar
  28. Macaluso E, Frith CD, Driver J (2000) Modulation of human visual cortex by crossmodal spatial attention. Science 289:1206–1208CrossRefPubMedGoogle Scholar
  29. Macaluso E, Frith CD, Driver J (2002) Crossmodal spatial influences of touch on extrastriate visual areas take current gaze direction into account. Neuron 34:647–658CrossRefPubMedGoogle Scholar
  30. Matelli M, Govoni P, Galletti C, Kutz DF, Luppino G (1998) Superior area 6 afferents from the superior parietal lobule in the macaque monkey. J Comp Neurol 402:327–352CrossRefPubMedGoogle Scholar
  31. Maunsell JHR, Van Essen DC (1983) The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey. J Neurosci 3:2563–2580PubMedGoogle Scholar
  32. McGurk J, MacDonald H (1976) Hearing lips and seeing voices. Nature 264:746–748PubMedGoogle Scholar
  33. Meredith MA, Stein BE (1983) Interactions among converging sensory inputs in the superior colliculus. Science 221:389–391PubMedGoogle Scholar
  34. Meredith MA, Stein BE (1986a) Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res 365:350–354CrossRefPubMedGoogle Scholar
  35. Meredith MA, Stein BE (1986b) Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol 56:640–662PubMedGoogle Scholar
  36. Meredith MA, Stein BE (1996) Spatial determinants of multisensory integration in cat superior colliculus neurons. J Neurophysiol 75:1843–1857PubMedGoogle Scholar
  37. Meredith MA, Nemitz JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 7:3215–3229PubMedGoogle Scholar
  38. Neppi-Mòdona M, Auclair D, Sirigu A, Duhamel J-R (2004) Spatial coding of the predicted impact location of a looming object. Curr Biol (in press)Google Scholar
  39. Pouget A, Sejnowski TJ (2001) Simulating a lesion in a basis function model of spatial representations: comparison with hemineglect. Psychol Rev 108:653–673CrossRefPubMedGoogle Scholar
  40. Pouget A, Deneve S, Duhamel J-R (2002) A computational perspective on the neural basis of multisensory spatial representations. Nat Rev Neurosci 3:741–747CrossRefPubMedGoogle Scholar
  41. Rizzolatti G, Scandolara C, Matelli M, Gentilucci M (1981) Afferent properties of periarcuate neurons in macaque monkeys I. Somatosensory responses. Behav Brain Res 2:125–146CrossRefPubMedGoogle Scholar
  42. Schaafsma SJ, Duysens J (1996) Neurons in the ventral intraparietal area of awake macaque monkey closely resemble neurons in the dorsal part of the medial superior temporal area in their responses to optic flow patterns. J Neurophysiol 76:4056–4068PubMedGoogle Scholar
  43. Schlack A, Hoffmann K-P, Bremmer F (2003) Selectivity of macaque area VIP for smooth pursuit eye movements. J NeurophysiolGoogle Scholar
  44. Spence C, Driver J (2002) Crossmodal space and crossmodal attention. Oxford University Press, OxfordGoogle Scholar
  45. Stein BE, Meredith MA (1993) The merging of the senses. MIT Press/Bradford Books, Cambridge, Mass.Google Scholar
  46. Stricanne B, Andesen RA, Mazzoni P (1996) Eye-centered, head-centered, and intermediate coding of remembered sound locations in area LIP. J Neurophysiol 76:2071–2076PubMedGoogle Scholar
  47. Wallace MT, Meredith MA, Stein BE (1992) Integration of multiple sensory modalities in cat cortex. Exp Brain Res 91:484–488PubMedGoogle Scholar
  48. Wallace MT, Wilkinson LK, Stein BE (1996) Representation and integration of multiple sensory inputs in primate superior colliculus. J Neurophysiol 76:1246–1266PubMedGoogle Scholar
  49. Zipser D, Andersen RA (1988) A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature 331:679–684CrossRefPubMedGoogle Scholar

Copyright information

© Marta O. Belardinelli and Springer-Verlag 2004

Authors and Affiliations

  • Marie Avillac
    • 1
  • Etienne Olivier
    • 3
  • Sophie Denève
    • 1
    • 2
  • Suliann Ben Hamed
    • 1
  • Jean-René Duhamel
    • 1
    Email author
  1. 1.Institut des Sciences Cognitives, CNRSBronFrance
  2. 2.Gatsby Computational UnitUniversity College LondonLondonUK
  3. 3.Laboratoire de NeurophysiologieUniversité Catholique de LouvainBrusselsBelgium

Personalised recommendations