Skip to main content

Advertisement

SpringerLink
Log in

The development of multisensory processes

  • Review
  • Published:
Cognitive Processing Aims and scope Submit manuscript

Abstract

To understand the development of sensory processes, it is necessary not only to look at the maturation of each of the sensory systems in isolation, but also to study the development of the nervous system’s capacity to integrate information across the different senses. It is through such multisensory integration that a coherent perceptual gestalt of the world comes to be generated. In the adult brain, multisensory convergence and integration take place at a number of brainstem and cortical sites, where individual neurons have been found that respond to multisensory stimuli with patterns of activation that depend on the nature of the stimulus complex and the intrinsic properties of the neuron. Parallels between the responses of these neurons and multisensory behavior and perception suggest that they are the substrates that underlie these cognitive processes. In both cat and monkey models, the development of these multisensory neurons and the appearance of their integrative capacity is a gradual postnatal process. For subcortical structures (i.e., the superior colliculus) this maturational process appears to be gated by the appearance of functional projections from regions of association cortex. The slow postnatal maturation of multisensory processes, coupled with its dependency on functional corticotectal connections, suggested that the development of multisensory integration may be tied to sensory experiences acquired during postnatal life. In support of this, eliminating experience in one sensory modality (i.e., vision) during postnatal development severely compromises the integration of multisensory cues. Research is ongoing to better elucidate the critical development antecedents for the emergence of normal multisensory capacity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8a–d
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Amlot R, Walker R, Driver J, Spence C (2003) Multimodal visual-somatosensory integration in saccade generation. Neuropsychologia 41:1–15

    Article  PubMed  Google Scholar 

  • Andersen RA, Buneo CA (2002) Intentional maps in posterior parietal cortex. Annu Rev Neurosci 25:189–220

    Article  CAS  PubMed  Google Scholar 

  • Andreassi JL, Greco JR (1975) Effects of bisensory stimulation on reaction time and the evoked cortical potential. Physiol Psychol 3:189–194

    Google Scholar 

  • Birch H, Lefford A (1963) Intersensory development in children. Monogr Soc Res Child Dev 28:1–47

    CAS  Google Scholar 

  • Birch H, Lefford A (1967) Visual differentiation, intersensory integration, and voluntary motor control. Monogr Soc Res Child Dev 32:1–82

    CAS  Google Scholar 

  • Bower TGR (1974) Development in infancy. Freeman, San Francisco

  • Brett-Green B, Fifkova E, Larue DT, Winer JA, Barth DS (2003) A multisensory zone in rat parietotemporal cortex: intra- and extracellular physiology and thalamocortical connections. J Comp Neurol 460:223–237

    Article  PubMed  Google Scholar 

  • Burnett LR, Stein BE, Chaponis D, Wallace MT (2004) Superior colliculus lesions preferentially disrupt multisensory orientation. Neuroscience 124:535–547

    Article  CAS  PubMed  Google Scholar 

  • Chalupa LM, Rhoades RW (1977) Responses of visual, somatosensory, and auditory neurones in the golden hamster’s superior colliculus. J Physiol 270:595–626

    CAS  PubMed  Google Scholar 

  • Clarey JC, Irvine DR (1986) Auditory response properties of neurons in the anterior ectosylvian sulcus of the cat. Brain Res 386:12–19

    Article  CAS  PubMed  Google Scholar 

  • Clemo HR, Stein BE (1982) Somatosensory cortex: a ‘new’ somatotopic representation. Brain Res 235:162–168

    Article  CAS  PubMed  Google Scholar 

  • Clemo HR, Stein BE (1983) Organization of a fourth somatosensory area of cortex in cat. J Neurophysiol 50:910–925

    CAS  PubMed  Google Scholar 

  • Colonius H, Arndt P (2001) A two-stage model for visual–auditory interaction in saccadic latencies. Percept Psychophys 63:126–147

    Google Scholar 

  • Corneil BD, Van Wanrooij M, Munoz DP, Van Opstal AJ (2002) Auditory-visual interactions subserving goal-directed saccades in a complex scene. J Neurophysiol 88:438–454

    CAS  PubMed  Google Scholar 

  • Cynader M, Berman N (1972) Receptive-field organization of monkey superior colliculus. J Neurophysiol 35:187–201

    CAS  PubMed  Google Scholar 

  • Diederich A, Colonius H, Bockhorst D, Tabeling S (2003) Visual-tactile spatial interaction in saccade generation. Exp Brain Res 148:328–337

    PubMed  Google Scholar 

  • Drager UC, Hubel DH (1975) Responses to visual stimulation and relationship between visual, auditory, and somatosensory inputs in mouse superior colliculus. J Neurophysiol 38:690–713

    PubMed  Google Scholar 

  • Duhamel JR, Colby CL, Goldberg ME (1998) Ventral intraparietal area of the macaque: congruent visual and somatic response properties. J Neurophysiol 79:126–136

    CAS  PubMed  Google Scholar 

  • Finlay BL, Schneps SE, Wilson KG, Schneider GE (1978) Topography of visual and somatosensory projections to the superior colliculus of the golden hamster. Brain Res 142:223–235

    Article  CAS  PubMed  Google Scholar 

  • Fogassi L, Gallese V, Fadiga L, Luppino G, Matelli M, Rizzolatti G (1996) Coding of peripersonal space in inferior premotor cortex (area F4). J Neurophysiol 76:141–157

    CAS  PubMed  Google Scholar 

  • Forster B, Cavina-Pratesi C, Aglioti SM, Berlucchi G (2002) Redundant target effect and intersensory facilitation from visual–tactile interactions in simple reaction time. Exp Brain Res 143:480–487

    Article  PubMed  Google Scholar 

  • Frassinetti F, Bolognini N, Ladavas E (2002) Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res 147:332–343

    Article  PubMed  Google Scholar 

  • Frens MA, Van Opstal AJ, Van der Willigen RF (1995) Spatial and temporal factors determine auditory–visual interactions in human saccadic eye movements. Percept Psychophys 57:802–816

    Google Scholar 

  • Fu KM, Johnston TA, Shah AS, Arnold L, Smiley J, Hackett TA, Garraghty PE, Schroeder CE (2003) Auditory cortical neurons respond to somatosensory stimulation. J Neurosci 23:7510–7515

    CAS  PubMed  Google Scholar 

  • Fuster JM, Bodner M, Kroger JK (2000) Cross-modal and cross-temporal association in neurons of frontal cortex. Nature 405:347–351

    Article  CAS  PubMed  Google Scholar 

  • Gibson JJ (1966) The senses considered as perceptual systems. Houghton Mifflin, Boston

  • Goldberg ME, Wurtz RH (1972) Activity of superior colliculus in behaving monkey. I. Visual receptive fields of single neurons. J Neurophysiol 35:542–559

    CAS  PubMed  Google Scholar 

  • Gordon B (1973) Receptive fields in deep layers of cat superior colliculus. J Neurophysiol 36:157–178

    CAS  PubMed  Google Scholar 

  • Graybiel A (1952) Oculogravic illusion. Ama Arch Opthalmol 48:605–615

    CAS  PubMed  Google Scholar 

  • Graziano MS, Hu XT, Gross CG (1997) Visuospatial properties of ventral premotor cortex. J Neurophysiol 77:2268–2292

    CAS  PubMed  Google Scholar 

  • Hairston WD, Wallace MT, Vaughan JW, Stein BE, Norris JL, Schirillo JA (2003) Visual localization ability influences cross-modal bias. J Cogn Neurosci 15:20–29

    Article  CAS  PubMed  Google Scholar 

  • Harrington LK, Peck CK (1998) Spatial disparity affects visual-auditory interactions in human sensorimotor processing. Exp Brain Res 122:247–252

    CAS  PubMed  Google Scholar 

  • Harris LR, Blakemore C, Donaghy M (1980) Integration of visual and auditory space in the mammalian superior colliculus. Nature 288:56–59

    CAS  PubMed  Google Scholar 

  • Hershenson M (1962) Reaction time as a measure of intersensory facilitation. J Exp Psychol 63:289–293

    Google Scholar 

  • Howard IP, Templeton WB (1966) Human spatial orientation. Wiley, New York

  • Hubel DH, Wiesel TN (1998) Early exploration of the visual cortex. Neuron 20:401–412

    CAS  PubMed  Google Scholar 

  • Huerta M, Harting J (1984) The mammalian superior colliculus: studies of its morphology and connections. In: Vanegas H (ed) Comparative neurology of the optic tectum. Plenum, New York, pp 687–773

  • Hughes HC, Reuter-Lorenz PA, Nozawa G, Fendrich R (1994) Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. J Exp Psychol Hum Percept Perform 20:131–153

    CAS  PubMed  Google Scholar 

  • Jassik-Gerschenfeld D (1965) Somesthetic and visual responses of superior colliculus neurones. Nature 208:898–900

    CAS  PubMed  Google Scholar 

  • Jiang H, Lepore F, Ptito M, Guillemot JP (1994a) Sensory modality distribution in the anterior ectosylvian cortex (AEC) of cats. Exp Brain Res 97:404–414

    CAS  PubMed  Google Scholar 

  • Jiang H, Lepore F, Ptito M, Guillemot JP (1994b) Sensory interactions in the anterior ectosylvian cortex of cats. Exp Brain Res 101:385–396

    CAS  PubMed  Google Scholar 

  • Jiang W, Wallace MT, Jiang H, Vaughan JW, Stein BE (2001) Two cortical areas mediate multisensory integration in superior colliculus neurons. J Neurophysiol 85:506–522

    CAS  PubMed  Google Scholar 

  • Jiang W, Jiang H, Stein BE (2002) Two corticotectal areas facilitate multisensory orientation behavior. J Cogn Neurosci 14:1240–1255

    Article  PubMed  Google Scholar 

  • Jousmaki V, Hari R (1998) Parchment-skin illusion: sound biased touch. Curr Biol 8:R190

    Google Scholar 

  • Kao CQ, McHaffie JG, Meredith MA, Stein BE (1994) Functional development of a central visual map in cat. J Neurophysiol 72:266–272

    CAS  PubMed  Google Scholar 

  • King AJ, Hutchings ME, Moore DR, Blakemore C (1988) Developmental plasticity in the visual and auditory representations in the mammalian superior colliculus. Nature 332:73–76

    CAS  PubMed  Google Scholar 

  • King AJ, Palmer AR (1983) Cells responsive to free-field auditory stimuli in guinea-pig superior colliculus: distribution and response properties. J Physiol 342:361–381

    CAS  PubMed  Google Scholar 

  • King AJ, Palmer AR (1985) Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus. Exp Brain Res 60:492–500

    CAS  PubMed  Google Scholar 

  • Knudsen EI, Brainard MS (1991) Visual instruction of the neural map of auditory space in the developing optic tectum. Science 253:85–87

    CAS  PubMed  Google Scholar 

  • Knudsen EI, Brainard MS (1995) Creating a unified representation of visual and auditory space in the brain. Annu Rev Neurosci 18:19–43

    CAS  PubMed  Google Scholar 

  • Lewkowicz DJ (1994) Development of intersensory perception in human infants. In: Lewkowicz DJ, Lickliter R (eds) The development of intersensory perception: comparative perspectives. Erlbaum, Hillsdale, N.J., pp 165–203

  • Lovelace CT, Stein BE, Wallace MT (2003) An irrelevent light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Cogn Brain 17:447–453

    Article  Google Scholar 

  • McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748

    CAS  PubMed  Google Scholar 

  • McHaffie JG, Stein BE (1982) Eye movements evoked by electrical stimulation in the superior colliculus of rats and hamsters. Brain Res 247:243–253

    Article  CAS  PubMed  Google Scholar 

  • Meredith MA, Clemo HR (1989) Auditory cortical projection from the anterior ectosylvian sulcus (Field AES) to the superior colliculus in the cat: an anatomical and electrophysiological study. J Comp Neurol 289:687–707

    CAS  PubMed  Google Scholar 

  • Meredith MA, Stein BE (1983) Interactions among converging sensory inputs in the superior colliculus. Science 221:389–391

    CAS  PubMed  Google Scholar 

  • Meredith MA, Stein BE (1986a) Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol 56:640–662

    CAS  PubMed  Google Scholar 

  • Meredith MA, Stein BE (1986b) Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res 365:350–354

    CAS  PubMed  Google Scholar 

  • Meredith MA, Stein BE (1990) The visuotopic component of the multisensory map in the deep laminae of the cat superior colliculus. Neuroscience 10:3727–3742

    CAS  PubMed  Google Scholar 

  • Meredith MA, Nemitz JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurons. 1 Temporal factors. J Neurosci 7:3215–3229

    CAS  PubMed  Google Scholar 

  • Meredith MA, Clemo HR, Stein BE (1991) Somatotopic component of the multisensory map in the deep laminae of the cat superior colliculus. J Comp Neurol 312:353–370

    CAS  PubMed  Google Scholar 

  • Meredith MA, Wallace MT, Stein BE (1992) Visual, auditory and somatosensory convergence in output neurons of the cat superior colliculus: multisensory properties of the tecto-reticulo-spinal projection. Exp Brain Res 88:181–186

    CAS  PubMed  Google Scholar 

  • Middlebrooks JC, Knudsen EI (1984) A neural code for auditory space in the cat’s superior colliculus. J Neurosci 4:2621–2634

    CAS  PubMed  Google Scholar 

  • Mucke L, Norita M, Benedek G, Creutzfeldt O (1982) Physiologic and anatomic investigation of a visual cortical area situated in the ventral bank of the anterior ectosylvian sulcus of the cat. Exp Brain Res 46:1–11

    CAS  PubMed  Google Scholar 

  • Norita M, Mucke L, Benedek G, Albowitz B, Katoh Y, Creutzfeldt OD (1986) Connections of the anterior ectosylvian visual area (AEV). Exp Brain Res 62:225–240

    Google Scholar 

  • Norton TT (1974) Receptive-field properties of superior colliculus cells and development of visual behavior in kittens. J Neurophysiol 37:674–690

    CAS  PubMed  Google Scholar 

  • Olson CR, Graybiel AM (1987) Ectosylvian visual area of the cat: location, retinotopic organization, and connections. J Comp Neurol 261:277–294

    Google Scholar 

  • Palmer AR, King AJ (1985) A monaural space map in the guinea-pig superior colliculus. Hear Res 17:267–280

    Article  CAS  PubMed  Google Scholar 

  • Perrault TJ Jr, Vaughan JW, Stein BE, Wallace MT (2003) Neuron-specific response characteristics predict the magnitude of multisensory integration. J Neurophysiol 90:4022–4026

    PubMed  Google Scholar 

  • Piaget J (1952) The origins of intelligence in children. International Universities Press, New York

  • Robinson DA (1972) Eye movements evoked by collicular stimulation in the alert monkey. Vis Res 12:1795–1808

    Article  CAS  PubMed  Google Scholar 

  • Schiller PH, Stryker M (1972) Single-unit recording and stimulation in superior colliculus of the alert rhesus monkey. J Neurophysiol 35:915–924

    CAS  PubMed  Google Scholar 

  • Schroeder CE, Lindsley RW, Specht C, Marcovici A, Smiley JF, Javitt DC (2001) Somatosensory input to auditory association cortex in the macaque monkey. J Neurophysiol 85:1322–1327

    CAS  PubMed  Google Scholar 

  • Sekuler R, Sekuler AB, Lau R (1997) Sound alters visual motion perception. Nature 385:308

    Article  CAS  PubMed  Google Scholar 

  • Shams L, Kamitani Y, Shimojo S (2000) Illusions. What you see is what you hear. Nature 408:788

    Article  CAS  PubMed  Google Scholar 

  • Shams L, Kamitani Y, Shimojo S (2002) Visual illusion induced by sound. Brain Res Cogn Brain Res 14:147–152

    Article  PubMed  Google Scholar 

  • Sparks DL (1986) Translation of sensory signals into commands for control of saccadic eye movements: role of primate superior colliculus. Physiol Rev 66:118–171

    CAS  PubMed  Google Scholar 

  • Sprague JM, Meikle TH Jr (1965) The role of the superior colliculus in visually guided behavior. Exp Neurol 11:115–146

    Google Scholar 

  • Stein BE (1978) Nonequivalent visual, auditory, and somatic corticotectal influences in cat. J Neurophysiol 41:55–64

    CAS  PubMed  Google Scholar 

  • Stein BE, Arigbede MO (1972) Unimodal and multimodal response properties of neurons in the cat superior colliculus. Exp Neurol 36:179–196

    Google Scholar 

  • Stein BE, Clamann HP (1981) Control of pinna movements and sensorimotor register in cat superior colliculus. Brain Behav Evol 19:180–192

    CAS  PubMed  Google Scholar 

  • Stein BE, Dixon JP (1979) Properties of superior colliculus neurons in the golden hamster. J Comp Neurol 183:269–284

    CAS  PubMed  Google Scholar 

  • Stein BE, Meredith MA (1993) The merging of the senses. MIT Press, Cambridge, Mass.

  • Stein BE, Labos E, Kruger L (1973) Sequence of changes in properties of neurons of superior colliculus of the kitten during maturation. J Neurophysiol 36:667–679

    CAS  PubMed  Google Scholar 

  • Stein BE, Goldberg SJ, Clamann HP (1976a) The control of eye movements by the superior colliculus in the alert cat. Brain Res 118:469–474

    Article  CAS  PubMed  Google Scholar 

  • Stein BE, Magalhaes-Castro B, Kruger L (1976b) Relationship between visual and tactile representations in cat superior colliculus. J Neurophysiol 39:401–419

    CAS  PubMed  Google Scholar 

  • Stein BE, Spencer RF, Edwards SB (1983) Corticotectal and corticothalamic efferent projections of SIV somatosensory cortex in cat. J Neurophysiol 50:896–909

    CAS  PubMed  Google Scholar 

  • Stein BE, Huneycutt WS, Meredith MA (1988) Neurons and behavior: the same rules of multisensory integration apply. Brain Res 448:355–358

    Article  CAS  PubMed  Google Scholar 

  • Stein B, Meredith M, Huneycutt W, McDade L (1989) Behavioral indices of multisensory integration: orientation to visual cues is affected by auditory stimuli. J Cogn Neurosci 1:12–24

    Google Scholar 

  • Sumby WH, Pollack I (1954) Visual contribution to speech intelligibility in noise. J Acoust Soc Am 26:212–215

    Google Scholar 

  • Taylor-Clarke M, Kennett S, Haggard P (2004) Persistence of visual-tactile enhancement in humans. Neurosci Lett 354:22–25

    Article  CAS  PubMed  Google Scholar 

  • Tiao YC, Blakemore C (1976) Functional organization in the superior colliculus of the golden hamster. J Comp Neurol 168:483–503

    CAS  PubMed  Google Scholar 

  • Tortelly A, Reinoso-Suarez F, Llamas A (1980) Projections from non-visual cortical areas to the superior colliculus demonstrated by retrograde transport of HRP in the cat. Brain Res 188:543–549

    Article  CAS  PubMed  Google Scholar 

  • Wallace MT, Stein BE (1994) Cross-modal synthesis in the midbrain depends on input from cortex. J Neurophysiol 71:429–432

    CAS  PubMed  Google Scholar 

  • Wallace MT, Stein BE (1996) Sensory organization of the superior colliculus in cat and monkey. Prog Brain Res 112:301–311

    CAS  PubMed  Google Scholar 

  • Wallace MT, Stein BE (1997) Development of multisensory neurons and multisensory integration in cat superior colliculus. J Neurosci 17:2429–2444

    CAS  PubMed  Google Scholar 

  • Wallace MT, Stein BE (2000) Onset of cross-modal synthesis in the neonatal superior colliculus is gated by the development of cortical influences. J Neurophysiol 83:3578–3582

    CAS  PubMed  Google Scholar 

  • Wallace MT, Stein BE (2001) Sensory and multisensory responses in the newborn monkey superior colliculus. J Neurosci 21:8886–8894

    CAS  PubMed  Google Scholar 

  • Wallace MT, Meredith MA, Stein BE (1992) Integration of multiple sensory modalities in cat cortex. Exp Brain Res 91:484–488

    CAS  PubMed  Google Scholar 

  • Wallace MT, Meredith MA, Stein BE (1993) Converging influences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. J Neurophysiol 69:1797–1809

    CAS  PubMed  Google Scholar 

  • Wallace MT, Wilkinson LK, Stein BE (1996) Representation and integration of multiple sensory inputs in primate superior colliculus. J Neurophysiol 76:1246–1266

    CAS  PubMed  Google Scholar 

  • Wallace MT, Meredith MA, Stein BE (1998) Multisensory integration in the superior colliculus of the alert cat. J Neurophysiol 80:1006–1010

    CAS  PubMed  Google Scholar 

  • Wallace MT, Ramachandran R, Stein BE (2004) A revised view of sensory cortical parcellation. Proc Natl Acad Sci 101:2167–2172

    Article  CAS  PubMed  Google Scholar 

  • Werner H (1973) Comparative psychology of mental development. International Universities Press, New York

  • Wickelgren BG (1971) Superior colliculus: some receptive field properties of bimodally responsive cells. Science 173:69–72

    CAS  PubMed  Google Scholar 

  • Xing J, Andersen RA (2000) Models of the posterior parietal cortex which perform multimodal integration and represent space in several coordinate frames. J Cogn Neurosci 12:601–614

    CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Neurobiology and Anatomy, Wake Forest University School of Medicine, Medical Center Boulevard, Winston-Salem, NC, 27157, USA

    Mark T. Wallace

Authors
  1. Mark T. Wallace
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark T. Wallace.

Additional information

Edited by Marie-Hélène Giard and Mark Wallace

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wallace, M.T. The development of multisensory processes. Cogn Process 5, 69–83 (2004). https://doi.org/10.1007/s10339-004-0017-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10339-004-0017-z

Keywords

Search

Navigation