Skip to main content
SpringerLink
Log in

Is the auditory sensory memory sensitive to visual information?

  • Research Article
  • Published:
Experimental Brain Research Aims and scope Submit manuscript

Abstract

The mismatch negativity (MMN) component of auditory event-related brain potentials can be used as a probe to study the representation of sounds in auditory sensory memory (ASM). Yet it has been shown that an auditory MMN can also be elicited by an illusory auditory deviance induced by visual changes. This suggests that some visual information may be encoded in ASM and is accessible to the auditory MMN process. It is not known, however, whether visual information affects ASM representation for any audiovisual event or whether this phenomenon is limited to specific domains in which strong audiovisual illusions occur. To highlight this issue, we have compared the topographies of MMNs elicited by non-speech audiovisual stimuli deviating from audiovisual standards on the visual, the auditory, or both dimensions. Contrary to what occurs with audiovisual illusions, each unimodal deviant elicited sensory-specific MMNs, and the MMN to audiovisual deviants included both sensory components. The visual MMN was, however, different from a genuine visual MMN obtained in a visual-only control oddball paradigm, suggesting that auditory and visual information interacts before the MMN process occurs. Furthermore, the MMN to audiovisual deviants was significantly different from the sum of the two sensory-specific MMNs, showing that the processes of visual and auditory change detection are not completely independent.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. As, on the one hand, the topography of the auditory MMN is well known and on the other hand, it would have needlessly lengthened the recording session, we chose not to conduct an auditory oddball paradigm

References

  • Alain C, Woods DL, Knight RT (1998) A distributed cortical network for auditory sensory memory in humans. Brain Res 812:23–37

    PubMed  Google Scholar 

  • Bental E, Dafny N, Feldman S (1968) Convergence of auditory and visual stimuli on single cells in the primary visual cortex of unanesthetized unrestrained cats. Exp Neurol 20:341–351

    PubMed  Google Scholar 

  • Bertelson P, Aschersleben G (1998) Automatic visual bias of perceived auditory location. Psychon Bull Rev 5:482–489

    Google Scholar 

  • Berti S, Schroger E (2004) Distraction effects in vision: behavioral and event-related potential indices. Neuroreport 15:665–669

    PubMed  Google Scholar 

  • Besle J, Fort A, Delpuech C, Giard M-H (2004) Bimodal speech: early suppressive visual effects in the human auditory cortex. Eur J Neurosci 20:2225–2234

    PubMed  Google Scholar 

  • Bruneau N, Roux S, Garreau B, Martineau J, Lelord G (1990) Cortical evoked potentials as indicators of Auditory-Visual Cross-Modal Association in young adults. Pavlov J Biol Sci 25:189–204

    Google Scholar 

  • Cahill L, Ohl F, Scheich H (1996) Alteration of auditory cortex activity with a visual stimulus through conditioning: a 2-deoxyglucose analysis. Neurobiol Learn Mem 65:213–222

    PubMed  Google Scholar 

  • Colin C, Radeau M, Soquet A, Demolin D, Colin F, Deltenre P (2002b) Mismatch negativity evoked by the McGurk–MacDonald effect: a phonetic representation within short-term memory. Clin Neurophysiol 113:495–506

    PubMed  Google Scholar 

  • Colin C, Radeau M, Soquet A, Dachy B, Deltenre P (2002a) Electrophysiology of spatial scene analysis: the mismatch negativity (MMN) is sensitive to the ventriloquism illusion. Clin Neurophysiol 113:507–518

    PubMed  Google Scholar 

  • Colin C, Radeau M, Soquet A, Deltenre P (2004) Generalization of the generation of an MMN by illusory McGurk percepts: voiceless consonants. Clin Neurophysiol 115:1989–2000

    PubMed  Google Scholar 

  • Czigler I, Balazs L, Winkler I (2002) Memory-based detection of task-irrelevant visual changes. Psychophysiology 39:869–873

    PubMed  Google Scholar 

  • Fort A, Giard M-H (2004) Multiple electrophysiological mechanisms of audio-visual integration in human perception. In: Calvert G, Spence C, Stein B (eds) The handbook of multisensory processes. MIT Press, Cambridge

    Google Scholar 

  • Fort A, Delpuech C, Pernier J, Giard MH (2002a) Dynamics of cortico-subcortical crossmodal operations involved in audio-visual object detection in humans. Cereb Cortex 12:1031–1039

    PubMed  Google Scholar 

  • Fort A, Delpuech C, Pernier J, Giard MH (2002b) Early auditory-visual interactions in human cortex during nonredundant target identification. Brain Res Cogn Brain Res 14:20–30

    PubMed  Google Scholar 

  • Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473–490

    PubMed  Google Scholar 

  • Giard MH, Perrin F, Pernier J (1990) Brain generators implicated in processing of auditory stimulus deviance A topographic ERP study. Psychophysiology 27:627–640

    PubMed  Google Scholar 

  • Heslenfeld DJ (2003) Visual mismatch negativity. In: Polich J (ed) Detection of change: event-related potential and fMRI findings. Kluwer Academic Publishers, Dordrecht, pp 41–60

    Google Scholar 

  • Kenemans JL, Jong TG, Verbaten MN (2003) Detection of visual change: mismatch or rareness?. Neuroreport 14:1239–1242

    PubMed  Google Scholar 

  • Kropotov JD, Näätänen R, Sevostianov AV, Alho K, Reinikainen K, Kropotova OV (1995) Mismatch negativity to auditory stimulus change recorded directly from the human temporal cortex. Psychophysiology 32:418–422

    PubMed  Google Scholar 

  • Lebib R, Papo D, de Bode S, Baudonniere PM (2003) Evidence of a visual-to-auditory cross-modal sensory gating phenomenon as reflected by the human P50 event-related brain potential modulation. Neurosci Lett 341:185–188

    PubMed  Google Scholar 

  • McGurk H, McDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748

    PubMed  Google Scholar 

  • Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res Cogn Brain Res 14:115–128

    PubMed  Google Scholar 

  • Möttönen R, Krause CM, Tiippana K, Sams M (2002) Processing of changes in visual speech in the human auditory cortex. Brain Res Cogn Brain Res 13:417–425

    PubMed  Google Scholar 

  • Näätänen R (1992) Attention and Brain Function. Hillsdale, NJ, USA

    Google Scholar 

  • Näätänen R, Winkler I (1999) The concept of auditory stimulus representation in cognitive neuroscience. Psychol Bull 125:826–859

    PubMed  Google Scholar 

  • Nyman G, Alho K, laurinen P, Paavilainen P, Radil T, Rainikainen K, Sams M, Näätänen R (1990) Mismatch negativity (MMN) for sequences of auditory and visual stimuli: evidence for a mechanism specific to the auditory modality. Electroencephalogr Clin Neurophysiol 77:436–444

    PubMed  Google Scholar 

  • Pazo-Alvarez P, Cadaveira F, Amenedo E (2003) MMN in the visual modality: a review. Biol Psychol 63:199–236

    PubMed  Google Scholar 

  • Pernier J, Perrin F, Bertrand O (1988) Scalp current density fields: concept and properties. Electroencephalogr Clin Neurophysiol 69:385–389

    PubMed  Google Scholar 

  • Perrin F, Pernier J, Bertrand O, Giard M-H (1987) Mapping of scalp potentials by surface spline interpolation. Electroencephalogr Clin Neurophysiol 66:75–81

    PubMed  Google Scholar 

  • Perrin F, Pernier J, Bertrand O, Echallier JF (1989) Spherical splines for scalp potential and current density mapping. Electroencephalogr Clin Neurophysiol 72:184–187

    PubMed  Google Scholar 

  • Ritter W, Deacon D, Gomes H, Javitt DC, Vaughan HG Jr (1995) The mismatch negativity of event-related potentials as a probe of transient auditory memory: a review. Ear Hear 16:52–67

    PubMed  Google Scholar 

  • Rosenblum LD, Fowler CA (1991) Audiovisual investigation of the loudness-effort effect for speech and nonspeech events. J Exp Psychol Hum Percept Perform 17:976–985

    PubMed  Google Scholar 

  • Saldana HM, Rosenblum LD (1993) Visual influences on auditory pluck and bow judgments. Percept Psychophys 54:406–416

    PubMed  Google Scholar 

  • Sams M, Aulanko R, Hamalainen H, Hari R, Lounasmaa OV, Lu ST, Simola J (1991) Seeing speech: visual information from lip movements modifies activity in the human auditory cortex. Neurosci Lett 127:141–145

    PubMed  Google Scholar 

  • Soto-Faraco S, Navarra J, Alsius A (2004) Assessing automaticity in audiovisual speech integration: evidence from the speeded classification task. Cognition 92:B13–B23

    PubMed  Google Scholar 

  • Stagg C, Hindley P, Tales A, Butler S (2004) Visual mismatch negativity: the detection of stimulus change. Neuroreport 15:659–663

    PubMed  Google Scholar 

  • Stekelenburg JJ, Vroomen J, de Gelder B (2004) Illusory sound shifts induced by the ventriloquist illusion evoke the mismatch negativity. Neurosci Lett 357:163–166

    PubMed  Google Scholar 

  • Vroomen J, Bertelson P, de Gelder B (2001) The ventriloquist effect does not depend on the direction of automatic visual attention. Percept Psychophys 63:651–659

    PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Univ. Lyon 2, Lyon, France

    Julien Besle

  2. Univ. Lyon 1, Lyon, France

    Julien Besle, Alexandra Fort & Marie-Hélène Giard

  3. IFNL IFR19, Lyon, France

    Julien Besle, Alexandra Fort & Marie-Hélène Giard

  4. INSERM U280, Mental Processes and Brain Activation, 69675, Bron Cedex, France

    Julien Besle, Alexandra Fort & Marie-Hélène Giard

Authors
  1. Julien Besle
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexandra Fort
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marie-Hélène Giard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julien Besle.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Besle, J., Fort, A. & Giard, MH. Is the auditory sensory memory sensitive to visual information?. Exp Brain Res 166, 337–344 (2005). https://doi.org/10.1007/s00221-005-2375-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-005-2375-x

Keywords

Search

Navigation