Single-Trial Multisensory Learning and Memory Retrieval
This chapter reviews our research investigating how the processing of unisensory visual objects is impacted by past experiences with multisensory auditory–visual versions of these objects. Single-trial exposure to multisensory objects during an orthogonal old/new discrimination task can incidentally modulate memory performance as well as brain activity. Performance was enhanced when the preceding multisensory stimulus had been a semantically congruent pairing, whereas performance was impaired by pairings with meaningless pure tones. Importantly, these opposing effects occurred despite identical performance patterns with the initial exposure to the objects (and presumably encoding thereof). In terms of brain activity, responses were enhanced to visual stimuli that had been previously encountered in a multisensory context within regions of the lateral occipital cortex beginning just 60 ms post-stimulus onset. Our collective findings indicate that single-trial learning of multisensory object associations occurs incidentally, is distinguishable from encoding processes, persists despite many intervening stimuli, and manifests as differential brain activity at early processing stages within visual cortices. The consequences of multisensory interactions thus persist over time to impact memory retrieval and object discrimination.
KeywordsAuditory Cortex Object Representation Rapid Serial Visual Presentation Repeated Presentation Repetition Blindness
- Besle J, Bertrand O, Giard MH (2009) Electrophysiological (EEG, sEEG, MEG) evidence for multiple audiovisual interactions in the human auditory cortex. Hear Res 258(1–2):143–151Google Scholar
- Cappe C, Morel A, Barone P, Rouiller EM (2009) The thalamocortical projection systems in primate: an anatomical support for multisensory and sensorimotor integrations. Cereb Cortex. 19(9):2025–2037, doi:10.1093/cercor/bhn228Google Scholar
- Holmes NP (2009) The principle of inverse effectiveness in multisensory integration: Some statistical considerations. Brain Topogr 21(3–4):168–176Google Scholar
- Naumer MJ, Doehrmann O, Müller NG, Muckli L, Kaiser J, Hein G (2009) Cortical plasticity of audio-visual object representations. Cereb Cortex. 19(7):1641–1653, doi: 10.1093/cercor/ bhn200Google Scholar
- Stein BE, Meredith MA (1993) The merging of the senses. MIT Press, Cambridge, MAGoogle Scholar