Abstract
Since its invention almost two decades ago functional magnetic resonance imaging (fMRI) has become the prime research methodology in human neuroscience. Its capabilities continue to evolve based on combined improvements of scanner hardware, experimental designs, and data analysis tools. Within the rapidly growing field of multisensory research the use of noninvasive neuroimaging techniques in general and fMRI in particular is also of increasing relevance. For several years, discussion in the multisensory fMRI community has mainly focused on principles of and statistical criteria for multisensory integration. The recent availability of more sophisticated experimental designs and increasingly sensitive (multivariate) analysis tools allows multisensory researchers to (noninvasively) differentiate between regional and neuronal convergence and to reveal the connectional basis of human multisensory integration.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Beauchamp MS (2005) Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 3:93–113
Beauchamp MS, Laconte S, Yasar N (2009) Distributed representation of single touches in somatosensory and visual cortex. Hum Brain Mapp 30:3163–3171
Beauchamp MS, Ro T (2008) Neural substrates of sound-touch synesthesia after a thalamic lesion. J Neurosci 28:13696–13702
Calvert GA (2001) Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11:1110–1123
Calvert GA, Hansen PC, Iversen SD, Brammer MJ (2001) Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage 14:427–438
Cappe C, Morel A, Barone P, Rouiller EM (2009) The thalamocortical projection systems in primate: an anatomical support for multisensory and sensorimotor interplay. Cerebral Cortex, Jan 15 [Epub ahead of print]
David O, Cosmelli D, Friston KJ (2004) Evaluation of different measures of functional connectivity using a neural mass model. Neuroimage 21:659–673
Deshpande G, Hu X, Stilla R, Sathian K (2008) Effective connectivity during haptic perception: a study using Granger causality analysis of functional magnetic resonance imaging data. Neuroimage 40:1807–1814
Doehrmann O, Naumer MJ (2008) Semantics and the multisensory brain: how meaning modulates processes of audio-visual integration. Brain Res 1242:136–150
Doehrmann O, Weigelt S, Altmann CF, Kaiser J, Naumer MJ (2010) Audio-visual fMRI adaptation reveals multisensory integration effects in object-related sensory cortices J Neurosci 30:3370–3379
Driver J, Noesselt T (2008) Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments. Neuron 57:11–23
Eckert MA, Kamdar NV, Chang CE, Beckmann CF, Greicius MD, Menon V (2008) A cross-modal system linking primary auditory and visual cortices: evidence from intrinsic fMRI connectivity analysis. Hum Brain Mapp 29:848–857
Etzel JA, Gazzola V, Keysers C (2008) Testing simulation theory with cross-modal multivariate classification of fMRI data. PLoS ONE 3:e3690
Fischl B, Sereno MI, Tootel RBH, Dale AM (1999) High-resolution intersubject averaging and a coordinate system for the cortical surface. Hum Brain Mapp 8:272–284
Friston KJ, Buechel C, Fink GR, Morris J, Rolls E, Dolan RJ (1997) Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 6:218–229
Friston KJ, Harrison L, Penny W (2003) Dynamic causal modelling. Neuroimage 19:1273–1302
Goebel R, Esposito F, Formisano E (2006) Analysis of Functional Image Analysis Contest (FIAC) data with BrainVoyager QX: from single-subject to cortically aligned group general linear model analysis and self-organizing group independent component analysis. Hum Brain Mapp 27:392–402
Goebel R, van Atteveldt N (2009) Multisensory functional magnetic resonance imaging: a future perspective. Exp Brain Res 198:153–164
Grill-Spector K, Malach R (2001) fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychol (Amst) 107:293–321
Haller S, Wetzel SG, Radue EW, Bilecen D (2006) Mapping continuous neuronal activation without an ON–OFF paradigm: initial results of BOLD ceiling fMRI. Eur J Neurosci 24:2672–2678
Hasson U, Skipper JI, Nusbaum HC, Small SL (2007) Abstract coding of audiovisual speech: beyond sensory representation. Neuron 56:1116–1126
Haynes JD, Rees G (2006) Decoding mental states from brain activity in humans. Nat Rev Neurosci 7:523–534
Haxby JV, Gobbini MI, Furey ML, Ishai A, Schouten JL, Pietrini P (2001) Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293:2425–2430
Hein G, Doehrmann O, Muller NG, Kaiser J, Muckli L, Naumer MJ (2007) Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J Neurosci 27:7881–7887
Hocking J, Price CJ (2008) The role of the posterior superior temporal sulcus in audiovisual processing. Cereb Cortex 18:2439–2449
James TW, Humphrey GK, Gati JS, Servos P, Menon RS, Goodale MA (2002) Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia 40:1706–1714
Kim S, James TW (2009) Enhanced effectiveness in visuo-haptic object-selective brain regions with increasing stimulus salience. Hum Brain Mapp, Oct 14 [Epub ahead of print]
Kriegeskorte N, Mur M, Bandettini P (2008a) Representational similarity analysis - connecting the branches of systems neuroscience. Front Syst Neurosci 2:4
Kriegeskorte N, Mur M, Ruff DA, Kiani R, Bodurka J, Esteky H, Tanaka K, Bandettini PA (2008b) Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron 60:1126–1141
Lacey S, Tal N, Amedi A, Sathian K (2009) A putative model of multisensory object representation. Brain Topogr 21:269–274
Laurienti PJ, Perrault TJ, Stanford TR, Wallace MT, Stein BE (2005) On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies. Exp Brain Res 166:289–297
Logothetis NK (2008) What we can do and what we cannot do with fMRI. Nature 453:869–878
Meienbrock A, Naumer MJ, Doehrmann O, Singer W, Muckli L (2007) Retinotopic effects during spatial audio-visual integration. Neuropsychologia 45:531–539
Mur M, Bandettini PA, Kriegeskorte N (2009) Revealing representational content with pattern-information fMRI--an introductory guide. Soc Cogn Affect Neurosci 4:101–109
Naumer MJ, Ratz L, Yalachkov Y, Polony A, Doehrmann O, Müller NG, Kaiser J, Hein G (2010) Visuo-haptic convergence in a cortico-cerebellar network. Eur J Neurosci (in press)
Naumer MJ, van den Bosch JJF (2009) Touching sounds: thalamo-cortical plasticity and the neural basis of multisensory integration. J Neurophysiol 102:7–8
Naumer MJ, van den Bosch JJF, Wibral M, Kohler A, Singer W, Kaiser J, van de Ven V, Muckli L (2009b) Audio-visual integration in the human brain: data-driven detection and independent validation.
Niogi SN, McCandliss BD (2006) Left lateralized white matter microstructure accounts for individual differences in reading ability and disability. Neuropsychologia 44:2178–2188
Noesselt T, Rieger JW, Schoenfeld MA, Kanowski M, Hinrichs H, Heinze HJ, Driver J (2007) Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J Neurosci 27:11431–11441
Noppeney U Characterization of multisensory integration with fMRI - experimental design, statistical analysis and interpretation. In: Wallace M, Murray M (eds) Frontiers in the neural bases of multisensory processes. Taylor and Francis Group, London (in press)
Noppeney U, Josephs O, Hocking J, Price CJ, Friston KJ (2008) The Effect of Prior Visual Information on Recognition of Speech and Sounds. Cereb Cortex 18:598–609
Pietrini P, Furey ML, Ricciardi E, Gobbini MI, Wu W-HC, Cohen L, Guazzelli M, Haxby JV (2004) Beyond sensory images: objectbased representation in the human ventral pathway. Proc Natl Acad Sci USA 101:5658–5663
Polony A, Ratz L, Doehrmann O, Kaiser J, Naumer MJ (2007) Audio-tactile integration of meaningful objects in the human brain. In: Annual Meeting of the International Multisensory Research Forum, Sydney, Australia
Roebroeck A, Formisano E, Goebel R (2005) Mapping directed influence over the brain using Granger causality and fMRI. Neuroimage 25:230–242
Roebroeck A, Formisano E, Goebel R (2009a) The identification of interacting networks in the brain using fMRI: Model selection, causality and deconvolution. Neuroimage, Sep 25 [Epub ahead of print]
Roebroeck A, Formisano E, Goebel R (2009b) Reply to Friston and David After comments on: The identification of interacting networks in the brain using fMRI: Model selection, causality and deconvolution. Neuroimage, Oct 31 [Epub ahead of print]
Rouw R, Scholte HS (2007) Increased structural connectivity in grapheme-color synesthesia. Nat Neurosci 10:792–797
Stein BE, Meredith MA (1993) The merging of the senses. Cambridge, Massachussetts: MIT Press
Stein BE, Stanford TR (2008) Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci 9:255–266
Stevenson RA, Geoghegan ML, James TW (2007) Superadditive BOLD activation in superior temporal sulcus with threshold nonspeech objects. Exp Brain Res 179:85–95
Stevenson RA, James TW (2009) Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 44:1210–1223
Stevenson RA, Kim S, James TW (2009) An additive-factors design to disambiguate neuronal and areal convergence: measuring multisensory interactions between audio, visual, and haptic sensory streams using fMRI. Exp Brain Res 198:183–194
Tal N, Amedi A (2009) Multisensory visual-tactile object related network in humans: insights gained using a novel crossmodal adaptation approach. Exp Brain Res 198:165–182
van Atteveldt N, Blau V, Blomert L, Goebel R (2008) fMR-adaptation reveals multisensory integration in human superior temporal cortex. In: Annual Meeting of the International Multisensory Research Forum, Hamburg, Germany
van Atteveldt NM, Formisano E, Blomert L, Goebel R (2007) The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb Cortex 17:962–974
van Atteveldt N, Roebroeck A, Goebel R (2009) Interaction of speech and script in human auditory cortex: Insights from neuroimaging and effective connectivity. Hear Res 258:152–164
van Essen DC, Drury HA, Joshi S, Miller MI (1998) Functional and structural mapping of human cerebral cortex: solutions are in the surfaces. Proc Natl Acad Sci U S A 95:788–95
von Kriegstein K, Giraud AL (2006) Implicit multisensory associations influence voice recognition. PLoS Biol 4:e326
von Kriegstein K, Kleinschmidt A, Sterzer P, Giraud AL (2005) Interaction of face and voice areas during speaker recognition. J Cogn Neurosci 17:367–376
Weigelt S, Muckli L, Kohler A (2008) Functional magnetic resonance adaptation in visual neuroscience. Rev Neurosci 19:363–380
Werner S, Noppeney U (2009) Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Nov 18 [Epub ahead of print]
Acknowledgments
This work was supported by the German Ministry of Education and Research (BMBF) and Frankfurt Medical School (Intramural Young Investigator Program to M.J.N.). We are grateful to Sarah Weigelt for helpful suggestions and Christoph Bledowski and Yavor Yalachkov for their helpful comments to an earlier version of this chapter.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer Science + Business Media, LLC
About this chapter
Cite this chapter
Naumer, M.J., van den Bosch, J.J.F., Polony, A., Kaiser, J. (2010). Multisensory Functional Magnetic Resonance Imaging. In: Kaiser, J., Naumer, M. (eds) Multisensory Object Perception in the Primate Brain. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-5615-6_6
Download citation
DOI: https://doi.org/10.1007/978-1-4419-5615-6_6
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-5614-9
Online ISBN: 978-1-4419-5615-6
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)