The noisy encoding of disparity model of the McGurk effect
- 1.1k Downloads
In the McGurk effect, incongruent auditory and visual syllables are perceived as a third, completely different syllable. This striking illusion has become a popular assay of multisensory integration for individuals and clinical populations. However, there is enormous variability in how often the illusion is evoked by different stimuli and how often the illusion is perceived by different individuals. Most studies of the McGurk effect have used only one stimulus, making it impossible to separate stimulus and individual differences. We created a probabilistic model to separately estimate stimulus and individual differences in behavioral data from 165 individuals viewing up to 14 different McGurk stimuli. The noisy encoding of disparity (NED) model characterizes stimuli by their audiovisual disparity and characterizes individuals by how noisily they encode the stimulus disparity and by their disparity threshold for perceiving the illusion. The model accurately described perception of the McGurk effect in our sample, suggesting that differences between individuals are stable across stimulus differences. The most important benefit of the NED model is that it provides a method to compare multisensory integration across individuals and groups without the confound of stimulus differences. An added benefit is the ability to predict frequency of the McGurk effect for stimuli never before seen by an individual.
KeywordsMcGurk effect Individual differences Speech perception Multisensory integration
This research was supported by NIH R01NS065395 to MSB. We thank Wei Ji Ma and xaq pitkow for helpful comments.
- Beauchamp, M. S., Nath, A. R., & Pasalar, S. (2010). fMRI-Guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect. Journal of Neuroscience, 30(7), 2414–2417. doi: 10.1523/JNEUROSCI.4865-09.2010 CrossRefPubMedCentralPubMedGoogle Scholar
- Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge; New York: Cambridge University Press.Google Scholar
- Massaro, D. W. (1998). Perceiving talking faces : from speech perception to a behavioral principle. Cambridge, Mass.: MIT Press.Google Scholar
- Mongillo, E. A., Irwin, J. R., Whalen, D. H., Klaiman, C., Carter, A. S., & Schultz, R. T. (2008). Audiovisual processing in children with and without autism spectrum disorders. Journal of Autism and Developmental Disorders, 38(7), 1349–1358. doi: 10.1007/s10803-007-0521-y CrossRefPubMedGoogle Scholar
- R Core Team. (2012). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.Google Scholar
- Saalasti, S., Katsyri, J., Tiippana, K., Laine-Hernandez, M., von Wendt, L., & Sams, M. (2012). Audiovisual speech perception and eye gaze behavior of adults with asperger syndrome. Journal of Autism and Developmental Disorders, 42(8), 1606–1615. doi: 10.1007/s10803-011-1400-0 CrossRefPubMedGoogle Scholar
- Stevenson, R. A., Siemann, J. K., Schneider, B. C., Eberly, H. E., Woynaroski, T. G., Camarata, S. M., & Wallace, M. T. (2014a). Multisensory temporal integration in autism spectrum disorders. Journal of Neuroscience, 34(3), 691–697. doi: 10.1523/JNEUROSCI.3615-13.2014 CrossRefPubMedCentralPubMedGoogle Scholar
- Stevenson, R. A., Siemann, J. K., Woynaroski, T. G., Schneider, B. C., Eberly, H. E., Camarata, S. M., & Wallace, M. T. (2014b). Brief report: Arrested development of audiovisual speech perception in autism spectrum disorders. Journal of Autism and Developmental Disorders, 44(6), 1470–1477. doi: 10.1007/s10803-013-1992-7 CrossRefPubMedGoogle Scholar
- Stevenson, R. A., Zemtsov, R. K., & Wallace, M. T. (2012). Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions. Journal of Experimental Psychology: Human Perception and Performance, 38(6), 1517–1529. doi: 10.1037/a0027339 PubMedCentralPubMedGoogle Scholar
- Taylor, N., Isaac, C., & Milne, E. (2010). A comparison of the development of audiovisual integration in children with autism spectrum disorders and typically developing children. Journal of Autism and Developmental Disorders, 40(11), 1403–1411. doi: 10.1007/s10803-010-1000-4 CrossRefPubMedGoogle Scholar