Journal of Autism and Developmental Disorders

, Volume 43, Issue 12, pp 2891–2902 | Cite as

Multisensory Speech Perception in Children with Autism Spectrum Disorders

  • Tiffany G. WoynaroskiEmail author
  • Leslie D. Kwakye
  • Jennifer H. Foss-Feig
  • Ryan A. Stevenson
  • Wendy L. Stone
  • Mark T. Wallace
Original Paper


This study examined unisensory and multisensory speech perception in 8–17 year old children with autism spectrum disorders (ASD) and typically developing controls matched on chronological age, sex, and IQ. Consonant–vowel syllables were presented in visual only, auditory only, matched audiovisual, and mismatched audiovisual (“McGurk”) conditions. Participants with ASD displayed deficits in visual only and matched audiovisual speech perception. Additionally, children with ASD reported a visual influence on heard speech in response to mismatched audiovisual syllables over a wider window of time relative to controls. Correlational analyses revealed associations between multisensory speech perception, communicative characteristics, and responses to sensory stimuli in ASD. Results suggest atypical speech perception is linked to broader behavioral characteristics of ASD.


Autism spectrum disorders Speech perception Multisensory integration Auditory Visual McGurk effect Sensory Communication 


  1. American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders-IV-TR. Washington, DC: APA.Google Scholar
  2. American Psychiatric Association. (2012). DSM-5 development: autism spectrum disorder. Retrieved August 8, 2012, from
  3. Bebko, J. M., Weiss, J. A., Demark, J. L., & Gomez, P. (2006). Discrimination of temporal synchrony in intermodal events by children with autism and children with developmental disabilities without autism. Journal of Child Psychology and Psychiatry, 47(1), 88–98. doi: 10.1111/j.1469-7610.2005.01443.x.PubMedCrossRefGoogle Scholar
  4. Bertone, A., Mottron, L., Jelenic, P., & Faubert, J. (2005). Enhanced and diminished visuo-spatial information processing in autism depends on stimulus complexity. Brain, 128(10), 2430–2441. doi: 10.1093/brain/awh561.PubMedCrossRefGoogle Scholar
  5. Billstedt, E., Carina Gillberg, I., & C, Gillberg. (2007). Autism in adults symptom patterns and early childhood predictors. Use of the DISCO in a community sample followed from childhood. Journal of Child Psychology and Psychiatry, 48(11), 1102–1110. doi: 10.1111/j.1469-7610.2007.01774.x.PubMedCrossRefGoogle Scholar
  6. Blake, R., Turner, L. M., Smoski, M. J., Pozdol, S. L., & Stone, W. L. (2003). Visual recognition of biological motion is impaired in children with autism. Psychological Science, 14(2), 151–157.PubMedCrossRefGoogle Scholar
  7. Brock, J., Brown, C., Boucher, J., & Rippon, G. (2002). The temporal binding deficit hypothesis of autism. Development and Psychopathology, 14(2), 209–224. doi: 10.1017/S0954579402002018.PubMedCrossRefGoogle Scholar
  8. Calvert, G. A., & Thesen, T. (2004). Multisensory integration: methodological approaches and emerging principles in the human brain. Journal of Physiology Paris, 98, 191–205. doi: 10.1016/j.jphysparis.2004.03.018.CrossRefGoogle Scholar
  9. Dawson, G., Webb, S. J., & McPartland, J. (2005). Understanding the nature of face processing impairment in autism: insights from behavioral and electrophysiological studies. Developmental Neuropsychology, 27(3), 403–424. doi: 10.1207/s15326942dn2703_6.PubMedCrossRefGoogle Scholar
  10. de Gelder, B., Vroomen, J., & Van der Heide, L. (1991). Face recognition and lip-reading in autism. European Journal of Cognitive Psychology, 3(1), 69–86.CrossRefGoogle Scholar
  11. Dunn, W. (1999). Sensory profile. San Antonio, TX: Psychological Corporation.Google Scholar
  12. Foss-Feig, J. H., Kwakye, L. D., Cascio, C. J., Burnette, C. P., Kadivar, H., Stone, W. L., et al. (2010). An extended multisensory temporal binding window in autism spectrum disorders. Experimental Brain Research, 203(2), 381–389. doi: 10.1007/s00221-010-2240-4.CrossRefGoogle Scholar
  13. Frith, U., & Happé, F. (1994). Autism: beyond “theory of mind”. Cognition, 50(1–3), 115–132. doi: 10.1016/0010-0277(94)90024-8.PubMedCrossRefGoogle Scholar
  14. Gentilucci, M., & Cattaneo, L. (2005). Automatic audiovisual integration in speech perception. Experimental Brain Research, 167(1), 66–75. doi: 10.1007/s00221-005-0008-z.CrossRefGoogle Scholar
  15. Gervais, H., Belin, P., Boddaert, N., Leboyer, M., Coez, A., Sfaello, I., et al. (2004). Abnormal cortical voice processing in autism. Nature Neuroscience, 7(8), 801–802. doi: 10.1038/nn1291nn1291.PubMedCrossRefGoogle Scholar
  16. Grossman, R. B., Schneps, M. H., & Tager-Flusberg, H. (2009). Slipped lips: onset asynchrony detection of auditory-visual language in autism. Journal of Child Psychology and Psychiatry, 50(4), 491–497. doi: 10.1111/j.1469-7610.2008.02002.x.PubMedCrossRefGoogle Scholar
  17. Happe, F. (2005). The weak central coherence account of autism. In Paul Volkmar & Cohen Klin (Eds.), Handbook of autism and pervasive developmental disorders, Vol 1: Diagnosis, development, neurobiology, and behavior (3rd ed) (pp. 640–649). New York: John Wiley & Sons.Google Scholar
  18. Happé, F., & Frith, U. (2006). The weak coherence account: Detail-focused cognitive style in autism spectrum disorders. Journal of Autism and Developmental Disorders, 36(1), 5–25. doi: 10.1007/s10803-005-0039-0.PubMedCrossRefGoogle Scholar
  19. Hutt, C., & Ounsted, C. (1966). The biological significance of gaze aversion with particular reference to the syndrome of infantile autism. Behavioral Science, 11(5), 346–356. doi: 10.1002/bs.3830110504.PubMedCrossRefGoogle Scholar
  20. Iarocci, G., & McDonald, J. (2006). Sensory integration and the perceptual experience of persons with autism. Journal of Autism and Developmental Disorders, 36(1), 77–90. doi: 10.1007/s10803-005-0044-3.PubMedCrossRefGoogle Scholar
  21. Iarocci, G., Rombough, A., Yager, J., Weeks, D. J., & Chua, R. (2010). Visual influences on speech perception in children with autism. Autism, 14(4), 305–320. doi: 10.1177/1362361309353615.PubMedCrossRefGoogle Scholar
  22. Irwin, J. R., Tornatore, L. A., Brancazio, L., & Whalen, D. H. (2011). Can children with autism spectrum disorders “hear” a speaking face? Child Development, 82(5), 1397–1403. doi: 10.1111/j.1467-8624.2011.01619.x.PubMedCrossRefGoogle Scholar
  23. Kellerman, G. R., Fan, J., & Gorman, J. M. (2005). Auditory abnormalities in autism: Toward functional distinctions among findings. CNS Spectrums, 10(9), 748–756.PubMedGoogle Scholar
  24. Klin, A. (1991). Young autistic children’s listening preferences in regard to speech: A possible characterization of the symptom of social withdrawal. Journal of Autism and Developmental Disorders, 21(1), 29–42.PubMedCrossRefGoogle Scholar
  25. Kuhl, P., Coffey-Corina, S., Padden, D., & Dawson, G. (2005). Links between social and linguistic processing of speech in preschool children with autism: Behavioral and electrophysiological measures. Developmental Science, 8(1), F1–F12. doi: 10.1111/j.1467-7687.2004.00384.x.PubMedCrossRefGoogle Scholar
  26. Kwakye, L. D., Foss-Feig, J. H., Cascio, C. J., Stone, W. L., & Wallace, M. T. (2011). Altered auditory and multisensory temporal processing in autism spectrum disorders. Frontiers in Integrative Neuroscience, 4, 1–11. doi: 10.3389/fnint.2010.00129.CrossRefGoogle Scholar
  27. Lansing, C. R., & McConkie, G. W. (2003). Word identification and eye fixation locations in visual and visual-plus-auditory presentations of spoken sentences. Attention, Perception, and Psychophysics, 65(4), 536–552.CrossRefGoogle Scholar
  28. Lord, C., Risi, S., Lambrecht, L., Cook, E. H, Jr, Leventhal, B. L., DiLavore, P. C., et al. (2000). The autism diagnostic observation schedule-generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30(3), 205–223.PubMedCrossRefGoogle Scholar
  29. Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism diagnostic interview-revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24(5), 659–685.PubMedCrossRefGoogle Scholar
  30. Macdonald, J., & McGurk, H. (1978). Visual influences on speech perception processes. Attention, Perception, & Psychophysics, 24(3), 253–257. doi: 10.3758/bf03206096.CrossRefGoogle Scholar
  31. Magnée, M., De Gelder, B., Van Engeland, H., & Kemner, C. (2008). Audiovisual speech integration in pervasive developmental disorder: Evidence from event-related potentials. Journal of Child Psychology and Psychiatry, 49(9), 995–1000. doi: 10.1111/j.1469-7610.2008.01902.x.PubMedCrossRefGoogle Scholar
  32. Massaro, D. (1998). Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, Massachusetts.: MIT Press.Google Scholar
  33. Massaro, D., & Bosseler, A. (2003). Perceiving speech by ear and eye: Multimodal integration by children with autism. Journal of Developmental and Learning Disorders, 7, 111–144.Google Scholar
  34. McGurk, H., & Macdonald, J. (1976). Hearing lips and seeing voices. Nature, 264(5588), 746–748.PubMedCrossRefGoogle Scholar
  35. Mongillo, E., Irwin, J., Whalen, D., Klaiman, C., Carter, A., & Schultz, R. (2008). Audiovisual processing in children with and without autism spectrum disorders. Journal of Autism and Developmental Disorders, 38(7), 1349–1358. doi: 10.1007/s10803-007-0521-y.PubMedCrossRefGoogle Scholar
  36. Nath, A. R., & Beauchamp, M. S. (2012). A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion. Neuroimage, 59(1), 781–787. doi: 10.1016/j.neuroimage.2011.07.024.PubMedCrossRefGoogle Scholar
  37. Osterling, J., & Dawson, G. (1994). Early recognition of children with autism: A study of first birthday home videotapes. Journal of Autism and Developmental Disorders, 24(3), 247–257. doi: 10.1007/bf02172225.PubMedCrossRefGoogle Scholar
  38. Pare, M., Richler, R. C., ten Hove, M., & Munhall, K. G. (2003). Gaze behavior in audiovisual speech perception: The influence of ocular fixations on the McGurk effect. Attention, Perception, & Psychophysics, 65(4), 553–567.CrossRefGoogle Scholar
  39. Powers, A. R, I. I. I., Hevey, M. A., & Wallace, M. T. (2012). Neural correlates of multisensory perceptual learning. Journal of Neuroscience, 32(18), 6263–6274. doi: 10.1523/JNEUROSCI.6138-11.201232/18/6263.PubMedCrossRefGoogle Scholar
  40. Powers, A. R, I. I. I., Hillock, A. R., & Wallace, M. T. (2009). Perceptual training narrows the temporal window of multisensory binding. The Journal of Neuroscience, 29(39), 12265–12274. doi: 10.1523/JNEUROSCI.3501-09.2009.PubMedCrossRefGoogle Scholar
  41. Rogers, S. J., Hepburn, S. L., Stackhouse, T., & Wehner, E. (2003). Imitation performance in toddlers with autism and those with other developmental disorders. Journal of Child Psychology and Psychiatry, 44(5), 763–781. doi: 10.1111/1469-7610.00162.PubMedCrossRefGoogle Scholar
  42. Rutter, M., Bailey, A., & Lord, C. (2003). Social communication questionnaire. Los Angeles, CA: Western Psychological Services.Google Scholar
  43. Samson, F., Mottron, L., Jemel, B., Belin, P., & Ciocca, V. (2006). Can spectro-temporal complexity explain the autistic pattern of performance on auditory tasks? Journal of Autism and Developmental Disorders, 36(1), 65–76. doi: 10.1007/s10803-005-0043-4.PubMedCrossRefGoogle Scholar
  44. Simmons, D. R., Robertson, A. E., McKay, L. S., Toal, E., McAleer, P., & Pollick, F. E. (2009). Vision in autism spectrum disorders. Vision Research, 49(22), 2705–2739. doi: 10.1016/j.visres.2009.08.005.PubMedCrossRefGoogle Scholar
  45. Smith, E. G., & Bennetto, L. (2007). Audiovisual speech integration and lipreading in autism. Journal of Child Psychology and Psychiatry, 48(8), 813–821. doi: 10.1111/j.1469-7610.2007.01766.x.PubMedCrossRefGoogle Scholar
  46. Stone, W. L., Ousley, O. Y., & Littleford, C. D. (1997). Motor imitation in young children with autism: What’s the object? Journal of Abnormal Child Psychology, 25(6), 475–485. doi: 10.1023/a:1022685731726.PubMedCrossRefGoogle Scholar
  47. Summerfield, Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 3–51). Hillsdale, NJ, England: Lawrence Erlbaum Associates, Inc.Google Scholar
  48. Tager-Flusberg, H., Paul, R., & Lord, C. (2005). Language and communication in autism. In F. R. Volkmar, R. Paul, A. Klin, & D. Cohen (Eds.), Handbook of autism and pervasive developmental disorders, Vol. 1: Diagnosis, development, neurobiology, and behavior (3rd ed.) (pp. 335–364). John Wiley & Sons Inc: Hoboken, NJ, US.Google Scholar
  49. Volkmar, F. R., & Mayes, L. C. (1990). Gaze behavior in autism. Development and Psychopathology, 2(01), 61–69. doi: 10.1017/S0954579400000596.CrossRefGoogle Scholar
  50. Wechsler, D. (1999). Wechsler abbreviated scale of intelligence. San Antonio, TX: Psychological Corporation.Google Scholar
  51. Williams, J., Massaro, D. W., Peel, N. J., Bosseler, A., & Suddendorf, T. (2004a). Visual-auditory integration during speech imitation in autism. Research in Developmental Disabilities, 25(6), 559–575. doi: 10.1016/j.ridd.2004.01.008.PubMedCrossRefGoogle Scholar
  52. Williams, J., Whiten, A., & Singh, T. (2004b). A systematic review of action imitation in autistic spectrum disorder. Journal of Autism and Developmental Disorders, 34(3), 285–299. doi: 10.1023/B:JADD.0000029551.56735.3a.PubMedCrossRefGoogle Scholar
  53. Stevenson, R., & Wallace, M. (in progress). Multisensory temporal integration: task and stimulus dependencies. Experimental Brain Research.Google Scholar
  54. Yoder, P., Molfese, D., Murray, M., & Key, A. (in press). Normative topographic ERP analyses of speed of speech processing and grammar before and after grammatical treatment. Journal of Developmental Neuropsychology.Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Tiffany G. Woynaroski
    • 1
    Email author
  • Leslie D. Kwakye
    • 2
  • Jennifer H. Foss-Feig
    • 3
  • Ryan A. Stevenson
    • 5
  • Wendy L. Stone
    • 4
  • Mark T. Wallace
    • 5
  1. 1.Department of Hearing and Speech SciencesVanderbilt UniversityNashvilleUSA
  2. 2.Neuroscience DepartmentOberlin CollegeOberlinUSA
  3. 3.Department of Psychology and Human DevelopmentVanderbilt UniversityNashvilleUSA
  4. 4.Department of Psychology and UW Autism CenterUniversity of WashingtonSeattleUSA
  5. 5.Department of Hearing and Speech Sciences, Psychology and Psychiatry, Vanderbilt Kennedy Center, Vanderbilt Brain InstituteVanderbilt UniversityNashvilleUSA

Personalised recommendations