Journal of Autism and Developmental Disorders

, Volume 38, Issue 7, pp 1349–1358 | Cite as

Audiovisual Processing in Children with and without Autism Spectrum Disorders

  • Elizabeth A. MongilloEmail author
  • Julia R. Irwin
  • D. H. Whalen
  • Cheryl Klaiman
  • Alice S. Carter
  • Robert T. Schultz
Original Paper


Fifteen children with autism spectrum disorders (ASD) and twenty-one children without ASD completed six perceptual tasks designed to characterize the nature of the audiovisual processing difficulties experienced by children with ASD. Children with ASD scored significantly lower than children without ASD on audiovisual tasks involving human faces and voices, but scored similarly to children without ASD on audiovisual tasks involving nonhuman stimuli (bouncing balls). Results suggest that children with ASD may use visual information for speech differently from children without ASD. Exploratory results support an inverse association between audiovisual speech processing capacities and social impairment in children with ASD.


Audiovisual processing Visual influence McGurk effect 



Support for this research came from a grant (U19-HD35482) from the NICHD, as part of the Collaborative Programs of Excellence in Autism (CPEA). Drs. Irwin and Whalen were supported by NIH grant HD-01994. We also acknowledge our gratitude for the diagnostic assessments provide by Ami Klin, Fred Volkmar and Diane Goudreau, the ASD participant recruitment by Tammy Babitz, and the careful proofreading of our final draft provided by Roxanne Mongillo. Finally, we wish to thank the children and families who gave their time so generously to facilitate this research project.

Supplementary material

10803_2007_521_MOESM1_ESM.doc (84 kb)
(DOC 83 kb)


  1. Baron-Cohen, S. (1998). Does the study of autism justify minimalist innate modularity? Learning & Individual Differences, 10(3), 179–191.CrossRefGoogle Scholar
  2. Best, C. T. (1995). A direct realist perspective on cross-language speech perception. In W. Strange & J. J. Jenkins (Eds.), Cross-langauge speech perception (pp. 171–204). Timonium, MD: York Press.Google Scholar
  3. Cassia, V. M., Turati, C., & Simion, F. (2004). Can a nonspecific bias toward top-heavy patterns explain newborns’ face preference? Psychological Science, 15(6), 379–383.PubMedCrossRefGoogle Scholar
  4. Cohen, J. D., MacWhinney, B., Flatt, M., & Provost, J. (1993). PsyScope: A new graphic interactive environment for designing psychology experiments. Behavioral Research Methods, Instruments, and Computers, 25(2), 257–271.Google Scholar
  5. Colombo, J., & Bundy, R. S. (1981). A method for the measurement of infant auditory selectivity. Infant Behavior & Development, 4, 219–223.CrossRefGoogle Scholar
  6. Constantino, J. N., Przybeck, T., Friesen, D., & Todd, R. D. (2000). Reciprocal social behavior in children with and without pervasive developmental disorder. Journal of Developmental & Behavioral Pediatrics, 21(1), 2–11.CrossRefGoogle Scholar
  7. Constantino, J. N., & Todd, R. D. (2000). Genetic structure of reciprocal social behavior. American Journal of Psychiatry, 157(12), 2043–2044.PubMedCrossRefGoogle Scholar
  8. Dawson, G., Meltzoff, A. N., Osterling, J., Rinaldi, J., & Brown, E. (1998). Children with autism fail to orient to naturally occurring social stimuli. Journal of Autism & Developmental Disorders, 28(6), 479–485.CrossRefGoogle Scholar
  9. de Gelder, B., Vroomen, J., & Van der Heide, L. (1991). Face Recognition and Lip-reading in Autism. European Journal of Experimental Psychology, 3(1), 69–86.Google Scholar
  10. Desjardins, R. N., & Werker, J. F. (2004). Is the integration of heard and seen speech mandatory for infants? Developmental Psychobiology, 45(4), 187–203.PubMedCrossRefGoogle Scholar
  11. Fowler, C. A., & Rosenblum, L. D. (1990). Duplex perception: A comparison of monosyllables and slamming doors. Journal of Experimental Psychology: Human Perception and Performance, 16, 742–754.PubMedCrossRefGoogle Scholar
  12. Fowler, C. A., & Smith, M. (1986). Speech perception as “vector analysis”: An Approach to the problems of segmentation and invariance. In J. Perkell & D. Klatt (Eds.), Invariance and variability in speech processes (pp. 123–136). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
  13. Frith, U., & Happe, F. (1994). Autism: Beyond “theory of mind’. Cognition, 50(1–3), 115–132.PubMedCrossRefGoogle Scholar
  14. Glenn, S. M., Cunningham, C. C., & Joyce, P. F. (1981). A study of auditory preferences in nonhandicapped infants and infants with Down’s syndrome. Child Development, 52(4), 1303–1307.PubMedCrossRefGoogle Scholar
  15. Green, K. P., & Kuhl, P. K. (1986). The Role of Visual Information from a Talker’s Face in Processing of Place and Manner Features in Speech. Journal of the Acoustical Society of America, 80(S–63).Google Scholar
  16. Heitanen, J. K., Manninen, P., Sams, M., & Surakka, V. (2001). Does audiovisual speech perception use information about facial configuration? European Journal of Cognitive Psychology, 13(3), 395–407.Google Scholar
  17. Irwin, J. R., Whalen, D. H., & Fowler, C. A. (2006). A sex difference in audiovisual integration of speech. Perception and Psychophysics, 68, 582–592.PubMedGoogle Scholar
  18. Johnson, M. H., Dziurawiec, S., Ellis, H., & Morton, J. (1991). Newborns’ preferential tracking of face-like stimuli and its subsequent decline. Cognition, 40(1–2), 1–19.PubMedCrossRefGoogle Scholar
  19. Jolliffe, T., & Baron-Cohen, S. (2001). A test of central coherence theory: Can adults with high-functioning autism or Asperger syndrome integrate objects in context? Visual Cognition, 8(1), 67–101.CrossRefGoogle Scholar
  20. Kikuchi, T., & Koga, S. (2001). Recognition of others’ and own facial expressions and production of facial expression: Children and adults with autism. Japanese Journal of Special Education, 39(2), 21–29.Google Scholar
  21. Klin, A. (1991). Young autistic children’s listening preferences in regard to speech: A possible characteristic of the symptom of social withdrawal. Journal of Autism and Developmental Disorders, 21, 29–42.PubMedCrossRefGoogle Scholar
  22. Klin, A. (1992). Listening preferences in regard to speech in four children with developmental disabilities. Journal of Child Psychology and Psychiatry, 33, 763–769.PubMedCrossRefGoogle Scholar
  23. Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002). Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry, 59(9), 809–816.PubMedCrossRefGoogle Scholar
  24. Kuhl, P. K., & Meltzoff, A. N. (1984a). Infants’ representations of events: Studies in imitation, cross-modal perception, and categorization. Paper presented at the fourth international conference on infant studies, New York.Google Scholar
  25. Kuhl, P. K., & Meltzoff, A. N. (1984b). The intermodal representation of speech in infants. Infant Behavior and Development, 7, 361–381.CrossRefGoogle Scholar
  26. Legerstee, M., Anderson, D., & Schaffer, A. (1998). Five- and eight-month-old infants recognize their faces and voices as familiar social stimuli. Child Development, 69, 37–50.PubMedCrossRefGoogle Scholar
  27. Liberman, A. M., Cooper, F. S., Shankweiler, D. P., & Studdert-Kennedy, M. (1967). Perception of the speech code. Psychological Review, 74, 431–461.PubMedCrossRefGoogle Scholar
  28. Liberman, A. M., & Whalen, D. H. (2000). On the relation of speech to language. Trends in Cognitive Science, 4, 187–196.CrossRefGoogle Scholar
  29. Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. (1999). Autism Diagnostic Observation Schedule-WPS (ADOS-WPS). Los Angeles, CA: Western Psychological Services.Google Scholar
  30. Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism Diagnostic Interview-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism & Developmental Disorders, 24(5), 659–685.CrossRefGoogle Scholar
  31. Loveland, K. A., Tunali-Kotoski, B., Chen, R., & Brelsford, K. A. (1995). Intermodal perception of affect in persons with autism or down syndrome. Development & Psychopathology, 7(3), 409–418.CrossRefGoogle Scholar
  32. MacDonald, J., Andersen, S., & Bachman, T. (2000). Hearing by eye: How much spatial degradation can be tolerated? Perception, 29, 1155–1166.PubMedCrossRefGoogle Scholar
  33. Massaro, D. W. (1984). Children’s perception of visual and auditory speech. Child Development, 55, 1777–1788.PubMedCrossRefGoogle Scholar
  34. Massaro, D. W. (1998). Perceiving talking faces: From Speech perception to a behavioral principle. Cambridge, MA: MIT Press.Google Scholar
  35. Massaro, D. W., & Bosseler, A. (2003). Perceiving speech by ear and eye: Multimodal integration by children with autism. Journal on Developmental and Learning Disorders, 7, 111–144.Google Scholar
  36. Massaro, D. W., & Cohen, M. M. (1983). Evaluation and integration of visual and auditory information in speech perception. Journal of Experimental Psychology: Human Perception and Performance, 9, 753–771.PubMedCrossRefGoogle Scholar
  37. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.PubMedCrossRefGoogle Scholar
  38. Osterling, J., & Dawson, G. (1994). Early recognition of children with autism: A study of first birthday home videotapes. Journal of Autism & Developmental Disorders, 24, 247–257.CrossRefGoogle Scholar
  39. Rosenblum, L. D., & Saldaña, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 22(2), 318–331.PubMedCrossRefGoogle Scholar
  40. Rosenblum, L. D., Schmuckler, M. A., & Johnson, J. A. (1997). The McGurk effect in infants. Perception & Psychophysics, 59(3), 347–357.Google Scholar
  41. Saldaña, H. M., & Rosenblum, L. D. (1993). Visual influences on auditory pluck and bow judgments. Perception & Psychophysics, 54(3), 406–416.Google Scholar
  42. Sams, M., Manninen, P., Suraka, V., Helin, P., & Kaettoe, R. (1998). McGurk effect in finnish syllables, isolated words and words in sentences: Effect of word meaning and sentence content. Speech Communication, 26(1–2), 75–87.CrossRefGoogle Scholar
  43. Schultz, R. T., Gauthier, I., Klin, A., Fulbright, R. K., Anderson, A. W., Volkmar, F., et al. (2000). Abnormal ventral temporal cortical activity during face discrimination among individuals with autism and asperger syndrome. Archives of General Psychiatry, 57(4), 331–340.PubMedCrossRefGoogle Scholar
  44. Schultz, R. T., Hunyadi, E., Conners, C., & Pasley, B. (2005). fMRI Study of Facial Expression Perception in Autism: The Amygdala, Fusiform Face Area and Their Functional Connectivity. Paper presented at the Annual meeting of the Organization for Human Brain Mapping, Toronto, CA.Google Scholar
  45. Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212–215.CrossRefGoogle Scholar
  46. Summerfield, Q. (1979). Use of visual information for phonetic perception. Phonetica, 36, 314–331.PubMedCrossRefGoogle Scholar
  47. Surprenant, A. M., & Watson, C. S. (2001). Individual differences in the processing of speech and nonspeech sounds by normal-hearing listeners. Journal of the Acoustical Society of America, 110, 2085–2095.PubMedCrossRefGoogle Scholar
  48. Tager-Flusberg, H. (1981). On the nature of linguistic functioning in early infantile autism. Journal of Autism and Developmental Disorders, 11, 45–56.PubMedCrossRefGoogle Scholar
  49. Tager-Flusberg, H. (1982). Pragmatic development and its implication for social interaction in autistic children. In D. Park (Ed.), Proceedings of the 1981 international conference on autism (pp. 103–107). Washington, DC: NSAC.Google Scholar
  50. Volkmar, F. R., & Lord, C. (1998). Diagnosis and definition of autism and other pervasive developmental disorders. In F. R. Volkmar (Ed.), Autism and pervasive developmental disorders. Cambridge: Cambridge University Press.Google Scholar
  51. Williams, J. H. G., Massaro, D. W., Peel, N. J., Bosseler, A., & Suddendorf, T. (2004). Visual-auditory integration during speech imitation in autism. Research in Developmental Disabilities, 25, 559–575.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Elizabeth A. Mongillo
    • 1
    Email author
  • Julia R. Irwin
    • 2
  • D. H. Whalen
    • 2
  • Cheryl Klaiman
    • 3
  • Alice S. Carter
    • 1
  • Robert T. Schultz
    • 3
  1. 1.Department of PsychologyUniversity of Massachusetts BostonBostonUSA
  2. 2.Haskins LaboratoriesNew HavenUSA
  3. 3.Yale School of Medicine Child Study CenterYale UniversityNew HavenUSA

Personalised recommendations