Fifteen children with autism spectrum disorders (ASD) and twenty-one children without ASD completed six perceptual tasks designed to characterize the nature of the audiovisual processing difficulties experienced by children with ASD. Children with ASD scored significantly lower than children without ASD on audiovisual tasks involving human faces and voices, but scored similarly to children without ASD on audiovisual tasks involving nonhuman stimuli (bouncing balls). Results suggest that children with ASD may use visual information for speech differently from children without ASD. Exploratory results support an inverse association between audiovisual speech processing capacities and social impairment in children with ASD.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
This token is a voiced dental fricative; the consonant is the initial consonant of “the”.
The total number of trials presented on the Vowel Task should have been 48, but a computer glitch resulted in the presentation of only 47 trials. The missing trial was one of a number of a–v stimuli featuring a combination of audio /a/ and visual /a/. All other combinations of audio /a/ and visual /a/ were presented as intended.
The Male/Female Face Classification Task was treated as separate from the AV Gender Match–Mismatch Task. Performance on the former was not co-varied out when analyzing performance on the latter, because the two tasks employed entirely different stimuli. The gender of the faces used in the AV Gender Match–Mismatch task was much more easily identifiable than the gender of the faces in the Male/Female Face Classification Task, because the former included color images in which the entire head, hair, etc. were clearly visible and the latter included black & white images in which the hair had been cropped out.
The tasks are referred to collectively as “perceptual” rather than “cross-modal” because only five of the six were cross-modal. The Male Female Face Classification Task was unimodal, as it included only a visual component and no auditory component.
Baron-Cohen, S. (1998). Does the study of autism justify minimalist innate modularity? Learning & Individual Differences, 10(3), 179–191.
Best, C. T. (1995). A direct realist perspective on cross-language speech perception. In W. Strange & J. J. Jenkins (Eds.), Cross-langauge speech perception (pp. 171–204). Timonium, MD: York Press.
Cassia, V. M., Turati, C., & Simion, F. (2004). Can a nonspecific bias toward top-heavy patterns explain newborns’ face preference? Psychological Science, 15(6), 379–383.
Cohen, J. D., MacWhinney, B., Flatt, M., & Provost, J. (1993). PsyScope: A new graphic interactive environment for designing psychology experiments. Behavioral Research Methods, Instruments, and Computers, 25(2), 257–271.
Colombo, J., & Bundy, R. S. (1981). A method for the measurement of infant auditory selectivity. Infant Behavior & Development, 4, 219–223.
Constantino, J. N., Przybeck, T., Friesen, D., & Todd, R. D. (2000). Reciprocal social behavior in children with and without pervasive developmental disorder. Journal of Developmental & Behavioral Pediatrics, 21(1), 2–11.
Constantino, J. N., & Todd, R. D. (2000). Genetic structure of reciprocal social behavior. American Journal of Psychiatry, 157(12), 2043–2044.
Dawson, G., Meltzoff, A. N., Osterling, J., Rinaldi, J., & Brown, E. (1998). Children with autism fail to orient to naturally occurring social stimuli. Journal of Autism & Developmental Disorders, 28(6), 479–485.
de Gelder, B., Vroomen, J., & Van der Heide, L. (1991). Face Recognition and Lip-reading in Autism. European Journal of Experimental Psychology, 3(1), 69–86.
Desjardins, R. N., & Werker, J. F. (2004). Is the integration of heard and seen speech mandatory for infants? Developmental Psychobiology, 45(4), 187–203.
Fowler, C. A., & Rosenblum, L. D. (1990). Duplex perception: A comparison of monosyllables and slamming doors. Journal of Experimental Psychology: Human Perception and Performance, 16, 742–754.
Fowler, C. A., & Smith, M. (1986). Speech perception as “vector analysis”: An Approach to the problems of segmentation and invariance. In J. Perkell & D. Klatt (Eds.), Invariance and variability in speech processes (pp. 123–136). Hillsdale, NJ: Lawrence Erlbaum Associates.
Frith, U., & Happe, F. (1994). Autism: Beyond “theory of mind’. Cognition, 50(1–3), 115–132.
Glenn, S. M., Cunningham, C. C., & Joyce, P. F. (1981). A study of auditory preferences in nonhandicapped infants and infants with Down’s syndrome. Child Development, 52(4), 1303–1307.
Green, K. P., & Kuhl, P. K. (1986). The Role of Visual Information from a Talker’s Face in Processing of Place and Manner Features in Speech. Journal of the Acoustical Society of America, 80(S–63).
Heitanen, J. K., Manninen, P., Sams, M., & Surakka, V. (2001). Does audiovisual speech perception use information about facial configuration? European Journal of Cognitive Psychology, 13(3), 395–407.
Irwin, J. R., Whalen, D. H., & Fowler, C. A. (2006). A sex difference in audiovisual integration of speech. Perception and Psychophysics, 68, 582–592.
Johnson, M. H., Dziurawiec, S., Ellis, H., & Morton, J. (1991). Newborns’ preferential tracking of face-like stimuli and its subsequent decline. Cognition, 40(1–2), 1–19.
Jolliffe, T., & Baron-Cohen, S. (2001). A test of central coherence theory: Can adults with high-functioning autism or Asperger syndrome integrate objects in context? Visual Cognition, 8(1), 67–101.
Kikuchi, T., & Koga, S. (2001). Recognition of others’ and own facial expressions and production of facial expression: Children and adults with autism. Japanese Journal of Special Education, 39(2), 21–29.
Klin, A. (1991). Young autistic children’s listening preferences in regard to speech: A possible characteristic of the symptom of social withdrawal. Journal of Autism and Developmental Disorders, 21, 29–42.
Klin, A. (1992). Listening preferences in regard to speech in four children with developmental disabilities. Journal of Child Psychology and Psychiatry, 33, 763–769.
Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002). Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry, 59(9), 809–816.
Kuhl, P. K., & Meltzoff, A. N. (1984a). Infants’ representations of events: Studies in imitation, cross-modal perception, and categorization. Paper presented at the fourth international conference on infant studies, New York.
Kuhl, P. K., & Meltzoff, A. N. (1984b). The intermodal representation of speech in infants. Infant Behavior and Development, 7, 361–381.
Legerstee, M., Anderson, D., & Schaffer, A. (1998). Five- and eight-month-old infants recognize their faces and voices as familiar social stimuli. Child Development, 69, 37–50.
Liberman, A. M., Cooper, F. S., Shankweiler, D. P., & Studdert-Kennedy, M. (1967). Perception of the speech code. Psychological Review, 74, 431–461.
Liberman, A. M., & Whalen, D. H. (2000). On the relation of speech to language. Trends in Cognitive Science, 4, 187–196.
Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. (1999). Autism Diagnostic Observation Schedule-WPS (ADOS-WPS). Los Angeles, CA: Western Psychological Services.
Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism Diagnostic Interview-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism & Developmental Disorders, 24(5), 659–685.
Loveland, K. A., Tunali-Kotoski, B., Chen, R., & Brelsford, K. A. (1995). Intermodal perception of affect in persons with autism or down syndrome. Development & Psychopathology, 7(3), 409–418.
MacDonald, J., Andersen, S., & Bachman, T. (2000). Hearing by eye: How much spatial degradation can be tolerated? Perception, 29, 1155–1166.
Massaro, D. W. (1984). Children’s perception of visual and auditory speech. Child Development, 55, 1777–1788.
Massaro, D. W. (1998). Perceiving talking faces: From Speech perception to a behavioral principle. Cambridge, MA: MIT Press.
Massaro, D. W., & Bosseler, A. (2003). Perceiving speech by ear and eye: Multimodal integration by children with autism. Journal on Developmental and Learning Disorders, 7, 111–144.
Massaro, D. W., & Cohen, M. M. (1983). Evaluation and integration of visual and auditory information in speech perception. Journal of Experimental Psychology: Human Perception and Performance, 9, 753–771.
McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.
Osterling, J., & Dawson, G. (1994). Early recognition of children with autism: A study of first birthday home videotapes. Journal of Autism & Developmental Disorders, 24, 247–257.
Rosenblum, L. D., & Saldaña, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 22(2), 318–331.
Rosenblum, L. D., Schmuckler, M. A., & Johnson, J. A. (1997). The McGurk effect in infants. Perception & Psychophysics, 59(3), 347–357.
Saldaña, H. M., & Rosenblum, L. D. (1993). Visual influences on auditory pluck and bow judgments. Perception & Psychophysics, 54(3), 406–416.
Sams, M., Manninen, P., Suraka, V., Helin, P., & Kaettoe, R. (1998). McGurk effect in finnish syllables, isolated words and words in sentences: Effect of word meaning and sentence content. Speech Communication, 26(1–2), 75–87.
Schultz, R. T., Gauthier, I., Klin, A., Fulbright, R. K., Anderson, A. W., Volkmar, F., et al. (2000). Abnormal ventral temporal cortical activity during face discrimination among individuals with autism and asperger syndrome. Archives of General Psychiatry, 57(4), 331–340.
Schultz, R. T., Hunyadi, E., Conners, C., & Pasley, B. (2005). fMRI Study of Facial Expression Perception in Autism: The Amygdala, Fusiform Face Area and Their Functional Connectivity. Paper presented at the Annual meeting of the Organization for Human Brain Mapping, Toronto, CA.
Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212–215.
Summerfield, Q. (1979). Use of visual information for phonetic perception. Phonetica, 36, 314–331.
Surprenant, A. M., & Watson, C. S. (2001). Individual differences in the processing of speech and nonspeech sounds by normal-hearing listeners. Journal of the Acoustical Society of America, 110, 2085–2095.
Tager-Flusberg, H. (1981). On the nature of linguistic functioning in early infantile autism. Journal of Autism and Developmental Disorders, 11, 45–56.
Tager-Flusberg, H. (1982). Pragmatic development and its implication for social interaction in autistic children. In D. Park (Ed.), Proceedings of the 1981 international conference on autism (pp. 103–107). Washington, DC: NSAC.
Volkmar, F. R., & Lord, C. (1998). Diagnosis and definition of autism and other pervasive developmental disorders. In F. R. Volkmar (Ed.), Autism and pervasive developmental disorders. Cambridge: Cambridge University Press.
Williams, J. H. G., Massaro, D. W., Peel, N. J., Bosseler, A., & Suddendorf, T. (2004). Visual-auditory integration during speech imitation in autism. Research in Developmental Disabilities, 25, 559–575.
Support for this research came from a grant (U19-HD35482) from the NICHD, as part of the Collaborative Programs of Excellence in Autism (CPEA). Drs. Irwin and Whalen were supported by NIH grant HD-01994. We also acknowledge our gratitude for the diagnostic assessments provide by Ami Klin, Fred Volkmar and Diane Goudreau, the ASD participant recruitment by Tammy Babitz, and the careful proofreading of our final draft provided by Roxanne Mongillo. Finally, we wish to thank the children and families who gave their time so generously to facilitate this research project.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Mongillo, E.A., Irwin, J.R., Whalen, D.H. et al. Audiovisual Processing in Children with and without Autism Spectrum Disorders. J Autism Dev Disord 38, 1349–1358 (2008). https://doi.org/10.1007/s10803-007-0521-y
- Audiovisual processing
- Visual influence
- McGurk effect