Audiovisual Processing in Children with and without Autism Spectrum Disorders

Abstract

Fifteen children with autism spectrum disorders (ASD) and twenty-one children without ASD completed six perceptual tasks designed to characterize the nature of the audiovisual processing difficulties experienced by children with ASD. Children with ASD scored significantly lower than children without ASD on audiovisual tasks involving human faces and voices, but scored similarly to children without ASD on audiovisual tasks involving nonhuman stimuli (bouncing balls). Results suggest that children with ASD may use visual information for speech differently from children without ASD. Exploratory results support an inverse association between audiovisual speech processing capacities and social impairment in children with ASD.

This is a preview of subscription content, log in to check access.

Notes

  1. 1.

    This token is a voiced dental fricative; the consonant is the initial consonant of “the”.

  2. 2.

    The total number of trials presented on the Vowel Task should have been 48, but a computer glitch resulted in the presentation of only 47 trials. The missing trial was one of a number of a–v stimuli featuring a combination of audio /a/ and visual /a/. All other combinations of audio /a/ and visual /a/ were presented as intended.

  3. 3.

    The Male/Female Face Classification Task was treated as separate from the AV Gender Match–Mismatch Task. Performance on the former was not co-varied out when analyzing performance on the latter, because the two tasks employed entirely different stimuli. The gender of the faces used in the AV Gender Match–Mismatch task was much more easily identifiable than the gender of the faces in the Male/Female Face Classification Task, because the former included color images in which the entire head, hair, etc. were clearly visible and the latter included black & white images in which the hair had been cropped out.

  4. 4.

    The tasks are referred to collectively as “perceptual” rather than “cross-modal” because only five of the six were cross-modal. The Male Female Face Classification Task was unimodal, as it included only a visual component and no auditory component.

References

  1. Baron-Cohen, S. (1998). Does the study of autism justify minimalist innate modularity? Learning & Individual Differences, 10(3), 179–191.

    Article  Google Scholar 

  2. Best, C. T. (1995). A direct realist perspective on cross-language speech perception. In W. Strange & J. J. Jenkins (Eds.), Cross-langauge speech perception (pp. 171–204). Timonium, MD: York Press.

    Google Scholar 

  3. Cassia, V. M., Turati, C., & Simion, F. (2004). Can a nonspecific bias toward top-heavy patterns explain newborns’ face preference? Psychological Science, 15(6), 379–383.

    PubMed  Article  Google Scholar 

  4. Cohen, J. D., MacWhinney, B., Flatt, M., & Provost, J. (1993). PsyScope: A new graphic interactive environment for designing psychology experiments. Behavioral Research Methods, Instruments, and Computers, 25(2), 257–271.

    Google Scholar 

  5. Colombo, J., & Bundy, R. S. (1981). A method for the measurement of infant auditory selectivity. Infant Behavior & Development, 4, 219–223.

    Article  Google Scholar 

  6. Constantino, J. N., Przybeck, T., Friesen, D., & Todd, R. D. (2000). Reciprocal social behavior in children with and without pervasive developmental disorder. Journal of Developmental & Behavioral Pediatrics, 21(1), 2–11.

    Article  Google Scholar 

  7. Constantino, J. N., & Todd, R. D. (2000). Genetic structure of reciprocal social behavior. American Journal of Psychiatry, 157(12), 2043–2044.

    PubMed  Article  Google Scholar 

  8. Dawson, G., Meltzoff, A. N., Osterling, J., Rinaldi, J., & Brown, E. (1998). Children with autism fail to orient to naturally occurring social stimuli. Journal of Autism & Developmental Disorders, 28(6), 479–485.

    Article  Google Scholar 

  9. de Gelder, B., Vroomen, J., & Van der Heide, L. (1991). Face Recognition and Lip-reading in Autism. European Journal of Experimental Psychology, 3(1), 69–86.

    Google Scholar 

  10. Desjardins, R. N., & Werker, J. F. (2004). Is the integration of heard and seen speech mandatory for infants? Developmental Psychobiology, 45(4), 187–203.

    PubMed  Article  Google Scholar 

  11. Fowler, C. A., & Rosenblum, L. D. (1990). Duplex perception: A comparison of monosyllables and slamming doors. Journal of Experimental Psychology: Human Perception and Performance, 16, 742–754.

    PubMed  Article  Google Scholar 

  12. Fowler, C. A., & Smith, M. (1986). Speech perception as “vector analysis”: An Approach to the problems of segmentation and invariance. In J. Perkell & D. Klatt (Eds.), Invariance and variability in speech processes (pp. 123–136). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  13. Frith, U., & Happe, F. (1994). Autism: Beyond “theory of mind’. Cognition, 50(1–3), 115–132.

    PubMed  Article  Google Scholar 

  14. Glenn, S. M., Cunningham, C. C., & Joyce, P. F. (1981). A study of auditory preferences in nonhandicapped infants and infants with Down’s syndrome. Child Development, 52(4), 1303–1307.

    PubMed  Article  Google Scholar 

  15. Green, K. P., & Kuhl, P. K. (1986). The Role of Visual Information from a Talker’s Face in Processing of Place and Manner Features in Speech. Journal of the Acoustical Society of America, 80(S–63).

  16. Heitanen, J. K., Manninen, P., Sams, M., & Surakka, V. (2001). Does audiovisual speech perception use information about facial configuration? European Journal of Cognitive Psychology, 13(3), 395–407.

    Google Scholar 

  17. Irwin, J. R., Whalen, D. H., & Fowler, C. A. (2006). A sex difference in audiovisual integration of speech. Perception and Psychophysics, 68, 582–592.

    PubMed  Google Scholar 

  18. Johnson, M. H., Dziurawiec, S., Ellis, H., & Morton, J. (1991). Newborns’ preferential tracking of face-like stimuli and its subsequent decline. Cognition, 40(1–2), 1–19.

    PubMed  Article  Google Scholar 

  19. Jolliffe, T., & Baron-Cohen, S. (2001). A test of central coherence theory: Can adults with high-functioning autism or Asperger syndrome integrate objects in context? Visual Cognition, 8(1), 67–101.

    Article  Google Scholar 

  20. Kikuchi, T., & Koga, S. (2001). Recognition of others’ and own facial expressions and production of facial expression: Children and adults with autism. Japanese Journal of Special Education, 39(2), 21–29.

    Google Scholar 

  21. Klin, A. (1991). Young autistic children’s listening preferences in regard to speech: A possible characteristic of the symptom of social withdrawal. Journal of Autism and Developmental Disorders, 21, 29–42.

    PubMed  Article  Google Scholar 

  22. Klin, A. (1992). Listening preferences in regard to speech in four children with developmental disabilities. Journal of Child Psychology and Psychiatry, 33, 763–769.

    PubMed  Article  Google Scholar 

  23. Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002). Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry, 59(9), 809–816.

    PubMed  Article  Google Scholar 

  24. Kuhl, P. K., & Meltzoff, A. N. (1984a). Infants’ representations of events: Studies in imitation, cross-modal perception, and categorization. Paper presented at the fourth international conference on infant studies, New York.

  25. Kuhl, P. K., & Meltzoff, A. N. (1984b). The intermodal representation of speech in infants. Infant Behavior and Development, 7, 361–381.

    Article  Google Scholar 

  26. Legerstee, M., Anderson, D., & Schaffer, A. (1998). Five- and eight-month-old infants recognize their faces and voices as familiar social stimuli. Child Development, 69, 37–50.

    PubMed  Article  Google Scholar 

  27. Liberman, A. M., Cooper, F. S., Shankweiler, D. P., & Studdert-Kennedy, M. (1967). Perception of the speech code. Psychological Review, 74, 431–461.

    PubMed  Article  Google Scholar 

  28. Liberman, A. M., & Whalen, D. H. (2000). On the relation of speech to language. Trends in Cognitive Science, 4, 187–196.

    Article  Google Scholar 

  29. Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. (1999). Autism Diagnostic Observation Schedule-WPS (ADOS-WPS). Los Angeles, CA: Western Psychological Services.

    Google Scholar 

  30. Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism Diagnostic Interview-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism & Developmental Disorders, 24(5), 659–685.

    Article  Google Scholar 

  31. Loveland, K. A., Tunali-Kotoski, B., Chen, R., & Brelsford, K. A. (1995). Intermodal perception of affect in persons with autism or down syndrome. Development & Psychopathology, 7(3), 409–418.

    Article  Google Scholar 

  32. MacDonald, J., Andersen, S., & Bachman, T. (2000). Hearing by eye: How much spatial degradation can be tolerated? Perception, 29, 1155–1166.

    PubMed  Article  Google Scholar 

  33. Massaro, D. W. (1984). Children’s perception of visual and auditory speech. Child Development, 55, 1777–1788.

    PubMed  Article  Google Scholar 

  34. Massaro, D. W. (1998). Perceiving talking faces: From Speech perception to a behavioral principle. Cambridge, MA: MIT Press.

    Google Scholar 

  35. Massaro, D. W., & Bosseler, A. (2003). Perceiving speech by ear and eye: Multimodal integration by children with autism. Journal on Developmental and Learning Disorders, 7, 111–144.

    Google Scholar 

  36. Massaro, D. W., & Cohen, M. M. (1983). Evaluation and integration of visual and auditory information in speech perception. Journal of Experimental Psychology: Human Perception and Performance, 9, 753–771.

    PubMed  Article  Google Scholar 

  37. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.

    PubMed  Article  Google Scholar 

  38. Osterling, J., & Dawson, G. (1994). Early recognition of children with autism: A study of first birthday home videotapes. Journal of Autism & Developmental Disorders, 24, 247–257.

    Article  Google Scholar 

  39. Rosenblum, L. D., & Saldaña, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 22(2), 318–331.

    PubMed  Article  Google Scholar 

  40. Rosenblum, L. D., Schmuckler, M. A., & Johnson, J. A. (1997). The McGurk effect in infants. Perception & Psychophysics, 59(3), 347–357.

    Google Scholar 

  41. Saldaña, H. M., & Rosenblum, L. D. (1993). Visual influences on auditory pluck and bow judgments. Perception & Psychophysics, 54(3), 406–416.

    Google Scholar 

  42. Sams, M., Manninen, P., Suraka, V., Helin, P., & Kaettoe, R. (1998). McGurk effect in finnish syllables, isolated words and words in sentences: Effect of word meaning and sentence content. Speech Communication, 26(1–2), 75–87.

    Article  Google Scholar 

  43. Schultz, R. T., Gauthier, I., Klin, A., Fulbright, R. K., Anderson, A. W., Volkmar, F., et al. (2000). Abnormal ventral temporal cortical activity during face discrimination among individuals with autism and asperger syndrome. Archives of General Psychiatry, 57(4), 331–340.

    PubMed  Article  Google Scholar 

  44. Schultz, R. T., Hunyadi, E., Conners, C., & Pasley, B. (2005). fMRI Study of Facial Expression Perception in Autism: The Amygdala, Fusiform Face Area and Their Functional Connectivity. Paper presented at the Annual meeting of the Organization for Human Brain Mapping, Toronto, CA.

  45. Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212–215.

    Article  Google Scholar 

  46. Summerfield, Q. (1979). Use of visual information for phonetic perception. Phonetica, 36, 314–331.

    PubMed  Article  Google Scholar 

  47. Surprenant, A. M., & Watson, C. S. (2001). Individual differences in the processing of speech and nonspeech sounds by normal-hearing listeners. Journal of the Acoustical Society of America, 110, 2085–2095.

    PubMed  Article  Google Scholar 

  48. Tager-Flusberg, H. (1981). On the nature of linguistic functioning in early infantile autism. Journal of Autism and Developmental Disorders, 11, 45–56.

    PubMed  Article  Google Scholar 

  49. Tager-Flusberg, H. (1982). Pragmatic development and its implication for social interaction in autistic children. In D. Park (Ed.), Proceedings of the 1981 international conference on autism (pp. 103–107). Washington, DC: NSAC.

  50. Volkmar, F. R., & Lord, C. (1998). Diagnosis and definition of autism and other pervasive developmental disorders. In F. R. Volkmar (Ed.), Autism and pervasive developmental disorders. Cambridge: Cambridge University Press.

    Google Scholar 

  51. Williams, J. H. G., Massaro, D. W., Peel, N. J., Bosseler, A., & Suddendorf, T. (2004). Visual-auditory integration during speech imitation in autism. Research in Developmental Disabilities, 25, 559–575.

    PubMed  Article  Google Scholar 

Download references

Acknowledgments

Support for this research came from a grant (U19-HD35482) from the NICHD, as part of the Collaborative Programs of Excellence in Autism (CPEA). Drs. Irwin and Whalen were supported by NIH grant HD-01994. We also acknowledge our gratitude for the diagnostic assessments provide by Ami Klin, Fred Volkmar and Diane Goudreau, the ASD participant recruitment by Tammy Babitz, and the careful proofreading of our final draft provided by Roxanne Mongillo. Finally, we wish to thank the children and families who gave their time so generously to facilitate this research project.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Elizabeth A. Mongillo.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(DOC 83 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Mongillo, E.A., Irwin, J.R., Whalen, D.H. et al. Audiovisual Processing in Children with and without Autism Spectrum Disorders. J Autism Dev Disord 38, 1349–1358 (2008). https://doi.org/10.1007/s10803-007-0521-y

Download citation

Keywords

  • Audiovisual processing
  • Visual influence
  • McGurk effect