Skip to main content
SpringerLink
Log in

Audiovisual Speech Perception and Eye Gaze Behavior of Adults with Asperger Syndrome

  • Original Paper
  • Published:
Journal of Autism and Developmental Disorders Aims and scope Submit manuscript

Abstract

Audiovisual speech perception was studied in adults with Asperger syndrome (AS), by utilizing the McGurk effect, in which conflicting visual articulation alters the perception of heard speech. The AS group perceived the audiovisual stimuli differently from age, sex and IQ matched controls. When a voice saying /p/ was presented with a face articulating /k/, the controls predominantly heard /k/. Instead, the AS group heard /k/ and /t/ with almost equal frequency, but with large differences between individuals. There were no differences in gaze direction or unisensory perception between the AS and control participants that could have contributed to the audiovisual differences. We suggest an explanation in terms of weak support from the motor system for audiovisual speech perception in AS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Note that for the behavioral results, all 16 participants of both groups were included in the analysis in order to maximize the group size. An analysis of the behavioral results of those 9 AS participants and 14 control participants who were excluded from the eye gaze analysis gave the same statistically significant results as the analysis with all 16 participants. The eye gaze results were similar when an analysis was conducted for the matched 9 AS and 9 control participant pairs.

References

  • American Psychiatric Association. (1994). Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) (4th ed.). Washington, DC: APA.

    Google Scholar 

  • Andersen, T. S., Tiippana, K., Laarni, J., Kojo, I., & Sams, M. (2009). The role of visual spatial attention in audiovisual speech perception. Speech Communication, 51, 184–193.

    Article  Google Scholar 

  • Attwood, T. (1998). Asperger’s syndrome: A guide for parents and professionals. London: Jessica Kingsley Publishers Ltd.

    Google Scholar 

  • Ayres, A. J. (1979). Sensory integration and the child. Los Angeles: Western Psychological Association.

    Google Scholar 

  • Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I. (2001). The ‘reading the mind in the eyes’ test revised version: A study with normal adults, and adults with Asperger syndrome or high-functioning autism. Journal of Child Psychology and Psychiatry, 42, 241–252.

    Article  PubMed  Google Scholar 

  • Brancazio, L., & Miller, J. L. (2005). Use of visual information in speech perception: Evidence for a visual rate effect both with and without a McGurk effect. Perception & Psychophysics, 67(5), 759–769.

    Article  Google Scholar 

  • Buchan, J. N., Paré, M., & Munhall, K. G. (2007). Spatial statistics of gaze fixations during dynamic face processing. Social Neuroscience, 2(1), 1–13.

    Article  PubMed  Google Scholar 

  • Cattaneo, L., Fabbri-Destro, M., Boria, S., Pieraccini, C., Monti, A., Cossu, G., et al. (2007). Impairment of actions chains in autism and its possible role in intention understanding. Proceeding of the National Academy of Sciences of the United States of America, 104(45), 17825–17830.

    Article  Google Scholar 

  • Dalton, K. M., Nacewicz, B. M., Johnstone, T., Schaefer, H. S., Gernsbacher, M. A., Goldsmith, H. H., et al. (2005). Gaze processing and neural circuitry of face processing in autism. Nature Neuroscience, 8(4), 519–526.

    PubMed  Google Scholar 

  • De Gelder, B., Vroomen, J., & van der Heide, L. (1991). Face recognition and lip-reading in autism. European Journal of Cognitive Psychology, 3, 69–86.

    Article  Google Scholar 

  • Ehlers, S., Gillberg, C., & Wing, L. (1999). A screening questionnaire for Asperger syndrome and other high-functioning autism spectrum disorders in school age children. Journal of Autism and Developmental Disorders, 29(2), 129–141.

    Article  PubMed  Google Scholar 

  • Frith, U. (1992). Autism and Asperger syndrome. Cambridge: Cambridge University Press.

    Google Scholar 

  • Frith, U. (2004). Emanuel Miller lecture: Confusions and controversies about Asperger syndrome. Journal of Child Psychology and Psychiatry, 45, 672–686.

    Article  PubMed  Google Scholar 

  • Grandin, T., & Scariano, M. M. (1992). Minun tarinani—Ulos autismista [Emergence: Labelled autism]. Jyväskylän yliopiston täydennyskoulutuskeskus, Jyväskylä.

  • Iacoboni, M., & Dapretto, M. (2006). Mirror neuron system and the consequences of its dysfunction. Nature Reviews Neuroscience, 7, 942–951.

    Article  PubMed  Google Scholar 

  • Iarocci, G., & McDonald, J. (2006). Sensory integration and the perceptual experience of persons with autism. Journal of Autism and Developmental Disorders, 36(1), 77–90.

    Article  PubMed  Google Scholar 

  • Irwin, J. R. (2007). Auditory and audiovisual speech perception in children with autism spectrum disorders. Acoustics Today, 3(4), 8–15.

    Google Scholar 

  • Kätsyri, J., Saalasti, S., Tiippana, K., von Wendt, L., & Sams, M. (2008). Impaired recognition of facial emotions from low-spatial frequences in Asperger syndrome. Neuropsychologia, 46, 1888–1897.

    Article  PubMed  Google Scholar 

  • Keane, B. P., Rosenthal, O., Chun, N., & Shams, L. (2010). Audiovisual integration in high functioning adults with autism. Research in Autism Spectrum Disorders, 4, 276–289.

    Article  Google Scholar 

  • Kirchner, J. C., Hatri, A., Heekeren, H., Dziobek, I. (2010). Autistic symptomatology, face processing abilities, and eye fixation patterns. Journal of Autism and Developmental Disorders (epub ahead of print). doi 10.1007/s10803-010-1032-9.

  • Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002a). Defining and quantifying the social phenotype in autism. American Journal of Psychiatry, 159(6), 895–908.

    Article  PubMed  Google Scholar 

  • Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002b). Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry, 59, 809–816.

    Article  PubMed  Google Scholar 

  • Klin, A., Volkmar, F. R., & Sparrow, S. S. (2000). Diagnostic issues in Asperger syndrome. In A. Klin, F. R. Volkmar, & S. S. Sparrow (Eds.), Asperger syndrome (pp. 25–71). New York: The Guilford Press.

    Google Scholar 

  • Liberman, A. M., & Mattingly, I. G. (1985). The motor theory of speech perception revised. Cognition, 21, 1–36.

    Article  PubMed  Google Scholar 

  • Lord, C., Rutter, M., Goode, S., Heemsbergen, J., Jordan, H., Mawhood, L., et al. (1989). Autism diagnostic observation schedule: A standardized observation of communicative and social behavior. Journal of Autism and Developmental Disorders, 19(2), 185–212.

    Google Scholar 

  • Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism diagnostic interview-revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24, 659–685.

    Google Scholar 

  • Massaro, D. W., & Bosseler, A. (2003). Perceiving speech by ear and eye: Multimodal Integration by children with autism. Journal of Development and Learning Disorders, 7, 111–144.

    Google Scholar 

  • McGurk, H., & MacDonald, J. W. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.

    Article  PubMed  Google Scholar 

  • Molholm, S., & Foxe, J. (2005). Look ‘hear’, primary auditory cortex is active during lip-reading. Neuroreport, 16, 123–124.

    Google Scholar 

  • Mongillo, E. A., Irwin, J., Whalen, D. H., Klaiman, C., Carter, A. S., & Schultz, R. T. (2008). Audiovisual processing in children with and without autism spectrum disorders. Journal of Autism and Developmental Disorders, 38(7), 1349–1358.

    Article  PubMed  Google Scholar 

  • Neumann, D., Spezio, M. L., Piven, J., & Adolphs, R. (2006). Looking you in the mouth: Abnormal gaze in autism resulting from impaired top-down modulation of visual attention. Scan, 1, 194–202.

    PubMed  Google Scholar 

  • Nishitani, N., Avikainen, S., & Hari, R. (2004). Abnormal imitation-related cortical activation sequences in Asperger’s syndrome. Annals of Neurology, 55(4), 558–562.

    Article  PubMed  Google Scholar 

  • Norbury, C. F., Brock, J., Cragg, L., Einav, S., Griffiths, H., & Nation, K. (2009). Eye-movement patterns are associated with communicative competence in autistic spectrum disorders. Journal of Child Psychology and Psychiatry, 50, 834–842.

    Article  PubMed  Google Scholar 

  • Oberman, L. M., & Ramachandran, V. S. (2008). Preliminary evidence for deficits in multisensory integration in autism spectrum disorders: The mirror neuron hypothesis. Social Neuroscience, 3(3–4), 348–355.

    Article  PubMed  Google Scholar 

  • Ojanen, V., Möttönen, R., Pekkola, J., Jääskeläinen, I. P., Joensuu, R., Autti, T., et al. (2005). Processing of audiovisual speech in Broca’s area. NeuroImage, 25(2), 333–338.

    Article  PubMed  Google Scholar 

  • Pekkola, J., Laasonen, M., Ojanen, V., Autti, T., Jääskeläinen, I. P., Kujala, T., et al. (2006). Perception of matching and conflicting audiovisual speech in dyslexic and fluent readers: and fMRI study at 3T. NeuroImage, 29(3), 797–807.

    Article  PubMed  Google Scholar 

  • Pelphrey, K. A., Sasson, N., Reznick, J. S., Paul, G., Goldman, B., & Piven, J. (2002). Visual scanning of faces in autism. Journal of Autism and Developmental Disorders, 32, 249–261.

    Article  PubMed  Google Scholar 

  • Rizzolatti, G., Fadiga, L., Gallese, V., & Fogassi, L. (1996). Premotor cortex and the recognition of motor actions. Cognitive Brain Research, 3, 131–141.

    Article  PubMed  Google Scholar 

  • Skipper, J. I., Nusbaum, H. C., & Small, S. L. (2005). Listening to talking faces: Motor cortical activation during speech perception. NeuroImage, 25(1), 76–89.

    Article  PubMed  Google Scholar 

  • Skipper, J. I., van Wassenhove, V., Nusbaum, H. C., & Small, S. L. (2007). Hearing lips and seeing voices: How cortical areas supporting speech production mediate audiovisual speech perception. Cerebral Cortex, 17(10), 2387–2399.

    Article  PubMed  Google Scholar 

  • Smith, E. G., & Bennetto, L. (2007). Audiovisual speech integration and lipreading in autism. Journal of Child Psychology and Psychiatry, 48(8), 813–821.

    Article  PubMed  Google Scholar 

  • Smits, R., ten Bosch, L., & Collier, R. (1996). Evaluation of various sets of acoustic cues for the perception of prevocalic stop consonants. I. Perception experiment. The Journal of the Acoustical Society of America, 100(6), 3852–3864.

    Article  PubMed  Google Scholar 

  • Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. The Journal of the Acoustical Society of America, 26(2), 212–215.

    Article  Google Scholar 

  • Taylor, N., Isaac, C., & Milne, E. (2010). A comparison of the development of audiovisual integration in children with autism spectrum disorders and typically developing children. Journal of Autism and Developmental Disorders, (Epub ahead of print). doi: 10.1007/s10803-010-1000-4.

  • Tiippana K., Hayes E. A., Möttönen R., Kraus N. & Sams M. (2010). The McGurk effect at various auditory signal-to-noise ratios in American and Finnish listeners. Proceedings of the AVSP2010, International Conference on Auditory-Visual Speech Processing (pp. 166–169).

  • Wetherill, G. B., & Levitt, H. (1965). Sequential estimation of points on a psychometric function. British Journal of Mathematical and Statistical Psychology, 18, 1–10.

    Article  PubMed  Google Scholar 

  • Williams, D. (1992). Nowbody nowhere. New York: Times Books.

    Google Scholar 

  • Williams, J. H. G., Massaro, D. W., Peel, N. J., Bosseler, A., & Suddendorf, T. (2004). Visual–auditory integration during speech imitation in autism. Research in Developmental Disabilities, 25, 559–575.

    Article  PubMed  Google Scholar 

  • Wing, L. (1981). The Asperger syndrome: A clinical account. Psychological Medicine, 11(1), 115–129.

    Article  PubMed  Google Scholar 

  • World Health Organization. (1992). International classification of diseases, 10th revision (ICD-10). Genova: WHO.

    Google Scholar 

Download references

Acknowledgments

We are obliged to the participants of the study. We would like to acknowledge the contibution of the anonymous Reviewers of this paper, whose comments led to considerable improvements. We thank Tapani Suihkonen for participating in the preparation of the experiment, Tuomas Tolvanen for his help in preprocessing the eye gaze tracking data, and Jari Lipsanen for statistical advice. Eira Jansson-Verkasalo Ph.D. and Minna Laakso Ph.D. gave useful comments on the manuscript and Taina Nieminen- von Wendt MD, Ph.D. helped with her diagnostic expertise. We also thank Professor Pirkko Oittinen for providing access to the eye tracking equipment. The study was funded by Langnet, the Finnish Graduate School of Language Studies, and by the Jenny and Antti Wihuri Foundation. We dedicate this study to the memory of Professor Lennart von Wendt.

Author information

Author notes

  1. Jari Kätsyri

    Present address: Knowledge Media Laboratory, Center for Knowledge and Innovation Research (CKIR), Aalto University School of Economics, Aalto, Finland

Authors and Affiliations

  1. Institute of Behavioural Sciences, University of Helsinki, P.O. Box 9, 00014, Helsinki, Finland

    Satu Saalasti & Kaisa Tiippana

  2. Department of Pediatric and Adolescent Medicine, Helsinki University Central Hospital, Helsinki, Finland

    Satu Saalasti & Lennart von Wendt

  3. Mind and Brain Laboratory, Department of Biomedical Engineering and Computational Science (BECS), Aalto University School of Science, Aalto, Finland

    Jari Kätsyri, Kaisa Tiippana & Mikko Sams

  4. Department of Media Technology, Aalto University School of Science, Aalto, Finland

    Mari Laine-Hernandez

Authors
  1. Satu Saalasti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jari Kätsyri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kaisa Tiippana
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mari Laine-Hernandez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lennart von Wendt
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mikko Sams
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Satu Saalasti.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Saalasti, S., Kätsyri, J., Tiippana, K. et al. Audiovisual Speech Perception and Eye Gaze Behavior of Adults with Asperger Syndrome. J Autism Dev Disord 42, 1606–1615 (2012). https://doi.org/10.1007/s10803-011-1400-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10803-011-1400-0

Keywords

Search

Navigation