Journal of Autism and Developmental Disorders

, Volume 42, Issue 8, pp 1606–1615 | Cite as

Audiovisual Speech Perception and Eye Gaze Behavior of Adults with Asperger Syndrome

  • Satu SaalastiEmail author
  • Jari Kätsyri
  • Kaisa Tiippana
  • Mari Laine-Hernandez
  • Lennart von Wendt
  • Mikko Sams
Original Paper


Audiovisual speech perception was studied in adults with Asperger syndrome (AS), by utilizing the McGurk effect, in which conflicting visual articulation alters the perception of heard speech. The AS group perceived the audiovisual stimuli differently from age, sex and IQ matched controls. When a voice saying /p/ was presented with a face articulating /k/, the controls predominantly heard /k/. Instead, the AS group heard /k/ and /t/ with almost equal frequency, but with large differences between individuals. There were no differences in gaze direction or unisensory perception between the AS and control participants that could have contributed to the audiovisual differences. We suggest an explanation in terms of weak support from the motor system for audiovisual speech perception in AS.


Asperger syndrome Autism spectrum disorders Multisensory integration Audiovisual speech Eye gaze behavior Perception 



We are obliged to the participants of the study. We would like to acknowledge the contibution of the anonymous Reviewers of this paper, whose comments led to considerable improvements. We thank Tapani Suihkonen for participating in the preparation of the experiment, Tuomas Tolvanen for his help in preprocessing the eye gaze tracking data, and Jari Lipsanen for statistical advice. Eira Jansson-Verkasalo Ph.D. and Minna Laakso Ph.D. gave useful comments on the manuscript and Taina Nieminen- von Wendt MD, Ph.D. helped with her diagnostic expertise. We also thank Professor Pirkko Oittinen for providing access to the eye tracking equipment. The study was funded by Langnet, the Finnish Graduate School of Language Studies, and by the Jenny and Antti Wihuri Foundation. We dedicate this study to the memory of Professor Lennart von Wendt.


  1. American Psychiatric Association. (1994). Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) (4th ed.). Washington, DC: APA.Google Scholar
  2. Andersen, T. S., Tiippana, K., Laarni, J., Kojo, I., & Sams, M. (2009). The role of visual spatial attention in audiovisual speech perception. Speech Communication, 51, 184–193.CrossRefGoogle Scholar
  3. Attwood, T. (1998). Asperger’s syndrome: A guide for parents and professionals. London: Jessica Kingsley Publishers Ltd.Google Scholar
  4. Ayres, A. J. (1979). Sensory integration and the child. Los Angeles: Western Psychological Association.Google Scholar
  5. Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I. (2001). The ‘reading the mind in the eyes’ test revised version: A study with normal adults, and adults with Asperger syndrome or high-functioning autism. Journal of Child Psychology and Psychiatry, 42, 241–252.PubMedCrossRefGoogle Scholar
  6. Brancazio, L., & Miller, J. L. (2005). Use of visual information in speech perception: Evidence for a visual rate effect both with and without a McGurk effect. Perception & Psychophysics, 67(5), 759–769.CrossRefGoogle Scholar
  7. Buchan, J. N., Paré, M., & Munhall, K. G. (2007). Spatial statistics of gaze fixations during dynamic face processing. Social Neuroscience, 2(1), 1–13.PubMedCrossRefGoogle Scholar
  8. Cattaneo, L., Fabbri-Destro, M., Boria, S., Pieraccini, C., Monti, A., Cossu, G., et al. (2007). Impairment of actions chains in autism and its possible role in intention understanding. Proceeding of the National Academy of Sciences of the United States of America, 104(45), 17825–17830.CrossRefGoogle Scholar
  9. Dalton, K. M., Nacewicz, B. M., Johnstone, T., Schaefer, H. S., Gernsbacher, M. A., Goldsmith, H. H., et al. (2005). Gaze processing and neural circuitry of face processing in autism. Nature Neuroscience, 8(4), 519–526.PubMedGoogle Scholar
  10. De Gelder, B., Vroomen, J., & van der Heide, L. (1991). Face recognition and lip-reading in autism. European Journal of Cognitive Psychology, 3, 69–86.CrossRefGoogle Scholar
  11. Ehlers, S., Gillberg, C., & Wing, L. (1999). A screening questionnaire for Asperger syndrome and other high-functioning autism spectrum disorders in school age children. Journal of Autism and Developmental Disorders, 29(2), 129–141.PubMedCrossRefGoogle Scholar
  12. Frith, U. (1992). Autism and Asperger syndrome. Cambridge: Cambridge University Press.Google Scholar
  13. Frith, U. (2004). Emanuel Miller lecture: Confusions and controversies about Asperger syndrome. Journal of Child Psychology and Psychiatry, 45, 672–686.PubMedCrossRefGoogle Scholar
  14. Grandin, T., & Scariano, M. M. (1992). Minun tarinaniUlos autismista [Emergence: Labelled autism]. Jyväskylän yliopiston täydennyskoulutuskeskus, Jyväskylä.Google Scholar
  15. Iacoboni, M., & Dapretto, M. (2006). Mirror neuron system and the consequences of its dysfunction. Nature Reviews Neuroscience, 7, 942–951.PubMedCrossRefGoogle Scholar
  16. Iarocci, G., & McDonald, J. (2006). Sensory integration and the perceptual experience of persons with autism. Journal of Autism and Developmental Disorders, 36(1), 77–90.PubMedCrossRefGoogle Scholar
  17. Irwin, J. R. (2007). Auditory and audiovisual speech perception in children with autism spectrum disorders. Acoustics Today, 3(4), 8–15.Google Scholar
  18. Kätsyri, J., Saalasti, S., Tiippana, K., von Wendt, L., & Sams, M. (2008). Impaired recognition of facial emotions from low-spatial frequences in Asperger syndrome. Neuropsychologia, 46, 1888–1897.PubMedCrossRefGoogle Scholar
  19. Keane, B. P., Rosenthal, O., Chun, N., & Shams, L. (2010). Audiovisual integration in high functioning adults with autism. Research in Autism Spectrum Disorders, 4, 276–289.CrossRefGoogle Scholar
  20. Kirchner, J. C., Hatri, A., Heekeren, H., Dziobek, I. (2010). Autistic symptomatology, face processing abilities, and eye fixation patterns. Journal of Autism and Developmental Disorders (epub ahead of print). doi  10.1007/s10803-010-1032-9.
  21. Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002a). Defining and quantifying the social phenotype in autism. American Journal of Psychiatry, 159(6), 895–908.PubMedCrossRefGoogle Scholar
  22. Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002b). Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry, 59, 809–816.PubMedCrossRefGoogle Scholar
  23. Klin, A., Volkmar, F. R., & Sparrow, S. S. (2000). Diagnostic issues in Asperger syndrome. In A. Klin, F. R. Volkmar, & S. S. Sparrow (Eds.), Asperger syndrome (pp. 25–71). New York: The Guilford Press.Google Scholar
  24. Liberman, A. M., & Mattingly, I. G. (1985). The motor theory of speech perception revised. Cognition, 21, 1–36.PubMedCrossRefGoogle Scholar
  25. Lord, C., Rutter, M., Goode, S., Heemsbergen, J., Jordan, H., Mawhood, L., et al. (1989). Autism diagnostic observation schedule: A standardized observation of communicative and social behavior. Journal of Autism and Developmental Disorders, 19(2), 185–212.Google Scholar
  26. Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism diagnostic interview-revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24, 659–685.Google Scholar
  27. Massaro, D. W., & Bosseler, A. (2003). Perceiving speech by ear and eye: Multimodal Integration by children with autism. Journal of Development and Learning Disorders, 7, 111–144.Google Scholar
  28. McGurk, H., & MacDonald, J. W. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.PubMedCrossRefGoogle Scholar
  29. Molholm, S., & Foxe, J. (2005). Look ‘hear’, primary auditory cortex is active during lip-reading. Neuroreport, 16, 123–124.Google Scholar
  30. Mongillo, E. A., Irwin, J., Whalen, D. H., Klaiman, C., Carter, A. S., & Schultz, R. T. (2008). Audiovisual processing in children with and without autism spectrum disorders. Journal of Autism and Developmental Disorders, 38(7), 1349–1358.PubMedCrossRefGoogle Scholar
  31. Neumann, D., Spezio, M. L., Piven, J., & Adolphs, R. (2006). Looking you in the mouth: Abnormal gaze in autism resulting from impaired top-down modulation of visual attention. Scan, 1, 194–202.PubMedGoogle Scholar
  32. Nishitani, N., Avikainen, S., & Hari, R. (2004). Abnormal imitation-related cortical activation sequences in Asperger’s syndrome. Annals of Neurology, 55(4), 558–562.PubMedCrossRefGoogle Scholar
  33. Norbury, C. F., Brock, J., Cragg, L., Einav, S., Griffiths, H., & Nation, K. (2009). Eye-movement patterns are associated with communicative competence in autistic spectrum disorders. Journal of Child Psychology and Psychiatry, 50, 834–842.PubMedCrossRefGoogle Scholar
  34. Oberman, L. M., & Ramachandran, V. S. (2008). Preliminary evidence for deficits in multisensory integration in autism spectrum disorders: The mirror neuron hypothesis. Social Neuroscience, 3(3–4), 348–355.PubMedCrossRefGoogle Scholar
  35. Ojanen, V., Möttönen, R., Pekkola, J., Jääskeläinen, I. P., Joensuu, R., Autti, T., et al. (2005). Processing of audiovisual speech in Broca’s area. NeuroImage, 25(2), 333–338.PubMedCrossRefGoogle Scholar
  36. Pekkola, J., Laasonen, M., Ojanen, V., Autti, T., Jääskeläinen, I. P., Kujala, T., et al. (2006). Perception of matching and conflicting audiovisual speech in dyslexic and fluent readers: and fMRI study at 3T. NeuroImage, 29(3), 797–807.PubMedCrossRefGoogle Scholar
  37. Pelphrey, K. A., Sasson, N., Reznick, J. S., Paul, G., Goldman, B., & Piven, J. (2002). Visual scanning of faces in autism. Journal of Autism and Developmental Disorders, 32, 249–261.PubMedCrossRefGoogle Scholar
  38. Rizzolatti, G., Fadiga, L., Gallese, V., & Fogassi, L. (1996). Premotor cortex and the recognition of motor actions. Cognitive Brain Research, 3, 131–141.PubMedCrossRefGoogle Scholar
  39. Skipper, J. I., Nusbaum, H. C., & Small, S. L. (2005). Listening to talking faces: Motor cortical activation during speech perception. NeuroImage, 25(1), 76–89.PubMedCrossRefGoogle Scholar
  40. Skipper, J. I., van Wassenhove, V., Nusbaum, H. C., & Small, S. L. (2007). Hearing lips and seeing voices: How cortical areas supporting speech production mediate audiovisual speech perception. Cerebral Cortex, 17(10), 2387–2399.PubMedCrossRefGoogle Scholar
  41. Smith, E. G., & Bennetto, L. (2007). Audiovisual speech integration and lipreading in autism. Journal of Child Psychology and Psychiatry, 48(8), 813–821.PubMedCrossRefGoogle Scholar
  42. Smits, R., ten Bosch, L., & Collier, R. (1996). Evaluation of various sets of acoustic cues for the perception of prevocalic stop consonants. I. Perception experiment. The Journal of the Acoustical Society of America, 100(6), 3852–3864.PubMedCrossRefGoogle Scholar
  43. Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. The Journal of the Acoustical Society of America, 26(2), 212–215.CrossRefGoogle Scholar
  44. Taylor, N., Isaac, C., & Milne, E. (2010). A comparison of the development of audiovisual integration in children with autism spectrum disorders and typically developing children. Journal of Autism and Developmental Disorders, (Epub ahead of print). doi:  10.1007/s10803-010-1000-4.
  45. Tiippana K., Hayes E. A., Möttönen R., Kraus N. & Sams M. (2010). The McGurk effect at various auditory signal-to-noise ratios in American and Finnish listeners. Proceedings of the AVSP2010, International Conference on Auditory-Visual Speech Processing (pp. 166–169).Google Scholar
  46. Wetherill, G. B., & Levitt, H. (1965). Sequential estimation of points on a psychometric function. British Journal of Mathematical and Statistical Psychology, 18, 1–10.PubMedCrossRefGoogle Scholar
  47. Williams, D. (1992). Nowbody nowhere. New York: Times Books.Google Scholar
  48. Williams, J. H. G., Massaro, D. W., Peel, N. J., Bosseler, A., & Suddendorf, T. (2004). Visual–auditory integration during speech imitation in autism. Research in Developmental Disabilities, 25, 559–575.PubMedCrossRefGoogle Scholar
  49. Wing, L. (1981). The Asperger syndrome: A clinical account. Psychological Medicine, 11(1), 115–129.PubMedCrossRefGoogle Scholar
  50. World Health Organization. (1992). International classification of diseases, 10th revision (ICD-10). Genova: WHO.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Satu Saalasti
    • 1
    • 2
    Email author
  • Jari Kätsyri
    • 3
    • 4
  • Kaisa Tiippana
    • 1
    • 4
  • Mari Laine-Hernandez
    • 5
  • Lennart von Wendt
    • 2
  • Mikko Sams
    • 4
  1. 1.Institute of Behavioural SciencesUniversity of HelsinkiHelsinkiFinland
  2. 2.Department of Pediatric and Adolescent MedicineHelsinki University Central HospitalHelsinkiFinland
  3. 3.Knowledge Media Laboratory, Center for Knowledge and Innovation Research (CKIR)Aalto University School of EconomicsAaltoFinland
  4. 4.Mind and Brain Laboratory, Department of Biomedical Engineering and Computational Science (BECS)Aalto University School of ScienceAaltoFinland
  5. 5.Department of Media TechnologyAalto University School of ScienceAaltoFinland

Personalised recommendations