Journal of Autism and Developmental Disorders

, Volume 43, Issue 1, pp 188–199 | Cite as

Eye-Tracking, Autonomic, and Electrophysiological Correlates of Emotional Face Processing in Adolescents with Autism Spectrum Disorder

  • Jennifer B. Wagner
  • Suzanna B. Hirsch
  • Vanessa K. Vogel-Farley
  • Elizabeth Redcay
  • Charles A. Nelson
Original Paper


Individuals with autism spectrum disorder (ASD) often have difficulty with social-emotional cues. This study examined the neural, behavioral, and autonomic correlates of emotional face processing in adolescents with ASD and typical development (TD) using eye-tracking and event-related potentials (ERPs) across two different paradigms. Scanning of faces was similar across groups in the first task, but the second task found that face-sensitive ERPs varied with emotional expressions only in TD. Further, ASD showed enhanced neural responding to non-social stimuli. In TD only, attention to eyes during eye-tracking related to faster face-sensitive ERPs in a separate task; in ASD, a significant positive association was found between autonomic activity and attention to mouths. Overall, ASD showed an atypical pattern of emotional face processing, with reduced neural differentiation between emotions and a reduced relationship between gaze behavior and neural processing of faces.


Autism spectrum disorder Eye-tracking Event-related potentials Pupillometry Emotional face processing 


  1. Anderson, C. J., & Colombo, J. (2009). Larger tonic pupil size in young children with autism spectrum disorder. Developmental Psychobiology, 51, 207–211.Google Scholar
  2. Ashwin, E., Ashwin, C., Rhydderch, D., Howells, J., & Baron-Cohen, S. (2009). Eagle-eyed visual acuity: An experimental investigation of enhanced perception in autism. Biological Psychiatry, 65, 17–21.PubMedCrossRefGoogle Scholar
  3. Ashwin, C., Chapman, E., Colle, L., & Baron-Cohen, S. (2006). Impaired recognition of negative basic emotions in autism: A test of the amygdala theory. Social Neuroscience, 1, 349–363.PubMedCrossRefGoogle Scholar
  4. Bal, E., Harden, E., Lamb, D., Vaughan Van Hecke, A., Denver, J. W., & Porges, S. W. (2010). Emotion recognition in children with autism spectrum disorders: Relations to eye gaze and autonomic state. Journal of Autism and Developmental Disorders, 40, 358–370.PubMedCrossRefGoogle Scholar
  5. Baron-Cohen, S., Wheelwright, S., & Jolliffe, T. (1997). Is there a “language of the eyes”? Evidence from normal adults, and adults with autism or Asperger Syndrome. Visual Cognition, 4, 311–331.CrossRefGoogle Scholar
  6. Batty, M., & Taylor, M. J. (2003). Early processing of the six basic facial emotional expressions. Cognitive Brain Research, 17, 613–620.PubMedCrossRefGoogle Scholar
  7. Bentin, S., Allison, T., Puce, A., Perez, E., & McCarthy, G. (1996). Electrophysiological studies of face perception in humans. Journal of Cognitive Neuroscience, 8, 551–565.PubMedCrossRefGoogle Scholar
  8. Berument, S., Rutter, M., Lord, C., Pickles, A., & Bailey, A. (1999). Autism screening questionnaire: Diagnostic validity. British Journal of Psychiatry, 175, 444–451.PubMedCrossRefGoogle Scholar
  9. Bradley, M. M., Miccoli, L., Escrig, M. A., & Lang, P. J. (2008). The pupil as a measure of emotional arousal and autonomic activation. Psychophysiology, 45, 602–607.PubMedCrossRefGoogle Scholar
  10. Cassia, V., Kuefner, D., Westerlund, A., & Nelson, C. (2006). A behavioural and ERP investigation of 3-month-olds’ face preferences. Neuropsychologia, 44, 2113–2125.PubMedCrossRefGoogle Scholar
  11. Celani, G., Battacchi, M. W., & Arcidiacono, L. (1999). The understanding of emotional meaning of facial expressions in people with autism. Journal of Autism and Developmental Disorders, 29, 57–66.PubMedCrossRefGoogle Scholar
  12. Critchley, H. D., Daly, E. M., Bullmore, E. T., Williams, S. C., Van Amelsvoort, T., Robertson, D. M., et al. (2000). The functional neuroanatomy of social behaviour: Changes in cerebral blood flow when people with autistic disorder process facial expressions. Brain: A. Journal of Neurology, 123, 2203–2212.Google Scholar
  13. Dalton, K. M., Nacewicz, B. M., Alexander, A. L., & Davidson, R. J. (2007). Gaze-fixation, brain activation, and amygdala volume in unaffected siblings of individuals with autism. Biological Psychiatry, 61, 512–520.PubMedCrossRefGoogle Scholar
  14. Dalton, K. M., Nacewicz, B. M., Johnstone, T., Schaefer, H. S., Gernsbacher, M. A., Goldsmith, H. H., et al. (2005). Gaze-fixation and the neural circuity of face processing in autism. Nature Reviews Neuroscience, 8, 519–526.Google Scholar
  15. Dawson, G., Carver, L., Meltzoff, A. N., Panagiotides, H., McPartland, J., & Webb, S. J. (2002). Neural correlates of face and object recognition in young children with autism spectrum disorder, developmental delay, and typical development. Child Development, 73, 700–717.PubMedCrossRefGoogle Scholar
  16. Dawson, G., Webb, S. J., Carver, L., Panagiotides, H., & McPartland, J. (2004). Young children with autism show atypical brain responses to fearful versus neutral facial expressions of emotion. Developmental Science, 7, 340–359.PubMedCrossRefGoogle Scholar
  17. de Wit, T. C., Falck-Ytter, T., & von Hofsten, C. (2008). Young children with autism spectrum disorder look differently at positive versus negative emotional faces. Research in Autism Spectrum Disorders, 2, 651–659.CrossRefGoogle Scholar
  18. Ekman, P., & Friesen, W. V. (1976). Pictures of facial affect. Palo Alto, CA: Consulting Psychologists Press.Google Scholar
  19. Hall, G. B. C., Szechtman, H., & Nahmias, C. (2003). Enhanced salience and emotion recognition in autism: A PET study. American Journal of Psychiatry, 160, 1439–1441.PubMedCrossRefGoogle Scholar
  20. Heinze, H. J., Luck, S. J., Mangun, G. R., & Hillyard, S. A. (1990). Visual event-related potentials index focused attention within bilateral stimulus arrays, I: Evidence for early selection. Electroencephalography and Clinical Neurophysiology, 75, 511–527.PubMedCrossRefGoogle Scholar
  21. Hileman, C. M., Henderson, H., Mundy, P., Newell, L., & Mark, J. (2011). Developmental and individual differences on the P1 and N170 ERP components in children with and without autism. Developmental Neuropsychology, 36, 214–236.PubMedCrossRefGoogle Scholar
  22. Hirstein, W., Iverson, P., & Ramachandran, V. S. (2001). Autonomic responses of autistic children to people and objects. Proceedings of the Royal Society of London B, 268, 1883–1888.CrossRefGoogle Scholar
  23. Itier, R. J., & Batty, M. (2009). Neural bases of eye and gaze processing: The core of social cognition. Neuroscience and Biobehavioral Reviews, 33, 843–863.PubMedCrossRefGoogle Scholar
  24. Itier, R. J., & Taylor, M. J. (2002). Inversion and contrast polarity reversal affect both encoding and recognition processes of unfamiliar faces: A repetition study using ERPs. Neuroimage, 15, 353–372.PubMedCrossRefGoogle Scholar
  25. Itier, R. J., & Taylor, M. J. (2004). N170 or N1? Spatiotemporal differences between object and face processing using ERPs. Cerebral Cortex, 14, 132–142.PubMedCrossRefGoogle Scholar
  26. Jeste, S. S., & Nelson, C. A. (2009). Event related potentials in the understanding of autism spectrum disorders: An analytical review. Journal of Autism and Developmental Disorders, 39, 495–510.PubMedCrossRefGoogle Scholar
  27. Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: A module in human extra striate cortex specialized for face perception. Journal of Neuroscience, 17, 4302–4311.PubMedGoogle Scholar
  28. Kaufman, A. S., & Kaufman, N. L. (2004). Kaufman Brief Intelligence Test (2nd ed.). Bloomington, MN: Pearson, Inc.Google Scholar
  29. Kemner, C., van Ewijk, L., van Engeland, H., & Hooge, I. (2008). Brief report: Eye movements during visual search tasks indicate enhanced stimulus discriminability in subjects with PDD. Journal of Autism and Developmental Disorders, 38, 553–557.PubMedCrossRefGoogle Scholar
  30. Laeng, B., Sirois, S., & Gredeback, G. (2012). Pupillometry: A window to the preconscious? Perspectives on Psychological Science, 7, 18–27.CrossRefGoogle Scholar
  31. Leppanen, J. M., Moulson, M. C., Vogel-Farley, V. K., & Nelson, C. A. (2007). An ERP study of emotional face processing in the adult and infant brain. Child Development, 78, 232–245.PubMedCrossRefGoogle Scholar
  32. Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. (2002). Autism Diagnostic Observation Schedule: ADOS: Manual. Los Angeles, CA: Western Psychological Services.Google Scholar
  33. Luck, S. J., Heinze, H. J., Mangun, G. R., & Hillyard, S. A. (1990). Visual event-related potentials index focused attention within bilateral stimulus arrays, II: Functional dissociation of P1 and N1 components. Electroencephalography and Clinical Neurophysiology, 75, 528–542.PubMedCrossRefGoogle Scholar
  34. Manjaly, Z. M., Bruning, N., Neufang, S., Stephan, K. E., Brieber, S., Marshall, J. C., et al. (2007). Neurophysiological correlates of relatively enhanced local visual search in autistic adolescents. Neuroimage, 35, 283–291.PubMedCrossRefGoogle Scholar
  35. McCleery, J. P., Askshoomoff, N., Dobkins, K. R., & Carver, L. J. (2009). Atypical face versus object processing and hemispheric asymmetries in 10-month-old infants at risk for autism. Biological Psychiatry, 66, 950–957.PubMedCrossRefGoogle Scholar
  36. McPartland, J., Dawson, G., Webb, S. J., Panagiotides, H., & Carver, L. J. (2004). Event related brain potentials reveal anomalies in temporal processing of faces in autism spectrum disorder. Journal of Child Psychology and Psychiatry, 45, 1235–1245.PubMedCrossRefGoogle Scholar
  37. McPartland, J. C., Webb, S. J., Keehn, B., & Dawson, G. (2011). Patterns of visual attention to faces and objects in autism spectrum disorder. Journal of Autism and Developmental Disorders, 41, 148–157.PubMedCrossRefGoogle Scholar
  38. Mottron, L., Dawson, M., & Soulieres, I. (2009). Enhanced perception in savant syndrome: Patterns, structure and creativity. Philosophical Transactions of the Royal Society B, 364, 1385–1391.CrossRefGoogle Scholar
  39. Nelson, C. A., & de Haan, M. (1996). Neural correlates of infants’ visual responsiveness to facial expressions of emotion. Developmental Psychobiology, 29, 577–595.PubMedCrossRefGoogle Scholar
  40. Nelson, C. A., & McCleery, J. P. (2008). Use of event-related potentials in the study of typical and atypical development. Journal of American Academy of Child and Adolescent Psychiatry, 47, 1252–1261.CrossRefGoogle Scholar
  41. Noland, J. S., Reznick, J. S., Stone, W. L., Walden, T., & Sheridan, E. H. (2010). Better working memory for non-social targets in infant siblings of children with Autism Spectrum Disorder. Developmental Science, 13, 244–251.PubMedCrossRefGoogle Scholar
  42. Ogai, M., Matsumoto, H., Suzuki, K., Ozawa, F., Fukudu, R., Uchiyama, I., et al. (2003). fMRI study of recognition of facial expressions in high-functioning autistic patients. NeuroReport, 14, 559–563.PubMedCrossRefGoogle Scholar
  43. Pelphrey, K. A., Sasson, N. J., Reznick, J., Paul, G., Goldman, B. D., & Piven, J. (2002). Visual scanning of faces in autism. Journal of Autism and Developmental Disorders, 32, 249–261.PubMedCrossRefGoogle Scholar
  44. Pierce, K., Muller, R. A., Ambrose, J., Allen, G., & Courchesne, E. (2001). Face processing occurs outside the fusiform ‘face area’ in autism: Evidence from functional MRI. Brain, 124, 2059–2073.PubMedCrossRefGoogle Scholar
  45. Porter, G., Troscianki, T., & Gilchrist, I. D. (2007). Effort during visual search and counting: Insights from pupillometry. Quarterly Journal of Experimental Psychology, 60, 211–229.CrossRefGoogle Scholar
  46. Rossignol, M., Philippot, P., Douilliez, C., Crommelinck, M., & Campanella, S. (2005). The perception of fearful and happy facial expression is modulated by anxiety: An event-related potential study. Neuroscience Letters, 377, 115–120.PubMedCrossRefGoogle Scholar
  47. Rossion, B., Delvenne, J. F., Debatisse, D., Goffaux, V., Bruyer, R., Crommelinck, M., et al. (1999). Spatio-temporal localization of the face inversion effect: An event related potentials study. Biological Psychology, 50, 173–189.PubMedCrossRefGoogle Scholar
  48. Rutherford, M. D., & Towns, A. M. (2008). Scan path differences and similarities during emotion perception in those with and without autism spectrum disorders. Journal of Autism and Developmental Disorders, 38, 1371–1381.PubMedCrossRefGoogle Scholar
  49. Sawyer, A. C. P., Williamson, P., & Young, R. L. (2012). Can gaze avoidance explain why individuals with Asperger’s syndrome can’t recognize emotions from facial expressions? Journal of Autism and Developmental Disorders, 42, 606–618.Google Scholar
  50. Schoen, S. A., Miller, L. J., Brett-Green, B., & Hepburn, S. L. (2008). Psychophysiology of children with autism spectrum disorder. Research in Autism Spectrum Disorders, 2, 417–429.CrossRefGoogle Scholar
  51. Siegle, G. J., Steinhauer, S. R., Carter, C. S., Ramel, W., & Thase, M. E. (2003). Do the seconds turn into hours? Relationships between sustained pupil dilation in response to emotional information and self-reported rumination. Cognitive Therapy and Research, 27, 365–383.CrossRefGoogle Scholar
  52. Siegle, G. J., Steinhauer, S. R., & Thase, M. E. (2004). Pupillary assessment and computational modeling of the stroop task in depression. International Journal of Psychophysiology, 52, 63–76.PubMedCrossRefGoogle Scholar
  53. Soulieres, I., Dawson, M., Samson, F., Barbeau, E. B., Sahyoun, C., Strangman, G. E., et al. (2009). Enhanced visual processing contributes to matrix reasoning in autism. Human Brain Mapping, 30, 4082–4107.PubMedCrossRefGoogle Scholar
  54. Tottenham, N., Tanaka, J., Leon, A., McCarry, T., Nurse, M., Hare, T., et al. (2009). The NimStim set of facial expressions: Judgments from untrained research participants. Psychiatry Research, 168, 242–249.PubMedCrossRefGoogle Scholar
  55. Vaughn Van Hecke, A., Lebow, J., Bal, E., Lamb, D., Harden, E., Kramer, A., et al. (2009). Electroencephalogram and heart rate regulation to familiar and unfamiliar people in children with autism spectrum disorders. Child Development, 80, 1118–1133.CrossRefGoogle Scholar
  56. Webb, S. J., Dawson, G., Bernier, R., & Panagiotides, H. (2006). ERP evidence of atypical face processing in young children with autism. Journal of Autism and Developmental Disorders, 36, 881–890.PubMedCrossRefGoogle Scholar
  57. Webb, S. J., Jones, E. J. H., Merkle, K., Murias, M., Greenson, J., Richards, T., et al. (2010). Response to familiar faces, newly familiar faces, and novel faces as assessed by ERPs is intact in adults with autism spectrum disorders. International Journal of Psychophysiology, 77, 106–117.PubMedCrossRefGoogle Scholar
  58. Webb, S. J., Merkle, K., Murias, M., Richards, T., Aylward, E., & Dawson, G. (2009). ERP responses differentiate inverted but not upright face processing in adults with ASD. Social, Cognitive & Affective Neuroscience. doi:10.1093/scan/nsp002.
  59. Wong, T. K. W., Fung, P. C. W., Chua, S. E., & McAlonan, G. M. (2008). Abnormal spatiotemporal processing of emotional facial expressions in childhood autism: Dipole source analysis of event-related potentials. European Journal of Neuroscience, 28, 407–416.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Jennifer B. Wagner
    • 1
    • 2
  • Suzanna B. Hirsch
    • 1
  • Vanessa K. Vogel-Farley
    • 1
  • Elizabeth Redcay
    • 3
  • Charles A. Nelson
    • 1
    • 2
  1. 1.Laboratories of Cognitive Neuroscience, Division of Developmental MedicineChildren’s Hospital BostonBostonUSA
  2. 2.Harvard Medical SchoolBostonUSA
  3. 3.Department of PsychologyUniversity of MarylandCollege ParkUSA

Personalised recommendations