Skip to main content
SpringerLink
Log in

Reduced eye gaze fixation during emotion recognition among patients with temporal lobe epilepsy

  • Original Communication
  • Published:
Journal of Neurology Aims and scope Submit manuscript

Abstract

Objectives

To investigate the facial scan patterns during emotion recognition (ER) through the dynamic facial expression task and the awareness of social interference test (TASIT) using eye tracking (ET) technology, and to find some ET indicators that can accurately depict the ER process, which is a beneficial supplement to existing ER assessment tools.

Method

Ninety-six patients with TLE and 88 healthy controls (HCs) were recruited. All participants watched the dynamic facial expression task and TASIT including a synchronized eye movement recording and recognized the emotion (anger, disgust, happiness, or sadness). The accuracy of ER was recorded. The first fixation time, first fixation duration, dwell time, and fixation count were selected and analyzed.

Results

TLE patients exhibited ER impairment especially for disgust (Z =  − 3.391; p = 0.001) and sadness (Z =  − 3.145; p = 0.002). TLE patients fixated less on the face, as evidenced by the reduced fixation count (Z =  − 2.549; p = 0.011) of the face and a significant decrease in the fixation count rate (Z =  − 1.993; p = 0.046). During the dynamic facial expression task, TLE patients focused less on the eyes, as evidenced by the decreased first fixation duration (Z =  − 4.322; p = 0.000), dwell time (Z =  − 4.083; p = 0.000), and fixation count (Z =  − 3.699; p = 0.000) of the eyes.

Conclusion

TLE patients had ER impairment, especially regarding negative emotions, which may be attributable to their reduced fixation on the eyes during ER, and the increased fixation on the mouth could be a compensatory effect to improve ER performance. Eye-tracking technology could provide the process indicators of ER, and is a valuable supplement to traditional ER assessment tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

Original data can be provided by the corresponding author on demand.

References

  1. Sen A, Capelli V, Husain M (2018) Cognition and dementia in older patients with epilepsy. Brain 141(6):1592–1608. https://doi.org/10.1093/brain/awy022

    Article  PubMed  PubMed Central  Google Scholar 

  2. Qi L, Zhao J, Zhao P, Zhang H, Zhong J, Pan P, Wang G, Yi Z, Xie L (2022) Theory of mind and facial emotion recognition in adults with temporal lobe epilepsy: a meta-analysis. Front Psychiatry 13:976439. https://doi.org/10.3389/fpsyt.2022.976439

    Article  PubMed  PubMed Central  Google Scholar 

  3. Bonora A, Benuzzi F, Monti G, Mirandola L, Pugnaghi M, Nichelli P, Meletti S (2011) Recognition of emotions from faces and voices in medial temporal lobe epilepsy. Epilepsy Behav 20(4):648–654. https://doi.org/10.1016/j.yebeh.2011.01.027

    Article  PubMed  Google Scholar 

  4. Hlobil U, Rathore C, Alexander A, Sarma S, Radhakrishnan K (2008) Impaired facial emotion recognition in patients with mesial temporal lobe epilepsy associated with hippocampal sclerosis (MTLE-HS): side and age at onset matters. Epilepsy Res 80(2–3):150–157. https://doi.org/10.1016/j.eplepsyres.2008.03.018

    Article  PubMed  Google Scholar 

  5. Stewart E, Catroppa C, Gonzalez L, Gill D, Webster R, Lawson J, Sabaz M, Mandalis A, Barton B, McLean S, Lah S (2019) Facial emotion perception and social competence in children (8 to 16 years old) with genetic generalized epilepsy and temporal lobe epilepsy. Epilepsy Behav 100(Pt B):106301. https://doi.org/10.1016/j.yebeh.2019.04.054

    Article  PubMed  Google Scholar 

  6. Meletti S, Benuzzi F, Rubboli G, Cantalupo G, Stanzani Maserati M, Nichelli P, Tassinari CA (2003) Impaired facial emotion recognition in early-onset right mesial temporal lobe epilepsy. Neurology 60(3):426–431. https://doi.org/10.1212/wnl.60.3.426

    Article  CAS  PubMed  Google Scholar 

  7. Meletti S, Benuzzi F, Cantalupo G, Rubboli G, Tassinari CA, Nichelli P (2009) Facial emotion recognition impairment in chronic temporal lobe epilepsy. Epilepsia 50(6):1547–1559. https://doi.org/10.1111/j.1528−1167.2008.01978.x

    Article  PubMed  Google Scholar 

  8. Steiger BK, Jokeit H (2017) Why epilepsy challenges social life. Seizure 44:194–198. https://doi.org/10.1016/j.seizure.2016.09.008

    Article  PubMed  Google Scholar 

  9. Myers L, Lancman M, Laban-Grant O, Lancman M, Jones J (2017) Socialization characteristics in persons with epilepsy. Epilepsy Behav 72:99–107. https://doi.org/10.1016/j.yebeh.2017.04.036

    Article  PubMed  Google Scholar 

  10. Amlerova J, Cavanna AE, Bradac O, Javurkova A, Raudenska J, Marusic P (2014) Emotion recognition and social cognition in temporal lobe epilepsy and the effect of epilepsy surgery. Epilepsy Behav 36:86–89. https://doi.org/10.1016/j.yebeh.2014.05.001

    Article  PubMed  Google Scholar 

  11. Tanaka A, Akamatsu N, Yamano M, Nakagawa M, Kawamura M, Tsuji S (2013) A more realistic approach, using dynamic stimuli, to test facial emotion recognition impairment in temporal lobe epilepsy. Epilepsy Behav 28(1):12–16. https://doi.org/10.1016/j.yebeh.2013.03.022

    Article  PubMed  Google Scholar 

  12. Cohn M, St-Laurent M, Barnett A, McAndrews MP (2015) Social inference deficits in temporal lobe epilepsy and lobectomy: risk factors and neural substrates. Soc Cogn Affect Neurosci 10(5):636–644. https://doi.org/10.1093/scan/nsu101

    Article  PubMed  Google Scholar 

  13. Bujarski KA, Flashman L, Li Z, Tosteson TD, Jobst BC, Thadani VM, Kobylarz EJ, Roberts DW, Roth RM (2016) Investigating social cognition in epilepsy using a naturalistic task. Epilepsia 57(9):1515–1520. https://doi.org/10.1111/epi.13477

    Article  PubMed  Google Scholar 

  14. Zhang X, Yin L, Cohn JF, Canavan S, Reale M, Horowitz A, Liu P (2013) A high-resolution spontaneous 3d dynamic facial expression database. In: 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG). IEEE, pp 1–6

  15. Motofei IG, Rowland DL (2014) The ventral-hypothalamic input route: a common neural network for abstract cognition and sexuality. BJU Int 113(2):296–303. https://doi.org/10.1111/bju.12399

    Article  PubMed  Google Scholar 

  16. Bal E, Harden E, Lamb D, Van Hecke AV, Denver JW, Porges SW (2010) Emotion recognition in children with autism spectrum disorders: relations to eye gaze and autonomic state. J Autism Dev Disord 40(3):358–370. https://doi.org/10.1007/s10803-009-0884−3

    Article  PubMed  Google Scholar 

  17. Gomez-Ibanez A, Urrestarazu E, Viteri C (2014) Recognition of facial emotions and identity in patients with mesial temporal lobe and idiopathic generalized epilepsy: an eye-tracking study. Seizure 23(10):892–898. https://doi.org/10.1016/j.seizure.2014.08.012

    Article  PubMed  Google Scholar 

  18. Metternich B, Gehrer NA, Wagner K, Geiger MJ, Schütz E, Schulze-Bonhage A, Heers M, Schönenberg M (2022) Eye-movement patterns during emotion recognition in focal epilepsy: an exploratory investigation. Seizure 100:95–102. https://doi.org/10.1016/j.seizure.2022.06.018

    Article  PubMed  Google Scholar 

  19. Liu YC, Tseng YH, Tsai MC, Yeh ZT (2018) Reliability and validity of the awareness of social inference test (TASIT)—Mandarin Chinese Version. Re Appl Psychol. https://doi.org/10.3966/156092512018040068002

    Article  Google Scholar 

  20. Edwards M, Stewart E, Palermo R, Lah S (2017) Facial emotion perception in patients with epilepsy: a systematic review with meta-analysis. Neurosci Biobehav Rev 83:212–225. https://doi.org/10.1016/j.neubiorev.2017.10.013

    Article  PubMed  Google Scholar 

  21. Monti G, Meletti S (2015) Emotion recognition in temporal lobe epilepsy: a systematic review. Neurosci Biobehav Rev 55:280–293. https://doi.org/10.1016/j.neubiorev.2015.05.009

    Article  PubMed  Google Scholar 

  22. Patel GH, Arkin SC, Ruiz-Betancourt DR, DeBaun HM, Strauss NE, Bartel LP, Grinband J, Martinez A, Berman RA, Leopold DA, Javitt DC (2021) What you see is what you get: visual scanning failures of naturalistic social scenes in schizophrenia. Psychol Med 51(16):2923–2932. https://doi.org/10.1017/S0033291720001646

    Article  PubMed  Google Scholar 

  23. Iakimova G, Moriano C, Farruggio L, Jover F (2016) Socio-demographic and clinical correlates of facial expression recognition disorder in the euthymic phase of bipolar patients. Can J Psychiatry 61(10):633–642. https://doi.org/10.1177/0706743716639927

    Article  PubMed  PubMed Central  Google Scholar 

  24. Gillespie DC, Duncan SE, Flewitt BI, Sacripante R, Chin RF (2023) Screening for anxiety, depression and suicidality by epilepsy specialists in adult services in Scotland. Epilepsy Behav 142:109187. https://doi.org/10.1016/j.yebeh.2023.109187

    Article  PubMed  Google Scholar 

  25. Mo F, Gu J, Zhao K, Fu X (2021) Confusion effects of facial expression recognition in patients with major depressive disorder and healthy controls. Front Psychol 12:703888. https://doi.org/10.3389/fpsyg.2021.703888

    Article  PubMed  PubMed Central  Google Scholar 

  26. Roy-Charland A, Perron M, Beaudry O, Eady K (2014) Confusion of fear and surprise: a test of the perceptual-attentional limitation hypothesis with eye movement monitoring. Cogn Emot 28(7):1214–1222. https://doi.org/10.1080/02699931.2013.878687

    Article  PubMed  Google Scholar 

  27. Wegrzyn M, Vogt M, Kireclioglu B, Schneider J, Kissler J (2017) Mapping the emotional face. How individual face parts contribute to successful emotion recognition. PLoS ONE 12(5):0177239. https://doi.org/10.1371/journal.pone.0177239

    Article  CAS  Google Scholar 

  28. Dutta P, Barman A (2020) Human emotion recognition from face images. Springer Nature, Berlin

    Book  Google Scholar 

  29. Baron-Cohen S, Jolliffe T, Mortimore C, Robertson M (1997) Another advanced test of theory of mind: evidence from very high functioning adults with autism or asperger syndrome. J Child Psychol Psychiatry 38(7):813–822. https://doi.org/10.1111/j.1469-7610.1997.tb01599.x

    Article  CAS  PubMed  Google Scholar 

  30. Baron-Cohen S, Wheelwright S, Hill J, Raste Y, Plumb I (2001) The “reading the mind in the eyes” Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. J Child Psychol Psychiatry 42(2):241–251

    Article  CAS  PubMed  Google Scholar 

  31. Morningstar M, French RC, Mattson WI, Englot DJ, Nelson EE (2021) Social brain networks: Resting-state and task-based connectivity in youth with and without epilepsy. Neuropsychologia 157:107882. https://doi.org/10.1016/j.neuropsychologia.2021.107882

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Cao R, Li X, Brandmeir NJ, Wang S (2021) Encoding of facial features by single neurons in the human amygdala and hippocampus. Commun Biol 4(1):1394. https://doi.org/10.1038/s42003-021-02917−1

    Article  PubMed  PubMed Central  Google Scholar 

  33. Meletti S, Cantalupo G, Benuzzi F, Mai R, Tassi L, Gasparini E, Tassinari CA, Nichelli P (2012) Fear and happiness in the eyes: an intra-cerebral event-related potential study from the human amygdala. Neuropsychologia 50(1):44–54. https://doi.org/10.1016/j.neuropsychologia.2011.10.020

    Article  PubMed  Google Scholar 

  34. Kegel LC, Frühholz S, Grunwald T, Mersch D, Rey A, Jokeit H (2021) Temporal lobe epilepsy alters neural responses to human and avatar facial expressions in the face perception network. Brain Behav 11(6):e02140. https://doi.org/10.1002/brb3.2140

    Article  PubMed  PubMed Central  Google Scholar 

  35. Tanaka JW, Sung A (2016) The “eye avoidance” hypothesis of autism face processing. J Autism Dev Disord 46(5):1538–1552. https://doi.org/10.1007/s10803-013−1976-7

    Article  PubMed  PubMed Central  Google Scholar 

  36. Dalton KM, Nacewicz BM, Johnstone T, Schaefer HS, Gernsbacher MA, Goldsmith HH, Alexander AL, Davidson RJ (2005) Gaze fixation and the neural circuitry of face processing in autism. Nat Neurosci 8(4):519–526. https://doi.org/10.1038/nn1421

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Poncet F, Soussignan R, Jaffiol M, Gaudelus B, Leleu A, Demily C, Franck N, Baudouin JY (2021) The spatial distribution of eye movements predicts the (false) recognition of emotional facial expressions. PLoS ONE 16(1):e0245777. https://doi.org/10.1371/journal.pone.0245777

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Yitzhak N, Pertzov Y, Aviezer H (2021) The elusive link between eye-movement patterns and facial expression recognition. Social Personal Psychol Compass. https://doi.org/10.1111/spc3.12621

    Article  Google Scholar 

  39. Beaudry O, Roy-Charland A, Perron M, Cormier I, Tapp R (2014) Featural processing in recognition of emotional facial expressions. Cogn Emot 28(3):416–432. https://doi.org/10.1080/02699931.2013.833500

    Article  PubMed  Google Scholar 

  40. Schurgin MW, Nelson J, Iida S, Ohira H, Chiao JY, Franconeri SL (2014) Eye movements during emotion recognition in faces. J Vis 14(13):14. https://doi.org/10.1167/14.13.14

    Article  CAS  PubMed  Google Scholar 

  41. Ekman P (1976) Pictures of facial affect. Consulting Psychologists Press

    Google Scholar 

  42. Sedda A, Rivolta D, Scarpa P, Burt M, Frigerio E, Zanardi G, Piazzini A, Turner K, Canevini MP, Francione S, Lo Russo G, Bottini G (2013) Ambiguous emotion recognition in temporal lobe epilepsy: the role of expression intensity. Cogn Affect Behav Neurosci 13(3):452–463. https://doi.org/10.3758/s13415-013-0153-y

    Article  PubMed  Google Scholar 

  43. Dunifon CM, Rivera S, Robinson CW (2016) Auditory stimuli automatically grab attention: Evidence from eye tracking and attentional manipulations. J Exp Psychol Hum Percept Perform 42(12):1947–1958. https://doi.org/10.1037/xhp0000276

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We are very grateful to all the participants in this study. Sincere thanks also go to Professor Yin from Binghamton University and Professor Ye from Fu Jen Catholic University, for providing the dynamic expression task and TASIT. We appreciate Professor Atkinson from Durham University for your body movement stimuli though we do not apply this task to this paper. Thank you to everyone who helped us directly and indirectly.

Funding

This study was supported by National Key Research and Development Program of China (2022YFC2503804), National Natural Science Foundation of China (82071461; 82271503), “From 0 to 1” Original Innovation Project of the Basic Frontier Scientific Research Program of the Chinese Academy of Sciences (29J20-015-III), and the Natural Science Foundation of Hunan Province, China (2023JJ30939).

Author information

Author notes

  1. Kailing Huang and Ziwei Tian contributed equally to this work.

Authors and Affiliations

  1. Department of Neurology, Xiangya Hospital, Central South University, Changsha, 410008, People’s Republic of China

    Kailing Huang, Qiong Zhang, Haojun Yang, Shirui Wen, Jie Feng, Weiting Tang & Li Feng

  2. National Clinical Research Center for Geriatric Disorders, Xiangya Hospital, Central South University, Changsha, Hunan, 410008, People’s Republic of China

    Kailing Huang, Qiong Zhang, Haojun Yang, Shirui Wen, Jie Feng, Weiting Tang & Li Feng

  3. Key Laboratory of Biomedical Spectroscopy of Xi’an, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an, 710119, China

    Ziwei Tian & Quan Wang

  4. Key Laboratory of Spectral Imaging Technology, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an, 710119, China

    Ziwei Tian & Quan Wang

  5. University of Chinese Academy of Sciences, Beijing, 101400, China

    Ziwei Tian

  6. Department of Neurology, Xiangya Hospital, Central South University (Jiangxi Branch), Nanchang, 330000, Jiangxi, China

    Li Feng

Authors
  1. Kailing Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziwei Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haojun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shirui Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jie Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Weiting Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Quan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Li Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Kailing Huang, Qiong Zhang, Haojun Yang, and Shirui Wen contributed to the acquisition of data and drafting of the manuscript for content and interpretation of data, Ziwei Tian contributed to the programming of the experimental paradigm; Jie Feng and Weiting Tang contributed to the conceptualization and study design; Li Feng contributed to study design and revision of manuscript. Quan Wang contributed to study design and data analysis.

Corresponding authors

Correspondence to Quan Wang or Li Feng.

Ethics declarations

Conflict of interest

The authors declare no competing financial interests.

Ethical approval

The studies involving human participants were reviewed and approved by the Ethics Committee of Xiangya Hospital of Central South University. The patients/participants provided their written informed consent to participate in this study.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOC 390 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, K., Tian, Z., Zhang, Q. et al. Reduced eye gaze fixation during emotion recognition among patients with temporal lobe epilepsy. J Neurol 271, 2560–2572 (2024). https://doi.org/10.1007/s00415-024-12202-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00415-024-12202-w

Keywords

Search

Navigation