Emotion Recognition

  • Stefano MelettiEmail author
Part of the Neuropsychiatric Symptoms of Neurological Disease book series (NSND)


The recognition of emotional signals from all sensory modalities is a critical component of human social interactions. It is through the understanding of the affective states of others that we can guide our own behavioral responses. Notably, facial expression provides the greatest amount of emotional cues that are useful in recognizing emotions, such as joy, anger, and fear. The temporal lobe – and the amygdala in particular – plays a crucial role in processing the appropriate autonomic and behavioral responses to relevant emotional stimuli. Only in the past decade, however, the role played by the antero-medial temporal lobe region has been demonstrated in decoding the emotions, mental states, and beliefs of others. In the field of epilepsy, this knowledge has several clinical, as well as speculative, implications. Indeed, temporal lobe epilepsy (TLE) is the most common type of focal epilepsy. It is frequently characterized by lesions or gliosis/atrophy (hippocampal sclerosis) involving the medial temporal lobe region, and antero-medial temporal lobe resection is the standard treatment for drug-resistant medial TLE. Consequently, the investigation of emotional and social competence in TLE patients has been the focus of several studies. Such studies have extended the scope of neuropsychological evaluation in TLE beyond the traditional evaluation of memory, language, and executive functions.


Amygdala Emotion Recognition Emotional Prosody Epilepsy Facial Expression fMRI Temporal Lobe Epilepsy Temporal Lobectomy Social Cognition 



I want to thank all the friends and colleagues that promoted the research in this field during the years, in particular Carlo Alberto Tassinari, Gaetano Cantalupo, Francesca Benuzzi, Giulia Monti, and Paolo Nichelli.


  1. 1.
    Darwin C, Ekman P, Prodger P. The expression of the emotions in man and animals. 3rd ed. London: Harper Collins; 1998.Google Scholar
  2. 2.
    Damasio A. The somatic marker hypothesis and the possible functions of the prefrontal cortex. Proc R Soc. 1996;B351:1420–31.Google Scholar
  3. 3.
    Schwartzkroin PA. Hippocampal slices in experimental and human epilepsy. Adv Neurol. 1986;44:991–1010.PubMedGoogle Scholar
  4. 4.
    Graebenitz S, Kedo O, Speckmann EJ, Gorji A, Panneck H, Hans V, et al. Interictal-like network activity and receptor expression in the epileptic human lateral amygdala. Brain. 2011;134:2929–47.PubMedCrossRefGoogle Scholar
  5. 5.
    Wieser HG, ILAE Commission on Neurosurgery of Epilepsy. ILAE Commission Report, Mesial temporal lobe epilepsy with hippocampal sclerosis. Epilepsia. 2004;45:695–714.PubMedCrossRefGoogle Scholar
  6. 6.
    Hermann BP, Seidenberg M, Schoenfeld J, Davies K. Neuropsychological characteristics of the syndrome of mesial temporal lobe epilepsy. Arch Neurol. 1997;54:369–76.PubMedCrossRefGoogle Scholar
  7. 7.
    Jokeit H, Ebner A. Long term effects of refractory temporal lobe epilepsy on cognitive abilities: a cross sectional study. J Neurol Neurosurg Psychiatry. 1999;67:44–50.PubMedCentralPubMedCrossRefGoogle Scholar
  8. 8.
    Helmstaedter C, Kurthen M. Memory and epilepsy: characteristics course and influence of drugs and surgery. Curr Opin Neurol. 2001;14:211–6.PubMedCrossRefGoogle Scholar
  9. 9.
    Helmstaedter C, Kurthen M, Lux S, Reuber M, Elger CE. Chronic epilepsy and cognition: a longitudinal study in temporal lobe epilepsy. Ann Neurol. 2003;54:425–32.PubMedCrossRefGoogle Scholar
  10. 10.
    Hermann B, Seidenberg M, Jones J. The neurobehavioural comorbidities of epilepsy: can a natural history be developed? Lancet Neurol. 2008;7:151–60.PubMedCrossRefGoogle Scholar
  11. 11.
    Helmstaedter C, Witt JA. Multifactorial etiology of interictal behavior in frontal and temporal lobe epilepsy. Epilepsia. 2012;53:1765–73.PubMedCrossRefGoogle Scholar
  12. 12.
    Anderson AK, Spencer DD, Fulbright RK, Phelps EA. Contribution of the anteromedial temporal lobes to the evaluation of facial emotion. Neuropsychology. 2000;14:526–36.PubMedCrossRefGoogle Scholar
  13. 13.
    Anderson AK, Phelps EA. Lesions of the human amygdala impair enhanced perception of emotionally salient events. Nature. 2001;411:305–9.PubMedCrossRefGoogle Scholar
  14. 14.
    Adolphs R, Tranel D, Damasio H, Damasio A. Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala. Nature. 1994;372:669–72.PubMedCrossRefGoogle Scholar
  15. 15.
    Adolphs R, Tranel D, Damasio H. Emotion recognition from faces and prosody following temporal lobectomy. Neuropsychology. 2001;15:396–404.PubMedCrossRefGoogle Scholar
  16. 16.
    Adolphs R. Neural systems for recognizing emotions. Curr Opin Neurobiol. 2002;12:169–77.PubMedCrossRefGoogle Scholar
  17. 17.
    Adolphs R. Fear faces and the human amygdala. Curr Opin Neurobiol. 2008;18:166–72.PubMedCentralPubMedCrossRefGoogle Scholar
  18. 18.
    Adolphs R. What does the amygdala contribute to social cognition? Ann N Y Acad Sci. 2010;1191:42–61.PubMedCentralPubMedCrossRefGoogle Scholar
  19. 19.
    Adolphs R. Cognitive neuroscience of human social behavior. Nat Rev Neurosci. 2013;4:165–78.CrossRefGoogle Scholar
  20. 20.
    LeDoux JE. The emotional brain: the mysterious underpinnings of emotional life. New York: Touchstone; 1996.Google Scholar
  21. 21.
    Harrison NA, Gray MA, Gianaros PJ, Critchley HD. The embodiment of emotional feelings in the brain. J Neurosci. 2010;30:12878–84.PubMedCentralPubMedCrossRefGoogle Scholar
  22. 22.
    Phelps EA. Emotion and cognition: insights from studies of the human amygdala. Annu Rev Psychol. 2006;57:27–53.PubMedCrossRefGoogle Scholar
  23. 23.
    Salzman CD, Fusi S. Emotion, cognition, and mental state representation in amygdala and prefrontal cortex. Annu Rev Neurosci. 2010;33:173–202.PubMedCentralPubMedCrossRefGoogle Scholar
  24. 24.
    Winston JS, O’Doherty J, Dolan RJ. Common and distinct neural responses during direct and incidental processing of multiple facial emotions. Neuroimage. 2003;20:84–97.PubMedCrossRefGoogle Scholar
  25. 25.
    Wiebe S, Blume WT, Girvin JP, Eliasziw M. A randomized controlled trial of surgery for temporal-lobe epilepsy. N Engl J Med. 2001;345:311–8.PubMedCrossRefGoogle Scholar
  26. 26.
    Meletti S, Benuzzi F, Rubboli G, Cantalupo G, Stanzani Maserati M, et al. Impaired facial emotion recognition in early-onset right mesial temporal lobe epilepsy. Neurology. 2003;60:426–31.PubMedCrossRefGoogle Scholar
  27. 27.
    Meletti S, Benuzzi F, Nichelli P, Tassinari CA. Damage to the right hippocampal-amygdala formation during early infancy and recognition of fearful faces. Neuropsychological and fMRI evidence in subjects with temporal lobe epilepsy. Ann N Y Acad Sci. 2003;1000:385–8.PubMedCrossRefGoogle Scholar
  28. 28.
    Meletti S, Benuzzi F, Cantalupo G, Rubboli G, Tassinari CA, Nichelli P. Facial emotion recognition impairment in chronic temporal lobe epilepsy. Epilepsia. 2009;50:1547–59.PubMedCrossRefGoogle Scholar
  29. 29.
    Broicher SD, Kuchukhidze G, Grunwald T, Kramer G, Kurthena M, Jokeit H. “Tell me how do I feel” – emotion recognition and theory of mind in symptomatic mesial temporal lobe epilepsy. Neuropsychologia. 2012;50:118–28.PubMedCrossRefGoogle Scholar
  30. 30.
    Giovagnoli AR, Franceschetti S, Reati F, Parente A, Maccagno C, et al. Theory of mind in frontal and temporal lobe epilepsy: cognitive and neural aspects. Epilepsia. 2011;52:1995–2002.PubMedCrossRefGoogle Scholar
  31. 31.
    Gosselin N, Peretz I, Noulhiane M, Hasboun D, Beckett C, et al. Impaired recognition of scary music following unilateral temporal lobe excision. Brain. 2005;128:628–40.PubMedCrossRefGoogle Scholar
  32. 32.
    Gosselin N, Peretz I, Hasboun D, Baulac M, Samson S. Impaired recognition of musical emotions and facial expressions following anteromedial temporal lobe excision. Cortex. 2011;47:1116–25.PubMedCrossRefGoogle Scholar
  33. 33.
    Bonora A, Benuzzi F, Monti G, Mirandola L, Pugnaghi M, et al. Recognition of emotions from faces and voices in medial temporal lobe epilepsy. Epilepsy Behav. 2011;20:648–54.PubMedCrossRefGoogle Scholar
  34. 34.
    Dellacherie D, Hasboun D, Baulac M, Belin P, Samson S. Impaired recognition of fear in voices and reduced anxiety after unilateral temporal lobe resection. Neuropsychologia. 2011;49:618–29.PubMedCrossRefGoogle Scholar
  35. 35.
    Ekman P, Friesen WV. Pictures of facial affect. Palo Alto: Consulting Psychologist Press; 1976.Google Scholar
  36. 36.
    Ekman P. The argument and evidence about universals in facial expressions of emotion. In: Wagner H, Manstead A, editors. Handbook of psychophysiology: emotion and social behavior. London: Wiley; 1989.Google Scholar
  37. 37.
    Ekman P. Facial expression and emotion. Am Psychol. 1993;48:384–92.PubMedCrossRefGoogle Scholar
  38. 38.
    Calder AJ, Young AW. Understanding the recognition of facial identity and facial expression. Nat Rev Neurosci. 2005;6:641–51.PubMedCrossRefGoogle Scholar
  39. 39.
    Calder A, Young AW, Rowland D, Perrett DI, Hodges RJ, Etcoff NL. Facial emotion recognition after bilateral amygdala damage: differentially severe impairment of fear. Cogn Neuropsychol. 1996;13:699–745.CrossRefGoogle Scholar
  40. 40.
    Reynders HJ, Broks P, Dickson JM, Lee CE, Turpin G. Investigation of social and emotion information processing in temporal lobe epilepsy with ictal fear. Epilepsy Behav. 2005;7:419–29.PubMedCrossRefGoogle Scholar
  41. 41.
    Walpole P, Isaac CL, Reynders HJ. A comparison of emotional and cognitive intelligences in people with and without temporal lobe epilepsy. Epilepsia. 2008;49:1470–4.PubMedCrossRefGoogle Scholar
  42. 42.
    Sedda A, Rivolta D, Scarpa P, Frigerio E, Burt M, Zanardi G, et al. Ambiguous emotions recognition in temporal lobe epilepsy: the role of expression intensity. Cogn Affect Behav Neurosci. 2013;13:452–63.PubMedCrossRefGoogle Scholar
  43. 43.
    Young AW, Hellawell DJ, Van deWal C, Johnson M. Facial expression processing after amygdalotomy. Neuropsychologia. 1996;34:31–9.PubMedCrossRefGoogle Scholar
  44. 44.
    Benuzzi F, Meletti S, Zamboni G, Calandra-Buonaura G, Serafini M, Lui F, et al. Impaired fear processing in right mesial temporal sclerosis: a fMRI study. Brain Res Bull. 2004;63:269–81.PubMedCrossRefGoogle Scholar
  45. 45.
    Yamada M, Muraib T, Satoa W, Namikib C, Miyamotoc T, Ohigashi Y. Emotion recognition from facial expressions in a temporal lobe epileptic patient with ictal fear. Neuropsychologia. 2005;43:434–41.PubMedCrossRefGoogle Scholar
  46. 46.
    Shaw P, Lawrence E, Bramham J, Brierley B, Radbourne C, David AS. A prospective study of the effects of anterior temporal lobectomy on emotion recognition and theory of mind. Neuropsychologia. 2007;45:2783–90.PubMedCrossRefGoogle Scholar
  47. 47.
    Brierley B, Medford N, Shaw P, David AS. Emotional memory and perception in temporal lobectomy patients with amygdala damage. J Neurol Neurosurg Psychiatry. 2004;75:593–9.PubMedCentralPubMedCrossRefGoogle Scholar
  48. 48.
    Golouboff N, Fiori N, Delalande O, Fohlen M, Dellatolas G, Jambaqué I. Impaired facial expression recognition in children with temporal lobe epilepsy: impact of early seizure onset on fear recognition. Neuropsychologia. 2008;46:1415–28.PubMedCrossRefGoogle Scholar
  49. 49.
    Pinabiaux C, Bulteau C, Fohlen M, Dorfmuller G, Chiron C, Hertz-Pannier L, et al. Impaired emotional memory recognition after early temporal lobe epilepsy surgery: the fearful face exception? Cortex. 2013;49:1386–93.PubMedCrossRefGoogle Scholar
  50. 50.
    Ammerlaan E, Hendriks PH, Colon AJ, Kessels RPC. Emotion perception and interpersonal behavior in epilepsy patients after unilateral amygdalohippocampectomy. Acta Neurobiol Exp. 2008;68:214–8.Google Scholar
  51. 51.
    Tanaka A, Akamatsu N, Yamano M, Nakagawa M, Kawamura M, Tsuji S. A more realistic approach using dynamic stimuli to test facial emotion recognition impairment in temporal lobe epilepsy. Epilepsy Behav. 2013;28:12–6.PubMedCrossRefGoogle Scholar
  52. 52.
    Scott SK, Young AW, Calder AJ, Hellawell DJ, Aggleton JP, Johnson M. Impaired auditory recognition of fear and anger following bilateral amygdala lesions. Nature. 1997;385:254–7.PubMedCrossRefGoogle Scholar
  53. 53.
    Fowler H, Baker GA, Tipples J, Hare DJ, Keller S, Chadwick DW, Young AW. Recognition of emotion with temporal lobe epilepsy and asymmetrical amygdala damage. Epilepsy Behav. 2006;9:164–72.PubMedCrossRefGoogle Scholar
  54. 54.
    Khalfa S, Guye M, Peretz I, Chapon F, Girard N, Chauvel P, Liegeois-Chauvel C. Evidence of lateralized anteromedial temporal structures involvement in musical emotion processing. Neuropsychologia. 2008;46:2485–93.PubMedCrossRefGoogle Scholar
  55. 55.
    Sanz-Martin A, Guevara MA, Corsi-Cabrera M, Ondarza-Rovira R, Ramos-Loyo J. Differential effect of left and right temporal lobectomy on emotional recognition and experience in patients with epilepsy. Rev Neurol. 2006;42:391–8.PubMedGoogle Scholar
  56. 56.
    Benton AH, Hamsher K, Varney N, Spreen O. Contribution to neuropsychological assessment. New York: Oxford University Press; 1983.Google Scholar
  57. 57.
    Kanwisher N, McDermott J, Chun MM. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J Neurosci. 1997;17:4302–11.PubMedGoogle Scholar
  58. 58.
    Allison T, Puce A, Spencer DD, McCarthy G. Electrophysiological studies of human face perception. I: potentials generated in occipitotemporal cortex by face and non-face stimuli. Cereb Cortex. 1999;9:415–30.PubMedCrossRefGoogle Scholar
  59. 59.
    Meletti S, Cantalupo G, Santoro F, Benuzzi F, Marliani AF, Tassinari CA, Rubboli G. Temporal lobe epilepsy and emotion recognition without amygdala: a case study of Urbach-Wiethe disease and review of the literature. Epileptic Disord. 2014;16:518–27.PubMedGoogle Scholar
  60. 60.
    Krolak-Salmon P, Henaff MA, Vighetto A, Bertrand O, Mauguiere F. Early amygdala reaction to fear spreading in occipital, temporal, and frontal cortex: a depth electrode ERP study in human. Neuron. 2004;42:665–76.PubMedCrossRefGoogle Scholar
  61. 61.
    Meletti S, Cantalupo G, Benuzzi F, Mai R, Tassi L, Gasparini E, et al. Fear and happiness in the eyes: an intra-cerebral event-related potential study from the human amygdala. Neuropsychologia. 2012;50:44–54.PubMedCrossRefGoogle Scholar
  62. 62.
    Carvajal F, Rubio S, Martin P, Serrano JM, Garcia-Sola R. Perception and recall of faces and facial expressions following temporal lobectomy. Epilepsy Behav. 2009;14:60–5.PubMedCrossRefGoogle Scholar
  63. 63.
    Bernasconi N, Bernasconi A, Caramanos Z, Antel SB, Andermann F, Arnold DL. Mesial temporal damage in temporal lobe epilepsy: a volumetric MRI study of the hippocampus amygdala and parahippocampal region. Brain. 2003;126:462–9.PubMedCrossRefGoogle Scholar
  64. 64.
    Chassoux F, Semah F, Bouilleret V, Landre E, Devaux B, Turak B, et al. Metabolic changes and electro-clinical patterns in mesio-temporal lobe epilepsy: a correlative study. Brain. 2004;127:164–74.PubMedCrossRefGoogle Scholar
  65. 65.
    Lee SK, Kim DW, Kim KK, Chung CK, Song IC, Chang KH. Effect of seizure on hippocampus in mesial temporal lobe epilepsy and neocortical epilepsy: an MRS study. Neuroradiology. 2005;47:916–23.PubMedCrossRefGoogle Scholar
  66. 66.
    Saykin AJ, Gur RC, Sussman NM, O’Connor MJ, Gur RE. Memory deficits before and after temporal lobectomy: effect of laterality and age of onset. Brain Cogn. 1989;9:191–200.PubMedCrossRefGoogle Scholar
  67. 67.
    Davies KG, Hermann BP, Dohan Jr FC, Foley KT, Bush AJ, Wyler AR. Relationship of hippocampal sclerosis to duration and age of onset of epilepsy and childhood febrile seizures in temporal lobectomy patients. Epilepsy Res. 1996;24:119–26.PubMedCrossRefGoogle Scholar
  68. 68.
    McClelland III S, Garcia RE, Peraza DM, Shih TT, Hirsch LJ, et al. Facial emotion recognition after curative nondominant temporal lobectomy in patients with mesial temporal sclerosis. Epilepsia. 2006;47:1337–42.PubMedCrossRefGoogle Scholar
  69. 69.
    Palermo R, Schmalzl L, Mohamed A, Bleasel A, Miller L. The effect of unilateral amygdala removals on detecting fear from briefly presented backward-masked faces. J Clin Exp Neuropsychol. 2010;32:123–31.PubMedCrossRefGoogle Scholar
  70. 70.
    Cantalupo G, Meletti S, Miduri A, Mazzotta S, Rios-Pohl L, Benuzzi F, et al. Facial emotion recognition in childhood: the effects of febrile seizures in the developing brain. Epilepsy Behav. 2013;29:211–6.PubMedCrossRefGoogle Scholar
  71. 71.
    Hlobil U, Rathore C, Alexander A, Sarma S, Radhakrishnan K. Impaired facial emotion recognition in patients with mesial temporal lobe epilepsy associated with hippocampal sclerosis (MTLE-HS): Side and age at onset matters. Epilepsy Res. 2008;80:150–7.PubMedCrossRefGoogle Scholar
  72. 72.
    Amlerova J, Cavanna AE, Bradac O, Javurkova A, Raudenska J, Marusic P. Emotion recognition and social cognition in temporal lobe epilepsy and the effect of epilepsy surgery. Epilepsy Behav. 2014;36:86–9.PubMedCrossRefGoogle Scholar
  73. 73.
    Benuzzi F, Zamboni G, Meletti S, Serafini M, Lui F, Baraldi P, et al. Recovery from emotion recognition impairment after temporal lobectomy. Front Neurol. 2014;5:92. doi: 10.3389/fneur.2014.00092.PubMedCentralPubMedCrossRefGoogle Scholar
  74. 74.
    Meletti S, Picardi A, De Risi M, Monti G, Esposito V, et al. The affective value of faces in patients achieving long-term seizure freedom after temporal lobectomy. Epilepsy Behav. 2014;36:97–101.PubMedCrossRefGoogle Scholar
  75. 75.
    Morris JS, Frith CD, Perrett DI, Rowland D, Young AW, et al. A differential neural response in the human amygdala to fearful and happy facial expressions. Nature. 1996;383:812–5.PubMedCrossRefGoogle Scholar
  76. 76.
    Breiter HC, Etcoff NL, Whalen PJ, Kennedy WA, Rauch SL, Buckner RL, et al. Response and habituation of the human amygdala during visual processing of facial expression. Neuron. 1996;17:875–87.PubMedCrossRefGoogle Scholar
  77. 77.
    Vuilleumier P, Richardson MP, Armony JL, Driver J, Dolan RJ. Distant influences of amygdala lesion on visual cortical activation during emotional face processing. Nat Neurosci. 2004;7:1271–8.PubMedCrossRefGoogle Scholar
  78. 78.
    Schacher M, Haemmerle B, Woermann FG, Okujava M, Huber D, Grunwald T, et al. Amygdala fMRI lateralizes temporal lobe epilepsy. Neurology. 2006;66:81–7.PubMedCrossRefGoogle Scholar
  79. 79.
    Bonelli SB, Powell R, Yogarajah M, Thompson PJ, Symms MR, et al. Preoperative amygdala fMRI in temporal lobe epilepsy. Epilepsia. 2009;50:217–27.PubMedCentralPubMedCrossRefGoogle Scholar
  80. 80.
    Broicher S, Kuchukhidze G, Grunwald T, Krämer G, Kurthen M, Trinka E, Jokeit H. Association between structural abnormalities and fMRI response in the amygdala in patients with temporal lobe epilepsy. Seizure. 2010;19:426–31.PubMedCrossRefGoogle Scholar
  81. 81.
    Labudda K, Mertens M, Steinkroeger C, Bien CG, Woermann FG. Lesion side matters – an fMRI study on the association between neural correlates of watching dynamic fearful faces and their evaluation in patients with temporal lobe epilepsy. Epilepsy Behav. 2014;31:321–8.PubMedCrossRefGoogle Scholar
  82. 82.
    Frühholz S, Hofstetter C, Cristinzio C, Saj A, Seeck M, et al. Asymmetrical effects of unilateral right or left amygdala damage on auditory cortical processing of vocal emotions. Proc Natl Acad Sci U S A. 2015;112(5):1583–8.PubMedCentralPubMedCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of Biomedical, Metabolic, and Neural ScienceUniversity of Modena e Reggio EmiliaModenaItaly
  2. 2.Neurology UnitNOCSAE Hospital – AUSLModenaItaly

Personalised recommendations