Considering cross-cultural context in the automatic recognition of emotions

  • Maria Alejandra Quiros-RamirezEmail author
  • Takehisa Onisawa
Original Article


Automatic recognition of emotions remains an ongoing challenge and much effort is being invested towards developing a system to solve this problem. Although several systems have been proposed, there is still none that considers the cultural context for emotion recognition. It remains unclear whether emotions are universal or culturally specific. A study on how culture influences the recognition of emotions is presented. For this purpose, a multicultural corpus for cross-cultural emotion analysis is constructed. Subjects from three different cultures—American, Asian and European—are recruited. The corpus is segmented and annotated. To avoid language artifacts, the emotion recognition model considers facial expressions, head movements, body motions and dimensional emotions. Three training and testing paradigms are carried out to compare cultural effects: intra-cultural, cross-cultural and multicultural emotion recognition. Intra-cultural and multicultural emotion recognition paradigms raised the best recognition results; cross-cultural emotion recognition rates were lower. These results suggest that emotion expression varies by culture, representing a hint of emotion specificity.


Affect Culture Universality Specificity Emotional corpus 


  1. 1.
    Anacleto JC, Pinatti de Carvalho AF (2008) Improving human-computer interaction by developing culture-sensitive applications based on common sense knowledge. In: Pinder S (ed) Advances in Human-Computer Interaction. I-Tech, Vienna, pp 1–30Google Scholar
  2. 2.
    Caridakis G, Wagner J, Raouzaiou A, Lingenfelser F, Karpouzis K, Andre E (2012) A cross-cultural, multimodal, affective corpus for gesture expressivity analysis. J Multimodal User Interfaces. doi: 10.1007/s12193-012-0112-x Google Scholar
  3. 3.
    Canu S, Grandvalet Y, Guigue V, Rakotomamonjy A (2005) SVM and kernel methods matlab toolbox. Percept Syst Inf 2:2Google Scholar
  4. 4.
    Cho J, Kato S, Itoh H (2009) Comparison of sensibilities of Japanese and Koreans in recognizing emotions from speech by using Bayesian networks. International Conference on System Man and Cybernetics, SMC 2009, pp 2866–2871Google Scholar
  5. 5.
    Darwin C (1872) The expression of the emotions in man and animals. John Murray, LondonCrossRefGoogle Scholar
  6. 6.
    Dan-Glauser ES, Scherer KR (2011) The Geneva affective picture database (GAPED): a new 730-picture database focusing on valence and normative significance. Behav Res Methods 43:468–477CrossRefGoogle Scholar
  7. 7.
    Ekman P (1994) Strong evidence for universals in facial expressions: a reply to Russell’s mistaken critique. Psychol Bull 115:268–287CrossRefGoogle Scholar
  8. 8.
    Gunes H, Schuller B (2013) Categorical and dimensional affect analysis in continuous input: current trends and future directions. Imag Vis Comput 31(2):120–136CrossRefGoogle Scholar
  9. 9.
    Haid J, Keltner D (1999) Culture and facial expression: open-ended methods find more expressions and a gradient recognition. Cogn Emot 13:225–266CrossRefGoogle Scholar
  10. 10.
    Harper R, Rodden T, Rogers Y, Sellen A (2008) Being human: human-computer interaction in the year 2020. Microsoft Research Ltda, EnglandGoogle Scholar
  11. 11.
    Heimgartner R, Kindermann H (2012) Revealing Cultural Influences in Human Computer Interaction by Analyzing Big Data in Interactions. Lecture Notes in Computer Science pp 572–583Google Scholar
  12. 12.
    Hoque M, Picard R (2011) Acted vs. natural frustration and delight: many people smile in natural frustration. IEEE International Conference on Automatic Face and Gesture Recognition doi: 10.1109/FG.2011.5771425
  13. 13.
    Jack RE, Blais C, Scheepers C, Schyns PG, Caldara R (2009) Cultural confusions show that facial expressions are not universal. Curr Biol 19:1543–1548CrossRefGoogle Scholar
  14. 14.
    Kamaruddin N, Wahab A, Quek C (2012) Cultural dependency analysis for understanding speech emotion. Expert Syst Appl 39:5115–5133CrossRefGoogle Scholar
  15. 15.
    Kamppuri M, Bednarik R, Tukiainen M (2006) The expanding focus of HCI: case culture. Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles pp 405–408Google Scholar
  16. 16.
    Kleinsmith A, De Silva P, Bianchi-Berthouze N (2006) Cross-cultural differences in recognizing affect from body posture. Interact Comput 18:1371–1389CrossRefGoogle Scholar
  17. 17.
    Mauss IB, Robinson MD (2009) Measures of emotion: a review. Cogn Emot 23:209–237CrossRefGoogle Scholar
  18. 18.
    Picard RW (1997) Affective Computing. The MIT Press, MassachusettsGoogle Scholar
  19. 19.
    Scherer KR, Clark-Polner E, Mortillaro M (2011) In the eye of the beholder? Universality and cultural specificity in the expression and perception of emotion. Int J Psychol 46:401–435CrossRefGoogle Scholar
  20. 20.
    Quiros-Ramirez MA, Onisawa T (2012) Assessing emotions in a cross-cultural context. System Man and Machine pp 2967–2972Google Scholar
  21. 21.
    Russell JA (1994) Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychol Bull 115:102–141CrossRefGoogle Scholar
  22. 22.
    Russell JA (1980) A circumplex model of affect. J Pers Soc Psychol 36:1161–1178CrossRefGoogle Scholar
  23. 23.
    Soleymani M, Lichtenauer M, Pun J, Pantic M (2011) A multi-modal affective database for affect recognition and implicit tagging. Int J Affect Comput pp 1–14Google Scholar
  24. 24.
    Wilting J, Krahmer E, Swerts M (2006) Real vs. acted emotional speech. Ninth International Conference on Spoken Language Processing. pp 17–21Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Maria Alejandra Quiros-Ramirez
    • 1
    Email author
  • Takehisa Onisawa
    • 1
  1. 1.Graduate School of Systems and Information EngineeringUniversity of TsukubaTsukubaJapan

Personalised recommendations