Emotion Twenty Questions: Toward a Crowd-Sourced Theory of Emotions

  • Abe Kazemzadeh
  • Sungbok Lee
  • Panayiotis G. Georgiou
  • Shrikanth S. Narayanan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6975)


This paper introduces a method for developing a socially-constructed theory of emotions that aims to reflect the aggregated judgments of ordinary people about emotion terms. Emotion Twenty Questions (EMO20Q) is a dialog-based game that is similar to the familiar Twenty Questions game except that the object of guessing is the name for an emotion, rather than an arbitrary object. The game is implemented as a dyadic computer chat application using the Extensible Messaging and Presence Protocol (XMPP). We describe the idea of a theory that is socially-constructed by design, or crowd-sourced, as opposed to the de facto social construction of theories by the scientific community. This paper argues that such a subtle change in paradigm is useful when studying natural usage of emotion words, which can mean different things to different people but still contain a shared, socially-defined meaning that can be arrived at through conversational dialogs. The game of EMO20Q provides a framework for demonstrating this shared meaning and, moreover, provides a standardized way for collecting the judgments of ordinary people. The paper offers preliminary results of EMO20Q pilot experiments, showing that such a game is feasible and that it generates a range of questions that can be used to describe emotions.


Natural Language Emotional Intelligence Emotion Word Question Type Formal Concept Analysis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Gordon, A., Kazemzadeh, A., Nair, A., Petrova, M.: Recognizing expressions of commonsense psychology in english text. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, ACL 2003 (2003)Google Scholar
  2. 2.
    Singh, P., Lin, T., Mueller, E.T., Lim, G., Perkins, T., Zhu, W.L.: Open mind common sense: Knowledge acquisition from the general public. In: Chung, S., et al. (eds.) CoopIS 2002, DOA 2002, and ODBASE 2002. LNCS, vol. 2519, pp. 1223–1237. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  3. 3.
    Devillers, L., Abrilian, S., Martin, J.-C.: Representing real-life emotions in audiovisual data with non basic emotional patterns and context features. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, pp. 519–526. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  4. 4.
    Mower, E., Metallinou, A., Lee, C.-C., Kazemzadeh, A., Busso, C., Lee, S., Narayanan, S.: Interpreting ambiguous emotional expressions. In: ACII Special Session: Recognition of Non-Prototypical Emotion from Speech-The Final Frontier?, Amsterdam, Netherlands (2009)Google Scholar
  5. 5.
    Howe, J.: The rise of crowdsourcing. Wired Magazine 14.06 (June 2006)Google Scholar
  6. 6.
    von Ahn, L., Dabbish, L.: Labeling images with a computer game. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2004)Google Scholar
  7. 7.
    Zhong, N., Liu, J., Yao, Y., Ohsuga, S.: Web intelligence. In: Computer Software and Applications Conference (2000)Google Scholar
  8. 8.
    Whissell, C.M.: The Dictionary of Affect in Language, pp. 113–131. Academic Press, London (1989)Google Scholar
  9. 9.
    Osgood, C.E., Suci, G.J., Tannenbaum, P.H.: The Measurement of Meaning. University of Illinois Press, US (1957)Google Scholar
  10. 10.
    Oudeyer, P.: The production and recognition of emotions in speech: features and algorithms. J. Hum. Comput. Stud. 59, 157–183 (2003)CrossRefGoogle Scholar
  11. 11.
    Russell, J.A., Mehrabian, A.: Evidence for a three-factor theory of emotions. Journal of Research in Personality 11, 273–294 (1977)CrossRefGoogle Scholar
  12. 12.
    Kazemzadeh, A., Lee, S., Narayanan, S.: An interval type-2 fuzzy logic system to translate between emotion-related vocabularies. In: Proceedings of Interspeech, Brisbane, Australia (September 2008)Google Scholar
  13. 13.
    Kazemzadeh, A.: Using interval type-2 fuzzy logic to translate emotion words from spanish to english. In: IEEE World Conference on Computational Intelligence (WCCI) FUZZ-IEEE Workshop (2010)Google Scholar
  14. 14.
    Enderton, H.B.: A Mathematical Introduction to Logic, 2nd edn. Academic Press, London (2001)zbMATHGoogle Scholar
  15. 15.
    Ganter, B., Wille, G.S.R. (eds.): Formal Concept Analysis: foundation and applications. Springer, Berlin (2005)Google Scholar
  16. 16.
    Kunegis, J., Lommatzsch, A., Bauckhage, C.: The slashdot zoo: Mining a social network with negative costs. In: World Wide Web Conference (WWW 2009), Madrid, pp. 741–750 (April 2009)Google Scholar
  17. 17.
    Kazemzadeh, A., Lee, S., Georgiou, P.G., Narayanan, S.: Determining what questions to ask, with the help of spectral graph theory. In: Proceedings of Interspeech (2011)Google Scholar
  18. 18.
    Kazemzadeh, A., Gibson, J., Georgiou, P., Lee, S., Narayanan, S.: Emo20q questioner agent. In: D´Mello, S., et al. (eds.) Proceedings of ACII (Interactive Event), vol. 6975, pp. 313–314. Springer, Heidelberg (2011), The interactive demo Scholar
  19. 19.
    Kazemzadeh, A., Lee, S., Narayanan, S.: An interval type-2 fuzzy logic model for the meaning of words in an emotional vocabulary (2011) (under review)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Abe Kazemzadeh
    • 1
  • Sungbok Lee
    • 1
  • Panayiotis G. Georgiou
    • 1
  • Shrikanth S. Narayanan
    • 1
  1. 1.University of Southern CaliforniaUSA

Personalised recommendations