Affective Intelligence: The Human Face of AI

  • Lori Malatesta
  • Kostas Karpouzis
  • Amaryllis Raouzaiou
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5640)


Affective computing has been an extremely active research and development area for some years now, with some of the early results already starting to be integrated in human-computer interaction systems. Driven mainly by research initiatives in Europe, USA and Japan and accelerated by the abundance of processing power and low-cost, unintrusive sensors like cameras and microphones, affective computing functions in an interdisciplinary fashion, sharing concepts from diverse fields, such as signal processing and computer vision, psychology and behavioral sciences, human-computer interaction and design, machine learning, and so on. In order to form relations between low-level input signals and features to high-level concepts such as emotions or moods, one needs to take into account the multitude of psychology and representation theories and research findings related to them and deploy machine learning techniques to actually form computational models of those. This chapter elaborates on the concepts related to affective computing, how these can be connected to measurable features via representation models and how they can be integrated into human-centric applications.


Emotion Recognition Human Face Action Tendency Participatory Design Affective Computing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andre, E., Klesen, M., Gebhard, P., Allen, S., Rist, T.: Integrating models of personality and emotions in lifelike characters. In: Paiva, A. (ed.) IWAI 1999. LNCS, vol. 1814, pp. 150–165. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  2. 2.
    Araujo, A.F.R.: Emotions influencing cognition. In: WAUME ’93, Workshop on Architectures Underlying Motivation and Emotion, The University of Birmingham, Birmingham, UK (1993)Google Scholar
  3. 3.
    Bates, J., Loyall, A.B., Reilly, W.S.: Integrating reactivity, goals, and emotion in a broad agent. In: Proceedings of the 14th Meeting of the Cognitive Science Society, Boulder, CO (1992)Google Scholar
  4. 4.
    Breckler, S.J.: Empirical validation of affect, behaviour and cognition as distinct attitude components. Journal of Personality and Social Psychology 47(6), 1191–1205 (1984)CrossRefGoogle Scholar
  5. 5.
    Breazeal, C.: Emotion and sociable humanoid robots. International Journal of Human-Computer Studies 59(1-2), 119–155 (2003)CrossRefGoogle Scholar
  6. 6.
    Canamero, D.: Issues in the design of emotional agents. In: Proceedings of Emotional and Intelligent: The Tangled Knot of Cognition, AAAI Fall Symposium, TR FS-98-03, pp. 49–54. AAAI Press, Menlo Park (1998)Google Scholar
  7. 7.
    Castelfranchi, C.: Affective appraisal versus cognitive evaluations in social emotions and interactions. In: Paiva, A. (ed.) Affective Interactions: Towards a New Generation of Affective Interfaces, Springer, New York (2000)Google Scholar
  8. 8.
    Cowie, R., Douglas-Cowie, E., Apolloni, B., Taylor, J., Romano, A., Fellenz, W.: What a neural net needs to know about emotion words. In: Mastorakis, N. (ed.) Computational intelligence and applications, pp. 109–114. World Scientific Engineering Society (1999)Google Scholar
  9. 9.
    Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., Taylor, J.G.: Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine 18, 32–80 (2001)CrossRefGoogle Scholar
  10. 10.
    Damasio, A.: Looking for Spinoza: Joy, Sorrow, and the Feeling Brain. Harcourt Press, Orlando (2003)Google Scholar
  11. 11.
    De Rosis, F., Pelachaud, C., Poggi, I., Carofiglio, V., Carolis, B.D.: From Greta’s mind to her face: modeling the dynamics of affective states in a conversational embodied agent. International Journal of Human-Computer Studies 59(1-2), 81–118 (2003)CrossRefGoogle Scholar
  12. 12.
    Ekman, P.: An argument for basic emotions. Cognition & Emotion 6, 169–200 (1992)CrossRefGoogle Scholar
  13. 13.
    Elliott, C., Rickel, J., Lester, J.C.: Lifelike pedagogical agents and affective computing: An exploratory synthesis. In: Veloso, M.M., Wooldridge, M.J. (eds.) Artificial Intelligence Today. LNCS (LNAI), vol. 1600, pp. 195–211. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  14. 14.
    Frijda, N.H.: The Emotions. Studies in Emotion and Social Interaction. Cambridge University Press, New York (1986)Google Scholar
  15. 15.
    Frijda, N.H.: Emotions and Beliefs: How Feelings Influence Thoughts. Cambridge University Press, Cambridge (2000)CrossRefGoogle Scholar
  16. 16.
    Frijda, N.H., Swagerman, J.: Can computers feel? Theory and design of an emotional system. Cognition and Emotion 1(3), 235–257 (1987)CrossRefGoogle Scholar
  17. 17.
    Gratch, J., Marsella, S.: A domain-independent framework for modeling emotion. Cognitive Systems Research 5, 269–306 (2004)CrossRefGoogle Scholar
  18. 18.
    Goldie, P.: On Personality. Rutledge, New York (2004)Google Scholar
  19. 19.
    Hudlicka, E.: To feel or not to feel: The role of affect in human-computer interaction. International Journal of Human-Computer Studies 59, 1–32 (2003)CrossRefGoogle Scholar
  20. 20.
    Hudlicka, E.: Increasing SIA architecture realism by modeling and adapting to affect and personality. In: Dautenhahn, K., Bond, A.H., Canamero, L., Edmonds, B. (eds.) Multiagent Systems, Artificial Societies, and Simulated Organizations, Kluwer Academic Publishers, Dordrecht (2002)Google Scholar
  21. 21.
    Ioannou, S., Raouzaiou, A., Tzouvaras, V., Mailis, T., Karpouzis, K., Kollias, S.: Emotion recognition through facial expression analysis based on a neurofuzzy network. Neural Networks (Special Issue on Emotion: Understanding & Recognition) 18(4), 423–435 (2005)CrossRefGoogle Scholar
  22. 22.
    Jones, R., Henninger, A., Chown, E.: Interfacing emotional behavior moderators with intelligent synthetic forces. In: Proceedings of the 11th Conference on Computer Generated Forces and Behaviour Representation, Orlando, FL (2002)Google Scholar
  23. 23.
    Karpouzis, K., Caridakis, G., Kessous, L., Amir, N., Raouzaiou, A., Malatesta, L., Kollias, S.: Modeling naturalistic affective states via facial, vocal, and bodily expressions recognition. In: Huang, T.S., Nijholt, A., Pantic, M., Pentland, A. (eds.) ICMI/IJCAI Workshops 2007. LNCS (LNAI), vol. 4451, pp. 91–112. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  24. 24.
    Kant, I.: Critique of Judgment (1790) (Trans. Werner S. Pluhar). Hackett Publishing, Indianapolis (1987)Google Scholar
  25. 25.
    Lazarus, R.S., Folkman, S.: Transactional theory and research on emotions and coping. European Journal of Personality 1, 141–169 (1987)CrossRefGoogle Scholar
  26. 26.
    Lazarus, R.S.: Emotion and Adaptation. Oxford University Press, New York (1991)Google Scholar
  27. 27.
    Leventhal, H., Scherer, K.R.: The relationship of emotion to cognition: a functional approach to a semantic controversy. Cognition and Emotion 1, 3–28 (1987)CrossRefGoogle Scholar
  28. 28.
    Malatesta, L., Raouzaiou, A., Karpouzis, K., Kollias, S.: MPEG-4 facial expression synthesis. Personal and Ubiquitous Computing (Special issue on Emerging Multimodal Interfaces) 13(1), 77–83 (2007)CrossRefzbMATHGoogle Scholar
  29. 29.
    Muller, M.J.: Participatory Design: The Third Space in HCI - Handbook of HCI. Erlbaum, Mahwah (2003)Google Scholar
  30. 30.
    Ortony, A., Collins, A., Clore, G.L.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1988)CrossRefGoogle Scholar
  31. 31.
    Ortony, A., Turner, T.J.: What’s basic about basic emotions? Psychological Review 97, 315–331 (1990)CrossRefGoogle Scholar
  32. 32.
    Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)CrossRefGoogle Scholar
  33. 33.
    Raouzaiou, A., Tsapatsoulis, N., Karpouzis, K., Kollias, S.: Parameterized facial expression synthesis based on MPEG-4. EURASIP Journal on Applied Signal Processing 2002(10), 1021–1038 (2002)CrossRefzbMATHGoogle Scholar
  34. 34.
    Roseman, I.J.: The structure of emotion antecedents: Individual and cross-cultural differences. New approaches to emotion structure and process. Symposium conducted at the 92nd Annual Convention, American Psychological Association, Toronto, Canada (1984)Google Scholar
  35. 35.
    Roseman, I.J.: The emotion system: Strategies for coping with crises and opportunities. Paper presented at Department of Psychology, The Graduate Center, City University of New York (1990)Google Scholar
  36. 36.
    Rusting, C.: Personality, mood, and cognitive processing of Emotional information: three conceptual frameworks. Psychological Bulletin 124, 165–196 (1998)CrossRefGoogle Scholar
  37. 37.
    Schuler, D., Namioka, A. (eds.): Participatory Design: Principles and Practices. Lawrence Erlbaum Associates, Hillsdale (1993)Google Scholar
  38. 38.
    Sloman, A.: Beyond shallow models of emotions. Cognitive Processing 2(1), 177–198 (2001)Google Scholar
  39. 39.
    Smith, C.A., Kirby, L.D.: Affect and appraisal. In: Forgas, J.P. (ed.) Feeling and Thinking: The Role of Affect in Social Cognition, Cambridge University Press, Cambridge (2000)Google Scholar
  40. 40.
    Scherer, K.R.: Toward a dynamic theory of emotion: The component process model of affective states. Geneva Studies in Emotion and Communication 1, 1–98 (1987)Google Scholar
  41. 41.
    Scherer, K.R.: Criteria for emotion-antecedent appraisal: A review. In: Hamilton, V., Bower, G.H., Frijda, N.H. (eds.) Cognitive perspectives on emotion and motivation, pp. 89–126. Kluwer Academic Publishers, Dordrecht (1988)CrossRefGoogle Scholar
  42. 42.
    Scherer, K.R.: Appraisal considered as a process of multi-level sequential checking. In: Scherer, K.R., Schorr, A., Johnstone, T. (eds.) Appraisal processes in emotion: Theory, methods, research, pp. 92–120. Oxford University Press, New York (2001)Google Scholar
  43. 43.
    Scherer, K.R.: Feelings integrate the central representation of appraisal-driven response organization in emotion. In: Manstead, A.S.R., Frijda, N.H., Fischer, A.H. (eds.) Feelings and emotions The Amsterdam symposium, pp. 136–157. Cambridge University Press, Cambridge (2004)CrossRefGoogle Scholar
  44. 44.
    Scherer, K.R.: Unconscious processes in emotion: The bulk of the iceberg. In: Niedenthal, P., Feldman-Barrett, L., Winkielman, P. (eds.) The unconscious in emotion, Guilford, New York (2005)Google Scholar
  45. 45.
    Velasquez, J.: Modeling Emotions and Other Motivations in Synthetic Agents. In: Proceedings of AAAI-97, Providence, RI, pp. 10–15 (1997)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2009

Authors and Affiliations

  • Lori Malatesta
    • 1
  • Kostas Karpouzis
    • 1
  • Amaryllis Raouzaiou
    • 1
  1. 1.National Technical University of AthensAthensGreece

Personalised recommendations