Sentic Maxine: Multimodal Affective Fusion and Emotional Paths

  • Isabelle Hupont
  • Erik Cambria
  • Eva Cerezo
  • Amir Hussain
  • Sandra Baldassarri
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7368)


The capability of perceiving and expressing emotions through different modalities is a key issue for the enhancement of human-agent interaction. In this paper, an architecture for the development of intelligent multimodal affective interfaces is presented. It is based on the integration of Sentic Computing, a new opinion mining and sentiment analysis paradigm based on AI and Semantic Web techniques, with a facial emotional classifier and Maxine, a powerful multimodal animation engine for managing virtual agents and 3D scenarios. One of the main distinguishing features of the system is that it does not simply perform emotional classification in terms of a set of discrete emotional labels but it operates in a novel continuous 2D emotional space, enabling the output of a continuous emotional path that characterizes user’s affective progress over time. Another key factor is the fusion methodology proposed, which is able to fuse any number of unimodal categorical modules, with very different time-scales, output labels and recognition success rates, in a simple and scalable way.


Sentic computing Facial expression analysis Sentiment analysis Multimodal fusion Embodied agents 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Casell, J., Sullivan, J., Prevost, S., Churchill, E. (eds.): Embodied Conversational Agents. MIT Press, Cambridge (2000)Google Scholar
  2. 2.
    Isbister, K.: Better game characters by design: A psychological approach, p. 83. Morgan Kaufmann (2006)Google Scholar
  3. 3.
    Yee, N., Bailenson, J., Urbanek, M., Chang, F., Merget, D.: The unbearable likeness of being digital: The persistence of nonverbal social norms in online virtual environments. Cyber Psychology & Behavior 10(1), 115–121 (2007)Google Scholar
  4. 4.
    Ochs, M., Pelachaud, C., Sadek, D.: An empathic virtual dialog agent to improve human-machine interaction. In: International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 89–96 (2008)Google Scholar
  5. 5.
    Boukricha, H., Becker, C., Wachsmuth, I.: Simulating empathy for the virtual human Max. In: International Workshop on Emotion and Computing in Conjunction With the German Conference on Artificial Intelligence, pp. 22–27 (2007)Google Scholar
  6. 6.
    Gilroy, S., Cavazza, M., Niiranen, M., Andre, E., Vogt, T., Urbain, J., Benayoun, M., Seichter, H., Billinghurst, M.: PAD-based multimodal affective fusion. In: ACII, pp. 1–8, 10–12 (2009)Google Scholar
  7. 7.
    Zeng, Z., Pantic, M., Huang, T.: Emotion Recognition Based on Multimodal Information. In: Affective Information Processing, pp. 241–265 (2009)Google Scholar
  8. 8.
    Kapoor, A., Burleson, W., Picard, R.: Automatic prediction of frustration. International Journal of Human-Computer Studies 65, 724–736 (2007)CrossRefGoogle Scholar
  9. 9.
    Gunes, H., Piccardi, M., Pantic, M.: From the lab to the real world: Affect recognition using multiple cues and modalities. In: Affective Computing: Focus on Emotion Expression, Synthesis, and Recognition, pp. 185–218 (2008)Google Scholar
  10. 10.
    Cambria, E., Olsher, D., Kwok, K.: Sentic Activation: A Two-Level Affective Common Sense Reasoning Framework. In: AAAI, Toronto (2012)Google Scholar
  11. 11.
    Hupont, I., Baldassarri, S., Cerezo, E.: Sensing facial emotions in a continuos 2D affective space. In: IEEE International Conference on System, Man and Cybernetics (2010)Google Scholar
  12. 12.
    Baldassarri, S., Cerezo, E., Seron, F.: Maxine: a platform for embodied Animated Agents. Computers & Graphics 32(4), 430–437 (2008)CrossRefGoogle Scholar
  13. 13.
    Cambria, E., Hussain, A.: Sentic Computing: Techniques, Tools, and Applications. Springer, Heidelberg (2012)Google Scholar
  14. 14.
    Cambria, E., Benson, T., Eckl, C., Hussain, A.: Sentic PROMs: Application of sentic computing to the development of a novel unified framework for measuring health-care quality. Expert Systems with Applications 39(12), 10533–10543 (2012)CrossRefGoogle Scholar
  15. 15.
    Cambria, E., Livingstone, A., Hussain, A.: The hourglass of emotions. In: Esposito, A., Vinciarelli, A., Hoffmann, R., Muller, V. (eds.) Cognitive Behavioral Systems. LNCS. Springer, Heidelberg (2012)Google Scholar
  16. 16.
    Whissell, C.: The dictionary of affect in language. In: Emotion: Theory, Research and Experience, The Measurement of Emotions, vol. 4. Academic, New York (1989)Google Scholar
  17. 17.
    Picard, R.: Affective Computing. The MIT Press (1997)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Isabelle Hupont
    • 1
  • Erik Cambria
    • 2
  • Eva Cerezo
    • 3
  • Amir Hussain
    • 4
  • Sandra Baldassarri
    • 3
  1. 1.Instituto Tecnológico de AragónSpain
  2. 2.National University of SingaporeSingapore
  3. 3.University of ZaragozaSpain
  4. 4.University of StirlingUnited Kingdom

Personalised recommendations