Affective Cognitive Modeling for Autonomous Agents Based on Scherer’s Emotion Theory

  • Christine L. Lisetti
  • Andreas Marpaung
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4314)


In this article, we propose the design of sensory motor level as part of a three-layered agent architecture inspired from the Multilevel Process Theory of Emotion (Leventhal 1979, 1980; Leventhal and Scherer, 1987). Our project aims at modeling emotions on an autonomous embodied agent, a more robust robot than our previous prototype. Our robot has been equipped with sonar and vision for obstacle avoidance as well as vision for face recognition, which are used when she roams around the hallway to engage in social interactions with humans. The sensory motor level receives and processes inputs and produces emotion-like states without any further willful planning or learning. We describe: (1) the psychological theory of emotion which inspired our design, (2) our proposed agent architecture, (3) the needed hardware additions that we implemented on the commercialized ActivMedia’s robot, (4) the robot’s multi-modal interface designed especially to engage humans in natural (and hopefully pleasant) social interaction, and finally (5) our future research efforts.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    ActivMedia (2002),
  2. 2.
    Arkin, R.C.: Behavior-Based Robotics. MIT Press, Cambridge (1998)Google Scholar
  3. 3.
    Breazeal, C., Scassellati, B.: Infant-like Social Interactions Between a Robot and a Human Caretaker. Special issue of Adaptive Behavior on Simulation Models of Social Agents, guest editor Kerstin Dautenhahn (2000)Google Scholar
  4. 4.
    Breazeal, C.: Emotion and sociable humanoid robots. International Journal of Human Computer Studies 59, 119–155 (2003)CrossRefGoogle Scholar
  5. 5.
    Brooks, R., Flynn, A.: Fast, Cheap, and Out of Control. AI Memo 1182, MIT AI Laboratory (1989)Google Scholar
  6. 6.
    Brown, S., Lisetti, C., Marpaung, A.: Cherry, the Little Red Robot with a Mission and a Personality. In: Working Notes of the AAAI Fall Symposium Series on Human-Robot Interaction, Cape Cod, MA, November 2002, AAAI Press, Menlo Park (2002)Google Scholar
  7. 7.
    Casper, J.: Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center. MS Thesis, Computer Science and Engineering, University of South Florida (April 2002)Google Scholar
  8. 8.
    Casper, J., Murphy, R.: Workflow study on human-robot interaction in USAR. In: Proceedings of ICRA 2002, pp. 1997–2003 (2002)Google Scholar
  9. 9.
    Ekman, P., Friesen, W.: The Facial Action Coding System. Consulting Psychologist Press, San Francisco (1978)Google Scholar
  10. 10.
    El-Nasr, M.S., Yen, J., Ioerger, T.: FLAME - A Fuzzy Logic Adaptive Model of Emotions. Automous Agents and Multi-agent Systems 3, 219–257 (2000)CrossRefGoogle Scholar
  11. 11.
    Grizard, A., Lisetti, C.: Generation of Facial Emotional Expressions Based on Psychological Theory. In: Notes of the 1st Workshop Emotion and Computing, 29th Annual German Conference on Artificial Intelligence, Universität Bremen, Germany (June 2006)Google Scholar
  12. 12.
    Identix Inc. (2002),
  13. 13.
    Leventhal, H.: A perceptual-motor processing model of emotion. In: Pilner, P., Blankenstein, K., Spigel, I.M. (eds.) Perception of emotion in self and others, vol. 5, pp. 1–46. Plenum, New York (1979)Google Scholar
  14. 14.
    Leventhal, H.: Toward a comprehensive theory of emotion. In: Berkowitz, L. (ed.) Advances in experimental social Psychology, vol. 13, pp. 139–207. Academic Press, New York (1980)CrossRefGoogle Scholar
  15. 15.
    Leventhal, H., Scherer, K.: The relationship of emotion to cognition: A functional approach to a semantic controversy. Cognition and Emotion 1(1), 3–28 (1987)CrossRefGoogle Scholar
  16. 16.
    Lisetti, C., et al.: A Social Informatics Approach to Human-Robot Interaction with an Office Service Robot. IEEE Transactions on Systems, Man, and Cybernetics - Special Issue on Human Robot Interaction 34(2) (2004)Google Scholar
  17. 17.
    Lisetti, C.L., Nasoz, F.: MAUI: A Multimodal Affective User Interface. In: Proceedings of the ACM Multimedia International Conference 2002, Juan les Pins, France (December 2002)Google Scholar
  18. 18.
    Marpaung, A.: Social Robots with Emotion State Generator Enhancing Human-Robot Interaction (HRI). Master’s thesis. University of Central Florida, in progress (2004)Google Scholar
  19. 19.
    Murphy, R.R.: Use of Scripts for Coordinating Perception and Action. In: Proceedings of IROS-96 (1996a)Google Scholar
  20. 20.
    Murphy, R.R.: Biological and Cognitive Foundations of Intelligent Sensor Fusion. IEEE Transactions on Systems, Man and Cybernetics 26(1), 42–51 (1996b)MathSciNetGoogle Scholar
  21. 21.
    Murphy, R.R.: Dempster-Shafer Theory for Sensor Fusion in Autonomous Mobile Robots. IEEE Transactions on Robotics and Automation 14(2) (1998)Google Scholar
  22. 22.
    Murphy, R.R.: Introduction to AI Robotics. MIT Press, Cambridge (2000)Google Scholar
  23. 23.
    Murphy, R.R., et al.: Emotion-Based Control of Cooperating Heterogeneous Mobile Robots. IEEE Transactions on Robotics and Automation 18 (2002)Google Scholar
  24. 24.
    Paleari, M., Lisetti, C.: Psychologically Grounded Avatar Expression. In: Notes of the 1st Workshop Emotion and Computing, 29th Annual German Conference on Artificial Intelligence, Universität Bremen, Germany (June 2006)Google Scholar
  25. 25.
    Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)Google Scholar
  26. 26.
    Scherer, K.: Emotion as a multicomponent process: A model and some cross-cultural data. In: Shaver, P. (ed.) Review of personality and social psychology. Vol. 5. Emotions, relationships and health, pp. 37–63. Sage, Beverly Hills (1984)Google Scholar
  27. 27.
    Scherer, K.: Vocal affect expression: A review and a model for future research. Psychological Bulletin 99, 143–165 (1986)CrossRefGoogle Scholar
  28. 28.
    Smith, C.A., Ellsworth, P.C.: Patterns of cognitive appraisal in emotion. Journal of Personality and Social Psychology 48, 813–838 (1985)CrossRefGoogle Scholar
  29. 29.
    Villon, O., Lisetti, C.: Toward Building Adaptive User’s Psycho-physiological Maps of Emotions using Bio-Sensors. In: Notes of the 1st Workshop Emotion and Computing, 29th Annual German Conference on Artificial Intelligence, Universität Bremen, Germany (June 2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Christine L. Lisetti
    • 1
  • Andreas Marpaung
    • 2
  1. 1.Multimedia Communications, Institut Eurecom, Sophia AntipolisFrance
  2. 2.Electrical Engineering & Computer Science, University of Central Florida, Orlando, FloridaUSA

Personalised recommendations