Skip to main content
SpringerLink
Log in

Animated Agents’ Facial Emotions: Does the Agent Design Make a Difference?

  • Conference paper
  • First Online:
Augmented Reality, Virtual Reality, and Computer Graphics (AVR 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11613))

Abstract

The paper reports ongoing research toward the design of multimodal affective pedagogical agents that are effective for different types of learners and applications. In particular, the work reported in the paper investigated the extent to which the type of character design (realistic versus stylized) affects students’ perception of an animated agent’s facial emotions, and whether the effects are moderated by learner characteristics (e.g. gender). Eighty-two participants viewed 10 animation clips featuring a stylized character exhibiting 5 different emotions, e.g. happiness, sadness, fear, surprise and anger (2 clips per emotion), and 10 clips featuring a realistic character portraying the same emotional states. The participants were asked to name the emotions and rate their sincerity, intensity, and typicality. The results indicated that for recognition, participants were slightly more likely to recognize the emotions displayed by the stylized agent, although the difference was not statistically significant. The stylized agent was on average rated significantly higher for facial emotion intensity, whereas the differences in ratings for typicality and sincerity across all emotions were not statistically significant. A significant difference in ratings was shown in regard to sadness (within typicality), happiness (within sincerity), fear, anger, sadness and happiness (within intensity) with the stylized agent rated higher. Gender was not a significant correlate across all emotions or for individual emotions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adamo-Villani, N., Beni, G.: Keyboard encoding of facial expressions. In: 8th International Conference on Information Visualization-HCI Symposium (IV04) Proceedings, pp. 324–328. IEEE, London (2004)

    Google Scholar 

  2. Adamo-Villani, N., Wilbur, R.: Two novel technologies for accessible Math and Science Education. IEEE Multimedia Spec. Issue Access. 15, 38–46 (2008)

    Article  Google Scholar 

  3. Adamo-Villani, N., Lestina, J., Anasingaraju, S.: Does character’s visual style affect viewer’s perception of signing avatars? In: Vincenti, G., Bucciero, A., Vaz de Carvalho, C. (eds.) eLEOT 2015. LNICST, vol. 160, pp. 1–8. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-28883-3_1

    Chapter  Google Scholar 

  4. Adolphs, R.: Perception and emotion: How we recognize facial expressions. Current Dir. Psychol. Sci. 15(5), 222–226 (2006)

    Article  Google Scholar 

  5. Albrecht, I., Schroeder, M., Haber, J., Seidel, H.-S.: Mixed feelings: expression of non-basic emotions in a muscle-based talking head. Virtual Real. (special issue) 8, 201–212 (2005)

    Article  Google Scholar 

  6. Bancroft, T.: Creating Characters with Personality. Watson-Guptil, New York (2016)

    Google Scholar 

  7. Baylor, A., Kim, S.: Designing nonverbal communication for pedagogical agents: when less is more. Comput. Human Behav. 25(2), 450–457 (2009)

    Article  Google Scholar 

  8. Becker, C., Wachsmuth, I.: Modeling primary and secondary emotions for a believable communication agent. In: Proceedings of International Workshop on Emotion and Computing, in Conjunction with the 29th annual German Conference on Artificial Intelligence (KI2006), Bremen, Germany, pp. 31–34 (2006)

    Google Scholar 

  9. Calder, A., Rhodes, G., Johnson, M., Haxby, J.: Oxford Handbook of Face Perception. Oxford University Press, Oxford (2011)

    Google Scholar 

  10. Cissel, K.: A study of the effects of computer animated character body style on perception of facial expression. Purdue University e-Pub, MS thesis (2015). https://docs.lib.purdue.edu/cgttheses/28/

  11. Courgeon, M., Buisine, S., Martin, J.C.: Impact of expressive wrinkles on perception of a virtual character’s facial expressions of emotions. In: Proceedings of 9th International Conference, IVA 2009, Netherlands, pp. 201–214 (2009)

    Google Scholar 

  12. D’Mello, S., Graesser, A.: Emotions during learning with AutoTutor. In: Durlach, P.J., Lesgold, A.M. (eds.) Adaptive Technologies for Training and Education, pp. 117–139. Cambridge University Press, New York (2012)

    Chapter  Google Scholar 

  13. D’Mello, S., Graesser, A.: AutoTutor and affective autotutor: learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Transact. Interact. Intell. Syst. 2(4), 23 (2012)

    Google Scholar 

  14. Davis, C., Kim, J., Kuratate, T., Burnham, D.: Making a thinking-talking head. In: Proceedings of the International Conference on Auditory-Visual Speech Processing (AVSP 2007), Hilvarenbeek. The Netherlands (2007)

    Google Scholar 

  15. Domagk, S.: Do pedagogical agents facilitate learner motivation and learning outcomes? The role of the appeal of agent’s appearance and voice. J. Media Psychol. 22(2), 84–97 (2010)

    Article  Google Scholar 

  16. Ekman, P., Friesen, V.W.: Unmasking the Face. A guide to Recognizing Emotions from Facial Expressions. Malor Books, Cambridge (1975)

    Google Scholar 

  17. Ekman, P., Friesen, V.W.: Manual for the Facial Action Coding System. Consulting Psychologists Press (1977)

    Google Scholar 

  18. Garcia-Rojas, A., et al.: Emotional face expression profiles supported by virtual human ontology. Comp. Anim. Virtual Worlds 17, 259–269 (2006). https://doi.org/10.1002/cav.130

    Article  Google Scholar 

  19. Guo, Y.R., Goh, H.L.: Affect in embodied pedagogical agents: meta-analytic review. J. Educat. Comput. Res. 53(1), 124–149 (2015)

    Article  Google Scholar 

  20. Holmes, J.: Designing agents to support learning by explaining. Comput. Educ. 48, 523–547 (2007)

    Article  Google Scholar 

  21. Huang, T.S., Cohen, I., Hong, P., Li, Y.: IA3: Intelligent Affective Animated Agents. In: Proceedings of ICIG 2000 (2000)

    Google Scholar 

  22. Hyde, J., Carter, E., Kiesler, S., Hodgins, J.K.: Perceptual effects of damped and exaggerated facial motion in animated characters. In: Proceedings of the 10th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2013) (2013)

    Google Scholar 

  23. Intense. (n.d.). In Merriam-Webster. https://www.merriam-webster.com/dictionary/intense

  24. Johnson, W.L., Rickel, J.: Steve: an animated pedagogical agent for procedural training in virtual environments. SIGART Bull. 8, 16–21 (1998)

    Article  Google Scholar 

  25. Kim, S., Baylor, A.: Pedagogical agents as social models to influence learner attitudes. Educat. Technol. 47(01), 23–28 (2007)

    Google Scholar 

  26. Kim, Y., Lim, J.: Gendered socialization with an embodied agent: creating a social and affable mathematics learning environment for middle-grade females. J. Educat. Psychol. 105(4), 1164–1174 (2013)

    Article  Google Scholar 

  27. Kim, Y., Wei, Q.: The impact of user attributes and user choice in an agent-based environment. Comput. Educat. 56, 505–514 (2011)

    Article  Google Scholar 

  28. Kim, Y., Baylor, A., Shen, E.: Pedagogical agents as learning companions: the impact of agent emotion and gender. J. Comput. Assist. Learn. 23, 220–234 (2007)

    Article  Google Scholar 

  29. Lester, J., Voerman, J., Towns, S.J., Callaway, C.: Cosmo: a life-like animated pedagogical agent with deictic believability. In: Notes of the IJCAI 1997 Workshop on Animated Interface Agents: Making Them Intelligent, Nagoya, Japan, pp. 61–70 (1997)

    Google Scholar 

  30. Lester, J., Stone, B., Stelling, G.: Lifelike pedagogical agents for mixed-initiative problem solving in constructivist learning environments. User Model. User-Adap. Inter. 9(1–2), 1–44 (1999)

    Article  Google Scholar 

  31. Lester, J., Towns, S.G., Fitzgerald, P.: Achieving Affective Impact: Visual Emotive Communication in Lifelike Pedagogical Agents. Int. J. Artif. Intell. Educ. 10(3–4), 278–291 (1999)

    Google Scholar 

  32. Lester, J., Converse, S., Kahler, S., Barlow, T., Stone, B., Bhogal, R.: The persona effect: affective impact of animated pedagogical agents. In: Proceedings of CHI 1997, pp. 359–366 (1997)

    Google Scholar 

  33. Lester, J., Converse, S., Stone, B., Kahler, S., Barlow, T.: Animated pedagogical agents and problem-solving effectiveness: a large-scale empirical evaluation. In: Proceedings of the 8th World Conference on Artificial Intelligence in Education, pp. 23–30 (1997)

    Google Scholar 

  34. Lisetti, C., Nasoz, F.: MAUI: a multimodal affective user interface. In: Proceedings 10th ACM Int’l Conf. Multimedia (Multimedia 2002), pp. 161–170 (2002)

    Google Scholar 

  35. Martinez, A., Benavente. R.: The AR Face Database. CVC Technical Report #24 (1998)

    Google Scholar 

  36. Mayer, R.E.: Principles based on social cues in multimedia learning: Personalization, voice, image, and embodiment principles. In Mayer, R.E. (ed.) The Cambridge handbook of multimedia learning, 2nd ed, pp. 345–368. Cambridge University Press, New York (2014)

    Google Scholar 

  37. Mayer, R.E., DaPra, S.: An embodiment effect in computer-based learning with animated pedagogical agent. J. Exp. Psychol. Appl. 18, 239–252 (2012)

    Article  Google Scholar 

  38. McCloud, S.: Understanding Comics: The Invisible Art. Northampton, Mass (1993)

    Google Scholar 

  39. McDonnell, R., Jorg, S., JMcHugh, J., Newell, F., O’Sullivan, C.: Evaluating the emotional content of human motions on real and virtual characters. In: Proceedings of the 5th Symposium on Applied Perception in Graphics and Visualization, pp. 67–73 (2008)

    Google Scholar 

  40. Morency, L.P., et al.: Demonstration: a perceptive virtual human interviewer for healthcare applications. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)

    Google Scholar 

  41. Moreno, R., Flowerday, T.: Students’ choice of animated pedagogical agents in science learning: a test of the similarity attraction hypothesis on gender and ethnicity. Contemp. Educ. Psychol. 31, 186–207 (2006)

    Article  Google Scholar 

  42. Moreno, R., Mayer, R.E.: Interactive Multimodal Learning Environments. Educ. Psychol. Rev. 19, 309–326 (2007)

    Article  Google Scholar 

  43. Nass, C., Isbister, K., Lee, E.J.: Truth is beauty: researching embodied conversational agents. Embodied Conversational Agents, pp. 374–402 (2000)

    Google Scholar 

  44. Paleari, M., Lisetti, C.: Psychologically grounded avatars expressions. In: Proceedings of First Workshop on Emotion and Computing at KI 2006, 29th Annual Conf. on Artificial Intelligence, Bremen, Germany (2006)

    Google Scholar 

  45. Pandzic, I.S., Forcheimer, R.: MPEG4 facial animation – The standard, implementations and applications. John Wiley, Sons, Chichester (2002)

    Book  Google Scholar 

  46. Pelachaud, C.: Modelling multimodal expression of emotion in a virtual agent. Phil. Trans. R. Soc. B 364, 3539–3548 (2009)

    Article  Google Scholar 

  47. Powers, D., Leibbrandt, R., Luerssen, M., Lewis, T., Lawson, M.: PETA-a pedagogical embodied teaching agent. In: Proceedings of PETRA08: 1st international conference on PErvasive Technologies Related to Assistive Environments, Athens (2008)

    Google Scholar 

  48. Riek, D.L., Rabinowitch, T.C., Chakrabarti, B., Robinson, P.: How anthropomorphism affects empathy toward robots. In: Proceedings of the 4th ACM/IEEE international conference on Human robot interaction, pp. 245–246 (2009)

    Google Scholar 

  49. Rosenberg-Kima, R.B., Baylor, A., Plant, A., Doerr, C.: Interface agents as social models for female students: the effects of agent visual presence and appearance on female students’ attitudes and beliefs. Comput. Hum. Behav. 24, 2741–2756 (2008)

    Article  Google Scholar 

  50. Ruttkay, Z., Dormann, C., Noot, H.: Embodied conversational agents on a common ground. In: Ruttkay, Z., Pelachaud, C. (eds.) From Brows to Trust. HIS, vol. 7, pp. 27–66. Springer, Dordrecht (2004). https://doi.org/10.1007/1-4020-2730-3_2

    Chapter  Google Scholar 

  51. Scherer, K.R.: Appraisal considered as a process of multilevel sequential checking. In: Scherer, K., Schorr, A., Johnstone, T. (eds.) Appraisal processes in emotion: theory, methods, research, pp. 92–119. Oxford University Press, New York (2001)

    Google Scholar 

  52. Schroeder, N.L., Adesope, O., Barouch Gilbert, R.: How effective are pedagogical agents for learning? A Meta-Analytic Review. J. Educ.Comput. Res. 49(1), 1–39 (2013)

    Article  Google Scholar 

  53. Sincere. (n.d). In Oxford Dictionaries Online. https://en.oxforddictionaries.com/definition/sincere

  54. Thomas, F., Johnston, O.: The Illusion of Life. Disney Animation. Disney Editions, Los Angeles (1995)

    Google Scholar 

  55. Typical. (n.d.). In Oxford Dictionaries online. https://en.oxforddictionaries.com/definition/typical

  56. Wallraven, C., Breidt, M., Cunningham, D.W., Bulthoff, H.H.: Evaluating the perceptual realism of animated facial expressions. ACM Transact. Appl. Percept. 23, 1–20 (2007). http://dl.acm.org/citation.cfm?id=127876456

    Google Scholar 

  57. Wang, N., Johnson, L., Mayer, R.E., Rizzo, P., Shaw, E., Collins, H.: The politeness effect: pedagogical agents and learning outcomes. Int. J. Hum. Comput. Stud. 66, 98–112 (2008)

    Article  Google Scholar 

  58. Welch, R., Blackmon, T., Liu, A., Stark, L.: The effects of pictorial realism, delay of visual feedback, and observer interactivity on the subjective sense of presence. Presence Teleoperators Virtual Environ. 5, 263–273 (1996)

    Article  Google Scholar 

  59. Willcox, G.: The feeling wheel: a tool for expanding awareness of emotions and increasing spontaneity and intimacy. Transact. Anal. J. 12(4), 274–276 (1982)

    Article  Google Scholar 

  60. Woo, H.L.: Designing multimedia learning environments using animated pedagogical agents: factors and issues. J. Comput. Assist. Learn. 25(3), 203–218 (2009)

    Article  Google Scholar 

  61. Zhou, L., Mohammed, A., Zhang, D.: Mobile personal information management agent: Supporting natural language interface and application integration. Inf. Process. Manage. 48(1), 23–31 (2012)

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by NSF-Cyberlearning award 1821894, and by Purdue University Instructional Innovation Grant 2017–2019. The authors would like to thank Purdue Statistical Consulting for their help with the statistical analysis of the data.

Author information

Authors and Affiliations

  1. Computer Graphics Technology, Purdue University, West Lafayette, IN, USA

    Nicoletta Adamo & Hazar N. Dib

  2. Psychological Sciences, Purdue University, West Lafayette, IN, USA

    Nicholas J. Villani

Authors
  1. Nicoletta Adamo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hazar N. Dib
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nicholas J. Villani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicoletta Adamo .

Editor information

Editors and Affiliations

  1. University of Salento, Lecce, Italy

    Lucio Tommaso De Paolis

  2. University of Paris-Sud, Orsay, France

    Patrick Bourdot

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Adamo, N., Dib, H.N., Villani, N.J. (2019). Animated Agents’ Facial Emotions: Does the Agent Design Make a Difference?. In: De Paolis, L., Bourdot, P. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2019. Lecture Notes in Computer Science(), vol 11613. Springer, Cham. https://doi.org/10.1007/978-3-030-25965-5_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-25965-5_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-25964-8

  • Online ISBN: 978-3-030-25965-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation