Skip to main content
SpringerLink
Log in

Multimodal Affective Pedagogical Agents for Different Types of Learners

  • Conference paper
  • First Online:
Intelligent Human Systems Integration 2021 (IHSI 2021)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1322))

Included in the following conference series:

Abstract

The paper reports progress on an NSF-funded project whose goal is to research and develop multimodal affective animated pedagogical agents (APA) for different types of learners. Although the preponderance of research on APA tends to focus on the cognitive aspects of online learning, this project explores the less-studied role of affective features. More specifically, the objectives of the work are to: (1) research and develop novel algorithms for emotion recognition and for life-like emotion representation in embodied agents, which will be integrated in a new system for creating APA to be embedded in digital lessons; and (2) develop an empirically grounded research base that will guide the design of affective APA that are effective for different types of learners. This involves conducting a series of experiments to determine the effects of the agent’s emotional style and emotional intelligence on a diverse population of students. The paper outlines the work conducted so far, e.g., development of a new system (and underlying algorithms) for producing affective APA. It also reports the findings from two preliminary studies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cheng, J., Zhou, W., Lei, X., Adamo, N., Benes, B.: The effects of body gestures and gender on viewer’s perception of animated pedagogical agent’s emotions. In: Kurosu, M. (ed.) Human-Computer Interaction. Multimodal and Natural Interaction. HCII 2020. Lecture Notes in Computer Science, vol. 12182. Springer, Cham (2020)

    Google Scholar 

  2. D’Mello, S., Graesser, A.: AutoTutor and affective AutoTutor: learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Trans. Interact. Intell. Syst. 2(4), Article 23 (2012)

    Google Scholar 

  3. Guo, Y.R., Goh, D.H.L.: Affect in embodied pedagogical agents: meta-analytic review. J. Educ. Comput. Res. 53(1), 124–149 (2015)

    Article  Google Scholar 

  4. Huang, T.S., Cohen, I., Hong, P., Li, Y.: IA3: intelligent affective animated agents. In: ICIG 2000 (2000)

    Google Scholar 

  5. Johnson, W.L., Lester, J.C.: Face-to-face interaction with pedagogical agents, twenty years later. Int. J. Artif. Intell. Educ. 26(1), 25–36 (2016)

    Article  Google Scholar 

  6. Kim, Y., Baylor, A.L., Shen, E: Pedagogical agents as learning companions: the impact of agent emotion and gender. J. Comput. Assist. Learn. 23, 220–234 (2007)

    Google Scholar 

  7. Lawson, A., Mayer, R.E., Adamo-Villani, N., Benes, B., Cheng, J., Lei, X.: Recognizing the emotional state of human and virtual instructors. Comput. Hum. Behav. 114, 106554 (2021)

    Article  Google Scholar 

  8. Lisetti, C.L., Nasoz, F.: MAUI: a multimodal affective user interface. In: 10th ACM International Conference on Multimedia (Multimedia 2002), pp. 161–170 (2002)

    Google Scholar 

  9. Morency, L.P., Stratou, G., DeVault, D., Hartholt, A., Lhommet, M., Lucas, G., Morbini, F., Georgila, K., Scherer, S., Gratch, J., Marsella, S., Traum, D., Rizzo, A.: Demonstration: a perceptive virtual human interviewer for healthcare applications. In: 29th AAAI Conference on Artificial Intelligence (2015)

    Google Scholar 

  10. Mori, M.: The uncanny valley. IEEE Rob. Autom. 19(2), 98–100 (2012). https://doi.org/10.1109/mra.2012.2192811

    Article  Google Scholar 

  11. Russell, J.A.: Core affect and the psychological construct of emotion. Psychol. Rev. 110, 145–172 (2003)

    Article  Google Scholar 

  12. Schroeder, N., Adesope, O.O., Gilbert, R.B.: How effective are pedagogical agents for learning? A meta-analytic review. J. Educ. Comput. Res. 49(1) (2013)

    Google Scholar 

  13. Zhou, L., Mohammed, A.S., Zhang, D.: Mobile personal information management agent: supporting natural language interface and application integration. Inf. Process. Manage. 48(1), 23–31 (2012)

    Article  Google Scholar 

  14. Zhou, W., Cheng, J., Lei, X., Benes, B., Adamo, N.: Deep learning-based emotion recognition from real-time videos. In: Kurosu, M. (ed.) Human-Computer Interaction. Multimodal and Natural Interaction. HCII 2020. Lecture Notes in Computer Science, vol. 12182. Springer, Cham (2020)

    Google Scholar 

Download references

Acknowledgments

The work reported in the paper is supported in part by NSF – Cyberlearning Collaborative Research: Multimodal Affective Pedagogical Agents for Different Types of Learners, Award #1821894.

Author information

Authors and Affiliations

  1. Department of Computer Graphics Technology, Purdue University, West Lafayette, IN, 47907, USA

    Nicoletta Adamo, Bedrich Benes, Xingyu Lei, Zhiquan Wang & Zachary Meyer

  2. Department of Psychological and Brain Sciences, University of California Santa Barbara, Santa Barbara, CA, 93106-9660, USA

    Richard E. Mayer & Alyssa Lawson

Authors
  1. Nicoletta Adamo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bedrich Benes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Richard E. Mayer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xingyu Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhiquan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zachary Meyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Alyssa Lawson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicoletta Adamo .

Editor information

Editors and Affiliations

  1. Università degli Studi di Palermo, Palermo, Palermo, Italy

    Dario Russo

  2. Institute for Advanced Systems Engineering, Florida, FL, USA

    Tareq Ahram

  3. University of Central Florida, Orlando, FL, USA

    Waldemar Karwowski

  4. Università degli Studi di Chieti-Pescara, Pescara, Italy

    Giuseppe Di Bucchianico

  5. Université de Reims Champagne-Ardenne, Reims, France

    Redha Taiar

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Adamo, N. et al. (2021). Multimodal Affective Pedagogical Agents for Different Types of Learners. In: Russo, D., Ahram, T., Karwowski, W., Di Bucchianico, G., Taiar, R. (eds) Intelligent Human Systems Integration 2021. IHSI 2021. Advances in Intelligent Systems and Computing, vol 1322. Springer, Cham. https://doi.org/10.1007/978-3-030-68017-6_33

Download citation

Publish with us

Policies and ethics

Search

Navigation