Virtual tutors are a promising technology, providing a rich interactive environment for children to learn in. However, the question of how they should behave in order to enhance pupils’ motivation remains unanswered. Using an embodied conversational agent platform, we tested human-computer interactions with 22 children aged 9–11 years. Children performed several numeracy exercises set by two different virtual agents. One agent provided solely verbal feedback (unimodal), while the other one combined facial expressions based on real muscle contractions with its verbal feedback (bimodal). Children then completed a perceived social support questionnaire. Qualitative and quantitative data were subjected to inferential statistical tests. Results showed that the overall duration of agent-pupil interactions varied, children found the bimodal agent more empathic, and produced significantly more correct answers. Moreover, there was a positive correlation between accuracy and mean reaction times for correct answers with the bimodal agent. The lack of a correlation for the unimodal agent is discussed in the light of empathy and motivation in social cognition.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Data available on request from the authors.
Askew, S., & Lodge, C. (2000). Gifts, ping-pong and loops: Linking feedback and learning. In S. Askew (Ed.), Feedback for learning (pp. 1–18). London: Routledge.
Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral & Brain Sciences, 22, 577–660.
Beer, J. M., Smarr, C. A., Fisk, A. D., & Rogers, W. A. (2015). Younger and older users’ recognition of virtual agent facial expressions. International Journal of Human-Computer Studies, 75, 1–20.
Berrada-Baby, Z., Oker, A., Courgeon, M., Urbach, M., Bazin, N., Amorim, M. A., et al. (2016). Patients with schizophrenia are less prone to interpret virtual others’ empathetic questioning as helpful. Psychiatry Research, 242, 67–74.
Boyatzis, C. J., Chazan, E., & Ting, C. Z. (1993). Preschool children's decoding of facial emotions. Journal of Genetic Psychology, 154(3), 375–382.
Castellano, G., Paiva, A., Kappas, A., Aylett, R., Hastie, H., Barendregt, W., et al. (2013). Towards empathic virtual and robotic tutors. In International conference on artificial intelligence in education (pp. 733–736). Berlin, Heidelberg: Springer.
Courgeon, M., Clavel, C., Tan, N., & Martin, J. C. (2011). Front view vs. side view of facial and postural expressions of emotions in a virtual character. In Transactions on edutainment VI (pp. 132–143). Berlin, Heidelberg: Springer.
Decety, J. (2005). Une anatomie de l’empathie. PSN, 3(1), 16–24.
Dias, J., & Paiva, A. (2005). Feeling and reasoning: A computational model for emotional characters. In Portuguese conference on artificial intelligence (pp. 127–140). Berlin, Heidelberg: Springer.
D'mello, S., & Graesser, A. (2012). AutoTutor and affective AutoTutor: Learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Transactions on Interactive Intelligent Systems, 2(4), 23.
Ekman, P., & Friesen, W. V. (1978). Manual for the facial action coding system. Palo Alto: Consulting Psychologists Press.
Ekman, P., Friesen, W. V., & Hager, J. C. (2002). Facial action coding system: The manual on CD ROM. A Human Face, Salt Lake City, 77–254.
Faivre, J., Frasson, C., & Nkambou, R. (2002). Gestion émotionnelle dans les systèmes tuteurs intelligents. In Technologies de l'information et de la communication dans les enseignements d'ingénieurs et dans l'industrie (pp. 73–80). Lyon: Institut National des Sciences Appliquées de Lyon.
Godwin, K., & Fisher, A. (2011). Allocation of attention in classroom environments: Consequences for learning. In L. Carlson, C. Hölscher, & T. Shipley (Eds.), Proceedings of the 33rd annual conference of the cognitive science society (pp. 2806–2811). Austin: Cognitive Science Society.
Gordon, A. T. (2011). Assessing social support in children: Development and initial validation of the social support questionnaire for children. LSU Doctoral Dissertations, 3529.
Herba, C., & Phillips, M. (2004). Annotation: Development of facial expression recognition from childhood to adolescence: Behavioural and neurological perspectives. Journal of Child Psychology and Psychiatry, 45(7), 1185–1198.
Huitt, W. (2001). Motivation to learn: An overview. Educational Psychology Interactive, 12.
Jennings, P. A., & Greenberg, M. T. (2009). The prosocial classroom: Teacher social and emotional competence in relation to student and classroom outcomes. Review of Educational Research, 79(1), 491–525.
Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254–284.
Lepper, M. R., & Henderlong, J. (2000). Turning “play” into “work” and “work” into “play”: 25 years of research into intrinsic vs. extrinsic motivation. In C. Sansone & J. M. Harackiewicz (Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and performance (pp. 257–307). New York: Academic Press.
Lepper, M. R., Sethi, S., Dialdin, D., & Drake, M. (1997). Intrinsic and extrinsic motivation: A developmental perspective. Developmental psychopathology: Perspectives on adjustment, risk, and disorder, 23–50.
Lepper, M. R., Corpus, J. H., & Iyengar, S. S. (2005). Intrinsic and extrinsic motivational orientations in the classroom: Age differences and academic correlates. Journal of Educational Psychology, 97(2), 184.
Lin, L., Atkinson, R. K., Christopherson, R. M., Joseph, S. S., & Harrison, C. J. (2013). Animated agents and learning: Does the type of verbal feedback they provide matter? Computers & Education, 67, 239–249.
Madaio, M., Cassell, J., & Ogan, A. (2017). The impact of peer tutors’ use of indirect feedback and instructions. Philadelphia, PA: International Society of the Learning Sciences.
Martin, G., & Pear, J. (2015). Behavior modification: What it is and how to do it (10th ed.). Englewood Cliffs: PrenticeHall, Inc..
McClure, E. B. (2000). A meta-analytic review of sex differences in facial expression processing and their development in infants, children, and adolescents. Psychological Bulletin, 126(3), 424–453.
Ochs, M., & Pelachaud, C. (2013). Socially aware virtual characters: The social signal of smiles [social sciences]. IEEE Signal Processing Magazine, 30(2), 128–132.
Oker, A., Courgeon, M., Prigent, E., Eyharabide, V., Bazin, N., Urbach, M., et al. (2015a). A virtual reality study of help recognition and metacognition with an affective agent. International Journal of Synthetic Emotions, 6(1), 60–73.
Oker, A., Prigent, E., Courgeon, M., Eyharabide, V., Urbach, M., Bazin, N., et al. (2015b). How and why affective and reactive virtual agents will bring new insights on social cognitive disorders in schizophrenia? An illustration with a virtual card game paradigm. Frontiers in Human Neuroscience, 9, 133.
Oker, A., Glas, N., Pecune, F., & Pelachaud, C. (2018). An embodied virtual agent platform for emotional Stroop effect experiments: A proof of concept. Biologically Inspired Cognitive Architectures, 24, 107–114.
Paiva, A., Dias, J., Sobral, D., Aylett, R., Woods, S., Hall, L., & Zoll, C. (2005). Learning by feeling: Evoking empathy with synthetic characters. Applied Artificial Intelligence, 19(3–4), 235–266.
Paiva, A., Dias, J., Sobral, D., Aylett, R., Sobreperez, P., Woods, S., et al. (2004, July). Caring for agents and agents that care: Building empathic relations with synthetic agents. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 1 (pp. 194–201). IEEE Computer Society.
Pecune, F., Cafaro, A., Chollet, M., Philippe, P., & Pelachaud, C. (2014). Suggestions for extending saiba with the vib platform. In 14th international conference on intelligent virtual agents (IVA) (pp. 16-20). Bielefeld eCollections.
Philippot, P., & Feldman, R. S. (1990). Age and social competence in preschoolers’ decoding of facial expression. British Journal of Social Psychology, 29(1), 43–54.
Poggi, I., Pelachaud, C., & De Rosis, F. (2000). Eye communication in a conversational 3D synthetic agent. AI Communications, 13(3), 169–181.
Ramaprasad, A. (1983). On the definition of feedback. Systems Research and Behavioral Science, 28(1), 4–13.
Richmond, V. P., & McCroskey, J. C. (1995). Non verbal behavior in interpersonal relations (3rd ed.). Boston: Allyn & Bacon.
Richmond, V. P., McCroskey, J. C., & Hickson, M. (2008). Nonverbal behavior in interpersonal relations. Boston: Allyn & Bacon.
Rizzolatti, G., Fadiga, L., Gallese, V., & Fogassi, L. (1996). Premotor cortex and the recognition of motor actions. Cognitive Brain Research, 3(2), 131–141.
Roda, C., Angehrn, A., & Nabeth, T. (2001). Dialogue systems for advanced learning: Applications and research. In: Proc. of BotShow’01 Conference. pp. 1–7. Paris.
Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67.
Schröder, M., Bevacqua, E., Cowie, R., Eyben, F., Gunes, H., Heylen, D., et al. (2011). Building autonomous sensitive artificial listeners. IEEE Transactions on Affective Computing, 3(2), 165–183.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.
Sidner, C. L., Kidd, C. D., Lee, C., & Lesh, N. (2004). Where to look: A study of human-robot engagement. In Proceedings of the 9th international conference on intelligent user interfaces (pp. 78-84). ACM.
Snowman, J., & McCown, R. (2011). Psychology applied to teaching. Boston: Nelson Education.
Sugai, G., Horner, R. H., Dunlap, G., Hieneman, M., Lewis, T. J., Nelson, C. M., et al. (2000). Applying positive behavior support and functional behavioral assessment in schools. Journal of Positive Behavior Interventions, 2(3), 131–143.
Sutton, R. E., & Wheatley, K. F. (2003). Teachers' emotions and teaching: A review of the literature and directions for future research. Educational Psychology Review, 15(4), 327–358.
Theonas, G., Hobbs, D., & Rigas, D. (2008). Employing virtual lecturers’ facial expressions in virtual educational environments. International Journal of Virtual Reality, 7(1), 31–44.
Van der Gaag, C., Minderaa, R. B., & Keysers, C. (2007). Facial expressions: What the mirror neuron system can and cannot tell us. Social Neuroscience, 2(3–4), 179–222.
Walker-Andrews, A. S. (1997). Infants’ perception of expressive behaviors: Differentiation of multimodal information. Psychological Bulletin, 121(3), 437.
Woolf, B. P., Arroyo, I., Muldner, K., Burleson, W., Cooper, D. G., Dolan, R., & Christopherson, R. M. (2010). The effect of motivational learning companions on low achieving students and students with disabilities. In International conference on intelligent tutoring systems (pp. 327–337). Berlin, Heidelberg: Springer.
This work was partially performed within the Labex SMART (ANR-11-LABX-65) supported by French state funds managed by the ANR within the “Investissements d’Avenir” program under reference ANR-11-IDEX-0004-02. It has also been partially funded by the French National Research Agency project MaClasse 3.0.
The authors declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and national research committee (local French authorities) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consents were obtained from all underage participants’ family or legal representatives and included in the study, as well as from the school’s authorities in France.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Oker, A., Pecune, F. & Declercq, C. Virtual tutor and pupil interaction: A study of empathic feedback as extrinsic motivation for learning. Educ Inf Technol (2020). https://doi.org/10.1007/s10639-020-10123-5
- Virtual agents