Skip to main content

The effects of virtual human gesture frequency and reduced video speed on satisfaction and learning outcomes

A Correction to this article was published on 12 July 2021

This article has been updated

Abstract

Educators use various strategies to increase listening comprehension for nonnative English speakers in the classroom and multimedia environments. Research on audio reduction has shown mixed results, whereas a study that enhanced (doubled) virtual human gesturing found increased listening comprehension with procedural information (Davis and Vincent, British Journal of Educational Technology 50:3252–3263, 2019). This research examined the use of virtual human gesture frequency (enhanced, average, no) and video speed (normal, reduced 25%) on participant satisfaction and learning outcomes with procedural information. Analysis based on 234 multinational university students indicated that normal video speed significantly increased satisfaction compared to reduced speed; satisfaction was rated significantly higher with agents that gestured compared with the no-gesture condition; and enhancing the gesture frequency significantly increased learning outcomes compared to the average and no-gesture conditions. These findings support previous studies that indicated enhanced gestures significantly increase the learning of procedural information. Also, agent gesturing increased the satisfaction with the agent, which supports systematic review findings that gesturing is a principal moderator for agent persona (Davis et al., Journal of Research on Technology in Education 53:89–106, 2021). However, this research provides evidence that a 25% reduction in video speed may be too slow to maintain satisfaction with advanced foreign language users and that less reduced rates such as 15% or 10% should to be considered. Finally, this research puts forth a gesture design framework for designers to create gesturing virtual humans in multimedia environments.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

Change history

References

  1. Adesope, O. O., & Nesbit, J. C. (2012). Verbal redundancy in multimedia learning environments: A meta-analysis. Journal of Educational Psychology, 104(1), 250.

    Google Scholar 

  2. Alibali, M. W., Heath, D. C., & Myers, H. J. (2001). Effects of visibility between speaker and listener on gesture production: Some gestures are meant to be seen. Journal of Memory and Language, 44(2), 169–188.

    Google Scholar 

  3. Blau, E. K. (1990). The effect of syntax, speed, and pauses on listening comprehension. TESOL Quarterly, 24(4), 746–753.

    Google Scholar 

  4. Carlotto, T., & Jaques, P. A. (2016). The effects of animated pedagogical agents in an English-as-a-foreign-language learning environment. International Journal of Human-Computer Studies, 95(November), 15–26.

    Google Scholar 

  5. Carrier, K. (1999). The social environment of second language listening: Does status play a role in comprehension? The Modern Language Journal, 83, 65–79.

    Google Scholar 

  6. Chaudron, C. (1988). Second language classrooms: Research on teaching and learning. Cambridge University Press.

    Google Scholar 

  7. Chiou, E. K., Schroeder, N. L., & Craig, S. D. (2020). How we trust, perceive, and learn from virtual humans: The influence of voice quality. Computers & Education, 146, 103756.

    Google Scholar 

  8. Choi, S., & Clark, R. E. (2006). Cognitive and affective benefits of an animated pedagogical agent for learning English as a second language. Journal of Educational Computing Research, 34(4), 441–466.

    Google Scholar 

  9. Church, R. B., Ayman-Nolley, S., & Mahootian, S. (2004). The role of gesture in bilingual education: Does gesture enhance learning? International Journal of Bilingual Education and Bilingualism, 7(4), 303–319.

    Google Scholar 

  10. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.

    Google Scholar 

  11. Conrad, L. (1989). The effects of time-compressed speech on native and EFL listening comprehension. Studies in Second Language Acquisition, 11(1), 1–16.

    Google Scholar 

  12. Craig, S. D., & Schroeder, N. L. (2017). Reconsidering the voice effect when learning from a virtual human. Computers & Education, 114, 193–205.

    Google Scholar 

  13. Davis, R. O. (2018). The impact of pedagogical agent gesturing in multimedia learning environments: A meta-analysis. Educational Research Review, 24, 193–209.

    Google Scholar 

  14. Davis, R., & Antonenko, P. (2017). Effects of pedagogical agent gestures on social acceptance and learning: Virtual real relationships in an elementary foreign language classroom. Journal of Interactive Learning Research, 28(4), 459–480.

    Google Scholar 

  15. Davis, R. O., Park, T., & Vincent, J. (2021). A systematic narrative review of agent persona on learning outcomes and design variables to enhance personification. Journal of Research on Technology in Education, 53(1), 89–106.

    Google Scholar 

  16. Davis, R. O., & Vincent, J. (2019). Sometimes more is better: Agent gestures, procedural knowledge and the foreign language learner. British Journal of Educational Technology, 50(6), 3252–3263.

    Google Scholar 

  17. Derwing, T., & Munro, M. J. (2001). What speaking rates do non-native listeners prefer? Applied Linguistics, 22(3), 324–337.

    Google Scholar 

  18. Dunsworth, Q., & Atkinson, R. K. (2007). Fostering multimedia learning of science: Exploring the role of an animated agent’s image. Computers & Education, 49(3), 677–690.

    Google Scholar 

  19. Elkhafaifi, H. (2005). The effects of prelistening activities on listening comprehension in Arabic learners. Foreign Language Annals, 38, 505–513.

    Google Scholar 

  20. Ellis, R. (1993). Naturally Simplified Input, Comprehension, and Second Language Acquisition.

  21. Flowerdew, J., & Miller, L. (1992). Student perceptions, problems and strategies in second language lecture comprehension. RELC Journal, 23(2), 60–80.

    Google Scholar 

  22. Frechette, C., & Moreno, R. (2010). The roles of animated pedagogical agents’ presence and nonverbal communication in multimedia learning environments. Journal of Media Psychology, 22(2), 61–72.

    Google Scholar 

  23. Goldin-Meadow, S., Kim, S., & Singer, M. (1999). What the teacher’s hands tell the student’s mind about math. Journal of Educational Psychology, 91(4), 720.

    Google Scholar 

  24. Griffiths, R. (1990). Facilitating listening comprehension through rate-control. RELC Journal, 21(1), 55–65.

    Google Scholar 

  25. Griffiths, R. (1992). Speech rate and listening comprehension: Further evidence of the relationship. TESOL Quarterly, 26(2), 385–390.

    Google Scholar 

  26. Grosjean, F. (1972). Le rôle joué par trois variables temporelles dans la compréhension orale de l'anglais étudié comme seconde langue et perception de la vitesse de lecture par des lecteurs et des auditeurs (Doctoral dissertation).

  27. Hinkel, E. (2006). Current perspectives on teaching the four skills. TESOL Quarterly, 40, 109–131.

    Google Scholar 

  28. Honing, H. (2006). Evidence for tempo-specific timing in music using a web-based experimental setup. Journal of Experimental Psychology: Human Perception and Performance, 32(3), 780.

    Google Scholar 

  29. Hostetter, A. B. (2008). Mind in motion: The gesture as simulated action framework (PhD thesis). The University of Wisconsin, Madison

  30. Hostetter, A. B. (2011). When do gestures communicate? A meta-analysis. Psychological Bulletin, 137(2), 297–315.

    Google Scholar 

  31. Jensen, E., & Vinther, T. (2003). Exact repetition as input enhancement in second language acquisition. Language Learning, 53, 373–428.

    Google Scholar 

  32. Kelly, S. D., McDevitt, T., & Esch, M. (2009). Brief training with co-speech gesture lends a hand to word learning in a foreign language. Language and Cognitive Processes, 24(2), 313–334.

    Google Scholar 

  33. Larsen-Freeman, D. (2003). Teaching language: From grammar to grammaring. Heinle & Heinle Pub.

    Google Scholar 

  34. Lester, J. C., Voerman, J. L., Towns, S. G., & Callaway, C. B. (1999). Deictic believability: Coordinated gesture, locomotion, and speech in lifelike pedagogical agents. Applied Artificial Intelligence, 13(4–5), 383–414.

    Google Scholar 

  35. Lewis, K., Lange, D., & Gillis, L. (2005). Transactive memory systems, learning, and learning transfer. Organization Science, 16(6), 581–598.

    Google Scholar 

  36. Mayer, R. E. (2009). Multimedia learning (2nd ed.). . Cambridge University Press.

    Google Scholar 

  37. Mayer, R. E., & DaPra, C. S. (2012). An embodiment effect in computer-based learning with animated pedagogical agents. Journal of Experimental Psychology: Applied, 18(3), 239.

    Google Scholar 

  38. McBride, K. (2011). The effect of rate of speech and distributed practice on the development of listening comprehension. Computer Assisted Language Learning, 24(2), 131–154.

    Google Scholar 

  39. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. University of Chicago Press.

    Google Scholar 

  40. Mecartty, F. (2000). Lexical and grammatical knowledge in reading and listening comprehension by foreign language learners of Spanish. Applied Language Learning, 11, 323–348.

    Google Scholar 

  41. Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91(2), 358.

    Google Scholar 

  42. Osada, N. (2004). Listening comprehension research: A brief review of the past thirty years. Dialogue, 3(1), 53–66.

    Google Scholar 

  43. Pastore, R. S. (2010). The effects of diagrams and time-compressed instruction on learning and learners’ perceptions of cognitive load. Educational Technology Research and Development, 58(5), 485–505.

    Google Scholar 

  44. Pastore, R. (2012). The effects of time-compressed instruction and redundancy on learning and learners’ perceptions of cognitive load. Computers & Education, 58(1), 641–651.

    Google Scholar 

  45. Rader, K. E. (1990). The effects of three different levels of word rate on the listening comprehension of third-quarter university Spanish students (Doctoral dissertation, The Ohio State University).

  46. Reeves, L., & Weisberg, R. W. (1994). The role of content and abstract information in analogical transfer. Psychological Bulletin, 115(3), 381.

    Google Scholar 

  47. Ritzhaupt, A. D., & Barron, A. (2008). Effects of time-compressed narration and representational adjunct images on cued-recall, content recognition, and learner satisfaction. Journal of Educational Computing Research, 39(2), 161–184.

    Google Scholar 

  48. Ritzhaupt, A. D., Gomes, N. D., & Barron, A. E. (2008). The effects of time-compressed audio and verbal redundancy on learner performance and satisfaction. Computers in Human Behavior, 24(5), 2434–2445.

    Google Scholar 

  49. Ritzhaupt, A. D., Pastore, R., & Davis, R. (2015). Effects of captions and time-compressed video on learner performance and satisfaction. Computers in Human Behavior, 45, 222–227.

    Google Scholar 

  50. Rothermich, K., Harris, H. L., Sewell, K., & Bobb, S. C. (2019). Listener impressions of foreigner-directed speech: A systematic review. Speech Communication, 112, 22–29.

    Google Scholar 

  51. Schlaug, G., Knorr, U., & Seitz, R. J. (1994). Inter-subject variability of cerebral activations in acquiring a motor skill: A study with positron emission tomography. Experimental Brain Research, 98(3), 523–534.

    Google Scholar 

  52. Seo, K. (2002). The effect of visuals on listening comprehension: A study of Japanese learners’ listening strategies. International Journal of Listening, 15, 57–81.

    Google Scholar 

  53. Stewart, M. A., & Pertusa, I. (2004). Gains to language learners from viewing target anguage close-captioned films. Foreign Language Annals, 37, 438–447.

    Google Scholar 

  54. Theune, M., & Brandhorst, C. J. (2009). To beat or not to beat: Beat gestures in direction giving. In S. Kopp & I. Wachsmuth (Eds.), Gesture in embodied communication and human-computer interaction (pp. 195–206). Springer.

    Google Scholar 

  55. Underwood, M., Kenworthy, J., & Rost, M. (1989). Teaching listening. (Vol. 117). Longman.

    Google Scholar 

  56. Vandergrift, L. (2007). Recent developments in second and foreign language listening comprehension research. Language Teaching, 40(3), 191–210.

    Google Scholar 

  57. Vandergrift, L., Goh, C. C., Mareschal, C. J., & Tafaghodtari, M. H. (2006). The metacognitive awareness listening questionnaire: Development and validation. Language Learning, 56(3), 431–462.

    Google Scholar 

  58. Verdugo, D., & Belmonte, I. A. (2007). Using digital stories to improve listening comprehension with Spanish young learners of English. Language Learning & Technology, 11(1), 87–101.

    Google Scholar 

  59. Willems, R. M., Özyürek, A., & Hagoort, P. (2007). When language meets action: The neural integration of gesture and speech. Cerebral Cortex, 17(10), 2322–2333.

    Google Scholar 

  60. Winke, P., Gass, S., & Sydorenko, T. (2010). The effects of captioning videos used for foreign language listening activities. Language Learning and Technology, 14(1), 65–86.

    Google Scholar 

  61. Zhao, Y. (1997). The effects of listeners’ control of speech rate on second language comprehension. Applied Linguistics, 18, 49–68.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Robert O. Davis.

Ethics declarations

Conflict of interest

There are no conflicts of interest to report for this research or the authors involved.

Informed consent

The research involved human participants. Informed consent was explained orally to the participants, and also presented in written form and agreed to before beginning the experiment.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original version of this article was revised: The original publication inadvertently referenced agent persona instead of satisfaction in the title and abstract (lines 11 and 17). The abstract misstated the use of structural equation modeling as the method of analysis.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Davis, R.O., Wan, L.L., Vincent, J. et al. The effects of virtual human gesture frequency and reduced video speed on satisfaction and learning outcomes. Education Tech Research Dev 69, 2331–2352 (2021). https://doi.org/10.1007/s11423-021-10010-x

Download citation

Keywords

  • Pedagogical agent
  • Virtual human
  • Procedural information
  • Gesture
  • Gesture frequency
  • Video speed