The Effect of Metaphoric Gestures on Schematic Understanding of Instruction Performed by a Pedagogical Conversational Agent
In this paper, we examine the impact of metaphoric gestures performed by Pedagogical Conversational Agent (PCA) on learners’ memorization of technical terms, understanding of relationships between abstract concepts, learning experience, and perception of the PCA. The study employed a one-factor three-level between-participants design where we manipulated gesture factor (speech-gesture match vs. speech-gesture mismatch vs. no-gesture). The data of 97 students were acquired in on-line learning environment. As the results, while there was no effect found on memorization of technical terms, we found that students showed accurate schematic understanding of the relationship between abstract concepts when the PCA used metaphoric gestures matched to speech content than when used gestures mismatched, and no gesture. Contrary to the result, we also found that students judged the PCA useful, helpful, and felt the PCA looked like a teacher when performed mismatched gestures to speech content than when performed matched gesture.
KeywordsPedagogical agent Metaphoric gesture Understanding Reliability
This work was partially supported by JSPS Grant-in-Aid for Young Scientists (B) Grant Number 25870698, JSPS Grant-in-Aid for Challenging Exploratory Research Grant Number 26540185, and Council for Science, Technology and Innovation(CSTI), Cross-ministerial Strategic Innovation Promotion Program (SIP), Structural Materials for Innovation (SM4I) (Funding agency:JST).
- 1.Argyle, M.: Bodily communication. Methuen, London (1975)Google Scholar
- 6.Buisine, S., Abrilian, S., Martin, J.C.: Evaluation of multimodal behaviour of embodied agents. In: Ruttkay, Z., Pelachaud, C. (eds.) From Brows to Trust, pp. 217–238. Springer, The Netherlands (2005)Google Scholar
- 11.Hasegawa, D., Ugurlu, Y., Sakuta, H.: A human-like embodied agent learning tour guide for e-learning systems. In: 2014 IEEE Global Engineering Education Conference (EDUCON), pp. 50–53, April 2014Google Scholar
- 18.Lester, J.C., Voerman, J.L., Towns, S.G., Callaway, C.B.: Cosmo: A life-like animated pedagogical agent with deictic believability. In: Working Notes of the IJCAI 1997 Workshop on Animated Interface Agents: Making ThemIntelligent, pp. 61–69. Citeseer, August 1997Google Scholar
- 19.McNeil, D.: Hand and Mind. University of Chicago Press, Chicago (1992)Google Scholar
- 22.Neff, M., Wang, Y., Abbott, R., Walker, M.: Evaluating the effect of gesture and language on personality perception in conversational agents. In: Allbeck, J., Badler, N., Bickermore, T., Pelachcaud, C., Safonova, A. (eds.) IVA 2010. LNCS, vol. 6356, pp. 222–235. Springer, Heidelberg (2010) CrossRefGoogle Scholar
- 23.Ogan, A., Finkelstein, S., Mayfield, E., D’Adamo, C., Matsuda, N., Cassell, J.: "Oh dear stacy!": social interaction, elaboration, and learning with teachable agents. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2012, pp. 39–48. ACM, New York, NY, USA (2012)Google Scholar
- 24.Popescu, V., Adamo-Villani, N., Wu, M.L., Rajasekaran, S.D., Alibali, M.W., Nathan, M., Cook, S.W.: Animation killed the video star. In: Proceedings of Gesture-based Interaction Design: Communication and Cognition, 2014 CHI Workshop, pp. 55–59 (2014)Google Scholar
- 28.Voerman, J.L., FitzGerald, P.J.: Deictic and emotive communication in animated pedagogical agents. In: Cassell, J., Sullican, J., Prevost, S., Churchill, E. (eds.) Embodied Conversational Agents, pp. 123–154. The MIT Press, Cambridge (2000)Google Scholar