Abstract
Pedagogical agents offer significant promise for engaging students in learning. In this paper, we investigate students’ conversational interactions with a pedagogical agent in a game-based learning environment for middle school science education. We utilize word embeddings of student-agent conversations along with features distilled from students’ in-game actions to induce predictive models of student engagement. An evaluation of the models’ accuracy and early prediction performance indicates that features derived from students’ conversations with the pedagogical agent yield the highest accuracy for predicting student engagement. Results also show that combining student problem-solving features and conversation features yields higher performance than a problem solving-only feature set. Overall, the findings suggest that student-agent conversations can greatly enhance student models for game-based learning environments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Burgoon, J., et al.: Application of expectancy violations theory to communication with and judgments about embodied agents during a decision-making task. Int. J. Hum.-Comput. Stud. 91, 24–36 (2016)
Dermouche, S., Pelachaud, C.: Engagement modeling in dyadic interaction. In: Proceedings of the 2019 International Conference on Multimodal Interaction, pp. 440–445 (2019)
Devlin, J., et al.: BERT: pre-training of deep bidirectional transformers for language understanding. North American Association for Computational Linguistics (NAACL) (2018)
Emerson, A., et al.: Multimodal learning analytics for game-based learning. Br. J. Educ. Technol. 51(5), 1505–1526 (2020)
Forbes-Riley, K., Litman, D.: Benefits and challenges of real-time uncertainty detection and adaptation in a spoken dialogue computer tutor. Speech Commun. 53(9–10), 1115–1136 (2011)
Geden, M., et al.: Predictive student modeling in game-based learning environments with word embedding representations of reflection. Int. J. Artif. Intell. Educ. 31(1), 1–23 (2021)
Gobert, J., Baker, R., Wixon, M.: Operationalizing and detecting disengagement within online science microworlds. Educ. Psychol. 50(1), 43–57 (2015)
Graesser, A.: Conversations with AutoTutor help students learn. Int. J. Artif. Intell. Educ. 26(1), 124–132 (2016)
Graesser, A., et al.: Advancing the science of collaborative problem solving. Psychol. Sci. Public Interest 19(2), 59–92 (2018)
Hirschberg, J., Manning, C.: Advances in natural language processing. Science 349(6245), 261–266 (2015)
Lin, Z., et al.: MinTL: minimalist transfer learning for task-oriented dialogue systems. arXiv preprint arXiv:2009.12005 (2020)
Min, W., et al.: Multimodal goal recognition in open-world digital games. In: 13th Artificial Intelligence and Interactive Digital Entertainment Conference, pp. 80–86 (2017)
Min, W., et al.: Predicting dialogue acts for intelligent virtual agents with multimodal student interaction data. International Educational Data Mining Society (2016)
O’Brien, H., Toms, E.: The development and evaluation of a survey to measure user engagement. J. Am. Soc. Inf. Sci. Technol. 61(1), 50–69 (2010)
Pezzullo, L.G., et al.: “Thanks Alisha, Keep in Touch”: gender effects and engagement with virtual learning companions. In: André, E., Baker, R., Hu, X., Rodrigo, M.M.T., du Boulay, B. (eds.) AIED 2017. LNCS (LNAI), vol. 10331, pp. 299–310. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61425-0_25
Pugh, S., et al.: Do speech-based collaboration analytics generalize across task contexts?. In: 12th International LAK Conference, pp. 208–218 (2022)
Sikström, P., et al.: How pedagogical agents communicate with students: a two-phase systematic review. Comput. Educ. 188, 104564 (2022)
Tegos, S., et al.: Conversational agents for academically productive talk: a comparison of directed and undirected agent interventions. Int. J. Comput.-Support. Collab. Learn. 11(4), 417–440 (2016)
Wiebe, E., et al.: Measuring engagement in video game-based environments: investigation of the User Engagement Scale. Comput. Hum. Behav. 32, 123–132 (2014)
Zhang, J., et al.: Investigating student interest and engagement in game-based learning environments. In: Rodrigo, M.M., Matsuda, N., Cristea, A.I., Dimitrova, V. (eds.) AIED 2022. LNCS, vol. 13355, pp. 711–716. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-11644-5
Acknowledgements
This research was supported by funding from the National Science Foundation under grants IIS 2016943, IIS 2016993, and IIS 1409639. Any opinions, findings, and conclusions expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Goslen, A. et al. (2023). Enhancing Engagement Modeling in Game-Based Learning Environments with Student-Agent Discourse Analysis. In: Wang, N., Rebolledo-Mendez, G., Dimitrova, V., Matsuda, N., Santos, O.C. (eds) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky. AIED 2023. Communications in Computer and Information Science, vol 1831. Springer, Cham. https://doi.org/10.1007/978-3-031-36336-8_105
Download citation
DOI: https://doi.org/10.1007/978-3-031-36336-8_105
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36335-1
Online ISBN: 978-3-031-36336-8
eBook Packages: Computer ScienceComputer Science (R0)