Abstract
Over the 40 years since its debut, computer-aided teaching and learning have grown from novelty to near pervasive, with a host of dynamic interactive digital systems underpinning many aspects of the modern student experience. While potentially transformative, the wholesale integration of computing systems into contemporary schooling has not yet delivered solutions to enduring problems in the education sector: so far few techniques enable the tracking of engagement levels in real time. This chapter addresses a new method to examine the engagement of learners by utilizing webcam and mouse information. Based on a study conducted with learners undertaking an online course, a dataset was collected including the mouse and keyboard usages, face images and screenshots. The learner’s engagement level was inferred via the eye gaze direction and mouse movements. The analytics suggests that the learning behavior data, which potentially reflects the learner’s engagement, could be an indicator to the learning outcome of individual tasks. We have also identified a few practical issues faced by real-time engagement examination systems. It provides a novel means of examining engagement quantitatively in online learning and brings new insights into innovative education solutions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Juhaňák, L., Zounek, J., RohlÃková, L.: Using process mining to analyze students’ quiz-taking behavior patterns in a learning management system. Computers in Human Behavior 92, 496–506 (2019)
Carlsson, M., Simovska, V.: Exploring learning outcomes of school-based health promotion—a multiple case study. Health Education Research 27(3), 437–447 (2012)
Norton, P., Hathaway, D.: Exploring two teacher education online learning designs: A classroom of one or many? Journal of Research on Technology in Education 40(4), 475–495 (2008)
Lavy, V.: Expanding school resources and increasing time on task: Effects of a policy experiment in israel on student academic achievement and behavior. Tech. rep., National Bureau of Economic Research (2012)
Sinatra, G.M., Heddy, B.C., Lombardi, D.: The challenges of defining and measuring student engagement in science (2015)
D’Mello, S., Dieterle, E., Duckworth, A.: Advanced, analytic, automated (AAA) measurement of engagement during learning. Educational psychologist 52(2), 104–123 (2017)
Henrie, C.R., Halverson, L.R., Graham, C.R.: Measuring student engagement in technology-mediated learning: A review. Computers & Education 90, 36–53 (2015)
D’mello, S.K., Graesser, A.: Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features. User Modeling and User-Adapted Interaction 20(2), 147–187 (2010)
Aslan, S., Alyuz, N., Tanriover, C., Mete, S.E., Okur, E., D’Mello, S.K., Arslan Esme, A.: Investigating the impact of a real-time, multimodal student engagement analytics technology in authentic classrooms. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–12 (2019)
Alyuz, N., Okur, E., Oktay, E., Genc, U., Aslan, S., Mete, S.E., Arnrich, B., Esme, A.A.: Semi-supervised model personalization for improved detection of learner’s emotional engagement. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction, pp. 100–107 (2016)
Salmeron-Majadas, S., Santos, O.C., Boticario, J.G.: An evaluation of mouse and keyboard interaction indicators towards non-intrusive and low cost affective modeling in an educational context. Procedia Computer Science 35, 691–700 (2014)
Lim, Y.M., Ayesh, A., Stacey, M.: Using mouse and keyboard dynamics to detect cognitive stress during mental arithmetic. In: Science and Information Conference, pp. 335–350. Springer (2014)
Estrada, J., Buhia, J., Guevarra, A., Forcado, M.R.: Keyboard and mouse: tools in identifying emotions during computer activities. In: International Conference on Big Data Technologies and Applications, pp. 115–123. Springer (2017)
Ally, M.: Foundations of educational theory for online learning. Theory and practice of online learning 2, 15–44 (2004)
Hu, Y., Wu, B., Gu, X.: An eye tracking study of high-and low-performing students in solving interactive and analytical problems. Journal of Educational Technology & Society 20(4), 300–311 (2017)
Khedher, A.B., Jraidi, I., Frasson, C.: Static and dynamic eye movement metrics for students’ performance assessment. Smart Learning Environments 5(1), 1–12 (2018)
Nutrition Australia Fact Sheets, http://www.nutritionaustralia.org/national/resources (2019)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, vol. 1, pp. I–I. IEEE (2001)
Bradski, G., Kaehler, A.: Learning OpenCV: Computer vision with the OpenCV library. O’Reilly Media, Inc. (2008)
Acknowledgements
We would like to thank all the people who contributed their time, data, and helpful advice to this study. We would appreciate ACER Australia as well for their support and collaboration in the research project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Yu, K., Xu, J., Ou, Y., Luo, L., Chen, F. (2022). Learner Engagement Examination Via Computer Usage Behaviors. In: Chen, F., Zhou, J. (eds) Humanity Driven AI. Springer, Cham. https://doi.org/10.1007/978-3-030-72188-6_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-72188-6_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-72187-9
Online ISBN: 978-3-030-72188-6
eBook Packages: Computer ScienceComputer Science (R0)