Abstract
In this paper, we present the CTAT (Cognitive Tutor Authoring Tools) implementation of a system-general model for the detection of students who “game the system”, a behavior in which students misuse intelligent tutors or other online learning environments in order to complete problems or otherwise advance without learning. We discuss how this publicly available detector can be used for both live detection of gaming behavior while students are using CTAT tutors and for retroactive application of the detector to historical data within LearnSphere. The goal of making this detector publicly available is to foster new research about how to best intervene when students game the system and to increase the large scale adoption of such detectors in the classroom.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The CTAT detector library is accessible from the CTAT detector wiki: https://github.com/d19fe8/CTAT-detector-plugins/wiki/CTAT-Detector-Library.
- 2.
References
Luckin, R., Holmes, W., Griffiths, M., Forcier, L.B.: Intelligence Unleashed: An Argument for AI in Education (2016)
Baker, R.S.J., Corbett, A.T., Roll, I., Koedinger, K.R.: Developing a generalizable detector of when students game the system. User Model. User Adap. Inter. 18, 287–314 (2008)
Beal, C.R., Qu, L., Lee, H.: Classifying learner engagement through integration of multiple data sources. In: Proceedings of the 21st National Conference on Artificial Intelligence, pp. 2–8 (2006)
Muldner, K., Burleson, W., Van de Sande, B., VanLehn, K.: An analysis of students’ gaming behaviors in an intelligent tutoring system: predictors and impact. User Model. User Adap. Inter. 21, 99–135 (2011)
Paquette, L., de Carvalho, A.M.J.B., Baker, R.S.: Towards understanding expert coding of student disengagement in online learning. In: Proceedings of the 36th Annual Meeting of the Cognitive Science Society, pp. 1126–1131 (2014)
Walonoski, J.A., Heffernan, N.T.: Detection and analysis of off-task gaming behavior in intelligent tutoring systems. In: Ikeda, M., Ashley, Kevin D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 382–391. Springer, Heidelberg (2006). https://doi.org/10.1007/11774303_38
Aleven, V., Sewall, J., Popescu, O., van Velsen, M., Demi, Sandra, Leber, B.: Reflecting on twelve years of its authoring tools research with CTAT. In: Design Recommendation for Adaptive Intelligent Tutoring Systems, vol. 3, pp. 263–283 (2015)
Koedinger, K.R., Corbett, A.T.: Cognitive tutors: technology bringing learning sciences to the classroom. In R.K. Sawyer (ed.) The Cambridge Handbook of the Learning Sciences, pp. 61–77 (2006)
Paquette, L., Baker, R.S., de Carvalho, A.M.J.B., Ocumpaugh, J.: Cross-system transfer of machine learned and knowledge engineered models of gaming the system. In: Proceedings of the 23rd Conference on User Modelling, Adaptation and Personalization, pp. 183–194 (2015)
Razzaq, L., et al.: The assistment project: blending assessment and assisting. In: Proceedings of the 12th Annual Conference on Artificial Intelligence in Education, pp. 555–562 (2005)
Koedinger, K.R., Baker, R.S.J., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J.: A data repository for the EDM community: the PSLC dataShop. In: Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J. (eds.) Handbook of Educational Data Mining, pp. 43–56 (2010)
Holstein, K., McLaren, B.M., Aleven, V.: Intelligent tutors as teachers’ aides: exploring needs for real-time analytics in blended classrooms. In: Proceedings of the 7th International Learning Analytics and Knowledge Conference, pp. 257–266 (2017)
Acknowledgement
We would like to thank Kenneth Holstein, Cindy Tipper, Peter Schaldenbrand and Vincent Aleven for their support during the implementation of our detector in the CTAT and LearnSphere platforms.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Paquette, L., Baker, R.S., Moskal, M. (2018). A System-General Model for the Detection of Gaming the System Behavior in CTAT and LearnSphere. In: Penstein Rosé, C., et al. Artificial Intelligence in Education. AIED 2018. Lecture Notes in Computer Science(), vol 10948. Springer, Cham. https://doi.org/10.1007/978-3-319-93846-2_47
Download citation
DOI: https://doi.org/10.1007/978-3-319-93846-2_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93845-5
Online ISBN: 978-3-319-93846-2
eBook Packages: Computer ScienceComputer Science (R0)