A System-General Model for the Detection of Gaming the System Behavior in CTAT and LearnSphere
In this paper, we present the CTAT (Cognitive Tutor Authoring Tools) implementation of a system-general model for the detection of students who “game the system”, a behavior in which students misuse intelligent tutors or other online learning environments in order to complete problems or otherwise advance without learning. We discuss how this publicly available detector can be used for both live detection of gaming behavior while students are using CTAT tutors and for retroactive application of the detector to historical data within LearnSphere. The goal of making this detector publicly available is to foster new research about how to best intervene when students game the system and to increase the large scale adoption of such detectors in the classroom.
KeywordsGaming the system System-general models Cognitive tutor authoring tool LearnSphere Student model
We would like to thank Kenneth Holstein, Cindy Tipper, Peter Schaldenbrand and Vincent Aleven for their support during the implementation of our detector in the CTAT and LearnSphere platforms.
- 1.Luckin, R., Holmes, W., Griffiths, M., Forcier, L.B.: Intelligence Unleashed: An Argument for AI in Education (2016)Google Scholar
- 3.Beal, C.R., Qu, L., Lee, H.: Classifying learner engagement through integration of multiple data sources. In: Proceedings of the 21st National Conference on Artificial Intelligence, pp. 2–8 (2006)Google Scholar
- 5.Paquette, L., de Carvalho, A.M.J.B., Baker, R.S.: Towards understanding expert coding of student disengagement in online learning. In: Proceedings of the 36th Annual Meeting of the Cognitive Science Society, pp. 1126–1131 (2014)Google Scholar
- 7.Aleven, V., Sewall, J., Popescu, O., van Velsen, M., Demi, Sandra, Leber, B.: Reflecting on twelve years of its authoring tools research with CTAT. In: Design Recommendation for Adaptive Intelligent Tutoring Systems, vol. 3, pp. 263–283 (2015)Google Scholar
- 8.Koedinger, K.R., Corbett, A.T.: Cognitive tutors: technology bringing learning sciences to the classroom. In R.K. Sawyer (ed.) The Cambridge Handbook of the Learning Sciences, pp. 61–77 (2006)Google Scholar
- 9.Paquette, L., Baker, R.S., de Carvalho, A.M.J.B., Ocumpaugh, J.: Cross-system transfer of machine learned and knowledge engineered models of gaming the system. In: Proceedings of the 23rd Conference on User Modelling, Adaptation and Personalization, pp. 183–194 (2015)Google Scholar
- 10.Razzaq, L., et al.: The assistment project: blending assessment and assisting. In: Proceedings of the 12th Annual Conference on Artificial Intelligence in Education, pp. 555–562 (2005)Google Scholar
- 11.Koedinger, K.R., Baker, R.S.J., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J.: A data repository for the EDM community: the PSLC dataShop. In: Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J. (eds.) Handbook of Educational Data Mining, pp. 43–56 (2010)Google Scholar
- 12.Holstein, K., McLaren, B.M., Aleven, V.: Intelligent tutors as teachers’ aides: exploring needs for real-time analytics in blended classrooms. In: Proceedings of the 7th International Learning Analytics and Knowledge Conference, pp. 257–266 (2017)Google Scholar