A System-General Model for the Detection of Gaming the System Behavior in CTAT and LearnSphere

  • Luc Paquette
  • Ryan S. Baker
  • Michal Moskal
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10948)


In this paper, we present the CTAT (Cognitive Tutor Authoring Tools) implementation of a system-general model for the detection of students who “game the system”, a behavior in which students misuse intelligent tutors or other online learning environments in order to complete problems or otherwise advance without learning. We discuss how this publicly available detector can be used for both live detection of gaming behavior while students are using CTAT tutors and for retroactive application of the detector to historical data within LearnSphere. The goal of making this detector publicly available is to foster new research about how to best intervene when students game the system and to increase the large scale adoption of such detectors in the classroom.


Gaming the system System-general models Cognitive tutor authoring tool LearnSphere Student model 



We would like to thank Kenneth Holstein, Cindy Tipper, Peter Schaldenbrand and Vincent Aleven for their support during the implementation of our detector in the CTAT and LearnSphere platforms.


  1. 1.
    Luckin, R., Holmes, W., Griffiths, M., Forcier, L.B.: Intelligence Unleashed: An Argument for AI in Education (2016)Google Scholar
  2. 2.
    Baker, R.S.J., Corbett, A.T., Roll, I., Koedinger, K.R.: Developing a generalizable detector of when students game the system. User Model. User Adap. Inter. 18, 287–314 (2008)CrossRefGoogle Scholar
  3. 3.
    Beal, C.R., Qu, L., Lee, H.: Classifying learner engagement through integration of multiple data sources. In: Proceedings of the 21st National Conference on Artificial Intelligence, pp. 2–8 (2006)Google Scholar
  4. 4.
    Muldner, K., Burleson, W., Van de Sande, B., VanLehn, K.: An analysis of students’ gaming behaviors in an intelligent tutoring system: predictors and impact. User Model. User Adap. Inter. 21, 99–135 (2011)CrossRefGoogle Scholar
  5. 5.
    Paquette, L., de Carvalho, A.M.J.B., Baker, R.S.: Towards understanding expert coding of student disengagement in online learning. In: Proceedings of the 36th Annual Meeting of the Cognitive Science Society, pp. 1126–1131 (2014)Google Scholar
  6. 6.
    Walonoski, J.A., Heffernan, N.T.: Detection and analysis of off-task gaming behavior in intelligent tutoring systems. In: Ikeda, M., Ashley, Kevin D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 382–391. Springer, Heidelberg (2006). Scholar
  7. 7.
    Aleven, V., Sewall, J., Popescu, O., van Velsen, M., Demi, Sandra, Leber, B.: Reflecting on twelve years of its authoring tools research with CTAT. In: Design Recommendation for Adaptive Intelligent Tutoring Systems, vol. 3, pp. 263–283 (2015)Google Scholar
  8. 8.
    Koedinger, K.R., Corbett, A.T.: Cognitive tutors: technology bringing learning sciences to the classroom. In R.K. Sawyer (ed.) The Cambridge Handbook of the Learning Sciences, pp. 61–77 (2006)Google Scholar
  9. 9.
    Paquette, L., Baker, R.S., de Carvalho, A.M.J.B., Ocumpaugh, J.: Cross-system transfer of machine learned and knowledge engineered models of gaming the system. In: Proceedings of the 23rd Conference on User Modelling, Adaptation and Personalization, pp. 183–194 (2015)Google Scholar
  10. 10.
    Razzaq, L., et al.: The assistment project: blending assessment and assisting. In: Proceedings of the 12th Annual Conference on Artificial Intelligence in Education, pp. 555–562 (2005)Google Scholar
  11. 11.
    Koedinger, K.R., Baker, R.S.J., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J.: A data repository for the EDM community: the PSLC dataShop. In: Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J. (eds.) Handbook of Educational Data Mining, pp. 43–56 (2010)Google Scholar
  12. 12.
    Holstein, K., McLaren, B.M., Aleven, V.: Intelligent tutors as teachers’ aides: exploring needs for real-time analytics in blended classrooms. In: Proceedings of the 7th International Learning Analytics and Knowledge Conference, pp. 257–266 (2017)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of Illinois at Urbana-ChampaignChampaignUSA
  2. 2.University of PennsylvaniaPhiladelphiaUSA
  3. 3.University of WarsawWarsawPoland

Personalised recommendations