Detecting Student Misuse of Intelligent Tutoring Systems
Recent research has indicated that misuse of intelligent tutoring software is correlated with substantially lower learning. Students who frequently engage in behavior termed “gaming the system” (behavior aimed at obtaining correct answers and advancing within the tutoring curriculum by systematically taking advantage of regularities in the software’s feedback and help) learn only 2/3 as much as similar students who do not engage in such behaviors. We present a machine-learned Latent Response Model that can identify if a student is gaming the system in a way that leads to poor learning. We believe this model will be useful both for re-designing tutors to respond appropriately to gaming, and for understanding the phenomenon of gaming better.
Unable to display preview. Download preview PDF.
- 1.Aleven, V., Koedinger, K.R.: Investigations into Help Seeking and Learning with a Cognitive Tutor. In: Luckin, R. (ed.) Papers of the AIED-2001 Workshop on Help Provision and Help Seeking in Interactive Learning Environments, pp. 47–58 (2001)Google Scholar
- 2.Aleven, V., McLaren, B., Roll, I., Koedinger, K.: Toward Tutoring Help Seeking: Applying Cognitive Modeling to Meta-Cognitive Skills. To appear at Intelligent Tutoring Systems Conference (2004)Google Scholar
- 3.Arbreton, A.: Student Goal Orientation and Help-Seeking Strategy Use. In: Karabenick, S.A. (ed.) Strategic Help Seeking: Implications For Learning And Teaching, pp. 95–116. Lawrence Erlbaum Associates, Mahwah (1998)Google Scholar
- 4.Baker, R.S., Corbett, A.T., Koedinger, K.R.: Learning to Distinguish Between Representations of Data: a Cognitive Tutor That Uses Contrasting Cases. To appear at International Conference of the Learning Sciences (2004)Google Scholar
- 5.Baker, R.S., Corbett, A.T., Koedinger, K.R., Wagner, A.Z.: Off-Task Behavior in the Cognitive Tutor Classroom: When Students “Game the System”. In: Proceedings of ACM CHI 2004: Computer-Human Interaction, pp. 383–390 (2004)Google Scholar
- 7.Corbett, A.T., Koedinger, K.R., Hadley, W.S.: Cognitive Tutors: From the Research Classroom to All Classrooms. In: Goodman, P. (ed.) Technology Enhanced Learning: Opportunities For Change, pp. 235–263. Lawrence Erlbaum Associates, Mahwah (2001)Google Scholar
- 8.de Vicente, A., Pain, H.: Informing the Detection of the Students’ Motivational State: an Empirical Study. In: Cerri, S.A., Gouarderes, G., Paraguacu, F. (eds.) Proceedings of the Sixth International Conference on Intelligent Tutoring Systems, pp. 933–943 (2002)Google Scholar
- 9.del Soldato, T., du Boulay, B.: Implementation of Motivational Tactics in Tutoring Systems. Journal of Artificial Intelligence in Education 6(4), 337–376 (1995)Google Scholar
- 11.Jacob, B.A., Levitt, S.D.: Catching Cheating Teachers: The Results of an Unusual Experiment in Implementing Theory. To appear in Brookings-Wharton Papers on Urban Affairs Google Scholar
- 12.Lloyd, J.W., Loper, A.B.: Measurement and Evaluation of Task-Related Learning Behavior: Attention to Task and Metacognition. School Psychology Review 15(3), 336–345 (1986)Google Scholar
- 15.Mostow, J., Aist, G., Beck, J., Chalasani, R., Cuneo, A., Jia, P., Kadaru, K.: A La Recherche du Temps Perdu, or As Time Goes By: Where Does the Time Go in a Reading Tutor that Listens? In: Sixth International Conference on Intelligent Tutoring Systems, pp. 320–329 (2002)Google Scholar
- 16.Ramsey, F.L., Schafer, D.W.: The Statistical Sleuth: A Course in Methods of Data Analysis. Duxbury Press, Belmont (1997) Section 12.3Google Scholar