Detecting Student Misuse of Intelligent Tutoring Systems

  • Ryan Shaun Baker
  • Albert T. Corbett
  • Kenneth R. Koedinger
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3220)

Abstract

Recent research has indicated that misuse of intelligent tutoring software is correlated with substantially lower learning. Students who frequently engage in behavior termed “gaming the system” (behavior aimed at obtaining correct answers and advancing within the tutoring curriculum by systematically taking advantage of regularities in the software’s feedback and help) learn only 2/3 as much as similar students who do not engage in such behaviors. We present a machine-learned Latent Response Model that can identify if a student is gaming the system in a way that leads to poor learning. We believe this model will be useful both for re-designing tutors to respond appropriately to gaming, and for understanding the phenomenon of gaming better.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Aleven, V., Koedinger, K.R.: Investigations into Help Seeking and Learning with a Cognitive Tutor. In: Luckin, R. (ed.) Papers of the AIED-2001 Workshop on Help Provision and Help Seeking in Interactive Learning Environments, pp. 47–58 (2001)Google Scholar
  2. 2.
    Aleven, V., McLaren, B., Roll, I., Koedinger, K.: Toward Tutoring Help Seeking: Applying Cognitive Modeling to Meta-Cognitive Skills. To appear at Intelligent Tutoring Systems Conference (2004)Google Scholar
  3. 3.
    Arbreton, A.: Student Goal Orientation and Help-Seeking Strategy Use. In: Karabenick, S.A. (ed.) Strategic Help Seeking: Implications For Learning And Teaching, pp. 95–116. Lawrence Erlbaum Associates, Mahwah (1998)Google Scholar
  4. 4.
    Baker, R.S., Corbett, A.T., Koedinger, K.R.: Learning to Distinguish Between Representations of Data: a Cognitive Tutor That Uses Contrasting Cases. To appear at International Conference of the Learning Sciences (2004)Google Scholar
  5. 5.
    Baker, R.S., Corbett, A.T., Koedinger, K.R., Wagner, A.Z.: Off-Task Behavior in the Cognitive Tutor Classroom: When Students “Game the System”. In: Proceedings of ACM CHI 2004: Computer-Human Interaction, pp. 383–390 (2004)Google Scholar
  6. 6.
    Corbett, A.T., Anderson, J.R.: Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge. User Modeling and User-Adapted Interaction 4, 253–278 (1995)CrossRefGoogle Scholar
  7. 7.
    Corbett, A.T., Koedinger, K.R., Hadley, W.S.: Cognitive Tutors: From the Research Classroom to All Classrooms. In: Goodman, P. (ed.) Technology Enhanced Learning: Opportunities For Change, pp. 235–263. Lawrence Erlbaum Associates, Mahwah (2001)Google Scholar
  8. 8.
    de Vicente, A., Pain, H.: Informing the Detection of the Students’ Motivational State: an Empirical Study. In: Cerri, S.A., Gouarderes, G., Paraguacu, F. (eds.) Proceedings of the Sixth International Conference on Intelligent Tutoring Systems, pp. 933–943 (2002)Google Scholar
  9. 9.
    del Soldato, T., du Boulay, B.: Implementation of Motivational Tactics in Tutoring Systems. Journal of Artificial Intelligence in Education 6(4), 337–376 (1995)Google Scholar
  10. 10.
    Donaldson, W.: Accuracy of d’ and A’ as Estimates of Sensitivity. Bulletin of the Psychonomic Society 31(4), 271–274 (1993)MathSciNetGoogle Scholar
  11. 11.
    Jacob, B.A., Levitt, S.D.: Catching Cheating Teachers: The Results of an Unusual Experiment in Implementing Theory. To appear in Brookings-Wharton Papers on Urban Affairs Google Scholar
  12. 12.
    Lloyd, J.W., Loper, A.B.: Measurement and Evaluation of Task-Related Learning Behavior: Attention to Task and Metacognition. School Psychology Review 15(3), 336–345 (1986)Google Scholar
  13. 13.
    Maris, E.: Psychometric Latent Response Models. Psychometrika 60(4), 523–547 (1995)MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Martin, J., vanLehn, K.: Student Assessment Using Bayesian Nets. International Journal of Human-Computer Studies 42, 575–591 (1995)CrossRefGoogle Scholar
  15. 15.
    Mostow, J., Aist, G., Beck, J., Chalasani, R., Cuneo, A., Jia, P., Kadaru, K.: A La Recherche du Temps Perdu, or As Time Goes By: Where Does the Time Go in a Reading Tutor that Listens? In: Sixth International Conference on Intelligent Tutoring Systems, pp. 320–329 (2002)Google Scholar
  16. 16.
    Ramsey, F.L., Schafer, D.W.: The Statistical Sleuth: A Course in Methods of Data Analysis. Duxbury Press, Belmont (1997) Section 12.3Google Scholar
  17. 17.
    Schofield, J.W.: Computers and Classroom Culture. Cambridge University Press, Cambridge (1995)CrossRefGoogle Scholar
  18. 18.
    Wood, H., Wood, D.: Help Seeking, Learning, and Contingent Tutoring. Computers and Education 33, 153–159 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Ryan Shaun Baker
    • 1
  • Albert T. Corbett
    • 1
  • Kenneth R. Koedinger
    • 1
  1. 1.Human-Computer Interaction InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations