Estimating Student Proficiency Using an Item Response Theory Model
Item Response Theory (IRT) models were investigated as a tool for student modeling in an intelligent tutoring system (ITS). The models were tested using real data of high school students using the Wayang Outpost, a computer-based tutor for the mathematics portion of the Scholastic Aptitude Test (SAT). A cross-validation framework was developed and three metrics to measure prediction accuracy were compared. The trained models predicted with 72% accuracy whether a student would answer a multiple choice problem correctly.
KeywordsMean Square Error Item Response Theory Item Response Theory Model Mean Absolute Error Intelligent Tutoring System
Unable to display preview. Download preview PDF.
- 5.Baker, R., Corbett, A., Koedinger, K., Wagner, A.: Off-Task Behavior in the Cognitive Tutor Classroom: When Students Game the System. In: Proceedings of the ACM CHI 2004 Conference on Human Factors in Computing Systems, pp. 383–390 (2004)Google Scholar
- 6.Barnes, T.: The Q-Matrix Method of Fault-Tolerant Teaching in Knowledge Assessment and Data Mining. Ph.D. Dissertation. North Carolina State University (2003)Google Scholar
- 7.Beck, J.: Engagement Tracing: Using Response Times to Model Student Disengagement. In: Proceedings of the International Conference on Artificial Intelligent and Education (2005)Google Scholar
- 13.Jonsson, A., Johns, J., Mehranian, H., Arroyo, I., Woolf, B., Barto, A., Fisher, D., Mahadevan, S.: Evaluating the Feasibility of Learning Student Models from Data. In: American Association for Artificial Intelligence Workshop on Educational Data Mining (2005)Google Scholar
- 14.Mayo, M., Mitrovic, A.: Optimising ITS Behavior with Bayesian Networks and Decision Theory. International Journal of Artificial Intelligence in Education 12, 124–153 (2001)Google Scholar
- 15.Rudner, L.: An Evaluation of Measurement Decision Theory, http://edres.org/mdt/home3.asp
- 16.Thissen, D., Wainer, H. (eds.): Test Scoring. Lawrence Erlbaum Associates, Mahwah (2001)Google Scholar