Predicting State Test Scores Better with Intelligent Tutoring Systems: Developing Metrics to Measure Assistance Required
The ASSISTment system was used by over 600 students in 2004-05 school year as part of their math class. While in  we reported student learning within the ASSISTment system, in this paper we focus on the assessment aspect. Our approach is to use data that the system collected through a year to tracking student learning and thus estimate their performance on a high-stake state test (MCAS) at the end of the year. Because our system is an intelligent tutoring system, we are able to log how much assistance students needed to solve problems (how many hints students requested and how many attempts they had to make). In this paper, our goal is to determine if the models we built by taking the assistance information into account could predict students’ test scores better. We present some positive evidence that shows our goal is achieved.
KeywordsASSISTment System Median Absolute Difference Intelligent Tutor System Pretest Score Original Item
Unable to display preview. Download preview PDF.
- 1.Baker, R.S., Roll, I., Corbett, A.T., Koedinger, K.R.: Do Performance Goals Lead Students to Game the System? In: Proceedings of the 12th International Conference on Artificial Intelligence and Education, pp. 57–64 (2005)Google Scholar
- 2.Beck, J.E., Jia, P., Mostow, J.: Automatically assessing oral reading fluency in a computer tutor that listens. Technology, Instruction, Cognition and Learning 2, 61–81 (2004)Google Scholar
- 3.Campione, J.C., Brown, A.L., Bryant, N.R.: Individual differences in learning and memory. In: Sternberg, R.J. (ed.) In Human abilities: An information-processing approach, pp. 103–126. W.H. Freeman, New York (1985)Google Scholar
- 4.Feng, M., Heffernan, N.T.: Informing Teachers Live about Student Learning: Reporting in the ASSISTment System. In: The 12th Annual Conference on Artificial Intelligence in Education Workshop on Usage Analysis in Learning Systems (2005)Google Scholar
- 5.Feng, M., Heffernan, N.T., Koedinger, K.R.: Addressing the Testing Challenge with a Web-Based E-Assessment System that Tutors as it Assesses. In: The 15th International World Wide Web Conference, Scotland (in press), http://web.cs.wpi.edu/~mfeng/pub/www06.pdf
- 7.Razzaq, L., Feng, M., Nuzzo-Jones, G., Heffernan, N.T., et al.: The ASSISTment Project: Blending Assessment and Assisting. In: Proceedings of the 12th Annual Conference on Artificial Intelligence in Education (2005)Google Scholar
- 8.Razzaq, L., Heffernan, N.T.: Scaffolding vs. Hints in the ASSISTment System. In: The 8th International Conference on Intelligent Tutoring Systems (in press), http://nth.wpi.edu/pubs_and_grants/ITS2006/Submissons/Leena/razzaq.doc
- 9.Singer, J.D., Willett, J.B.: Applied Longitudinal Data Analysis: Modelling Change and Occurrence. Oxford University Press, New York (2003)Google Scholar