Predicting State Test Scores Better with Intelligent Tutoring Systems: Developing Metrics to Measure Assistance Required

  • Mingyu Feng
  • Neil T. Heffernan
  • Kenneth R. Koedinger
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4053)


The ASSISTment system was used by over 600 students in 2004-05 school year as part of their math class. While in [7] we reported student learning within the ASSISTment system, in this paper we focus on the assessment aspect. Our approach is to use data that the system collected through a year to tracking student learning and thus estimate their performance on a high-stake state test (MCAS) at the end of the year. Because our system is an intelligent tutoring system, we are able to log how much assistance students needed to solve problems (how many hints students requested and how many attempts they had to make). In this paper, our goal is to determine if the models we built by taking the assistance information into account could predict students’ test scores better. We present some positive evidence that shows our goal is achieved.


ASSISTment System Median Absolute Difference Intelligent Tutor System Pretest Score Original Item 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baker, R.S., Roll, I., Corbett, A.T., Koedinger, K.R.: Do Performance Goals Lead Students to Game the System? In: Proceedings of the 12th International Conference on Artificial Intelligence and Education, pp. 57–64 (2005)Google Scholar
  2. 2.
    Beck, J.E., Jia, P., Mostow, J.: Automatically assessing oral reading fluency in a computer tutor that listens. Technology, Instruction, Cognition and Learning 2, 61–81 (2004)Google Scholar
  3. 3.
    Campione, J.C., Brown, A.L., Bryant, N.R.: Individual differences in learning and memory. In: Sternberg, R.J. (ed.) In Human abilities: An information-processing approach, pp. 103–126. W.H. Freeman, New York (1985)Google Scholar
  4. 4.
    Feng, M., Heffernan, N.T.: Informing Teachers Live about Student Learning: Reporting in the ASSISTment System. In: The 12th Annual Conference on Artificial Intelligence in Education Workshop on Usage Analysis in Learning Systems (2005)Google Scholar
  5. 5.
    Feng, M., Heffernan, N.T., Koedinger, K.R.: Addressing the Testing Challenge with a Web-Based E-Assessment System that Tutors as it Assesses. In: The 15th International World Wide Web Conference, Scotland (in press),
  6. 6.
    Grigorenko, E.L., Sternberg, R.J.: Dynamic Testing. Psychological Bulletin 124, 75–111 (1998)CrossRefGoogle Scholar
  7. 7.
    Razzaq, L., Feng, M., Nuzzo-Jones, G., Heffernan, N.T., et al.: The ASSISTment Project: Blending Assessment and Assisting. In: Proceedings of the 12th Annual Conference on Artificial Intelligence in Education (2005)Google Scholar
  8. 8.
    Razzaq, L., Heffernan, N.T.: Scaffolding vs. Hints in the ASSISTment System. In: The 8th International Conference on Intelligent Tutoring Systems (in press),
  9. 9.
    Singer, J.D., Willett, J.B.: Applied Longitudinal Data Analysis: Modelling Change and Occurrence. Oxford University Press, New York (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Mingyu Feng
    • 1
  • Neil T. Heffernan
    • 1
  • Kenneth R. Koedinger
    • 2
  1. 1.Computer Science DepartmentWorcester Polytechnic InstituteWorcesterUSA
  2. 2.Human-Computer Interaction InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations