Human-Machine Student Model Discovery and Improvement Using DataShop

  • John C. Stamper
  • Kenneth R. Koedinger
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6738)


We show how data visualization and modeling tools can be used with human input to improve student models. We present strategies for discovering potential flaws in existing student models and use them to identify improvements in a Geometry model. A key discovery was that the student model should distinguish problem steps requiring problem decomposition planning and execution from problem steps requiring just execution of problem decomposition plans. This change to the student model better fits student data not only in the original data set, but also in two other data sets from different sets of students. We also show how such student model changes can be used to modify a tutoring system, not only in terms of the usual student model effects on the tutor’s problem selection, but also in driving the creation of new problems and hint messages.


data mining machine learning cognitive modeling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cen, H., Koedinger, K.R., Junker, B.: Learning Factors Analysis – A General Method for Cognitive Model Evaluation and Improvement. In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 164–175. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  2. 2.
    Clark, R.E., Feldon, D., van Merriënboer, J., Yates, K., Early, S.: Cognitive task analysis. In: Spector, J., Merrill, M., van Merriënboer, J., Driscoll, M. (eds.) Handbook of Research on Educational Communications and Technology, Mahwah, NJ, pp. 577–593 (2007)Google Scholar
  3. 3.
    Collins, M., Dasgupta, S., Schapire, R.: A generalization of PCA to the exponential family. In: Procs. of the 14th Conf. on Neural Info. Processing Systems, NIPS (2001)Google Scholar
  4. 4.
    Corbett, A.T., Anderson, J.R.: Knowledge tracing: Modeling the acquisition of Procedural knowledge. User Modeling and User-Adapted Interaction, 253–278 (1995)Google Scholar
  5. 5.
    Heffernan, N., Koedinger, K.: A developmental model for algebra symbolization: The results of a difficulty factors assessment. In: Gernsbacher, M.A., Derry, S.J. (eds.) Procs. of the 20th Annual Conf. of the Cognitive Science Society, Mahwah,NJ, pp. 484–489 (1998)Google Scholar
  6. 6.
    Koedinger, K.R., Baker, R.S.J.d., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J.: A Data Repository for the EDM commuity: The PSLC DataShop. In: Romero, V., Pechenizkiy, B. (eds.) Handbook of Educational Data Mining. CRC Press, Boca Raton (2010)Google Scholar
  7. 7.
    Koedinger, K., McLaughlin, E.: Seeing language learning inside the math: Cognitive analysis yields transfer. In: Procs. of the 32nd Ann. Conf. of the Cogitive Science Society (2010)Google Scholar
  8. 8.
    Koedinger, K.R., Nathan, M.J.: The real story behind story problems: Effects of representations on quantitative reasoning. The Jrnl of the Learning Sciences 13(2), 129–164 (2004)CrossRefGoogle Scholar
  9. 9.
    Lee, R.L.: Cognitive task analysis: A meta-analysis of comparative studies. Unpublished doctoral dissertation, University of Southern California, Los Angeles, CA (2003)Google Scholar
  10. 10.
    Tatsuoka, K.K.: Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement 20, 345–354 (1983)CrossRefGoogle Scholar
  11. 11.
    VanLehn, K.: The behavior of tutoring systems. Intl Jrnl of AIED 16, 227–265 (2006)Google Scholar
  12. 12.
    Villano, M.: Probabilistic student models: Bayesian Belief Networks and Knowledge Space Theory. In: Procs. of the 2nd International Conference on ITS, pp. 491–498. Springer, Heidelberg (1992)Google Scholar
  13. 13.
    Wilson, M., de Boeck, P.: Descriptive and explanatory item response models. In: de Boeck, P., Wilson, M. (eds.) Explanatory Item Response Models, pp. 43–74. Springer, Heidelberg (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • John C. Stamper
    • 1
  • Kenneth R. Koedinger
    • 1
  1. 1.Human-Computer Interaction InstituteCarnegie Mellon UniversityUSA

Personalised recommendations