User Modeling and User-Adapted Interaction

, Volume 18, Issue 4, pp 349–382

A multifactor approach to student model evaluation

  • Michael V. Yudelson
  • Olga P. Medvedeva
  • Rebecca S. Crowley
Original Paper

DOI: 10.1007/s11257-007-9046-5

Cite this article as:
Yudelson, M.V., Medvedeva, O.P. & Crowley, R.S. User Model User-Adap Inter (2008) 18: 349. doi:10.1007/s11257-007-9046-5


Creating student models for Intelligent Tutoring Systems (ITS) in novel domains is often a difficult task. In this study, we outline a multifactor approach to evaluating models that we developed in order to select an appropriate student model for our medical ITS. The combination of areas under the receiver-operator and precision-recall curves, with residual analysis, proved to be a useful and valid method for model selection. We improved on Bayesian Knowledge Tracing with models that treat help differently from mistakes, model all attempts, differentiate skill classes, and model forgetting. We discuss both the methodology we used and the insights we derived regarding student modeling in this novel domain.


Student modelingIntelligent tutoring systemsKnowledge TracingMethodologyDecision theoryModel evaluationModel selectionIntelligent medical training systemsMachine learningProbabilistic modelsBayesian modelsHidden Markov Models

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  • Michael V. Yudelson
    • 1
    • 2
  • Olga P. Medvedeva
    • 1
  • Rebecca S. Crowley
    • 1
    • 3
    • 4
    • 5
  1. 1.Department of Biomedical InformaticsUniversity of Pittsburgh School of MedicinePittsburghUSA
  2. 2.School of Information SciencesUniversity of PittsburghPittsburghUSA
  3. 3.Intelligent Systems ProgramUniversity of PittsburghPittsburghUSA
  4. 4.Department of PathologyUniversity of Pittsburgh School of MedicinePittsburghUSA
  5. 5.UPMC Cancer PavilionPittsburghUSA