User Modeling and User-Adapted Interaction

, Volume 18, Issue 4, pp 349–382

A multifactor approach to student model evaluation

Authors

  • Michael V. Yudelson
    • Department of Biomedical InformaticsUniversity of Pittsburgh School of Medicine
    • School of Information SciencesUniversity of Pittsburgh
  • Olga P. Medvedeva
    • Department of Biomedical InformaticsUniversity of Pittsburgh School of Medicine
    • Department of Biomedical InformaticsUniversity of Pittsburgh School of Medicine
    • Intelligent Systems ProgramUniversity of Pittsburgh
    • Department of PathologyUniversity of Pittsburgh School of Medicine
    • UPMC Cancer Pavilion
Original Paper

DOI: 10.1007/s11257-007-9046-5

Cite this article as:
Yudelson, M.V., Medvedeva, O.P. & Crowley, R.S. User Model User-Adap Inter (2008) 18: 349. doi:10.1007/s11257-007-9046-5

Abstract

Creating student models for Intelligent Tutoring Systems (ITS) in novel domains is often a difficult task. In this study, we outline a multifactor approach to evaluating models that we developed in order to select an appropriate student model for our medical ITS. The combination of areas under the receiver-operator and precision-recall curves, with residual analysis, proved to be a useful and valid method for model selection. We improved on Bayesian Knowledge Tracing with models that treat help differently from mistakes, model all attempts, differentiate skill classes, and model forgetting. We discuss both the methodology we used and the insights we derived regarding student modeling in this novel domain.

Keywords

Student modelingIntelligent tutoring systemsKnowledge TracingMethodologyDecision theoryModel evaluationModel selectionIntelligent medical training systemsMachine learningProbabilistic modelsBayesian modelsHidden Markov Models

Copyright information

© Springer Science+Business Media B.V. 2008