Learning Factors Analysis – A General Method for Cognitive Model Evaluation and Improvement

  • Hao Cen
  • Kenneth Koedinger
  • Brian Junker
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4053)


A cognitive model is a set of production rules or skills encoded in intelligent tutors to model how students solve problems. It is usually generated by brainstorming and iterative refinement between subject experts, cognitive scientists and programmers. In this paper we propose a semi-automated method for improving a cognitive model called Learning Factors Analysis that combines a statistical model, human expertise and a combinatorial search. We use this method to evaluate an existing cognitive model and to generate and evaluate alternative models. We present improved cognitive models and make suggestions for improving the intelligent tutor based on those models.


Success Probability Cognitive Model Production Rule Intelligent Tutor System Final Probability 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baffes, P., Mooney, R.J.: A Novel Application of Theory Refinement to Student Modeling. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI 1996), Portland OR, pp. 403–408 (1996)Google Scholar
  2. 2.
    Barnes, T.: The Q-matrix Method: Mining Student Response Data for Knowledge. In: Proceedings of AAAI 2005 Educational Data Mining Workshop (2005)Google Scholar
  3. 3.
    Cen, H., Koedinger, K., Junker, B.: Automating Cognitive Model Improvement by A*Search and Logistic Regression. In: Proceedings of AAAI 2005 Educational Data Mining Workshop (2005)Google Scholar
  4. 4.
    Corbett, A.T., Anderson, J.R., O’Brien, A.T.: Student Modelling in the ACT Programming Tutor. In: Cognitively Diagnostic Assessment. Erlbaum, Hillsdale (1995)Google Scholar
  5. 5.
    Corbett, A.T., Anderson, J.R.: Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction 4, 253–278 (1995)CrossRefGoogle Scholar
  6. 6.
    Croteau, E.A., Heffernan, N.T., Koedinger, K.R.: Why are algebra word problems difficult? Using tutorial log files and the power law of learning to select the best fitting cognitive model. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 240–250. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Draney, K., Pirolli, P., Wilson, M.: A Measurement Model for a Complex Cognitive Skill. In: Cognitively Diagnostic Assessment. Erlbaum, Hillsdale (1995)Google Scholar
  8. 8.
    Junker, B.W., Koedinger, K., Trottini, M.: Finding Improvements in Student Models for Intelligent Tutoring Systems via Variable Selection for a Linear Logistic Test Model. Presented at annual meeting of Psychometric Society Vancouver BC (2000)Google Scholar
  9. 9.
  10. 10.
    Koedinger, K.R., Mathan, S.: Distinguishing Qualitatively Different Kinds of Leaning Using Log Files and Learning Curves. In: Proc. of Intelligent Tutoring Systems (2004)Google Scholar
  11. 11.
    Koedinger, K.R., Nathan, M.J.: The Real Story Behind Story Problems: Effects Of Representations On Quantitative Reasoning. The Journal of the Learning Sciences 13(2), 129–164 (2004)CrossRefGoogle Scholar
  12. 12.
    Koedinger, K.R., Anderson, J.R., Hadley, W.H., Mark, M.A.: Intelligent Tutoring Goes to School in the Big City. In: Proceedings of the 7th World Conference on Art, Intelligence and Education, AACE (1995)Google Scholar
  13. 13.
    Newell, A., Rosenbloom, P.: Mechanisms of Skill Acquisition and the Law of Practice. In: Anderson, J. (ed.) Cognitive Skills and Their Acquisition. Erlbaum, Hillsdale (1981)Google Scholar
  14. 14.
    Russell, S., Norvig, P.: Artificial Intelligence, 2nd edn. Prentice-Hall, Englewood Cliffs (2003)Google Scholar
  15. 15.
    Tatsuoka, K.: Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement 20(4), 345–354 (1983)CrossRefGoogle Scholar
  16. 16.
    Ur, S., VanLehn, K.: STEPS: A Simulated, Tutorable Physics Student. Journal of Artificial Intelligence in Education 6(4), 405–437 (1995)Google Scholar
  17. 17.
    Wasserman, L.: All of Statistics: A Concise Course in Statistical Inference. Springer, Heidelberg (2004)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Hao Cen
    • 1
  • Kenneth Koedinger
    • 1
  • Brian Junker
    • 1
  1. 1.Carnegie Mellon UniversityForbes, PittsburghUSA

Personalised recommendations