Advertisement

A Matrix Factorization Method for Mapping Items to Skills and for Enhancing Expert-Based Q-Matrices

  • Michel C. Desmarais
  • Rhouma Naceur
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7926)

Abstract

Uncovering the right skills behind question items is a difficult task. It requires a thorough understanding of the subject matter and of the cognitive factors that determine student performance. The skills definition, and the mapping of item to skills, require the involvement of experts. We investigate means to assist experts for this task by using a data driven, matrix factorization approach. The two mappings of items to skills, the expert on one side and the matrix factorization on the other, are compared in terms of discrepancies, and in terms of their performance when used in a linear model of skills assessment and item outcome prediction. Visual analysis shows a relatively similar pattern between the expert and the factorized mappings, although differences arise. The prediction comparison shows the factorization approach performs slightly better than the original expert Q-matrix, giving supporting evidence to the belief that the factorization mapping is valid. Implications for the use of the factorization to design better item to skills mapping are discussed.

Keywords

student models skills assessment alternating least squares matrix factorization latent skills cognitive modeling 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Barnes, T.: Novel derivation and application of skill matrices: The Q-matrix method. Handbook on Educational Data Mining (2010)Google Scholar
  2. 2.
    Cetintas, S., Si, L., Xin, Y.P., Hord, C.: Predicting correctness of problem solving in ITS with a temporal collaborative filtering approach. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010, Part I. LNCS, vol. 6094, pp. 15–24. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    De La Torre, J.: An empirically based method of q-matrix validation for the dina model: Development and applications. Journal of Educational Measurement 45(4), 343–362 (2008)CrossRefGoogle Scholar
  4. 4.
    DeCarlo, L.T.: On the Analysis of Fraction Subtraction Data: The DINA Model, Classification, Latent Class Sizes, and the Q-Matrix. Applied Psychological Measurement 35, 8–26 (2011)CrossRefGoogle Scholar
  5. 5.
    Desmarais, M.C.: Conditions for effectively deriving a q-matrix from data with non-negative matrix factorization. In: Conati, C., Ventura, S., Calders, T., Pechenizkiy, M. (eds.) 4th International Conference on Educational Data Mining, EDM 2011, Eindhoven, Netherlands, June 6-8, pp. 41–50 (2011)Google Scholar
  6. 6.
    Desmarais, M.C.: Mapping question items to skills with non-negative matrix factorization. ACM KDD-Explorations 13(2), 30–36 (2011)CrossRefGoogle Scholar
  7. 7.
    Desmarais, M.C., Beheshti, B., Naceur, R.: Item to skills mapping: Deriving a conjunctive Q-matrix from data. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) ITS 2012. LNCS, vol. 7315, pp. 454–463. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  8. 8.
    Koedinger, K.R., McLaughlin, E.A., Stamper, J.C.: Automated student model improvement. In: Proceedings of the 5th International Conference on Educational Data Mining, pp. 17–24 (2012)Google Scholar
  9. 9.
    Lan, A.S., Waters, A.E., Studer, C., Baraniuk, R.G.: Sparse factor analysis for learning and content analytics. arXiv preprint arXiv:1303.5685 (2013)Google Scholar
  10. 10.
    Li, N., Cohen, W.W., Matsuda, N., Koedinger, K.R.: A machine learning approach for automatic student model discovery. In: Proceedings of the 4th International Conference on Educational Data Mining, pp. 31–40 (2011)Google Scholar
  11. 11.
    Liu, J., Xu, G., Ying, Z.: Data-driven learning of q-matrix. Applied Psychological Measurement 36(7), 548–564 (2012), http://apm.sagepub.com/content/36/7/548.abstract CrossRefGoogle Scholar
  12. 12.
    Robitzsch, A., Kiefer, T., George, A., Uenlue, A., Robitzsch, M.: Package CDM (2012), http://cran.r-project.org/web/packages/CDM/index.html
  13. 13.
    Stamper, J.C., Barnes, T., Croy, M.J.: Extracting student models for intelligent tutoring systems. In: AAAI 2007, pp. 1900–1901. AAAI Press (2007)Google Scholar
  14. 14.
    Tatsuoka, K.K.: Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement 20, 345–354 (1983)CrossRefGoogle Scholar
  15. 15.
    Tatsuoka, K.: Cognitive Assessment: An Introduction to the Rule Space Method. Routledge Academic (2009)Google Scholar
  16. 16.
    Tatsuoka, K.: Analysis of errors in fraction addition and subtraction problems. Computer-based Education Research Laboratory, University of Illinois (1984)Google Scholar
  17. 17.
    Thai-Nghe, N., Drumond, L., Horváth, T., Nanopoulos, A., Schmidt-Thieme, L.: Matrix and tensor factorization for predicting student performance. In: Verbraeck, A., Helfert, M., Cordeiro, J., Shishkov, B. (eds.) CSEDU 2011 - Proceedings of the 3rd International Conference on Computer Supported Education, Noordwijkerhout, Netherlands, May 6-8, vol. 1, pp. 69–78. SciTePress (2011)Google Scholar
  18. 18.
    Thai-Nghe, N., Horváth, T., Schmidt-Thieme, L.: Factorization models for forecasting student performance. In: Conati, C., Ventura, S., Pechenizkiy, M., Calders, T. (eds.) Proceedings of EDM 2011, The 4th International Conference on Educational Data Mining, Eindhoven, Netherlands, July 6-8, pp. 11–20 (2011), http://www.educationaldatamining.org
  19. 19.
    Toscher, A., Jahrer, M.: Collaborative filtering applied to educational data mining. Tech. rep., KDD Cup 2010: Improving Cognitive Models with Educational Data Mining (2010)Google Scholar
  20. 20.
    Winters, T.: Educational Data Mining: Collection and Analysis of Score Matrices for Outcomes-Based Assessment. Ph.D. thesis, University of California Riverside (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Michel C. Desmarais
    • 1
  • Rhouma Naceur
    • 1
  1. 1.École Polytechnique de MontréalMontréalCanada

Personalised recommendations