Adaptive Testing Using a General Diagnostic Model
In online learning platforms such as MOOCs, computerized assessment needs to be optimized in order to prevent boredom and dropout of learners. Indeed, they should spend as little time as possible in tests and still receive valuable feedback. It is actually possible to reduce the number of questions for the same accuracy with computerized adaptive testing (CAT): asking the next question according to the past performance of the examinee. CAT algorithms are divided in two categories: summative CATs, that measure the level of examinees, and formative CATs, that provide feedback to the examinees at the end of the test by specifying which knowledge components need further work. In this paper, we formalize the problem of test-size reduction by predicting student performance, and propose a new hybrid CAT algorithm GenMA based on the general diagnostic model, that is both summative and formative. Using real datasets, we compare our model to popular CAT models and show that GenMA achieves better accuracy while using fewer questions than the existing models.
This work is supported by the Paris-Saclay Institut de la Société Numérique funded by the IDEX Paris-Saclay, ANR-11-IDEX-0003-02.
- 1.Zernike, K.: Obama administration calls for limits on testing in schools (2015)Google Scholar
- 2.Council, G.M.A.: Profile of GMAT Candidates - Executive Summary (2013)Google Scholar
- 3.Huebner, A.: An overview of recent developments in cognitive diagnostic computer adaptive assessments. Pract. Assess. Res. Eval. 15(3), n3 (2010)Google Scholar
- 4.Yan, D., von Davier, A.A., Lewis, C.: Computerized multistage testing (2014)Google Scholar
- 6.Davier, M.: A general diagnostic model applied to language testing data. ETS Research Report Series, p. i-35 (2005)Google Scholar
- 8.Bergner, Y., Droschler, S., Kortemeyer, G., Rayyan, S., Seaton, D., Pritchard, D.E.: Model-based collaborative filtering analysis of student response data: machine-learning item response theory. In: International Educational Data Mining Society (2012)Google Scholar
- 11.Xu, X., Chang, H., Douglas, J.: A simulation study to compare CAT strategies for cognitive diagnosis. In: Annual Meeting of the American Educational Research Association, Chicago (2003)Google Scholar
- 17.Rupp, A., Levy, R., Dicerbo, K.E., Sweet, S.J., Crawford, A.V., Calico, T., Benson, M., Fay, D., Kunze, K.L., Mislevy, R.J.: Putting ECD into practice: the interplay of theory and data in evidence models within a digital learning environment. JEDM-J. Educ. Data Min. 4, 49–110 (2012)Google Scholar
- 19.Lynch, D., Howlin, C.P.: Real world usage of an adaptive testing algorithm to uncover latent knowledge (2014)Google Scholar
- 20.Mandin, S., Guin, N.: Basing learner modelling on an ontology of knowledge and skills. In: 2014 IEEE 14th International Conference on Advanced learning technologies (iCALT), pp. 321–323. IEEE (2014)Google Scholar
- 21.Kickmeier-Rust, M.D., Albert, D.: Competence-based knowledge space theory. In: Measuring and Visualizing Learning in the Information-Rich Classroom, p. 109 (2015)Google Scholar