High Speed High Stakes Scoring Rule
In this paper we will present the results of a three year subsidized research project investigating the performance of a new scoring rule for digital assessment. The scoring rule incorporates response time and accuracy in an adaptive environment. The project aimed to assess the validity and reliability of the ability estimations generated with the new scoring rule. It was also assessed whether the scoring rule was vulnerable for individual differences. Results show a strong validity and reliability in several studies within different domains: e.g. math, statistics and chess. We found no individual differences in the performance of the HSHS scoring rule for risk taking behavior and performance anxiety, nor did we find any performance differences for gender.
Keywords“computer adaptive testing” “speed accuracy trade-off” “scoring rule” “digital assessment” validity reliability CAT DIF
Unable to display preview. Download preview PDF.
- 1.Barkhof, J., Bekker, T., Bersma, M., Groenendijk, E., Maza, S.: Oefening Baart Kunst (unpublished research report). University of Amsterdam, Netherlands (2013)Google Scholar
- 6.Holzinger, K.J.: On Scoring Multiple Response Tests. Journal of Educational Measurement 15, 445–447 (1924)Google Scholar
- 7.Jansen, B.R.J., Louwerse, J., Straatemeier, M., Van der Ven, S.H.G., Klinkenberg, S., Van der Maas, H.L.J.: The influence of practicing maths with a computer-adaptive program on math anxiety, perceived math competence, and math performance. Learning and Individual Differences 24, 190–197 (2013)CrossRefGoogle Scholar
- 9.Klinkenberg, S., Van der Maas, H.L.J.: A dynamic paired comparison based computer adaptive testing method. Unpublished manuscript (2013)Google Scholar
- 12.Özen, S., Pronk, A., Sanchez Maceiras, S., Stel, N., Van Wersch, T.: De Invloed van de HSHS scoreregel op het Meten van Werkelijke Vaardigheid (Unpublished research report). University of Amsterdam, Netherlands (2012)Google Scholar
- 18.Van der Maas, H.L.J., Wagenmakers, E.J.: The Amsterdam Chess Test: a psychometric analysis of chess expertise. American Journal of Psychology 118, 29–60 (2005)Google Scholar