Optimal Scores: An Alternative to Parametric Item Response Theory and Sum Scores
- 188 Downloads
The aim of this paper is to discuss nonparametric item response theory scores in terms of optimal scores as an alternative to parametric item response theory scores and sum scores. Optimal scores take advantage of the interaction between performance and item impact that is evident in most testing data. The theoretical arguments in favor of optimal scoring are supplemented with the results from simulation experiments, and the analysis of test data suggests that sum-scored tests would need to be longer than an optimally scored test in order to attain the same level of accuracy. Because optimal scoring is built on a nonparametric procedure, it also offers a flexible alternative for estimating item characteristic curves that can fit items that do not show good fit to item response theory models.
This research was funded by the Swedish Research Council Grant 2014-578.
- Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental tests (pp. 395–479). Reading, MA: Addison-Wesley.Google Scholar
- Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
- Ramsay, J. O. (1997). A functional approach to modeling test data. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 351–367). New York: Springer.Google Scholar
- Ramsay, J. O. (2000). TestGraf: A program for the graphical analysis of multiple choice test and questionnaire data [Computer software and manual]. Montreal: Department of Psychology, McGill University.Google Scholar
- Ramsay, J. O., & Silverman, B. W. (2002). Functional models for test items. In J. O. Ramsay & B. W. Silverman Applied functional data analysis. New York: Springer-Verlag.Google Scholar
- Ramsay, J. O., & Silverman, B. W. (2005). Functional data analysis. New York: Springer.Google Scholar
- Ramsay, J. O., & Wiberg, M. (2017b). Breaking through the sum scoring barrier. In L. A. van der Ark, M. Wiberg, S. A. Culpepper, J. A. Douglas, & W.-C. Wang (Eds.), Quantitative psychology—81st annual meeting of the psychometric society, Asheville, North Carolina, 2016 (pp. 151–158). New York: Springer.Google Scholar
- Rossi, N., Wang, X., & Ramsay, J. O. (2002). Nonparametric item response function estimates with the EM algorithm. Journal of the Behavioral and Educational Sciences, 27, 291–317.Google Scholar