Skip to main content
SpringerLink
Log in

Optimal Scores: An Alternative to Parametric Item Response Theory and Sum Scores

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

The aim of this paper is to discuss nonparametric item response theory scores in terms of optimal scores as an alternative to parametric item response theory scores and sum scores. Optimal scores take advantage of the interaction between performance and item impact that is evident in most testing data. The theoretical arguments in favor of optimal scoring are supplemented with the results from simulation experiments, and the analysis of test data suggests that sum-scored tests would need to be longer than an optimally scored test in order to attain the same level of accuracy. Because optimal scoring is built on a nonparametric procedure, it also offers a flexible alternative for estimating item characteristic curves that can fit items that do not show good fit to item response theory models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental tests (pp. 395–479). Reading, MA: Addison-Wesley.

    Google Scholar 

  • Lee, Y.-S. (2007). A comparison of methods for nonparametric estimation of item characteristic curves for binary items. Applied Psychological Measurement, 37, 121–134.

    Article  Google Scholar 

  • Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Mokken, R. J. (1997). Nonparametric models for dichotomous responses. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 351–367). New York: Springer.

    Chapter  Google Scholar 

  • Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56, 611–630.

    Article  Google Scholar 

  • Ramsay, J. O. (1997). A functional approach to modeling test data. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 351–367). New York: Springer.

    Google Scholar 

  • Ramsay, J. O. (2000). TestGraf: A program for the graphical analysis of multiple choice test and questionnaire data [Computer software and manual]. Montreal: Department of Psychology, McGill University.

    Google Scholar 

  • Ramsay, J. O., Hooker, G., & Graves, S. (2009). Functional data analysis in R and Matlab. New York: Springer.

    Book  Google Scholar 

  • Ramsay, J. O., & Silverman, B. W. (2002). Functional models for test items. In J. O. Ramsay & B. W. Silverman Applied functional data analysis. New York: Springer-Verlag.

  • Ramsay, J. O., & Silverman, B. W. (2005). Functional data analysis. New York: Springer.

    Book  Google Scholar 

  • Ramsay, J. O., & Wiberg, M. (2017a). A strategy for replacing sum scores. Journal of Educational and Behavioral Statistics, 42(3), 282–307.

    Article  Google Scholar 

  • Ramsay, J. O., & Wiberg, M. (2017b). Breaking through the sum scoring barrier. In L. A. van der Ark, M. Wiberg, S. A. Culpepper, J. A. Douglas, & W.-C. Wang (Eds.), Quantitative psychology—81st annual meeting of the psychometric society, Asheville, North Carolina, 2016 (pp. 151–158). New York: Springer.

    Google Scholar 

  • Rossi, N., Wang, X., & Ramsay, J. O. (2002). Nonparametric item response function estimates with the EM algorithm. Journal of the Behavioral and Educational Sciences, 27, 291–317.

    Google Scholar 

  • Wiberg, M., & Bränberg, K. (2015). Kernel equating under the non-equivalent groups with covariates design. Applied Psychological Measurement, 39(5), 349–361.

    Article  PubMed  PubMed Central  Google Scholar 

  • Woods, C. M. (2006). Ramsay-curve item response theory (RC-IRT) to detect and correct for nonnormal latent variables. Psychological Methods, 11, 253–270.

    Article  PubMed  Google Scholar 

  • Woods, C. M., & Thissen, D. (2006). Item response theory with estimation of the latent population distribution using spline-based densities. Psychometrika, 71, 281–301.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This research was funded by the Swedish Research Council Grant 2014-578.

Author information

Authors and Affiliations

  1. USBE, Department of Statistics, Umeå University, 90187 , Umeå, Sweden

    Marie Wiberg

  2. Department of Psychology, McGill University, Montreal, Canada

    James O. Ramsay

  3. Department of Mathematics and Statistics, McGill University, Montreal, Canada

    Juan Li

Authors
  1. Marie Wiberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James O. Ramsay
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marie Wiberg.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (zip 42 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wiberg, M., Ramsay, J.O. & Li, J. Optimal Scores: An Alternative to Parametric Item Response Theory and Sum Scores. Psychometrika 84, 310–322 (2019). https://doi.org/10.1007/s11336-018-9639-4

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-018-9639-4

Keywords

Search

Navigation