Advertisement

Psychometrika

, Volume 75, Issue 2, pp 331–350 | Cite as

Estimating Difficulty from Polytomous Categorical Data

  • Javier RevueltaEmail author
Article

Abstract

A comprehensive analysis of difficulty for multiple-choice items requires information at different levels: the test, the items, and the alternatives. This paper introduces a new parameterization of the nominal categories model (NCM) for analyzing difficulty at these three levels. The new parameterization is referred to as the NE–NCM and is statistically equivalent to the NCM. The NE–NCM is applied to a sample of responses from a logical analysis test. The results suggest that the individuals execute a self-terminated response process that is mostly determined by working memory load.

Keywords

nested effects parameterization nominal categories model generalized logit-linear item response model identifiability polytomous item response theory 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B.N. Petrov & F. Csaki (Eds.), The second international symposium on information theory. Budapest: Akadmiai Kiado. Google Scholar
  2. Arendasy, M., & Sommer, M. (2005). The effect of different types of perceptual manipulations on the dimensionality of automatically generated figural matrices. Intelligence, 33, 307–324. CrossRefGoogle Scholar
  3. Bechger, T.M., Maris, G., Verstralen, H.H.F.M., & Verhelts, N.D. (2005). The Nedelsky model for multiple-choice items. In L.A. van der Ark, M.A. Croon & Sijtsma (Eds.), New developments in categorical data analysis for the social and behavioral sciences. Mahwah: Lawrence Erlbaum Associates. Google Scholar
  4. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In F.M. Lord & M.R. Novick (Eds.), Statistical theories of mental tests scores. Reading: Addison–Wesley. Google Scholar
  5. Bock, R.D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29–51. CrossRefGoogle Scholar
  6. Bock, R.D. (1975). Multivariate statistical methods in the behavioral sciences. New York: Wiley. Google Scholar
  7. Bock, R.D. (1997). The nominal categories model. In W.J. van der Linden & R.K. Hambleton (Eds.), Handbook of modern item response theory. New York: Springer. Google Scholar
  8. Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: application of an EM algorithm. Psychometrika, 46, 443–459. CrossRefGoogle Scholar
  9. Carpenter, P.A., Just, M.A., & Shell, P. (1990). What one intelligence test measures: a theoretical account of the processing in the Raven progressive matrices test. Psychological Review, 97(3), 404–431. CrossRefPubMedGoogle Scholar
  10. Haladyna, T. (2004). Developing and validating multiple-choice test items (3rd ed.). Mahwah: Lawrence Erlbaum Associates. Google Scholar
  11. Hambleton, R.K., & Swaminathan, H. (1985). Item response theory: principles and applications. Boston: Kluwer Academic. Google Scholar
  12. Hutchinson, T.P. (1991). Ability, partial information, guessing: statistical modelling applied to multiple-choice items. Adelaide: Rumsby Scientific Publishing. Google Scholar
  13. Maxwell, S.E., & Delaney, H.D. (1990). Designing experiments and analyzing data. Belmont: Wadsworth. Google Scholar
  14. Maydeu-Olivares, A., & Joe, H. (2006). Limited information goodness–of–fit testing in multidimensional contingency tables. Psychometrika, 71, 713–732. CrossRefGoogle Scholar
  15. Mislevy, R.J. (1984). Estimating latent distributions. Psychometrika, 49, 359–381. CrossRefGoogle Scholar
  16. Nocedal, J., & Wright, S.J. (2006). Numerical optimization. New York: Springer. Google Scholar
  17. Revuelta, J. (2004). Analysis of distractor difficulty in multiple-choice items. Psychometrika, 69, 217–234. CrossRefGoogle Scholar
  18. Revuelta, J. (2005). An item response model for nominal data based on the rising selection ratios criterion. Psychometrika, 70, 305–324. CrossRefGoogle Scholar
  19. Revuelta, J. (2008). The generalized logit-linear item response model for binary-designed items. Psychometrika, 73, 385–405. CrossRefGoogle Scholar
  20. Revuelta, J. (2009). Identifiability and equivalence of GLLIRM models. Psychometrika, 74, 257–272. CrossRefGoogle Scholar
  21. Rothenberg, T. (1971). Identification in parametric models. Econometrica, 39, 577–591. CrossRefGoogle Scholar
  22. San Martin, E., del Pino, G., & De Boeck, P. (2006). IRT models for ability-based guessing. Applied Psychological Measurement, 30, 183–203. CrossRefGoogle Scholar
  23. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464. CrossRefGoogle Scholar
  24. Sternberg, R.J. (2000). Handbook of intelligence. Cambridge: Cambridge University Press. Google Scholar
  25. Thissen, D. (1982). Marginal maximum likelihood estimation for the one parameter logistic model. Psychometrika, 47, 175–186. CrossRefGoogle Scholar
  26. Thissen, D., Chen, W.H., & Bock, R.D. (2003). Multilog 7.0. Scientific Software International. Lincolnwood, IL. Google Scholar
  27. Thissen, D., & Steinberg, L. (1984). A response model for multiple-choice items. Psychometrika, 49, 501–519. CrossRefGoogle Scholar
  28. Thissen, D., & Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51, 567–577. CrossRefGoogle Scholar
  29. Thissen, D., Steinberg, L., & Fitzpatrick, A.R. (1989). Multiple-choice items: the distractors are also part of the item. Journal of Educational Measurement, 26, 161–176. CrossRefGoogle Scholar
  30. Townsend, J.T., & Nozawa, G. (1997). Serial exhaustive models can violate the race model inequality: implications for architecture and capacity. Psychological Review, 104, 595–602. CrossRefGoogle Scholar

Copyright information

© The Psychometric Society 2010

Authors and Affiliations

  1. 1.Department of Social Psychology and MethodologyAutonoma University of MadridMadridSpain

Personalised recommendations