Psychometrika

, Volume 69, Issue 2, pp 217–234 | Cite as

Analysis of distractor difficulty in multiple-choice items

Theory And Methods

Abstract

Two psychometric models are presented for evaluating the difficulty of the distractors in multiple-choice items. They are based on the criterion of rising distractor selection ratios, which facilitates interpretation of the subject and item parameters. Statistical inferential tools are developed in a Bayesian framework: modal a posteriori estimation by application of an EM algorithm and model evaluation by monitoring posterior predictive replications of the data matrix. An educational example with real data is included to exemplify the application of the models and compare them with the nominal categories model.

Key words

multiple-choice items item-response models EM algorithm MCMC simulation Posterior Predictive Checks 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abramowitz, M., & Stegun, I.A. (1965).Handbook of Mathematical Functions. New York: Dover Publications Inc.Google Scholar
  2. Bock, R.D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories.Psychometrika, 37, 29–51.Google Scholar
  3. Bock, R.D. (1997). The nominal categories model. In W.J. van der Linden & R.K. Hambleton (Eds.),Handbook of Modern Item Response Theory. New York: Springer.Google Scholar
  4. Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm.Psychometrika, 46, 443–459.Google Scholar
  5. Bock, R.D., & Jones, L.V. (1968).The Measurement and Prediction of Judgement and Choice. San Francisco: Holden-Day.Google Scholar
  6. Brooks, S., & Gelman, A. (1998). General methods for monitoring convergence of iterative simulation.Journal of Computational and Graphical Statistics, 7, 434–455.Google Scholar
  7. Chib, S., & Greenberg, E. (1995). Understanding the Metropolis-Hastings algorithm.The American Statistician, 49, 327–335.Google Scholar
  8. Dempster, A.P., Laird, N.M., & Rubin D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion).Journal of the Royal Statistical Society, series B, 39, 1–38.Google Scholar
  9. Fahrmeir, L., & Tutz, G. (1994).Multivariate Statistical Modelling Based on Generalized Linear Models. New York: Springer-Verlag.Google Scholar
  10. Gelman, A., Carlin, J.B., Stern, H.S., & Rubin, D.B. (1995).Bayesian Data Analysis. London: Chapman and Hall.Google Scholar
  11. Gelman, A., Meng, X.L., & Stern, H.S. (1996). Posterior predictive assessment of model fitness via realized discrepancies.Statistica Sinica, 6, 733–807.Google Scholar
  12. Gill, P.E., Murray, W., & Wright, M.H. (1981).Practical Optimization. London: Academic Press.Google Scholar
  13. Goodman, L.A., & Kruskal, W.H. (1979).Measures of Associations for Cross Classifications. New York: Springer-Verlag.Google Scholar
  14. Hoijtink, H., & Molenaar, I. W. (1997). A multidimensional item response model: Constrained latent class analysis using the Gibbs sampler and Posterior Predictive Checks.Psychometrika, 62, 171–189.Google Scholar
  15. Louis, T.A. (1982). Finding the observed information matrix when using the EM algorithm.Journal of the Royal Statistical Society, series B, 44, 226–233.Google Scholar
  16. Love, T.E. (1997). Distractor selection ratios.Psychometrika, 62, 51–62.Google Scholar
  17. Luce, R.D. (1959).Individual Choice Behavior: A Theoretical Analysis. New York: Wiley.Google Scholar
  18. McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. In P. Zarembka.Frontiers in Econometrics. New York: Academic Press.Google Scholar
  19. Mellenberg, G.J. (1995). Conceptual notes on models for polytomous item responses.Applied Psychological Measurement, 19, 91–100.Google Scholar
  20. Nedelsky, L. (1954). Absolute grading standards for objective tests.Educational and Psychological Measurement, 14, 3–19.Google Scholar
  21. Press, W.H., Teukolsky, S.A., Veterling, W.T., & Flannery, B.P. (1992).Numerical Recipes in C, 2nd ed. New York: Cambridge University Press.Google Scholar
  22. Ramsay, J.O. (1991) Kernel smoothing approaches to nonparametric item characteristic curve estimation.Psychometrika, 56, 611–630.Google Scholar
  23. Ramsay, J.O. (2000).TestGraf. A Program for the Graphical Analysis of Multiple Choice Test and Questionnaire Data. McGill University. Available on-line at http://www.psych.mcgill.ca/faculty/ramsay.htmlGoogle Scholar
  24. Tierney, L. (1994). Markov chains for exploring posterior distributions.The Annals of Statistics, 22, 1701–1762.Google Scholar
  25. van Onna, M.J.H. (2002). Bayesian estimation and model selection in ordered latent class models for polytomous items.Psychometrika, 67, 519–538.Google Scholar

Copyright information

© The Psychometric Society 2004

Authors and Affiliations

  1. 1.Departamento de Psicologia Social y MetodologiaUniversidad Autónoma de Madrid, CantoblancoMadridSpain

Personalised recommendations