Psychometrika

, Volume 54, Issue 3, pp 427–450

Weighted likelihood estimation of ability in item response theory

  • Thomas A. Warm
Article

Abstract

Applications of item response theory, which depend upon its parameter invariance property, require that parameter estimates be unbiased. A new method, weighted likelihood estimation (WLE), is derived, and proved to be less biased than maximum likelihood estimation (MLE) with the same asymptotic variance and normal distribution. WLE removes the first order bias term from MLE. Two Monte Carlo studies compare WLE with MLE and Bayesian modal estimation (BME) of ability in conventional tests and tailored tests, assuming the item parameters are known constants. The Monte Carlo studies favor WLE over MLE and BME on several criteria over a wide range of the ability scale.

Key words

maximum likelihood estimation unbiased estimation statistical bias Bayesian modal estimation item response theory tailored testing adaptive testing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Akaike, H. (1978). A New Look At The Bayes Procedure.Biometrika, 65(1), 53–59.Google Scholar
  2. Anderson, J. A., & Richardson, S. C. (1979). Logistic discrimination and bias correction in maximum likelihood estimation.Technometrics, 21(1), 71–78.Google Scholar
  3. Andrews, D. F., Bickel, P. J., Hampel, F. R., Huber, P. J., Rogers, W. H., & Tukey, J. W. (1972).Robust estimates of location. Princeton, NJ: Princeton University Press.Google Scholar
  4. Baker, F. B. (1984). Ability metric transformations involved in vertical equating under item response theory.Applied Psychological Measurement, 8(3), 261–271.Google Scholar
  5. Barndorff-Nielsen, O. (1983). On a formula for the distribution of the maximum likelihood estimator.Biometrika, 70(2), 343–365.Google Scholar
  6. Basu, A. P., & Ghosh, J. K. (1980). Asymptotic properties of a solution to the likelihood equation with life-testing applications,Journal of the American Statistical Association, 75(370), 410–414.Google Scholar
  7. Bock, R. D. (1983). The discrete Bayesian. In H. Wainer & S. Messick (Eds.),Principals of modern psychological measurement. A festschrift for Frederick M. Lord (pp. 103–115). NJ: Lawrence Erlbaum.Google Scholar
  8. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm.Psychometrika, 46, 443–459.Google Scholar
  9. Bock, R. D., & Mislevy, R. J. (1981).Biweight estimates of latent ability. Unpublished manuscript.Google Scholar
  10. Bradley, R. A., & Gart, J. J. (1962). The asymptotic properties of ML estimators when sampling from associated populations.Biometrika, 49, 205–214.Google Scholar
  11. Cox, D. R., & Hinkley, D. V. (1974).Theoretical statistics. New York: Chapman & Hall. (Distributed by Halsted Press, New York)Google Scholar
  12. Efron, B., & Hinkley, D. V. (1978). Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information.Biometrika, 65(3), 457–487.Google Scholar
  13. Goldstein, H. (1980). Dimensionality, bias, independence and measurement scale problems in latent trait test score models.British Journal of Mathematical and Statistical Psychology, 33, 234–246.Google Scholar
  14. Hoadley, B. (1971). Asymptotic properties of maximum likelihood estimators for the independent not identically distributed case.Annals of Mathematical Statistics, 42(6), 1977–1991.Google Scholar
  15. Jeffreys, H. (1961). Theory of Probability (3rd ed.). Oxford: Clarendon.Google Scholar
  16. Jones, D. H. (1982).Redescending M-type estimators of latent ability (Program Statistics Tech. Rep. No. 82-30). Princeton, NJ: Educational Testing Service.Google Scholar
  17. Kendall, M. G., & Stuart, A. (1973).The advanced theory of statistics (Vol. 2). New York: Hafner.Google Scholar
  18. Kendall, M. G., & Stuart, A. (1977).The advanced theory of statistics (Vol. 1). London: Charles Griffin.Google Scholar
  19. Lord, F. M. (1980).Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  20. Lord, F. M. (1983a). Unbiased estimators of ability parameters, of their variance, and of their parallel-forms reliability.Psychometrika, 48, 233–245.Google Scholar
  21. Lord, F. M. (1983b).Memorandum for: Ms. Stocking, Ms. M. Wang, Ms. Wingersky. Subject: Sampling variance and bias for MLE and Bayesian estimation of α. August 26, 1983 (Internal Memorandum) Princeton, NJ: Educational Testing Service.Google Scholar
  22. Lord, F. M. (1984).Maximum likelihood and Bayesian parameter estimation in item response theory (Research Rep. No. RR-84-30-ONR). Princeton, NJ: Educational Testing Service.Google Scholar
  23. Lord, F. M., & Novick, M. (1967).Statistical theories of mental test scores. Reading, MA: Addison-Wesley.Google Scholar
  24. Owen, R. J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing.Journal of the American Statistical Association, 70, 351–356.Google Scholar
  25. Quenouille, M. H. (1956). Notes on Bias in Estimation.Biometrika, 43, 353–360.Google Scholar
  26. Samejima, F. (1980).Is Bayesian estimation proper for estimating the individual's ability (Research Rep. 80-3). Knoxville, TN: University of Tennessee, Department of Psychology.Google Scholar
  27. Schaefer, R. L. (1983). Bias correction in maximum likelihood logistic regression.Statistics in Medicine, 2, 71–78.Google Scholar
  28. Wainer, H., & Wright, B. D. (1980). Robust estimation of ability in the Rasch model.Psychometrika, 45, 373–391.Google Scholar
  29. Weiss, D. J., & McBride, J. R. (1984). Bias and information of Bayesian adaptive testing.Applied Psychological Measurement, 8(3), 273–285.Google Scholar

Copyright information

© The Psychometric Society 1989

Authors and Affiliations

  • Thomas A. Warm
    • 1
  1. 1.FAA Academy AAC-934, Mike Monroney Aeronautical CenterOklahoma City

Personalised recommendations