Abstract
We introduce two simple empirical approximate Bayes estimators (EABEs)—\(\widetilde{d}_N (x)\) and\(\widetilde\delta _N (x)\)—for estimating domain scores under binomial and hypergeometric distributions, respectively. Both EABEs (derived from corresponding marginal distributions of observed test scorex without relying on knowledge of prior domain score distributions) have been proven to hold Δ-asymptotic optimality in Robbins' sense of convergence in mean. We found that, where\(\widetilde{d}^* _N\) and\(\widetilde\delta ^* _N\) are the monotonized versions of\(\widetilde{d}_N\) and\(\widetilde\delta _N\) under Van Houwelingen's monotonization method, respectively, the convergence rate of the overall expected loss of Bayes risk in either\(\widetilde{d}^* _N\) or\(\widetilde\delta ^* _N\) depends on test length, sample size, and ratio of test length to size of domain items. In terms of conditional Bayes risk,\(\widetilde{d}^* _N\) and\(\widetilde\delta ^* _N\) outperform their maximum likelihood counterparts over the middle range of domain scales. In terms of mean-squared error, we also found that: (a) given a unimodal prior distribution of domain scores,\(\widetilde\delta ^* _N\) performs better than both\(\widetilde{d}^* _N\) and a linear EBE of the beta-binomial model when domain item size is small or when test items reflect a high degree of heterogeneity; (b)\(\widetilde{d}^* _N\) performs as well as\(\widetilde\delta ^* _N\) when prior distribution is bimodal and test items are homogeneous; and (c) the linear EBE is extremely robust when a large pool of homogeneous items plus a unimodal prior distribution exists.
Similar content being viewed by others
References
American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (1985).Standards for educational and psychological tests. Washington, DC: American Psychological Association.
Berk, R. (1980). A consumer's guide to criterion-referenced test reliability.Journal of Educational Measurement, 17, 323–349.
Chung, K. L. (1974).A course in probability theory. New York: Academic Press.
Cressie, N. (1982). A useful empirical Bayes identity.The Annals of Statistics, 10, 625–629.
Cressie, N., & Seheult, A. (1985). Empirical Bayes estimation in sampling inspection.Biometrika, 72, 451–458.
Deely, J. J., & Lindley, D. V. (1981). Bayes Empirical Bayes.Journal of the American Statistical Association, 76, 833–841.
Johnson, N., & Kotz, S. (1969).Discrete distribution in statistics: Distributions. New York: Wiley.
Keats, J. A., & Lord, F. M. (1962). A theoretical distribution for mental test scores.Psychometrika, 27, 59–72.
Lin, M. H., Hsiung, C. A., & Hsiao, C. F. (1994). A computing program for monotonizing two empirical Bayes estimators in binomial and hypergeometric data distributions.Psychometrika, 59, 423–424.
Lord, F. M., & Novick, M. R. (1968).Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
Maritz, J. S., & Lwin, T. (1975). Construction of simple empirical Bayes estimators.Journal of the Royal Statistical Society, Series B, 39, 421–425.
Maritz, J., & Lwin, J. (1989).Empirical Bayes methods. London: Chapman and Hall.
Meredith, W., & Kearns, J. (1973). Empirical Bayes point estimates of latent trait scores without knowledge of the trait distribution.Psychometrika, 38, 533–554.
Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures.Psychological Bulletin, 105(1), 156–166.
Millman, J. (1974). Criterion referenced measurement. In W. J. Popham (Ed.).Evaluation in education: Current application. Berkeley, CA: McCutcheon.
Mood, A., Graybill, F., & Boes, D. (1974).Introduction to the theory of statistics. New York: McGraw-Hill.
Nichols, W. G., & Tsokos, C. P. (1972). Empirical Bayes point estimation in a family of probability distributions.International Statistical Review, 40, 147–151.
Popham, W. J. (1984). Specifying the domain of content or behaviors. In R. A. Berk (Ed.).A guide to criterion-referenced test construction (pp. 29–48). Baltimore: Johns Hopkins University Press.
Robbins, H. (1964). The empirical Bayes approach to statistical decision problems.Annals Mathematical Statistics, 35, 1–20.
Rutherford, J. R., & Krutchkoff, R. G. (1969). Some empirical Bayes techniques in point estimation.Biometrika, 56, 133–137.
van der Linden, W. J. (1979). Binomial test models and item difficulty.Applied Psychological Measurement, 3, 401–411.
Van Houwelingen, J. C. (1977). Monotonizing empirical Bayes estimators for a class of discrete distributions with monotone likelihood ratio.Statistica Neerlandica, 31, 95–104.
Wilcox, R. R. (1979). A lower bound to the probability of choosing the optimal passing scores for a mastery test when there is an external criterion.Psychometrika, 44, 245–249.
Author information
Authors and Affiliations
Additional information
The authors are indebted to both anonymous reviewers, especially Reviewer 2, and the Editor for their invaluable comments and suggestions. Thanks are also due to Yuan-Chin Chang and Chin-Fu Hsiao for their help with our simulation and programming work.
Rights and permissions
About this article
Cite this article
Lin, MH., Hsiung, C.A. Empirical bayes estimates of domain scores under binomial and hypergeometric distributions for test scores. Psychometrika 59, 331–359 (1994). https://doi.org/10.1007/BF02296128
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02296128