, Volume 56, Issue 2, pp 177–196 | Cite as

Randomization-based inference about latent variables from complex samples

  • Robert J. Mislevy


Standard procedures for drawing inferences from complex samples do not apply when the variable of interestϑ cannot be observed directly, but must be inferred from the values of secondary random variables that depend onϑ stochastically. Examples are proficiency variables in item response models and class memberships in latent class models. Rubin's “multiple imputation” techniques yield approximations of sample statistics that would have been obtained, hadϑ been observable, and associated variance estimates that account for uncertainty due to both the sampling of respondents and the latent nature ofϑ. The approach is illustrated with data from the National Assessment for Educational Progress.

Key words

complex samples item response theory latent structure missing data multiple imputation National Assessment of Educational Progress sample surveys 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Andersen, E. B., & Madsen, M. (1977). Estimating the parameters of a latent population distribution.Psychometrika, 42, 357–374.Google Scholar
  2. Beaton, A. E. (1987).The NAEP 1983/84 technical report (NAEP Report 15-TR-20). Princeton: Educational Testing Service.Google Scholar
  3. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: An application of an EM algorithm.Psychometrika, 46, 443–459.Google Scholar
  4. Bock, R. D., Mislevy, R. J., & Woodson, C. E. M. (1982). The next stage in educational assessment.Educational Researcher, 11, 4–11, 16.Google Scholar
  5. Cassel, C. M., Särndal, C. E., & Wretman, J. H. (1977).Foundations of inference in survey sampling. New York: Wiley.Google Scholar
  6. Cochran, W. G. (1977).Sampling techniques. New York: Wiley.Google Scholar
  7. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion).Journal of the Royal Statistical Society, Series B, 39, 1–38.Google Scholar
  8. Ford, B. L. (1983). An overview of hot-deck procedures. In W. G. Madow, I. Olkin, & D. B. Rubin (Eds.),Incomplete data in sample surveys, Volume 2, Theory and bibliographies (pp. 185–207). New York: Academic Press.Google Scholar
  9. Goldstein, H., & McDonald, R. P. (1988). A general model for the analysis of multilevel data.Psychometrika, 53, 455–467.Google Scholar
  10. Johnson, E. G. (1987). Estimation of uncertainty due to sampling variation. In A. E. Beaton,The NAEP 1983/84 technical report (NAEP Report 15-TR-20, pp. 505–512). Princeton: Educational Testing Service.Google Scholar
  11. Kelley, T. L. (1923).Statistical method. New York: Macmillan.Google Scholar
  12. Kirsch, I. S., & Jungeblut, A. (1986).Literacy: Profile of America's young adults (Final Report, No. 16-PL-01). Princeton, NJ: National Assessment of Education Progress.Google Scholar
  13. Laird, N. M. (1978). Nonparametric maximum likelihood estimation of a mixing distribution.Journal of the American Statistical Association, 73, 805–811.Google Scholar
  14. Little, R. J. A., & Rubin, D. B. (1987).Statistical analysis with missing data. New York: Wiley.Google Scholar
  15. Lord, F. M. (1965). A strong true-score theory, with applications.Psychometrika, 30, 239–270.Google Scholar
  16. Lord, F. M. (1969). Estimating true score distributions in psychological testing (An empirical Bayes problem).Psychometrika, 34, 259–299.Google Scholar
  17. Mislevy, R. J. (1984). Estimating latent distributions.Psychometrika, 49, 359–381.Google Scholar
  18. Mislevy, R. J. (1985). Estimation of latent group effects.Journal of the American Statistical Association, 80, 993–997.Google Scholar
  19. Mislevy, R. J., & Bock, R. D. (1983).BILOG: Item analysis and test scoring with binary logistic models [computer program]. Mooresville, IN: Scientific Software.Google Scholar
  20. Mislevy, R. J., & Sheehan, K. M. (1987). Marginal estimation procedures. In A. E. Beaton,The NAEP 1983/84 technical report (NAEP Report 15-TR-20, pp. 293–360). Princeton: Educational Testing Service.Google Scholar
  21. Muthén, B., & Satorra, A. (1989). Multilevel aspects of varying parameters in structural models. In R. D. Bock (Ed.),Multilevel analysis of educational data (pp. 87–99). San Diego: Academic Press.Google Scholar
  22. National Assessment of Educational Progress (1985).The reading report card: Progress toward excellence in our schools. Princeton: Educational Testing Service.Google Scholar
  23. Rigdon, S. E., & Tsutakawa, R. K. (1983). Parameter estimation in latent trait models.Psychometrika, 48, 567–574.Google Scholar
  24. Rubin, D. B. (1977). Formalizing subjective notions about the effect of nonrespondents in sample surveys.Journal of the American Statistical Association, 72, 538–543.Google Scholar
  25. Rubin, D. B. (1987).Multiple imputation for nonresponse in surveys. New York: Wiley.Google Scholar
  26. Sanathanan, L., & Blumenthal, N. (1978). The logistic model and estimation of latent structure.Journal of the American Statistical Association, 73, 794–798.Google Scholar
  27. Sheehan, K. M. (1985).M-GROUP: Estimation of group effects in multivariate models [computer program]. Princeton: Educational Testing Service.Google Scholar
  28. Tanner, M., & Wong, W. (1987). The calculation of posterior distributions by data augmentation (with discussion).Journal of the American Statistical Association, 82, 528–550.Google Scholar
  29. Zwick, R. (1987). Assessing the dimensionality of NAEP reading data.Journal of Educational Measurement, 24, 293–308.Google Scholar

Copyright information

© The Psychometric Society 1991

Authors and Affiliations

  • Robert J. Mislevy
    • 1
  1. 1.Educational Testing ServicePrinceton

Personalised recommendations