Psychometrika

, Volume 52, Issue 4, pp 589–617 | Cite as

A nonparametric approach for assessing latent trait unidimensionality

  • William Stout
Article

Abstract

Assuming a nonparametric family of item response theory models, a theory-based procedure for testing the hypothesis of unidimensionality of the latent space is proposed. The asymptotic distribution of the test statistic is derived assuming unidimensionality, thereby establishing an asymptotically valid statistical test of the unidimensionality of the latent trait. Based upon a new notion of dimensionality, the test is shown to have asymptotic power 1. A 6300 trial Monte Carlo study using published item parameter estimates of widely used standardized tests indicates conservative adherence to the nominal level of significance and statistical power averaging 81 out of 100 rejections for examinee sample sizes and psychological test lengths often incurred in practice.

Key words

unidimensionality local independence test of unidimensionality latent trait theory latent structure analysis item response theory large sample theory asymptotic distribution theory nonparametric test 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bartholomew, D. J. (1980). Factor analysis for categorical data.Journal of the Royal Statistical Society, Series B, 42, 293–321.Google Scholar
  2. Bejar, I. I. (1980). A procedure of investigating the unidimensionality of achievement tests based on item parameter estimates.Journal of Educational Measurement, 17, 283–296.Google Scholar
  3. Bock, R. D. (1984, September). Contributions of empirical Bayes and marginal maximum likelihood methods to the measurement of individual differences. Proceedings of the 23rd International Conference of Psychology, Acapulco, Mexico.Google Scholar
  4. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm.Psychometrika, 46, 443–459.Google Scholar
  5. Bock, R. D., Gibbons, R., & Murake, E. (1985).Full information factor analysis (MRC Report No. 85-1). Washington, DC: Office of Naval Research.Google Scholar
  6. Christoffersson, A. (1975). Factor analysis of dichotomous variables.Psychometrika, 40, 5–32.Google Scholar
  7. Chung, K. L. (1974).A course in probability theory, (2nd ed.). New York: Academic Press.Google Scholar
  8. Divgi, D. R. (1981, April). Potential pitfalls of item response theory. Paper presented at annual meeting of National Council of Measurement in Education, Los Angeles.Google Scholar
  9. Drasgow, F. (1987). A study of the measurement bias of two standardized psychological tests.Journal of Applied Psychology, 72, 19–30.Google Scholar
  10. Drasgow, F., & Lissak, R. (1983). Modified parallel analysis: A procedure for examining the latent dimensionality of dichotomously scored item responses.Journal of Applied Psychology, 68, 363–373.Google Scholar
  11. Goldstein, H. (1980). Dimensionality, bias, independence, and measurement scale problems in latent trait score models.British Journal of Mathematical and Statistical Psychology, 33, 234–246.Google Scholar
  12. Hambleton, R. K., & Swaminathan (1985).Item Response Theory: Principles and Applications. Boston: Kluwer-Nijhoff.Google Scholar
  13. Hambleton, R. K., & Traub, R. E. (1973). Analysis of empirical data using two logistic latent trait models.British Journal of Mathematical and Statistical Psychology, 26, 195–211.Google Scholar
  14. Hattie, J. (1985). Methodology Review: Assessing unidimensionality of tests and items.Applied Psychology Measurement, 9, 139–164.Google Scholar
  15. Holland, P. W. (1981). When are item response models consistent with observed data?Psychometrika, 46, 79–92.Google Scholar
  16. Holland, P. W., & Rosenbaum, P. R. (1986).Conditional association and unidimensionality in monotone latent variable models.Annals of Statistics, 14, 1523–1543.Google Scholar
  17. Hulin, C. L., Drasgow, F., & Parsons, L. K. (1983).Item Response Theory. Homewood, IL: Dow Jones-Irwin.Google Scholar
  18. Humphreys, L. C. (1985). General intelligence: An integration of factor, test, and simplex theory. In B. J. Wolman (Ed.),Handbook of intelligence: Theories, measurements, and applications (pp. 201–224). New York: John Wiley and Sons.Google Scholar
  19. Lord, F. M. (1957). A significance test for the hypothesis that two variables measure the same trait except for errors of measurement.Psychometrika, 22, 207–220.Google Scholar
  20. Lord, F. M. (1968). An analysis of the verbal scholastic aptitude test using Birnbaum's three-parameter logistic model.Educational and Psychological Measurement, 28, 989–1020.Google Scholar
  21. Lord, F. M. (1980).Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  22. Lord, F. M., & Novick, M. R. (1968).Statistical theories of mental test scores. Reading, MA: Addison-Wesley.Google Scholar
  23. Lumsden, J. (1976). Test theory. In Rosenzwieg, M. R. and Porter, L. W. (Eds.)Annual Review of Psychology. Palo Alto, CA: Annual Reviews.Google Scholar
  24. McDonald, R. P. (1967a). Nonlinear factor analysis.Psychometrika Monograph No. 15, 32(4, Pt. 2).Google Scholar
  25. McDonald, R. P. (1967b). Numerical methods for polynomial models in nonlinear factor analysis.Psychometrika, 32, 77–112.Google Scholar
  26. McDonald, R. P. (1981). The dimensionality of test and items.British Journal of Mathematical and Statistical Psychology, 34, 100–117.Google Scholar
  27. McDonald, R. P. (1983). Exploratory and confirmatory factor analysis. In H. Warner & S. Messick (Eds.),Principles of modern psychological measurement, Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  28. McDonald, R. P., & Ahlawat, K. S. (1974). Difficulty factors in binary data.British Journal of Mathematical and Statistical Psychology, 27, 82–99.Google Scholar
  29. McNemar, Q. (1946). Opinion-attitude methodology.Psychological Bulletin, 43, 289–374.Google Scholar
  30. Meredith, W. (1965). Some results on a general stochastic model for mental tests.Psychometrika, 30, 419–440.Google Scholar
  31. Mislevy, R. J., & Bock, R. D. (1984). Item operating characteristics of the Armed Services Aptitute Battery (ASVAB), Form 8A, (Tech. Rep. N00014-83-C-0283). Washington, DC: Office of Naval Research.Google Scholar
  32. Molenaar, I. W. (1983). Some improved diagnostics for failure of the Rasch model.Psychometrika, 48, 49–72.Google Scholar
  33. Muthén, B. (1978). Contributions to factor analysis of dichotomous variables.Psychometrika, 43, 551–560.Google Scholar
  34. Nandakumar, R. (1987).Refinement of Stout's procedure for assessing latent trait unidimensionality. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign.Google Scholar
  35. Reckase, M. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications.Journal of Educational Statistics, 4, 207–230.Google Scholar
  36. Rosenbaum, P. R. (1984). Testing the conditional independence and monotonicity assumptions of item response theory.Psychometrika, 49, 425–436.Google Scholar
  37. Sarrazin, G. (1983, July).The detection of item bias for different cultural groups using latent trait and chi-square methods. Paper presented at the Joint Meeting of the Psychometric Society and the Classification Society, Jouy-en-Joses, France.Google Scholar
  38. Serfling, R. J. (1980).Approximation theories of mathematical statistics. New York: John Wiley.Google Scholar
  39. Stout, W. F. (1974).Almost sure convergence, New York: Academic Press.Google Scholar
  40. Stout, W. F. (1984).A statistical test of unidimensionality for binary data with applications (Tech. Rep. N00014-82-K-0486). Washington, DC: Office of Naval Research.Google Scholar
  41. Stout, W. F. (in press). A nonparametric multidimensional IRT approach with applications to ability estimation and test bias.Psychometrika.Google Scholar
  42. van den Wollenberg, A. L. (1982). Two new test statistics for the Rasch model.Psychometrika, 47, 123–140.Google Scholar

Copyright information

© The Psychometric Society 1987

Authors and Affiliations

  • William Stout
    • 1
  1. 1.Department of Statistics and MathematicsUniversity of Illinois at Urbana-ChampaignUSA

Personalised recommendations