Psychometrika

, Volume 69, Issue 3, pp 333–353 | Cite as

Higher-order latent trait models for cognitive diagnosis

Theory and Methods

Abstract

Higher-order latent traits are proposed for specifying the joint distribution of binary attributes in models for cognitive diagnosis. This approach results in a parsimonious model for the joint distribution of a high-dimensional attribute vector that is natural in many situations when specific cognitive information is sought but a less informative item response model would be a reasonable alternative. This approach stems from viewing the attributes as the specific knowledge required for examination performance, and modeling these attributes as arising from a broadly-defined latent trait resembling theϑ of item response models. In this way a relatively simple model for the joint distribution of the attributes results, which is based on a plausible model for the relationship between general aptitude and specific knowledge. Markov chain Monte Carlo algorithms for parameter estimation are given for selected response distributions, and simulation results are presented to examine the performance of the algorithm as well as the sensitivity of classification to model misspecification. An analysis of fraction subtraction data is provided as an example.

Key words

cognitive diagnosis item response theory latent class model Markov chain Monte Carlo 

References

  1. Best, N.G., Cowles, M.K., & Vines, S.K. (1995). CODA: Convergence diagnosis and output analysis software for Gibbs sampling output (Version 0.30). [Computer software]. Cambridge: MRC Biostatistics Unit.Google Scholar
  2. Bock, R.D., & Aitken, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm.Psychometrika, 46, 443–459.CrossRefGoogle Scholar
  3. Casella, G., & George, E.I. (1992). Explaining the Gibbs sampler.The American Statistician, 46, 167–174.Google Scholar
  4. Chib, S., & Greenberg, E. (1995). Understanding the Metropolis-Hastings algorithm.The American Statistician, 49, 327–335.Google Scholar
  5. DiBello, L.V., Stout, W.F., & Roussos, L.A. (1995). Unified cognitive/psychometric diagnostic assessment likelihood-based classification techniques. In P.D. Nichols, S.F. Chipman, & R.L. Brennan,Cognitively diagnostic assessment (pp. 361–389). Hillsdale, NJ: Erlbaum.Google Scholar
  6. Doignon, J.P., & Falmagne, J.C. (1999).Knowledge spaces. New York: Springer-Verlag.Google Scholar
  7. Doornik, J.A. (2002). Object-oriented matrix programming using Ox (Version 3.1). [Computer software]. London: Timberlake Consultants Press.Google Scholar
  8. Draney, K.L., Pirolli, P., & Wilson, M. (1995). A measurement model for complex cognitive skill. In P.D. Nichols, S.F. Chipman, R.L. Brennan (Eds.),Cognitively diagnostic assessment (pp. 103–125). Hillsdale, NJ: Erlbaum.Google Scholar
  9. Embretson, S. (1984). A general latent trait model for response processes.Psychometrika, 49, 175–186.Google Scholar
  10. Embretson, S. (1997). Multicomponent response models. In W.J. van der Linden & R.K. Hambleton (Eds.),Handbook of modern item response theory (pp. 305–321). New York: Springer-Verlag.Google Scholar
  11. Everitt, B. S. (1998).The Cambridge dictionary of statistics. Cambridge, UK: Cambridge University Press.Google Scholar
  12. Fischer, G. H. (1995). The linear logistic test model. In G. H. Fischer & I. W. Molenaar (Eds.),Rasch models: Foundations, recent developments, and applications (pp. 131–155) New York: Springer-Verlag.Google Scholar
  13. Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images.IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.Google Scholar
  14. Gelman, A., & Rubin, D.B. (1992). Inference from iterative simulation using multiple sequences (with discussion).Statistical Science, 7, 457–511.Google Scholar
  15. Haertel, E.H. (1989). Using restricted latent class models to map the skill structure of achievement items.Journal of Educational Measurement, 26, 333–352.CrossRefGoogle Scholar
  16. Hagenaars, J.A. (1990).Categorical longitudinal data: Loglinear panel, trend, and cohort analysis. Newbury Park, CA: Sage.Google Scholar
  17. Hagenaars, J.A. (1993).Loglinear models with latent variables. Newbury Park, CA: Sage.Google Scholar
  18. Hartz, S. (2002).A Bayesian framework for the Unified Model for assessing cognitive abilities: Blending theory with practicality. Unpublished doctoral dissertation.Google Scholar
  19. Junker, B.W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory.Applied Psychological Measurement, 25, 258–272.Google Scholar
  20. Macready, G.B., & Dayton, C. M. (1977). The use of probabilistic models in the assessment of mastery.Journal of Educational Statistics, 33, 379–416.Google Scholar
  21. Maris, E. (1999). Estimating multiple classification latent class models.Psychometrika, 64, 187–212.CrossRefGoogle Scholar
  22. Mislevy, R.J. (1996). Test theory reconceived.Journal of Educational Measurement, 33, 379–416.CrossRefGoogle Scholar
  23. Muthén, B. (1978). Contribution to factor analysis of binary variables.Psychometrika, 43, 551–560.CrossRefGoogle Scholar
  24. Patz, R.J., & Junker, B.W. (1999a). A straightforward approach to Markov chain Monte Carlo methods for item response theory.Journal of Educational and Behavioral Statistics, 24, 146–178.Google Scholar
  25. Patz, R.J., & Junker, B.W. (1999b). Applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses.Journal of Educational and Behavioral Statistics, 24, 342–366.Google Scholar
  26. Raftery, A.E. (1995). Bayesian model selection in social research.Sociological Methodology, 25, 111–163.Google Scholar
  27. Raftery, A.E. (1996). Hypothesis testing and model selection. In R. W. Gilks, S. Richardson, & D. J. Spiegelhalter (Eds.),Markov chain Monte Carlo in practice (pp. 163–187). London: Chapman & Hall.Google Scholar
  28. Reckase, M. (1997). A linear logistic multidimensional model for dichotomous item response data. In W.J. van der Linden & R.K. Hambleton (Eds.),Handbook of modern item response theory (pp. 271–286). New York: Springer-Verlag.Google Scholar
  29. Tatsuoka, C. (2002). Data-analytic methods for latent partially ordered classification models.Journal of the Royal Statistical Society Series C (Applied Statistics), 51, 337–350.CrossRefGoogle Scholar
  30. Tatsuoka, K. (1985). A probabilistic model for diagnosing misconceptions in the pattern classification approach.Journal of Educational Statistics, 12, 55–73.Google Scholar
  31. Tatsuoka, K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. In N. Frederiksen, R. Glaser, A. Lesgold, & Safto, M. (Eds.),Monitoring skills and knowledge acquisition (pp. 453–488). Hillsdale, NJ: Erlbaum.Google Scholar
  32. Tatsuoka, K. (1995). Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. In P.D. Nichols, S.F. Chipman, & R.L. Brennan (Eds.),Cognitively diagnostic assessment (pp. 327–359). Hillsdale, NJ: Erlbaum.Google Scholar

Copyright information

© The Psychometric Society 2004

Authors and Affiliations

  1. 1.Department of Educational PsychologyRutgers, The State University of New JerseyNew BrunswickUSA
  2. 2.University of IllinoisUSA

Personalised recommendations