Advertisement

Psychometrika

, Volume 84, Issue 1, pp 19–40 | Cite as

On the Identifiability of Diagnostic Classification Models

  • Guanhua Fang
  • Jingchen LiuEmail author
  • Zhiliang Ying
Article
First Online:
  • 94 Downloads

Abstract

This paper establishes fundamental results for statistical analysis based on diagnostic classification models (DCMs). The results are developed at a high level of generality and are applicable to essentially all diagnostic classification models. In particular, we establish identifiability results for various modeling parameters, notably item response probabilities, attribute distribution, and Q-matrix-induced partial information structure. These results are stated under a general setting of latent class models. Through a nonparametric Bayes approach, we construct an estimator that can be shown to be consistent when the identifiability conditions are satisfied. Simulation results show that these estimators perform well under various model settings. We also apply the proposed method to a dataset from the National Epidemiological Survey on Alcohol and Related Conditions (NESARC).

Keywords

identifiability diagnostic classification models Dirichlet allocation 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This research is supported in part by NSF IIS-1633360 and SES-1826540.

Supplementary material

11336_2018_9658_MOESM1_ESM.pdf (214 kb)
Supplementary material 1 (pdf 213 KB)
11336_2018_9658_MOESM2_ESM.rdata (5 kb)
Supplementary material 2 (Rdata 4 KB)
11336_2018_9658_MOESM3_ESM.r (5 kb)
Supplementary material 3 (R 4 KB)

References

  1. Allman, E. S., Matias, C., & Rhodes, J. A. (2009). Identifiability of parameters in latent structure models with many observed variables. The Annals of Statistics, 37, 3099–3132.CrossRefGoogle Scholar
  2. Chen, Y., Culpepper, S. A., Chen, Y., & Douglas, J. (2018). Bayesian estimation of the DINA \(Q\) matrix. Psychometrika, 83, 89–108.CrossRefGoogle Scholar
  3. Chen, Y., Liu, J., Xu, G., & Ying, Z. (2015a). Statistical analysis of \(Q\)-matrix based diagnostic classification models. Journal of the American Statistical Association, 110, 850–866.CrossRefGoogle Scholar
  4. Chen, Y., Liu, J., & Ying, Z. (2015b). Online item calibration for \(Q\)-matrix in CD-CAT. Applied Psychological Measurement, 39, 5–15.CrossRefGoogle Scholar
  5. Chiu, C.-Y., Douglas, J. A., & Li, X. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74, 633.CrossRefGoogle Scholar
  6. De La Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of educational and behavioral statistics, 34, 115–130.CrossRefGoogle Scholar
  7. De La Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199.CrossRefGoogle Scholar
  8. De La Torre, J., & Douglas, J. A. (2004). Higher order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353.CrossRefGoogle Scholar
  9. DeCarlo, L. T. (2011). On the analysis of fraction subtraction data: The DINA model, classification, latent class sizes, and the \(Q\)-matrix. Applied Psychological Measurement, 35, 8–26.CrossRefGoogle Scholar
  10. DiBello, L. V., Stout, W. F., & Roussos, L. A. (1995). Unified cognitive/psychometric diagnostic assessment likelihood-based classification techniques. In S. F. Chipman, P. D. Nichols, & R. L. Brennan (Eds.), Cognitively diagnostic assessment (pp. 361–389). Hillsdale, NJ: Erlbaum.Google Scholar
  11. Dunson, D., & Xing, C. (2009). Nonparametric Bayes modeling of multivariate categorical data. Journal of the American Statistical Association, 104, 1042–1051.CrossRefGoogle Scholar
  12. Grant, B. F., Kaplan, K., Shepard, J., & Moore, T. (2003). Source and accuracy statement for wave 1 of the 2001–2002 national epidemiologic survey on alcohol and related conditions. Bethesda, MD: National Institute on Alcohol Abuse and Alcoholism.Google Scholar
  13. Gu, Y., & Xu, G. (2018). The sufficient and necessary condition for the identifiability and estimability of the DINA model. Psychometrika.  https://doi.org/10.1007/s11336-018-9619-8.
  14. Gu, Y., Liu, J., Xu, G., & Ying, Z. (2018). Hypothesis testing of the \(Q\)-matrix. Psychometrika, 83, 515–537.CrossRefGoogle Scholar
  15. Hartz, S. M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Doctoral Dissertation, University of Illinois, Urbana-Champaign.Google Scholar
  16. Henson, R., Templin, J., & Willse, J. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.CrossRefGoogle Scholar
  17. Iza, M., Wall, M., Heimberg, R., Rodebaugh, T., Schneier, F., Liu, S.-M., et al. (2014). Latent structure of social fears and social anxiety disorders. Psychological medicine, 44, 361–370.CrossRefGoogle Scholar
  18. Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.CrossRefGoogle Scholar
  19. Kruskal, J. (1977). Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear Algebra and its Application, 18, 95–138.CrossRefGoogle Scholar
  20. Leighton, J. P., Gierl, M. J., & Hunka, S. M. (2004). The attribute hierarchy model for cognitive assessment: A variation on Tatsuoka’s rule-space approach. Journal of Educational Measurement, 41, 205–237.CrossRefGoogle Scholar
  21. Liu, J., Xu, G., & Ying, Z. (2012). Data-driven learning of \(Q\)-matrix. Applied Psychological Measurement, 36, 609–618.Google Scholar
  22. Liu, J., Xu, G., & Ying, Z. (2013). Theory of the self-learning \(Q\)-matrix. Bernoulli: Official Journal of the Bernoulli Society for Mathematical Statistics and Probability, 19, 1790.CrossRefGoogle Scholar
  23. Roussos, L. A., Templin, J. L., & Henson, R. A. (2007). Skills diagnosis using IRT-based latent class models. Journal of Educational Measurement, 44, 293–311.CrossRefGoogle Scholar
  24. Rupp, A. A., & Templin, J. (2008a). The effects of \(Q\)-matrix misspecification on parameter estimates and classification accuracy in the DINA model. Educational and Psychological Measurement, 68, 78–96.CrossRefGoogle Scholar
  25. Rupp, A. A., & Templin, J. L. (2008b). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement: Interdisciplinary Research and Perspective, 6, 219–262.Google Scholar
  26. Rupp, A. A., Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York: Guilford Press.Google Scholar
  27. Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Statistica Sinica, 4, 639–650.Google Scholar
  28. Stout, W. (2007). Skills diagnosis using IRT-based continuous latent trait models. Journal of Educational Measurement, 44, 313–324.CrossRefGoogle Scholar
  29. Tatsuoka, K. K. (1985). A probabilistic model for diagnosing misconceptions in the pattern classification approach. Journal of Educational Statistics, 12, 55–73.CrossRefGoogle Scholar
  30. Tatsuoka, K. K. (2009). Cognitive assessment: An introduction to the rule space method. Boca Raton: CRC Press.Google Scholar
  31. Templin, J., He, X., Roussos, L., & Stout, W. (2003). The pseudo-item method: a simple technique for analysis of polytomous data with the fusion model. External diagnostic research group technical report.Google Scholar
  32. Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305.CrossRefGoogle Scholar
  33. Van der Vaart, A. W. (1998). Asymptotic statistics. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  34. von Davier, M. (2005). A general diagnosis model applied to language testing data. Research report: Educational testing service.Google Scholar
  35. Vonesh, E. F., & Chinchilli, V. G. (1997). Linear and nonlinear models for the analysis of repeated measurements. London: Chapman and Hall.Google Scholar
  36. Walker, S. G. (2007). Sampling the dirichlet mixture model with slices. Communications in Statistics-Simulation and Computation, 36, 45–54.CrossRefGoogle Scholar
  37. Xu, G. (2017). Identifiability of restricted latent class models with binary responses. The Annals of Statistics, 45, 675–707.CrossRefGoogle Scholar
  38. Xu, G., & Shang, Z. (2018). Identifying latent structures in restricted latent class models. Journal of the American Statistical Association, 113(523), 1284–1295.CrossRefGoogle Scholar
  39. Xu, G., & Zhang, S. (2016). Identifiability of diagnostic classification models. Psychometrika, 81, 625–649.CrossRefGoogle Scholar

Copyright information

© The Psychometric Society 2019

Authors and Affiliations

  1. 1.Columbia UniversityNew YorkUSA

Personalised recommendations