Skip to main content
SpringerLink
Log in

A Unified Theory of the Completeness of Q-Matrices for the DINA Model

  • Published:
Journal of Classification Aims and scope Submit manuscript

Abstract

Diagnostic classification models in educational measurement describe ability in a knowledge domain as a composite of specific binary skills called “cognitive attributes,” each of which an examinee may or may not have mastered. Attribute Hierarchy Models (AHMs) account for the possibility that attributes are dependent by imposing a hierarchical structure such that mastery of one or more attributes is a prerequisite of mastering one or more other attributes. Thus, the number of meaningfully defined attribute combinations is reduced, so that constructing a complete Q-matrix may be challenging. (The Q-matrix of a cognitively diagnostic test documents which attributes are required for solving which item; the Q-matrix is said to be complete if it guarantees the identifiability of all realizable proficiency classes among examinees.) For structured Q-matrices (i.e., the item attribute profiles are restricted to reflect the hierarchy postulated to underlie the attributes), the conditions of completeness have been established. However, sometimes, a structured Q-matrix cannot be assembled because the items of the test in question have attribute profiles that do not conform to the prerequisite structure imposed by the postulated attribute hierarchy. A Q-matrix composed of such items is called “unstructured.” In this article, the completeness conditions of unstructured Q-matrices for the DINA model are presented. Specifically, there exists an entire range of Q-matrices that are all complete for DINA-AHMs. Thus, the theoretical results presented here can be combined with extant insights about Q-completeness for models without attribute hierarchies into a unified framework on the completeness of Q-matrices for the DINA model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Cai, Y., Tu, D., & Ding, S. (2018). Theorems and methods of a complete Q matrix with attribute hierarchies under restricted Q-matrix design. Frontiers in Psychology, 9, Article 1413.

  • Chen, Y., Culpepper, S.A., Chen, Y., & Douglas, J. (2018). Bayesian estimation of DINA Q matrix. Psychometrika, 83, 89–108.

    Article  MathSciNet  Google Scholar 

  • Chen, Y., Liu, J., Xu, G., & Ying, Z. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110, 850–866.

    Article  MathSciNet  Google Scholar 

  • Chiu, C.-Y. (2013). Statistical refinement of the Q-matrix in cognitive diagnosis. Applied Psychological Measurement, 37, 598–618.

    Article  Google Scholar 

  • Chiu, C.-Y., Douglas, J., & Li, X. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74, 633–665.

    Article  MathSciNet  Google Scholar 

  • Chiu, C.-Y., & Köhn, H.-F. (2015). Consistency of cluster analysis for cognitive diagnosis: The DINO model and the DINA model revisited. Applied Psychological Measurement, 39, 465–479.

    Article  Google Scholar 

  • Chiu, C.-Y., & Köhn, H.-F. (2016). The Reduced RUM as a logit model:, Parameterization and constraints. Psychometrika, 81, 350–370.

    Article  MathSciNet  Google Scholar 

  • Chung, M. (2014). Estimating the Q-matrix for Cognitive Diagnosis Models in a Bayesian Framework. Doctoral dissertation. Columbia University.

  • Chung, M. (2019). A Gibbs sampling algorithm that estimates the Q-matrix for the DINA model. Journal of Mathematical Psychology, 93, 1–8.

    Article  MathSciNet  Google Scholar 

  • DeCarlo, L.T. (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model. Applied Psychological Measurement, 36, 447–468.

    Article  Google Scholar 

  • de la Torre, J. (2008). An empirically based method of Q-matrix validation for the DINA model: Development and applications. Journal of Educational Measurement, 45, 343–362.

    Article  Google Scholar 

  • de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199.

    Article  MathSciNet  Google Scholar 

  • de la Torre, J., & Chiu, C.-Y. (2016). A general method of empirical Q-matrix validation. Psychometrika, 81, 253–73.

    Article  MathSciNet  Google Scholar 

  • de la Torre, J., & Douglas, J.A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353.

    Article  MathSciNet  Google Scholar 

  • de la Torre, J., Hong, Y., & Deng, W. (2010). Factors affecting the item parameter estimation and classification accuracy of the DINA model. Journal of Educational Measurement, 47, 227–249.

    Article  Google Scholar 

  • DiBello, L.V., Roussos, L.A., & Stout, W.F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In Rao, C.R., & Sinharay, S. (Eds.) Handbook of Statistics: Vol.26. Psychometrics (pp. 979–1030). Amsterdam: Elsevier.

  • Fu, J., & Li, Y. (2007). An integrative review of cognitively diagnostic psychometric models. Chicago: Paper presented at the Annual Meeting of the National Council on Measurement in Education.

    Google Scholar 

  • Gierl, M. (2007). Making diagnostic inferences about cognitive attributes using the rule-space model and attribute hierarchy method. Journal of Educational Measurement, 44, 325–340.

    Article  Google Scholar 

  • Gross, J., & George, A.C. (2014). On prerequisite relations between attributes in noncompensatory diagnostic classification. Methodology, 10(3), 100–107.

    Article  Google Scholar 

  • Gu, Y., & Xu, G. (2020). Partial identifiability of restricted latent class models. Annals of Statistics, 48(4), 2082–2107.

    Article  MathSciNet  Google Scholar 

  • Gu, Y., & Xu, G. (2019a). The sufficient and necessary condition for the identifiability and estimability of the DINA model. Psychometrika, 84, 468–483.

    Article  MathSciNet  Google Scholar 

  • Gu, Y., & Xu, G. (2019b). Identifiability of hierarchical latent attribute models. https://arxiv.org/pdf/1906.07869.pdf.

  • Haberman, S.J., & von Davier, M. (2007). Some notes on models for cognitively based skill diagnosis. In Rao, C.R., & Sinharay, S. (Eds.) Handbook of Statistics: Vol.26. Psychometrics (pp. 1031–1038). Amsterdam: Elsevier.

  • Haertel, E.H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 333–352.

    Article  Google Scholar 

  • Haertel, E.H., & Wiley, D.E. (1994). Representation of ability structure: Implications for testing. In Fredriksen, N., Mislevy, R., & Bejar, I. (Eds.) Testing theory for a new generation of tests (pp. 359–384). Hillsdale: Erlbaum.

  • Hartz, S.M. (2002). A Bayesian framework for the Unified Model for assessing cognitive abilities: Blending theory with practicality (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No.3044108).

  • Hartz, S.M., & Roussos, L.A. (2008). The Fusion Model for skill diagnosis: Blending theory with practicality. (Research report No. RR-08-71). Princeton: Educational Testing Service.

    Google Scholar 

  • Henson, R.A., Templin, J.L., & Willse, J.T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.

    Article  MathSciNet  Google Scholar 

  • Junker, B.W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.

    Article  MathSciNet  Google Scholar 

  • Köhn, H.-F., & Chiu, C.-Y. (2017). A procedure for assessing completeness of the Q-matrix of cognitive diagnosis models. Psychometrika, 82, 112–132.

    Article  MathSciNet  Google Scholar 

  • Köhn, H.F., & Chiu, C.-Y. (2019). Attribute hierarchy models in cognitive diagnosis: identfiability of the latent attribute space and conditions for completeness of the Q-matrix. Journal of Classfication, 36, 541–565.

    Article  Google Scholar 

  • Leighton, J., & Gierl, M. (2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Leighton, J.P., Gierl, M.J., & Hunka, S. (2004). The attribute hierarchy model: An approach for integrating cognitive theory with assessment practice. Journal of Educational Measurement, 41, 205–236.

    Article  Google Scholar 

  • Liu, C.-W., Andersson, B., & Skrondal, A. (2020). A constrained Metropolis-Hastings Robbins-Monro algorithm for Q matrix estimation in DINA models. Psychometrika, 85, 322–357.

    Article  MathSciNet  Google Scholar 

  • Liu, R., Huggins-Manley, A.C., & Bradshaw, L. (2016). The impact of q-matrix designs on diagnostic classification accuracy in the presence of attribute hierarchies. Educational and Psychological Measurement, 76, 220–240.

    Google Scholar 

  • Liu, J., Xu, G., & Ying, Z. (2012). Data-driven learning of Q-matrix. Applied Psychological Measurement, 36, 548–564.

    Article  Google Scholar 

  • Liu, J., Xu, G., & Ying, Z. (2013). Theory of the self-learning Q-matrix. Bernoulli, 19, 1790–1817.

    MathSciNet  MATH  Google Scholar 

  • Macready, G.B., & Dayton, C.M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 33, 379–416.

    Google Scholar 

  • Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 187–212.

    Article  MathSciNet  Google Scholar 

  • Nichols, P.D., Chipman, S.E., & Brennan, R.L. (1995). Cognitively diagnostic assessment. Mahwah: Erlbaum.

    Google Scholar 

  • Rupp, A.A., & Templin, J.L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement, 6, 219–262.

    Google Scholar 

  • Rupp, A.A., Templin, J.L., & Henson, R.A. (2010). Diagnostic Measurement. Theory, Methods, and Applications. New York: Guilford.

    Google Scholar 

  • Sessoms, J., & Henson, R.A. (2018). Applications of Diagnostic Classification Models: A literature review and critical commentary. Measurement: Interdisciplinary Research and Perspectives, 16(1), 1–17.

    Google Scholar 

  • Tatsuoka, K.K. (1985). A probabilistic model for diagnosing misconception in the pattern classification approach. Journal of Educational and Behavioral Statistics, 12, 55–73.

    Article  Google Scholar 

  • Tatsuoka, K.K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. In Frederiksen, N., Glaser, R., Lesgold, A., & Shafto, M. (Eds.) Diagnostic monitoring of skill and knowledge acquisition (pp. 453–488). Erlbaum: Hillsdale.

  • Tatsuoka, K.K. (2009). Cognitive assessment. An introduction to the Rule Space Method. New York: Routledge/Taylor & Francis.

    Book  Google Scholar 

  • Templin, J., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79, 317–339.

    Article  MathSciNet  Google Scholar 

  • von Davier, M. (2005). A general diagnostic model applied to language testing data (Research report No. RR-05-16). Princeton, NJ: Educational Testing Service.

  • von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287–301.

    Article  MathSciNet  Google Scholar 

  • Zhang, S. S., DeCarlo, L. T., & Ying, Z. (2013). Non-identifiability, equivalence classes, and attribute-specific classification in Q-matrix based cognitive diagnosis models. preprint, arXiv:1303.0426.

Download references

Author information

Authors and Affiliations

  1. University of Illinois at Urbana-Champaign, Champaign, IL, USA

    Hans Friedrich Köhn

  2. Rutgers, The State University of New Jersey, New Brunswick, NJ, USA

    Chia-Yi Chiu

Authors
  1. Hans Friedrich Köhn
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chia-Yi Chiu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chia-Yi Chiu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Köhn, H., Chiu, CY. A Unified Theory of the Completeness of Q-Matrices for the DINA Model. J Classif 38, 500–518 (2021). https://doi.org/10.1007/s00357-021-09384-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00357-021-09384-7

Keywords

Search

Navigation