Skip to main content

Hierarchical Diagnostic Classification Models: A Family of Models for Estimating and Testing Attribute Hierarchies

Abstract

Although latent attributes that follow a hierarchical structure are anticipated in many areas of educational and psychological assessment, current psychometric models are limited in their capacity to objectively evaluate the presence of such attribute hierarchies. This paper introduces the Hierarchical Diagnostic Classification Model (HDCM), which adapts the Log-linear Cognitive Diagnosis Model to cases where attribute hierarchies are present. The utility of the HDCM is demonstrated through simulation and by an empirical example. Simulation study results show the HDCM is efficiently estimated and can accurately test for the presence of an attribute hierarchy statistically, a feature not possible when using more commonly used DCMs. Empirically, the HDCM is used to test for the presence of a suspected attribute hierarchy in a test of English grammar, confirming the data is more adequately represented by hierarchical attribute structure when compared to a crossed, or nonhierarchical structure.

This is a preview of subscription content, access via your institution.

Figure 1.
Figure 2.
Figure 3.

References

  • Bradshaw, L.P., & Templin, J. (2013). Combining scaling and classification: a psychometric model for scaling ability and diagnosing misconceptions. Psychometrika. doi:10.1007/S11336-013-9350-4.

    Google Scholar 

  • Buck, G., & Tatsuoka, K.K. (1998). Application of the rule-space procedure to language testing: examining attributes of a free response listening test. Language Testing, 15, 119–157.

    Google Scholar 

  • Choi, H.-J. (2010). A model that combines diagnostic classification assessment with mixture item response theory models. Unpublished doctoral dissertation, University of Georgia, Athens, Georgia.

  • Croon, M. (1990). Latent class analysis with ordered latent classes. British Journal of Mathematical & Statistical Psychology, 43, 171–192.

    Article  Google Scholar 

  • Cui, Y., & Leighton, J.P. (2009). The hierarchy consistency index: evaluating person fit for cognitive diagnostic assessment. Journal of Educational Measurement, 46, 429–449.

    Article  Google Scholar 

  • de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199.

    Article  Google Scholar 

  • Gierl, M. J., Cui, Y., & Hunka, S. (2007a). Using connectionist models to evaluate examinees’ response patterns on tests. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.

  • Gierl, M.J., Leighton, J.P., & Hunka, S.M. (2007b). Using the attribute hierarchy method to make diagnostic inferences about respondents’ cognitive skills. In J.P. Leighton & M.J. Gierl (Eds.), Cognitive diagnostic assessment for education: theory and applications (pp. 242–274). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Gierl, M.J., Cui, Y., & Zhou, J. (2009). Reliability of attribute-based scoring in cognitive diagnostic assessment. Journal of Educational Measurement, 46, 293–313.

    Article  Google Scholar 

  • Haertel, E.H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 333–352.

    Article  Google Scholar 

  • Hartz, S. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: blending theory with practicality. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign.

  • Henson, R., & Templin, J. (2005). Hierarchical log-linear modeling of the joint skill distribution. Unpublished manuscript.

  • Henson, R., & Templin, J. (2007). Large-scale language assessment using cognitive diagnosis models. Paper presented at the annual meeting of the National Council for Measurement in Education in Chicago, Illinois.

  • Henson, R., Templin, J., & Willse, J. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.

    Article  Google Scholar 

  • Junker, B.W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.

    Article  Google Scholar 

  • Lazarsfeld, P.F., & Henry, N.W. (1968). Latent structure analysis. Boston: Houghton Mifflin.

    Google Scholar 

  • Leighton, J.P. & Gierl, M.J. (Eds.) (2007). Cognitive diagnostic assessment for education: theory and applications. Cambridge: Cambridge University Press.

    Google Scholar 

  • Leighton, J.P., Gierl, M.J., & Hunka, S.M. (2004). The attribute hierarchy model for cognitive assessment: a variation on Tatsuoka’s rule-space approach. Journal of Educational Measurement, 41, 205–237.

    Article  Google Scholar 

  • Leighton, J.P., Cui, Y., & Corr, M.K. (2009). Testing expert-based and student-based cognitive models: an application of the attribute hierarchy method and hierarchy consistency index. Applied Measurement in Education, 22, 229–254.

    Article  Google Scholar 

  • Lindsay, B., Clogg, C.C., & Grego, J. (1991). Semiparametric estimation in the Rasch model and related exponential response models, including a simple latent class model for item analysis. Journal of the American Statistical Association, 86, 96–107.

    Article  Google Scholar 

  • Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 197–212.

    Article  Google Scholar 

  • Maydeu-Olivares, A., & Joe, H. (2005). Limited- and full-information estimation and goodness-of-fit testing in 2n contingency tables: a unified framework. Journal of the American Statistical Association, 100, 1009–1020.

    Article  Google Scholar 

  • Muthén, L.K., & Muthén, B.O. (2013). Mplus user’s guide (Version 6.1) [Computer software and manual]. Los Angeles: Muthén & Muthén.

    Google Scholar 

  • Roussos, L., DiBello, L., Stout, W., Hartz, S., Henson, R., & Templin, J. (2007). The fusion model skills diagnosis system. In J. Leighton & M. Gierl (Eds.), Cognitive diagnostic assessment in education (pp. 275–318). New York: Cambridge University Press.

    Chapter  Google Scholar 

  • Rupp, A., Templin, J., & Henson, R. (2010). Diagnostic measurement: theory, methods, and applications. New York: Guilford.

    Google Scholar 

  • Shapiro, A. (1985). Asymptotic distribution of test statistics in the analysis of moment structures under inequality constraints. Biometrika, 72, 133–144.

    Article  Google Scholar 

  • Stoel, R.D., Garre, F.G., Dolan, C., & van den Wittenboer, G. (2006). On the likelihood ratio test in structural equation modeling when parameters are subject to boundary constraints. Psychological Methods, 11, 439–455.

    PubMed  Article  Google Scholar 

  • Tatsuoka, K.K. (1983). Rule space: an approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.

    Article  Google Scholar 

  • Tatsuoka, K.K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. In N. Fredrickson, R.L. Glaser, A.M. Lesgold, & M.G. Shafto (Eds.), Diagnostic monitoring of skills and knowledge acquisition (pp. 453–488). Hillsdale: Erlbaum.

    Google Scholar 

  • Tatsuoka, K.K. (1993). Item construction and psychometric models appropriate for constructed responses. In R.E. Bennett & W.C. Ward (Eds.), Construction versus choice in cognitive measurement (pp. 107–133). Hillsdale: Erlbaum.

    Google Scholar 

  • Tatsuoka, K.K. (1995). Architecture of knowledge structures and cognitive diagnosis: a statistical pattern recognition and classification approach. In P.D. Nichols, S.F. Chipman, & R.L. Brennan (Eds.), Cognitively diagnostic assessment (pp. 327–359). Hillsdale: Erlbaum.

    Google Scholar 

  • Tatsuoka, K.K. (2009). Cognitive assessment: an introduction to the rule space method. New York: Routledge.

    Google Scholar 

  • Templin, J. (2006). CDM user’s guide. Unpublished manuscript.

  • Templin, J., & Bradshaw, L. (2013). Measuring the reliability of diagnostic classification model examinee estimates. Journal of Classification, 30, 251–275.

    Article  Google Scholar 

  • Templin, J., & Henson, R. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11(3), 287–305.

    PubMed  Article  Google Scholar 

  • Templin, J., & Hoffman, L. (2013). Obtaining diagnostic classification model estimates using Mplus. Educational Measurement: Issues and Practice, 32(2), 37–50.

    Article  Google Scholar 

  • Templin, J., Rupp, A., Henson, R., Jang, E., & Ahmed, M. (2008). Nominal response diagnostic models. Paper presented at the annual meeting of the National Council on Measurement in Education in New York, NY.

  • Verbeke, G., & Molenberghs, G. (2000). Linear mixed models for longitudinal data. New York: Springer.

    Google Scholar 

  • von Davier, M. (2005). A general diagnostic model applied to language testing data (ETS Research Report RR-05-16).

  • von Davier, M., & Yamamoto, K. (2004). A class of models for cognitive diagnosis. Paper presented at the 4th Spearman Conference in Philadelphia, PA.

  • Xu, X., & von Davier, M. (2008). Fitting the structured general diagnostic model to NAEP data (RR-08-27). Princeton: Educational Testing Service.

    Google Scholar 

Download references

Acknowledgements

This research was supported by the National Science Foundation under grants DRL-0822064, SES-0750859, and SES-1030337. The opinions expressed are those of the authors and do not necessarily reflect the views of NSF.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jonathan Templin.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Templin, J., Bradshaw, L. Hierarchical Diagnostic Classification Models: A Family of Models for Estimating and Testing Attribute Hierarchies. Psychometrika 79, 317–339 (2014). https://doi.org/10.1007/s11336-013-9362-0

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-013-9362-0

Key words

  • diagnostic classification models
  • cognitive diagnosis
  • attribute hierarchies
  • LCDM
  • latent class models