Abstract
Nonparametric cognitive diagnosis methods are useful in cognitive diagnosis modeling for calibration efficiency, especially when sample size is small or large, or the latent attributes are more complex. This article proposes the Mantel-Haenszel chi-squared statistic as an index for detecting the misspecification of latent attributes as well as testlet effects in nonparametric cognitive diagnosis methods. The proposed theoretical considerations are augmented by simulation studies conducted to assess the performance of the Mantel-Haenszel statistic under various conditions within the nonparametric diagnosis framework, with a special focus on situations were the set of latent abilities assumed to underlie the data was underspecified.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ayers, E., Nugent, R., & Dean, N. (2008). Skill set profile clustering based on student capability vectors computed from online tutoring data. In R. S. J. de Baker, T. Barnes, & J. E. Beck (Eds.), Educational data mining 2008: proceedings of the 1st International Conference on Educational Data Mining (pp. 210–217). Montréal: International Data Mining Society.
Barnes, T. (2003). The Q-matrix method of fault-tolerant teaching in knowledge assessment and data mining. North Carolina State University, USA. http://www.lib.ncsu.edu/resolver/1840.16/4612. Accessed March 27, 2011
Chen, W., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265–289.
Chen, J., de la Torre, J., & Zhang, Z. (2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50, 123–140.
Chen, Y., Liu, J., Xu, G., & Ying, Z. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110(510), 850–866.
Chiu, C. Y., & Douglas, J. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns. Journal of Classification, 30(2), 225–250.
Chiu, C., Douglas, J., & Li, X. (2009). Cluster analysis for cognitive diagnosis: theory and applications. Psychometrika, 74, 633–655.
de la Torre, J., & Douglas, J. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353.
Holland, W. P., & Rosenbaum, P. R. (1985). Conditional association and unidimensionality in monotone latent variable models (Research Report No. 85 – 47). Princeton: Educational Testing Service.
Holland, W. P., & Thayer, D. T. (1988). Differential item performance and the Mantel Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129–145). Hillsdale: LEA.
Junker, B. W. (2011). The role of nonparametric analysis in assessment modeling: then and now. In N. J. Dorans & S. Sinharay (Eds.), Looking back: proceedings of a conference in honor of Paul W. Holland (pp. 67–85). New York: Springer-Verlag.
Kunina-Habenicht, O., Rupp, A. A., & Wilhelm, O. (2012). The impact of model misspecification on parameter estimation and item-fit assessment in log-linear diagnostic classification models. Journal of Educational Measurement, 49, 59–81.
Li, S., Simon, R. M., & Gart, J. J. (1979). Small sample properties of the Mantel-Haenszel test. Biometrika, 66, 181–183.
Lim, Y. S. & Drasgow, F. (2017 – accepeted). Nonparametric calibration of item-by-attribute matrix. Multivariate Behavioral Research.
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading: Addison-Wesley.
Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of National Cancer Institute, 22, 719–748.
Park, Y., & Lee, Y. (2011). Diagnostic cluster analysis of mathematics skills. IERI Monograph Series: Issues and Methodologies in Large-Scale Assessments, 4, 75–107.
Rosenbaum, P. R. (1984). Testing the conditional independence and monotonicity assumption of item response theory. Psychometrika, 49, 425–436.
Rupp, A., Templin, J., & Henson, R. (2010). Diagnostic assessment: theory, methods, and applications. New York: Guilford.
Tatsuoka, K. (1983). Rule space: an approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.
Wainer, H., & Kiely, G. (1987). Item clusters and computerized adaptive testing: a case study for testlets. Journal of Educational Measurement, 24, 195–201.
Wang, S., & Douglas, J. (2015). Consistency of nonparametric classification in cognitive diagnosis. Psychometrika, 80, 85–100.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lim, Y.S., Drasgow, F. Conditional Independence and Dimensionality of Cognitive Diagnostic Models: a Test for Model Fit. J Classif 36, 295–305 (2019). https://doi.org/10.1007/s00357-018-9287-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00357-018-9287-5