Incorporating Student Covariates in Cognitive Diagnosis Models
- 586 Downloads
In educational measurement, cognitive diagnosis models have been developed to allow assessment of specific skills that are needed to perform tasks. Skill knowledge is characterized as present or absent and represented by a vector of binary indicators, or the skill set profile. After determining which skills are needed for each assessment item, a model is specified for the relationship between item responses and skill set profiles. Cognitive diagnosis models are often used for diagnosis, that is, for classifying students into the different skill set profiles. Generally, cognitive diagnosis models do not exploit student covariate information. However, investigating the effects of student covariates, such as gender, SES, or educational interventions, on skill knowledge mastery is important in education research, and covariate information may improve classification of students to skill set profiles. We extend a common cognitive diagnosis model, the DINA model, by modeling the relationship between the latent skill knowledge indicators and covariates. The probability of skill mastery is modeled as a logistic regression model, possibly with a student-level random intercept, giving a higher-order DINA model with a latent regression. Simulations show that parameter recovery is good for these models and that inclusion of covariates can improve skill diagnosis. When applying our methods to data from an online tutor, we obtain reasonable and interpretable parameter estimates that allow more detailed characterization of groups of students who differ in their predicted skill set profiles.
KeywordsCognitive diagnosis model Collateral information Concomitant variables Covariates DIF DINA Higher order model Random effect Skill diagnosis
Unable to display preview. Download preview PDF.
- AYERS, E., NUGENT, R., and DEAN, N. (2009), “A Comparison of Student Skill Knowledge Estimates”, in Educational Data Mining 2009: 2nd International Conference on Educational Data Mining, Proceedings, eds. T. Barnes, M. Desmarais, C. Romero, and S.Ventura, Cordoba, Spain, pp.1-10, http://www.educationaldatamining.org/EDM2009/uploads/proceedings/ayers.pdf.
- BARNES, T.M. (2003), “The Q-matrix Method of Fault-Tolerant Teaching in Knowledge Assessment and Data Mining”, Ph.D. Thesis, Department of Computer Science, North Carolina State University, NC.Google Scholar
- BOZARD, J. (2010), “Invariance Testing in Diagnostic Classification Models”, Masters Thesis, The University of Georgia, Athens, GA.Google Scholar
- CHIU, C. (2008), “Cluster Analysis for Cognitive Diagnosis: Theory and Applications”, Ph.D. Thesis, Educational Psychology, University of Illinois, Urbana Champaign, IL.Google Scholar
- DE LA TORRE, J., and CHIU, C.Y. (2009), “A Generalized Index of Item Discrimination for Cognitive Diagnosis Models”, paper presented at the International Meeting of the Psychometric Society, Cambridge, England.Google Scholar
- HARTZ, S. (2002), “A Bayesian Framework for the Unified Model for Assessing Cognitive Abilities: Blending Theory with Practicality”, Ph.D Thesis, University of Illinois, Urbana-Champaign, IL.Google Scholar
- HEFFERNAN, N.T., KOEDINGER, K.R., and JUNKER, B.W. (2001), “Using Web-Based Cognitive Assessment Systems for Predicting Student Performance on State Exams”, research proposal to the Institute of Educational Statistics, US Department of Education; Department of Computer Science at Worcester Polytechnic Institute, Worcester County, MA.Google Scholar
- LI, F., and COHEN, A.S. (2006), “A Higher-Order DINA Rasch Model For Detection of Differential Item Functioning”, paper presented at the annual meeting of the Pacific Rim Objective Measurement Symposium, Hong Kong, People’s Republic of China.Google Scholar
- MUTHÉN, L. K., and MUTHÉN, B. O. (2010), Mplus User’s Guide (Sixth Ed.), Los Angeles, CA: Muthén & Muthén.Google Scholar
- R DEVELOPMENT CORE TEAM. (2004), “R: A Language and Environment for Statistical Computing”, R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0, http://www.R-project.org.
- RUPP, A., TEMPLIN, J., and HENSON, R. (2010), Diagnostic Measurement: Theory, Methods, and Applications, New York: The Guildford Press.Google Scholar
- SMIT, A., KELDERMAN, H., and VAN DER FLIER, H. (1999), “Collateral Information and Mixed Rasch Models”, Methods of Psychological Research Online, 4, 19–32.Google Scholar
- SMIT, A., KELDERMAN, H., and VAN DER FLIER, H. (2000), “The Mixed Birnbaum Model: Estimation using Collateral Information”, Methods of Psychological Research Online, 5, 31–43.Google Scholar
- SPIEGELHALTER, D.J., THOMAS, A., and BEST, N.G. (2003), WinBUGS: Bayesian Inference Using Gibbs Sampling, Manual Version 1.4, Cambridge: Medical Research Council Biostatistics Unit.Google Scholar
- STATACORP. (2009), Stata Statistical Software: Release 11, College Station, TX: Stata-Corp LP.Google Scholar
- TEMPLIN, J. (2004), “Generalized Linear Mixed Proficiency Models”, Ph.D. Thesis, University of Illinois, Urbana-Champaign, IL.Google Scholar
- TEMPLIN, J., HENSON, R., and DOUGLAS, J. (2007), “General Theory and Estimation of Cognitive Diagnosis Models: Using Mplus to DeriveModel Estimates”, Manuscript under Review.Google Scholar
- THISSEN, D., STEINBERG, L., and WAINER, H. (1988), “Use of Item Response Theory in the Study of Group Differences in Trace Lines”, in Test Validity, eds. H. Wainer and H. Braun, Hillsdale, NJ: Erlbaum, pp. 147–169.Google Scholar
- VON DAVIER, M. (2010), “Hierarchical Mixtures of Diagnostic Models”, Psychological Test and Assessment Modeling, 52, 8–28.Google Scholar
- XU, X., and VON DAVIER, M. (2008), “Fitting the Structural Diagnostic Model to NAEP Data”, Research Report RR-08-27, Princeton, NJ: Educational Testing Service.Google Scholar
- ZHANG, W. (2006), “Detecting Differential Item Functioning Using the DINA Model”, Ph.D. Thesis, The University of North Carolina at Greensboro, Greensboro, NC.Google Scholar