Abstract
This paper presents a noncompensatory latent trait model, the multicomponent latent trait model for diagnosis (MLTM-D), for cognitive diagnosis. In MLTM-D, a hierarchical relationship between components and attributes is specified to be applicable to permit diagnosis at two levels. MLTM-D is a generalization of the multicomponent latent trait model (MLTM; Whitely in Psychometrika, 45:479–494, 1980; Embretson in Psychometrika, 49:175–186, 1984) to be applicable to measures of broad traits, such as achievement tests, in which component structure varies between items. Conditions for model identification are described and marginal maximum likelihood estimators are presented, along with simulation data to demonstrate parameter recovery. To illustrate how MLTM-D can be used for diagnosis, an application to a large-scale test of mathematics achievement is presented. An advantage of MLTM-D for diagnosis is that it may be more applicable to large-scale assessments with more heterogeneous items than are latent class models.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: application of an EM algorithm. Psychometrika, 46, 443–445.
Bolt, D.M., & Lall, V.F. (2003). Estimation of compensatory and noncompensatory multidimensional item response models using Markov chain Monte Carlo. Applied Psychological Measurement, 27, 395–414.
Cai, L. (2010). Metropolis-Hastings Robbins-Monroe algorithm for confirmatory factor analysis. Journal of Educational and Behavioral Statistics, 35, 307–335.
Cai, L., du Toit, S.H.C., & Thissen, D. (2012). IRTPRO: Flexible, multidimensional, multiple category IRT modeling. Chicago: Scientific Software International. Computer software.
Cho, S.-J., & Rabe-Hesketh, S. (2011). Alternating imputation posterior estimation of models with crossed random effects. Computational Statistics & Data Analysis, 55, 12–25.
Daniel, R.C., & Embretson, S.E. (2010). Designing cognitive complexity in mathematical problem solving items. Applied Psychological Measurement, 348–364.
DeCarlo, L.T. (2011). The analysis of fraction subtraction data: the DINA model, classification, latent class sizes, and the Q-matrix. Applied Psychological Measurement, 35, 8–26.
de la Torre, J., & Chiu, C.-Y. (2009). Q-matrix validation under the generalized DINA model framework. In The international meeting of the psychometric society, Cambridge, St John’s College, July 20th–24th.
de la Torre, J., & Douglas, J.A. (2004). Higher order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353.
DiBello, L.V., Stout, W.F., & Roussos, L.A. (1995). Unified cognitive/psychometric diagnostic assessment likelihood-based classification techniques. In P.D. Nichols, S.F. Chipman, & R.L. Brennan (Eds.), Cognitively diagnostic assessment (pp. 327–359). Erlbaum: Hillsdale.
Embretson, S.E. (1984). A general multicomponent latent trait model for response processes. Psychometrika, 49, 175–186.
Embretson, S.E. (1995). Working memory capacity versus general central processes in intelligence. Intelligence, 20, 169–189.
Embretson, S.E. (1999). Generating items during testing: psychometric issues and models. Psychometrika, 64, 407–433.
Embretson, S.E. (2010). An adaptive testing system for diagnosing sources of mathematics difficulties. Project R305A100234. Washington, Institute of Educational Sciences.
Embretson, S.E., & Daniel, R.C. (2008). Understanding and quantifying cognitive complexity level in mathematical problem solving items. Psychology Science, 50, 328–344.
Embretson, S.E., & McCollam, K.M. (2000). A multicomponent Rasch model for covert processes. In M. Wilson & G. Engelhard (Eds.), Objective measurement V. Norwood: Ablex.
Fischer, G.H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359–374.
Gorin, J. (2007). Test design with cognition in mind. Educational Measurement, Issues and Practice, 4, 21–35.
Haertel, E.H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 333–352.
Hartz, S. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: blending theory with practice. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign.
Henson, R.A., Templin, J.L., & Willse, J.T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.
Junker, B.W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.
Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 197–212.
Mislevy, R.J., & Bock, R.D. (1990). BILOG 3: item analysis and test scoring with binary logistic models. Chicago: Scientific Software International.
National Mathematics Advisory Panel (2008). Foundations for success: final report. U.S. Department of Education.
Pellegrino, J., Goldman, S., DiBello, L., Gomez, K., & Stout, W. (2011). Evaluating the cognitive, psychometric and instructional affordances of curriculum-embedded assessments: a comprehensive validity-based approach. Chicago: Learning Sciences Institute, University of Illinois.
Poggio, J.P. (2011). Indicators, benchmarks and standards reflected in items for mathematical achievement tests in middle school (Technical Report KU-10012011). Institute of Educational Science Mathematics Diagnosis Project, Lawrence, Kansas.
Roussos, L., DiBello, L., Henson, R., Jang, E., & Templin, J. (2010). Skills diagnosis for education and psychology with IRT-based parametric latent class models. In S.E. Embretson (Ed.), Measuring psychological constructs: advances in model-based approaches. Washington: American Psychological Association.
Rupp, A., & Templin, J. (2008). The effects of Q-matrix misspecification on parameter estimates and classification accuracy in the DINA model. Educational and Psychological Measurement, 68, 78–96.
Rupp, A.A., Templin, J., & Henson, R.J. (2010). Diagnostic measurement: theory, methods, and applications. New York: Guilford Press.
Stout, W. (2007). Skills diagnosis using IRT-based continuous latent trait models. Journal of Educational Measurement, 44, 313–324.
Tatsuoka, K.K. (1985). A probabilistic model for diagnosing misconceptions in the pattern classification approach. Journal of Educational Statistics, 12, 55–73.
Templin, J.L., & Henson, R.A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305.
von Davier, M. (2005). A general diagnostic model applied to language testing data (Research Report RR-05-16). Princeton: ETS.
von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical & Statistical Psychology, 61, 287–307.
von Davier, M. (2009). Some notes on the reinvention of latent structure models as diagnostic classification models. Measurement: Interdisciplinary Research and Perspectives, 7(1), 67–74.
von Davier, M., & Sinharay, S. (2010). Stochastic approximation for latent regression item response models. Journal of Educational and Behavioral Statistics, 35, 174–193.
von Davier, M., & Yamamoto, K. (2004). A class of models for cognitive diagnosis. Paper presented at the 4th Spearman conference. Educational testing service: the inn at Penn. Philadelphia, October.
von Davier, M., & Yamamoto, K. (2007). Mixture distribution Rasch models and hybrid Rasch models. In M. Davier & C.H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models. New York: Springer.
Webb, N.L. (1999). Alignment of science and mathematics standards and assessments in four states (Research Monograph No. 18). Madison, National Institute for Science Education University of Wisconsin-Madison.
Whitely, S.E. (1980). Multicomponent latent trait models for ability tests. Psychometrika, 45, 479–494.
Yang, X. (2010). Construct theory-driven cognitive diagnostic testing in the domain of algebra story problems. Shanghai: National Science Foundation of China.
Yang, X., & Embretson, S.E. (2007). Construct validity and cognitive diagnostic assessment. In J.P. Leighton & M. Gierl (Eds.), Cognitive diagnostic assessment for education. New York: Cambridge University Press.
Author information
Authors and Affiliations
Corresponding author
Additional information
Preparation of this paper was partially supported by Institute of Educational Sciences Grant R305A100234, Susan Embretson, PI.
Rights and permissions
About this article
Cite this article
Embretson, S.E., Yang, X. A Multicomponent Latent Trait Model for Diagnosis. Psychometrika 78, 14–36 (2013). https://doi.org/10.1007/s11336-012-9296-y
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-012-9296-y