Skip to main content

When Cognitive Diagnosis Meets Computerized Adaptive Testing: CD-CAT


Computerized adaptive testing (CAT) is a mode of testing which enables more efficient and accurate recovery of one or more latent traits. Traditionally, CAT is built upon Item Response Theory (IRT) models that assume unidimensionality. However, the problem of how to build CAT upon latent class models (LCM) has not been investigated until recently, when Tatsuoka (J. R. Stat. Soc., Ser. C, Appl. Stat. 51:337–350, 2002) and Tatsuoka and Ferguson (J. R. Stat., Ser. B 65:143–157, 2003) established a general theorem on the asymptotically optimal sequential selection of experiments to classify finite, partially ordered sets. Xu, Chang, and Douglas (Paper presented at the annual meeting of National Council on Measurement in Education, Montreal, Canada, 2003) then tested two heuristics in a simulation study based on Tatsuoka’s theoretical work in the context of computerized adaptive testing. One of the heuristics was developed based on Kullback–Leibler information, and the other based on Shannon entropy. In this paper, we showcase the application of the optimal sequential selection methodology in item selection of CAT that is built upon cognitive diagnostic models. Two new heuristics are proposed, and are compared against the randomized item selection method and the two heuristics investigated in Xu et al. (Paper presented at the annual meeting of National Council on Measurement in Education, Montreal, Canada, 2003). Finally, we show the connection between the Kullback–Leibler-information-based approaches and the Shannon-entropy-based approach, as well as the connection between algorithms built upon LCM and those built upon IRT models.

This is a preview of subscription content, access via your institution.


  • Chang, H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213–229.

    Article  Google Scholar 

  • Chang, H., & Zhang, J. (2002). Hypergeometric family and item overlap rates in computerized adaptive testing. Psychometrika, 67, 387–398.

    Article  Google Scholar 

  • Cover, T.M., & Thomas, J.A. (1991). Elements of information theory. New York: Wiley.

    Book  Google Scholar 

  • Embretson, S.E. (2001). The second century of ability testing: Some predictions and speculations. Retrievable at

  • Haertel, E.H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301–321.

    Article  Google Scholar 

  • Haertel, E.H. (1984). An application of latent class models to assessment data. Applied Psychological Measurement, 8, 333–346.

    Article  Google Scholar 

  • Haertel, E.H., & Wiley, D.E. (1993). Presentations of ability structures: Implications for testing. In N. Frederiksen, R.J. Mislevey, & I.I. Bejar (Eds.). Test theory for a new generation of tests (pp. 359–384). Hillsdale: Erlbaum.

    Google Scholar 

  • Hambleton, R. & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer-Nijhoff.

    Google Scholar 

  • Hartz, S. (2002). A Bayesian framework for the Unified Model for assessing cognitive abilities: blending theory with practice. Unpublished doctoral thesis, University of Illinois at Urbana-Champaign.

  • Hartz, S., Roussos, L., & Stout, W. (2002). Skill diagnosis: Theory and practice [Computer software user manual for Arpeggio software]. Princeton: ETS.

    Google Scholar 

  • Henson, R., & Douglas, J. (2005). Test construction for cognitive diagnosis. Applied Psychological Measurement, 29, 262–277.

    Article  Google Scholar 

  • Junker, B.W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.

    Article  Google Scholar 

  • Karelitz, T.M., & de la Torre, J. (2008). When do measurement models produce diagnostic information? An investigation of the assumptions of cognitive diagnosis modeling. In National Council on Measurement in Education Annual Meeting in New York, NY.

  • Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale: Erlbaum.

    Google Scholar 

  • McGlohen, M.K. (2004). The application of cognitive diagnosis and computerized adaptive testing to a large-scale assessment. Unpublished doctoral thesis, University of Texas at Austin.

  • McGlohen, M.K., & Chang, H. (2008). Combining computer adaptive testing technology with cognitively diagnostic assessment. Behavioral Research Methods, 40, 808–821.

    Article  Google Scholar 

  • Mislevy, R.J., Almond, R.G., Yan, D., & Steinberg, L.S. (1999). Bayes nets in educational assessment: Where do the numbers come from? In K.B. Laskey & H. Prade (Eds.). Proceedings of the fifteenth conference on uncertainty in artificial intelligence (pp. 437–446). San Mateo: Morgan Kaufmann.

    Google Scholar 

  • Rupp, A., & Templin, J. (2008). The effects of Q-matrix misspecification on parameter estimates and classification accuracy in the DINA model. Educational and Psychological Measurement, 68, 78–96.

    Article  Google Scholar 

  • Shannon, C.E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423.

    Google Scholar 

  • Stocking, M.L. (1994). Three practical issues for modern adaptive testing item pools (ETS Research Rep. No. 94-5). Princeton: Educational Testing Service.

  • Tatsuoka, C. (2002). Data analytic methods for latent partially ordered classification models. Journal of the Royal Statistical Society, Series C (Applied Statistics), 51, 337–350.

    Article  Google Scholar 

  • Tatsuoka, C., & Ferguson, T. (2003). Sequential classification on partially ordered sets. Journal of Royal Statistics, Series B, 65, 143–157.

    Article  Google Scholar 

  • Tatsuoka, K.K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.

    Article  Google Scholar 

  • Tatsuoka, K.K. (1995). Architecture of knowledge structures and cognitive diagnosis: a statistical pattern classification approach. In P. Nichols, S. Chipman, & R. Brennan (Eds.). Cognitively diagnostic assessments (pp. 327–359). Hillsdale: Erlbaum.

    Google Scholar 

  • Templin, J. (2006). CDM: cognitive diagnosis modeling using Mplus, user guide. Retrievable at:

  • Templin, J., & Henson, R. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305.

    Article  PubMed  Google Scholar 

  • Thissen, D., & Mislevy, R.J. (2000). Testing algorithms. In H. Wainer (Eds.). Computerized adaptive testing: A primer (pp. 101–133). Hillsdale: Erlbaum.

    Google Scholar 

  • van der Linden, W.J. (1998). Bayesian item-selection criteria for adaptive testing. Psychometrika, 63, 201–216.

    Article  Google Scholar 

  • von Davier, M. (2005). A general diagnostic model applied to language testing data (ETS Research Rep. No. RR-05-16). Princeton: ETS.

  • Xu, X., Chang, H., & Douglas, J. (2003). Computerized adaptive testing strategies for cognitive diagnosis. Paper presented at the annual meeting of National Council on Measurement in Education, Montreal, Canada.

  • Xu, X., & Douglas, J. (2006). Computerized adaptive testing under nonparametric IRT models. Psychometrika, 71, 121–137.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Ying Cheng.

Additional information

The author would like to thank the editors, anonymous reviewers, and Drs. Hua-Hua Chang and Jeff Douglas for their constructive suggestions.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Cheng, Y. When Cognitive Diagnosis Meets Computerized Adaptive Testing: CD-CAT. Psychometrika 74, 619–632 (2009).

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


  • optimal sequential selection
  • latent class model
  • computerized adaptive testing
  • cognitive diagnosis
  • item response theory