Advertisement

An Attribute-Specific Item Discrimination Index in Cognitive Diagnosis

  • Lihong SongEmail author
  • Wenyi Wang
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 265)

Abstract

There lacks an item quality index as a measure of item’s correct classification rates of attributes. The purpose of this study is to propose an attribute-specific item discrimination index as a measure of correct classification rate of attributes based on a q-vector, item parameters, and the distribution of attribute patterns. First, an attribute-specific item discrimination index was introduced. Second, a heuristic method was presented using the new index for test construction. The first simulation results showed that the new index performed well in that their values matched closely with the simulated correct classification rates of attributes across different conditions. The second simulation study results showed that the heuristic method based on the sum of the attributes’ indices yielded comparable performance to the famous CDI. The new index provides test developers with a useful tool to evaluate the quality of diagnostic items. It will be valuable to explore the applications and advantages of using the new index for developing an item selection algorithm or a termination rule in cognitive diagnostic computerized adaptive testing.

Keywords

Cognitive diagnosis Item discrimination index Correct classification rate Test construction The deterministic inputs Noisy “and” gate model 

Notes

Acknowledgments

This research was supported by the Key Project of National Education Science “Twelfth Five Year Plan” of Ministry of Education of China (Grant No. DHA150285). The authors would like to thank the editor Steve Culpepper for reviewing an earlier version of this work.

References

  1. Chang, H.-H. (2015). Psychometrics behind computerized adaptive testing. Psychometrika, 80(1), 1–20.MathSciNetCrossRefGoogle Scholar
  2. Chang, H.-H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20(3), 213–229.CrossRefGoogle Scholar
  3. Chen, P., Xin, T., Wang, C., & Chang, H.-H. (2012). On-line calibration methods for the DINA model with independent attributes in CD-CAT. Psychometrika, 77(2), 201–222.MathSciNetCrossRefGoogle Scholar
  4. Chen, Y., Liu, Y., & Xu, S. (2018). Mutual information reliability for latent class analysis. Applied Psychological Measurement, 42(6), 460–477.CrossRefGoogle Scholar
  5. Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive testing: CD–CAT. Psychometrika, 74(4), 619–632.MathSciNetCrossRefGoogle Scholar
  6. Chiu, C.-Y., Douglas, J. A., & Li, X.-D. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74(4), 633–665.MathSciNetCrossRefGoogle Scholar
  7. Cui, Y., Gierl, M. J., & Chang, H.-H. (2012). Estimating classification consistency and accuracy for cognitive diagnostic assessment. Journal of Educational Measurement, 49(1), 19–38.CrossRefGoogle Scholar
  8. de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199.MathSciNetCrossRefGoogle Scholar
  9. de la Torre, J., Hong, Y., & Deng, W. L. (2010). Factors affecting the item parameter estimation and classification accuracy of the DINA model. Journal of Educational Measurement, 47(2), 227–249.CrossRefGoogle Scholar
  10. Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26(4), 301–321.CrossRefGoogle Scholar
  11. Henson, R., & Douglas, J. (2005). Test construction for cognitive diagnosis. Applied Psychological Measurement, 29(4), 262–277.MathSciNetCrossRefGoogle Scholar
  12. Henson, R., Roussos, L., Douglas, J., & He, X. (2008). Cognitive diagnostic attribute-level discrimination indices. Applied Psychological Measurement, 32(4), 275–288.MathSciNetCrossRefGoogle Scholar
  13. Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.MathSciNetCrossRefGoogle Scholar
  14. Huebner, A., & Wang, C. (2011). A note on comparing examinee classification methods for cognitive diagnosis models. Educational and Psychological Measurement, 71(2), 407–419.CrossRefGoogle Scholar
  15. Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272.MathSciNetCrossRefGoogle Scholar
  16. Kuo, B.-C., Pai, H.-S., & de la Torre, J. (2016). Modified cognitive diagnostic index and modified attribute-level discrimination index for test construction. Applied Psychological Measurement, 40(5), 315–330.CrossRefGoogle Scholar
  17. Lehmann, E. L., & Casella, G. (1998). Theory of point estimation (2nd ed.). New York: Springer.zbMATHGoogle Scholar
  18. Rupp, A. A., Templin, J. L., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York: The Guilford Press.Google Scholar
  19. Sun, J., Xin, T., Zhang, S., & de la Torre, J. (2013). A polytomous extension of the generalized distance discriminating method. Applied Psychological Measurement, 37(7), 503–521.CrossRefGoogle Scholar
  20. Theodoridis, S., & Koutroumbas, K. (2009). Pattern recognition (4th ed.). London: Elsevier.zbMATHGoogle Scholar
  21. von Davier, M. (2005). A general diagnostic model applied to language testing data (ETS RR-05–16). Princeton, NJ: Educational Testing Service.Google Scholar
  22. Wang, W. Y., Song, L. H., Chen, P., Meng, Y. R., & Ding, S. L. (2015). Attribute-level and pattern-level classification consistency and accuracy indices for cognitive diagnostic assessment. Journal of Educational Measurement, 52(4), 457–476.CrossRefGoogle Scholar
  23. Wang, W. Y., Song, L. H., & Ding, S. L. (2018). An item discrimination index and its application in cognitive diagnostic assessment on a classification oriented view. Journal of Psychological Science, 41(2), 475–483. Google Scholar
  24. Wang, W. Y., Song, L. H., Chen, P., & Ding, S. L. (2019). An item-level expected classification accuracy and its applications in cognitive diagnostic assessment. Journal of Educational Measurement, 56(1), 51–75.Google Scholar
  25. Xia, M. L., Mao, X. Z., & Yang, R. (2018). Cognitive diagnosis models under polytomous attributes and polytomous item. Journal of Jiangxi Normal University (Natural Science), 42(2), 134–138.zbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Jiangxi Normal UniversityNanchangPeople’s Republic of China

Personalised recommendations