Advertisement

Estimating CDMs Using MCMC

  • Xiang Liu
  • Matthew S. JohnsonEmail author
Chapter
Part of the Methodology of Educational Measurement and Assessment book series (MEMA)

Abstract

In this chapter, we provide a brief survey of Markov chain Monte Carlo (MCMC) methods used in estimating Cognitive Diagnostic Models (CDMs). MCMC techniques have been widely used for the Bayesian estimation of psychometric models. MCMC algorithms and general purpose MCMC software has been facilitating the development of modern psychometric models that are otherwise difficult to fit (Levy R, J Probab Stat 1–18, 2009. Retrieved from http://www.hindawi.com/journals/jps/2009/537139/, https://doi.org/10.1155/2009/ 537139). We introduce a Gibbs sampler for fitting the saturated Log-linear CDM model (LCDM, Henson RA, Templin JL, Willse JT, Psychometrika, 74(2):191–210, 2009. Retrieved from https://doi.org/10.1007/s11336-008-9089-5). The utility of Bayesian inference is demonstrated by analyzing the Examination for the Certificate of Proficiency in English (ECPE) dataset.

References

  1. Brooks, S., & Gelman, A. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7(4), 434–455. https://doi.org/10.1080/10618600.1998.10474787 Google Scholar
  2. Brooks, S., Gelman, A., Jones, G. L., & Meng, X.-L. (2011). Handbook of Markov chain Monte Carlo. Handbook of Markov chain Monte Carlo. https://doi.org/10.1201/b10905
  3. Cai, L. (2010). High-dimensional exploratory item factor analysis by a Metropolis-Hastings Robbins-Monro algorithm. Psychometrika, 75(1), 33–57. https://doi.org/10.1007/s11336-009-9136-x CrossRefGoogle Scholar
  4. Chen, Y., Culpepper, S. A., Chen, Y., & Douglas, J. (2018). Bayesian estimation of the DINA Q matrix. Psychometrika, 83(1), 89–108. Retrieved from https://doi.org/10.1007/s11336-017-9579-4 CrossRefGoogle Scholar
  5. Chen, Y., Liu, J., Xu, G., & Ying, Z. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110(510), 850–866. Retrieved from https://doi.org/10.1080/01621459.2014.934827 CrossRefGoogle Scholar
  6. Chung, M.-t. (2014). Estimating the Q-matrix for cognitive diagnosis models in a Bayesian framework. Doctoral dissertation, Columbia University. Retrieved from https://search.proquest.com/docview/1548332406
  7. Culpepper, S. A. (2015) Bayesian estimation of the DINA model with Gibbs sampling. Journal of Educational and Behavioral Statistics, 40(5), 454–476. Retrieved from https://doi.org/10.3102/1076998615595403 CrossRefGoogle Scholar
  8. DeCarlo, L. T. (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model. Applied Psychological Measurement, 36(6), 447–468. Retrieved from https://doi.org/10.1177/0146621612449069 CrossRefGoogle Scholar
  9. DeCarlo, L. T., & Kinghorn, B. R. (2016). An Exploratory Approach to the Q-Matrix Via Bayesian Estimation. (Paper presented at the meeting of the National Council on Measurement in Education, Washington, DC)Google Scholar
  10. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B, 39(1), 1–38. Retrieved from http://www.jstor.org/stable/2984875, https://doi.org/10.2307/2984875
  11. de la Torre, J. (2008). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34(1), 115–130. Retrieved from https://doi.org/10.3102/1076998607309474 CrossRefGoogle Scholar
  12. de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76(2), 179–199. https://doi.org/10.1007/s11336-011-9207-7 CrossRefGoogle Scholar
  13. de la Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69(3), 333–353. Retrieved from https://doi.org/10.1007/BF02295640 CrossRefGoogle Scholar
  14. Gelfand, A. E., & Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85(410), 398–409. Retrieved from https://doi.org/10.1080/01621459.1990.10476213 CrossRefGoogle Scholar
  15. Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian data analysis (3 ed.). Boca Raton: Chapman and Hall/CRC.Google Scholar
  16. Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science. https://doi.org/10.1214/ss/1177011136
  17. Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-6(6), 721–741. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/22499653,  https://doi.org/10.1109/TPAMI.1984.4767596
  18. George, A. C., Robitzsch, A., Kiefer, T., Groß, J., & Ünlü, A. (2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software, 74(2). Retrieved from http://www.jstatsoft.org/v74/i02/  https://doi.org/10.18637/jss.v074.i02
  19. Geweke, J. (1992). Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments. In J. M. Bernardo, J. O. Berger, A. P. Dawid, & A. F. M. Smith (Eds.), Bayesian statistics 4. Oxford: Oxford University Press.Google Scholar
  20. Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika, 82(4), 711–732. Retrieved from  https://doi.org/10.1093/biomet/82.4.711 CrossRefGoogle Scholar
  21. Hartz, S. M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Doctoral dissertation, University of Illinois at Urbana-Champaign. Retrieved from http://hdl.handle.net/2142/87393
  22. Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1), 97–109. Retrieved from http://www.jstor.org/stable/2334940 CrossRefGoogle Scholar
  23. Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74(2), 191–210. Retrieved from https://doi.org/10.1007/s11336-008-9089-5 CrossRefGoogle Scholar
  24. Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272. Retrieved from https://doi.org/10.1177/01466210122032064 CrossRefGoogle Scholar
  25. Levy, R. (2009). The rise of Markov chain Monte Carlo estimation for psychometric modeling. Journal of Probability and Statistics, 2009, 1–18. Retrieved from http://www.hindawi.com/journals/jps/2009/537139/, https://doi.org/10.1155/2009/537139
  26. Li, F., Cohen, A., Bottge, B., & Templin, J. (2016). A latent transition analysis model for assessing change in cognitive skills. Educational and Psychological Measurement, 76(2), 181–204. Retrieved from https://doi.org/10.1177/0013164415588946 CrossRefGoogle Scholar
  27. Liu, J., Xu, G., & Ying, Z. (2012). Data-driven learning of Q-matrix. Applied Psychological Measurement, 36(7), 548–564. Retrieved from https://doi.org/10.1177/0146621612456591 CrossRefGoogle Scholar
  28. Liu, J., Xu, G., & Ying, Z. (2013). Theory of self-learning Q-matrix. Bernoulli, 19(5), 1790–1817. Retrieved from https://doi.org/10.3150/12-BEJ430 CrossRefGoogle Scholar
  29. Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. (2009) The BUGS project: Evolution, critique and future directions. Statistics in Medicine, 28(25), 3049–3067. Retrieved from  https://doi.org/10.1002/sim.3680 CrossRefGoogle Scholar
  30. Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953). Equation of state calculations by fast computing machines. Journal Chemical Physics. https://doi.org/10.1063/1.1699114
  31. Neal, R. M. (1998). Probabilistic inference using Markov chain Monte Carlo methods. Technical Report, 1, 1–144. Retrieved from papers2://publication/uuid/0C88167E-5379-4E4E-A9E4-007ABA4F716D, https://doi.org/10.1021/np100920q
  32. Park, J. Y., Johnson, M. S., & Lee, Y. S. (2015). Posterior predictive model checks for cognitive diagnostic models. International Journal of Quantitative Research in Education, 2(3/4), 244. Retrieved from http://www.inderscience.com/link.php?id=71738,  https://doi.org/10.1504/IJQRE.2015.071738
  33. Plummer, M. (2005). JAGS: Just another Gibbs sampler. In Proceedings of the 3rd International Workshop on Distributed Statistical Computing (dsc 2003). Google Scholar
  34. Roberts, G. O., Gelman, A., & Gilks, W. R. (1997). Weak convergence and optimal scaling of random walk Metropolis algorithms. Annals of Applied Probability, 7(1), 110–120.  https://doi.org/10.1214/aoap/1034625254 CrossRefGoogle Scholar
  35. Rubin, D. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. The Annals of Statistics, 12(4), 1151–1172. Retrieved from http://projecteuclid.org/euclid.aos/1176346785,  https://doi.org/10.1214/aos/1176346785
  36. Sinharay, S. (2005). Assessing fit of unidimensional item response theory models using a Bayesian approach. Journal of Educational Measurement, 42(4), 375–394. Retrieved from https://doi.org/10.1111/j.1745-3984.2005.00021.x CrossRefGoogle Scholar
  37. Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & van der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society. Series B, 64(4), 583–639.CrossRefGoogle Scholar
  38. Spiegelhalter, D. J., Thomas, A., Best, N., & Lunn, D. (2014). OpenBUGS User Manual (Vol. 164). Retrieved from http://www.openbugs.net/Manuals/Manual.html
  39. Tatsuoka, K. K. (1983) Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20(4), 345–354. Retrieved from https://doi.org/10.1111/j.1745-3984.1983.tb00212.x CrossRefGoogle Scholar
  40. Templin, J., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79(2), 317–339. Retrieved from https://doi.org/10.1007/s11336-013-9362-0 CrossRefGoogle Scholar
  41. Templin, J., & Hoffman, L. (2013). Obtaining diagnostic classification model estimates using Mplus. Educational Measurement: Issues and Practice, 32(2), 37–50.  https://doi.org/10.1111/emip.12010 CrossRefGoogle Scholar
  42. Thomas, A., Spiegelhalter, D. J., & Gilks, W. R. (1992). BUGS: A program to perform Bayesian inference using Gibbs sampling. In J. M. Bernardo, J. O. Berger, A. P. Dawid, & A. F. M. Smith (Eds.), Bayesian statistics 4. Oxford: Oxford University Press.Google Scholar
  43. von Davier, M. (2008). A general diagnostic model applied to language testing data. The British Journal of Mathematical and Statistical Psychology, 61(Pt 2), 287–307. https://doi.org/10.1348/000711007X193957 CrossRefGoogle Scholar
  44. von Davier, M. (2014). The log-linear cognitive diagnostic model (LCDM) as a special case of the general diagnostic model (GDM). ETS Research Report Series, 2014(2), 1–13. Retrieved from  https://doi.org/10.1002/ets2.12043 CrossRefGoogle Scholar
  45. Xu, G., & Zhang, S. (2016). Identifiability of diagnostic classification models. Psychometrika, 81(3), 625–649. https://doi.org/10.1007/s11336-015-9471-z CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Human Development, Teachers CollegeColumbia UniversityNew YorkUSA
  2. 2.Educational Testing ServicePrincetonUSA

Personalised recommendations