Skip to main content

Advertisement

SpringerLink
Log in

Using Penalized EM Algorithm to Infer Learning Trajectories in Latent Transition CDM

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

Cognitive diagnostic models (CDMs) have arisen as advanced psychometric models in the past few decades for assessments that intend to measure students’ mastery of a set of attributes. Recently, quite a few studies attempted to extend CDMs to longitudinal versions, and they all tended to model transition probabilities from non-mastery to mastery or vice versa for each attribute separately, with an exception of a few studies (e.g., Chen et al. 2018; Madison & Bradshaw 2018). However, these pioneering works have not taken into consideration the attribute relationships and the ever-changing attributes in a learning period. In this paper, we consider a profile-level latent transition CDM (TCDM hereafter), which can not only identify transition probabilities across the same attributes over time, but also the transition pathways across different attributes. Two versions of the penalized expectation-maximization (PEM) algorithms are proposed to shrink the probabilities associated with impermissible transition pathways to 0 and, thereby, help explore attribute relationships in a longitudinal setting. Simulation results reveal that PEM with group penalty holds great promise for identifying learning trajectories.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. MATLAB R2018b is used for the study. When \(K_{1}+K_{2}=15\) and \(\left| {{\varvec{T}}}^{\gamma } \right| =100\) for instance, it returns “Inf.”

  2. These sample sizes were chosen in proportion of to the total number of latent classes.

  3. Gu and Xu (2019) used the threshold value \(\rho _{N}=\frac{1}{2N}\), and we considered a smaller threshold to avoid shrinking too many entries to 0.

References

  • Bolt, D. M., & Lall, V. F. (2003). Estimation of compensatory and noncompensatory multidimensional item response models using Markov chain Monte Carlo. Applied Psychological Measurement, 27(6), 395–414.

    Article  Google Scholar 

  • Briggs, D. C., Alonzo, A. C., Schwab, C., & Wilson, M. (2006). Diagnostic assessment with ordered multiple-choice items. Educational Assessment, 11(1), 33–63.

    Article  Google Scholar 

  • Broman, K. W., & Speed, T. P. (2002). A model selection approach for the identification of quantitative trait loci in experimental crosses. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(4), 641–656.

    Article  Google Scholar 

  • Chen, J., & Chen, Z. (2008). Extended Bayesian information criteria for model selection with large model spaces. Biometrika, 95(3), 759–771.

    Article  Google Scholar 

  • Chen, Y., Culpepper, S. A., Wang, S., & Douglas, J. (2018). A hidden Markov model for learning trajectories in cognitive diagnosis with application to spatial rotation skills. Applied Psychological Measurement, 42(1), 5–23.

    Article  PubMed  Google Scholar 

  • Chung, H., Flaherty, B. P., & Schafer, J. L. (2006). Latent class logistic regression: Application to marijuana use and attitudes among high school seniors. Journal of the Royal Statistical Society: Series A (Statistics in Society), 169(4), 723–743.

    Article  Google Scholar 

  • Chung, H., Lanza, S. T., & Loken, E. (2008). Latent transition analysis: Inference and estimation. Statistics in Medicine, 27(11), 1834–1854.

    Article  PubMed  PubMed Central  Google Scholar 

  • Cizek, G. J., Bunch, M. B., & Koons, H. (2004). Setting performance standards: Contemporary methods. Educational Measurement: Issues and Practice, 23(4), 31–31.

    Article  Google Scholar 

  • Choi, I., & Wilson, M. (2016). An investigation on the measurement model for complex learning progressions: Structured constructs model for the continuous latent trait (SCM-C). Paper presented at the annual meeting of National Council on Measurement in Education.

  • Collins, L. M., & Lanza, S. T. (2010). Latent class analysis with covariates (pp. 149–177). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences.

  • Collins, L. M., & Wugalter, S. E. (1992). Latent class models for stage-sequential dynamic latent variables. Multivariate Behavioral Research, 27(1), 131–157.

    Article  Google Scholar 

  • Corcoran, T. B., Mosher, F. A., & Rogat, A. (2009). Learning progressions in science: An evidence-based approach to reform.

  • Daro, P., Mosher, F. A., & Corcoran, T. B. (2011). Learning trajectories in mathematics: A foundation for standards, curriculum, assessment, and instruction.

  • De La Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76(2), 179–199.

    Article  Google Scholar 

  • Deng, N., Roussos, L., & LaFond, L. (2017). Multidimensional modeling of learning progression-based vertical scales. Paper presented at the annual meeting of the National Council on Measurement in Educaiton, San Antonio, TX.

  • Diakow, R., Irribarra, D., & Wilson, M. (2011). Analyzing the complex structure of a learning progression: Structured construct models. Paper presented at the annual meeting of the national council of measurement in education, New Orleans, LA.

  • Embretson, S. (1984). A general latent trait model for response processes. Psychometrika, 49(2), 175–186.

    Article  Google Scholar 

  • Embretson, S. E. (1997). Multicomponent response models. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 305–321). Berlin: Springer.

    Chapter  Google Scholar 

  • Embretson, S. E., & Yang, X. (2013). A multicomponent latent trait model for diagnosis. Psychometrika, 78(1), 14–36.

    Article  PubMed  Google Scholar 

  • Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360.

    Article  Google Scholar 

  • Fan, J., & Peng, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. The Annals of Statistics, 32(3), 928–961.

    Article  Google Scholar 

  • Fan, Y., & Tang, C. Y. (2013). Tuning parameter selection in high dimensional penalized likelihood. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 75(3), 531–552.

    Article  Google Scholar 

  • Gu, Y., & Xu, G. (2019). Learning attribute patterns in high-dimensional structured latent attribute models. Journal of Machine Learning, 20, 1–58.

    Google Scholar 

  • Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26(4), 301–321.

    Article  Google Scholar 

  • Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, inference, and prediction. Berlin: Springer.

    Book  Google Scholar 

  • Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74(2), 191.

    Article  Google Scholar 

  • Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272.

    Article  Google Scholar 

  • Kaya, Y., & Leite, W. L. (2017). Assessing change in latent skills across time with longitudinal cognitive diagnosis modeling: An evaluation of model performance. Educational and Psychological Measurement, 77(3), 369–388.

    Article  PubMed  Google Scholar 

  • Lanza, S. T., Collins, L. M., Schafer, J. L., & Flaherty, B. P. (2005). Using data augmentation to obtain standard errors and conduct hypothesis tests in latent class and latent transition analysis. Psychological Methods, 10(1), 84.

    Article  PubMed  Google Scholar 

  • Lappan, G., & Briars, D. (1995). How should mathematics be taught? In I. Carl (Ed.), Prospects for school mathematics. Reston, VA: National Council of Teachers of Mathematics.

  • Lehrer, R., & Wilson, M. (2011). Developing assessments of data modeling: Construct maps as boundary objects. Paper presented at the annual meeting of the American Educational Research Association, New Orleans.

  • Li, F., Cohen, A., Bottge, B., & Templin, J. (2016). A latent transition analysis model for assessing change in cognitive skills. Educational and Psychological Measurement, 76(2), 181–204.

    Article  PubMed  Google Scholar 

  • Lu, J., & Wang, C. (2019). A sequential exploratory approach to learning attribute hierarchies from data. Paper presented at the International meeting of psychometric society. Chile: Santiago.

  • Madison, M. J., & Bradshaw, L. (2018). Evaluating intervention effects in a diagnostic classification model framework. Journal of Educational Measurement, 55(1), 32–51.

    Article  Google Scholar 

  • Martineau, J. A. (2006). Distorting value added: The use of longitudinal, vertically scaled student achievement data for growth-based, value-added accountability. Journal of Educational and Behavioral Statistics, 31(1), 35–62.

    Article  Google Scholar 

  • Nylund-Gibson, K., Grimm, R., Quirk, M., & Furlong, M. (2014). A latent transition mixture model using the three-step specification. Structural Equation Modeling: A Multidisciplinary Journal, 21(3), 439–454.

    Article  Google Scholar 

  • Pellegrino, J. W., Chudowsky, N., & Glaser, R. (Eds.). (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.

    Google Scholar 

  • Pham, D., Monroe, S., Bauer, M., Wylie, C., & Wells, C. (2017). Evaluating a learning progression theory: An application of MIRT. Paper presented at the annual meeting of the National Council on Measurement in Educaiton, San Antonio, TX.

  • Popham, W. J. (2008). Formative assessment: Seven stepping-stones to success. Principal Leadership, 9(1), 16–20.

    Google Scholar 

  • Popham, W. J. (2011). Transformative assessment in action: An inside look at applying the process. Alexandria: ASCD.

    Google Scholar 

  • Roussos, L. A., DiBello, L. V., Stout, W., Hartz, S. M., Henson, R. A., & Templin, J. (2007). The fusion model skills diagnosis system (pp. 275–318). Cognitive diagnostic assessment for education: Theory and applications.

  • Rupp, A., Templin, J., & Henson, R. (2010). Diagnostic assessment: Theory, methods, and applications. New York: Guilford.

    Google Scholar 

  • Rupp, A. A., & Templin, J. L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement, 6(4), 219–262.

    Google Scholar 

  • Shao, J. (1997). An asymptotic theory for linear model selection. Statistica Sinica, 7, 221–242.

    Google Scholar 

  • Shepard, L. A. (2009). Commentary: Evaluating the validity of formative and interim assessment. Educational Measurement: Issues and Practice, 28(3), 32–37.

    Article  Google Scholar 

  • Siegmund, D. (2004). Model selection in irregular problems: Applications to mapping quantitative trait loci. Biometrika, 91(4), 785–800.

    Article  Google Scholar 

  • Simon, M. A., & Tzur, R. (2004). Explicating the role of mathematical tasks in conceptual learning: An elaboration of the hypothetical learning trajectory. Mathematical Thinking and Learning, 6(2), 91–104.

    Article  Google Scholar 

  • Sinharay, S. (2006). Model diagnostics for Bayesian networks. Journal of Educational and Behavioral Statistics, 31(1), 1–33.

    Article  Google Scholar 

  • Sinharay, S., & Almond, R. G. (2007). Assessing fit of cognitive diagnostic models a case study. Educational and Psychological Measurement, 67(2), 239–257.

    Article  Google Scholar 

  • Tatsuoka, C. (2002). Data analytic methods for latent partially ordered classification models. Journal of the Royal Statistical Society: Series C (Applied Statistics), 51(3), 337–350.

    Google Scholar 

  • Wang, C., & Lu, J. (2020). Learning attribute hierarchies from data: Two exploratory approaches. Journal of Educational and Behavioral Statistics. https://doi.org/10.3102/1076998620931094

  • Wang, C., & Nydick, S. W. (2015). Comparing two algorithms for calibrating the restricted non-compensatory multidimensional IRT model. Applied Psychological Measurement, 39(2), 119–134.

    Article  PubMed  Google Scholar 

  • Wang, C., Zheng, C., & Chang, H. H. (2014). An enhanced approach to combine item response theory with cognitive diagnosis in adaptive testing. Journal of Educational Measurement, 51(4), 358–380.

    Article  Google Scholar 

  • Wang, H., Li, B., & Leng, C. (2009). Shrinkage tuning parameter selection with a diverging number of parameters. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(3), 671–683.

    Article  Google Scholar 

  • Wang, S., Yang, Y., Culpepper, S. A., & Douglas, J. A. (2018). Tracking skill acquisition with cognitive diagnosis models: A higher-order, hidden Markov model with covariates. Journal of Educational and Behavioral Statistics, 43, 57–87.

    Article  Google Scholar 

  • Wilson, M. (2004). Constructing measures: An item response modeling approach. Abingdon: Routledge.

    Book  Google Scholar 

  • Xu, G., Wang, C., & Shang, Z. (2016). On initial item selection in cognitive diagnostic computerized adaptive testing. British Journal of Mathematical and Statistical Psychology, 69(3), 291–315.

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by University of Washington Royalty Research Fund A143697.

Author information

Authors and Affiliations

  1. Measurement and Statistics, College of Education, University of Washington, 312E Miller Hall, 2012 Skagit Ln, Seattle, WA 98105, USA

    Chun Wang

Authors
  1. Chun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chun Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Tables 7 and 8.

Table 7 Eight profiles and their orders based on 3 attributes
Full size table
Table 8 Sixty-four profiles and their orders based on 8 attributes
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C. Using Penalized EM Algorithm to Infer Learning Trajectories in Latent Transition CDM. Psychometrika 86, 167–189 (2021). https://doi.org/10.1007/s11336-020-09742-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-020-09742-1

Keywords

Search

Navigation