Abstract
A multilevel latent transition analysis (LTA) with a mixture IRT measurement model (MixIRTM) is described for investigating the effectiveness of an intervention. The addition of a MixIRTM to the multilevel LTA permits consideration of both potential heterogeneity in students’ response to instructional intervention as well as a methodology for assessing stage sequential change over time at both student and teacher levels. Results from an LTA–MixIRTM and multilevel LTA–MixIRTM were compared in the context of an educational intervention study. Both models were able to describe homogeneities in problem solving and transition patterns. However, ignoring a multilevel structure in LTA–MixIRTM led to different results in group membership assignment in empirical results. Results for the multilevel LTA–MixIRTM indicated that there were distinct individual differences in the different transition patterns. The students receiving the intervention treatment outscored their business as usual (i.e., control group) counterparts on the curriculum-based Fractions Computation test. In addition, 27.4 % of the students in the sample moved from the low ability student-level latent class to the high ability student-level latent class. Students were characterized differently depending on the teacher-level latent class.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
To present the individual differences for persons j nested within group s at a time point t, \(\theta_{g_{t}h}\) can be presented as \(\theta_{jstg_{t}h}\). For simplicity, the subscripts, j, s, and t, are dropped.
References
Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B.N. Petrov & F. Caski (Eds.), Second international symposium on information theory (pp. 267–281). Budapest: Academiai Kiado.
Andersen, E.B. (1985). Estimating latent correlations between repeated testings. Psychometrika, 50, 3–16.
Asparouhov, T., & Muthén, B. (2008). Multilevel mixture models. In G.R. Hancock & K.M. Samuelsen (Eds.), Advances in latent variable mixture models (pp. 27–51). Charlotte: Information Age Publishing
Ball, T.L. (1990). Prospective elementary and secondary teachers understanding of division. Journal for Research in Mathematics Education, 21, 132–144.
Bottge, B.A. (1999). Effects of contextualized math instruction on problem solving of average and below-average achieving students. Journal of Special Education, 33, 81–92.
Bottge, B.A., Heinrichs, M., Mehta, Z., & Hung, Y. (2002). Weighing the benefits of anchored math instruction for students with disabilities in general education classes. Journal of Special Education, 35, 186–200.
Bottge, B.A., Heinrichs, M., Mehta, Z.D., Rueda, E., Hung, Y., & Danneker, J. (2004). Teaching mathematical problem solving to middle school students in math, technology education, and special education classrooms. RMLE Online, 27. http://www.nmsa.org/portals/0/pdf/publications/RMLE/rmle_vol27_no1_article1.pdf. Retrieved July 20, 2006.
Bottge, B.A., Rueda, E., Serlin, R.C., Hung, Y., & Kwon, J. (2007). Shrinking achievement differences with anchored math problems: challenges and possibilities. Journal of Special Education, 41, 31–49.
Bottge, B., Ma, X., Toland, M., Gassaway, L., & Butler, M. (2012). Effects of enhanced anchored instruction on middle school students with disabilities in math. Department of Special Education & Rehabilitation Counseling, University of Kentucky, Lexington, KY.
Bozdogan, H. (1987). Model selection and Akaike’s information criterion (AIC): the general theory and its analytical extensions. Psychometrika, 52, 345–370.
Cho, S.-J., & Cohen, A.S. (2010). A multilevel mixture IRT model with an application to DIF. Journal of Educational and Behavioral Statistics, 35, 336–370.
Cho, S.-J., Cohen, A.S., Kim, S.-H., & Bottge, B. (2010). Latent transition analysis with a mixture IRT measurement model. Applied Psychological Measurement, 34, 583–604.
Cho, S.-J., & Rabe-Hesketh, S. (2011). Alternating imputation posterior estimation of models with crossed random effects. Computational Statistics & Data Analysis, 55, 12–25.
Cho, S.-J., Athay, M., & Preacher, K. J. (2012a). Measuring change for a multidimensional test using a generalized explanatory longitudinal item response model. British Journal of Mathematical and Statistical Psychology. doi:10.1111/j.2044-8317.2012.02058.x.
Cho, S.-J., Cohen, A. S., & Kim, S.-H. (2012b). Markov chain Monte Carlo estimation of a mixture item response theory model. Journal of Statistical Computation and Simulation. doi:10.1080/00949655.2011.603090.
Cognition Technology Group at Vanderbilt (1990). Anchored instruction and its relationship to situated cognition. Educational Researcher, 19, 2–10.
Cognition Technology Group at Vanderbilt (1997). The Jasper project: lessons in curriculum, instruction, assessment, and professional development. Mahwah: Erlbaum.
Collins, L.M., & Wugalter, S.E. (1992). Latent class models for stage-sequential dynamic latent variables. Multivariate Behavioral Research, 27, 131–157.
CCSSI-M (Common Core State Standards Initiative—Mathematics) (2011). http://www.corestandards.org. Retrieved December 5, 2011.
Congdon, P. (2003). Applied Bayesian modeling. New York: Wiley.
De Boeck, P., Wilson, M., & Acton, G.S. (2005). A conceptual and psychometric framework for distinguishing categories and dimensions. Psychological Review, 112, 129–158.
De Boeck, P., Cho, S.-J., & Wilson, M. (2011). Explanatory secondary dimension modeling of latent DIF. Applied Psychological Measurement, 35, 583–603.
Embretson, S.E., & Reise, S.P. (2000). Item response theory for psychologists. Mahwah: Lawrence-Erlbaum.
Frühwirth-Schnatter, S. (2006). Finite mixture and Markov switching models. New York: Springer.
Graham, J.W., Collins, L.M., Wugalter, S.E., Chung, N.K., & Hansen, W.B. (1991). Modeling transition in latent stage-sequential processes: a substance use prevention example. Journal of Consulting and Clinical Psychology, 59, 48–57.
Heinen, T. (1996). Latent classes and discrete latent trait models. Thousand Oaks: Sage Publications.
Henry, K.L., & Muthén, B. (2010). Multilevel latent class analysis: an application of adolescent smoking typologies with individual and contextual predictors. Structural Equation Modeling. A Multidisciplinary Journal, 17, 193–215.
Hickey, D.T., Moore, A.L., & Pellegrino, J.W. (2001). The motivational and academic consequences of elementary mathematics environments: do constructivist innovations and reforms make a difference? American Educational Research Journal, 38, 611–652.
Lee van Horn, M., Fagan, A.A., Jaki, T., Brown, E.C., Hawkins, D., Arthur, M.W., Abbott, R.D., & Catalano, R.F. (2008). Using multilevel mixtures to evaluate intervention effects in group randomized trials. Multivariate Behavioral Research, 43, 289–326.
Li, F., Cohen, A.S., Kim, S.-H., & Cho, S.-J. (2009). Model selection methods for dichotomous mixture IRT models. Applied Psychological Measurement, 33, 353–373.
Maxwell, S.E., & Tiberio, S. (2007). Multilevel models of change: fundamental concepts and relationships to mixed models and latent growth-curve models. In A. Ong & M.H.M. van Dulmen (Eds.), Oxford handbook of methods in positive psychology (pp. 439–452). New York: Oxford University Press.
McDonald, R.P. (1967). Psychometric monographs: Vol. 15. Nonlinear factor analysis. Richmond: Psychometric Corporation. Retrieved from http://www.psychometrika.org/journal/online/MN15.pdf.
McLachlan, G., & Peel, D. (2000). Finite mixture models. New York: Wiley.
Muthén, B., Brown, C.H., Jo, B., Khoo, S.-T., Yang, C.C., Wang, C.-P., Kellam, S.G., Carlin, J.B., & Liao, J. (2002). General growth mixture modeling for randomized preventive interventions. Biostatistics, 3, 459–475.
Muthén, B., & Asparouhov, T. (2006). Item response mixture modeling: application to tobacco dependence criteria. Addictive Behaviors, 31, 1050–1066.
Muthén, L.K., & Muthén, B.O. (2006–2010). Mplus [computer program]. Los Angeles: Muthén & Muthén.
Nylund, K.L., Asparouhov, T., & Muthén, B.O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: a Monte Carlo simulation study. Structural Equation Modeling, 14, 535–569.
Paek, I., Cho, S.-J., & Cohen, A.S. (2010). A note on scaling linking in mixture IRT model applications. Paper presented at the National Council on Measurement in Education. Denver.
Raudenbush, S.W., & Bryk, A.G. (2002). Hierarchical linear models: applications and data analysis methods (2nd ed.). Thousand Oaks: Sage.
Rijmen, F., & De Boeck, P. (2005). A relation between a between-item multidimensional IRT model and the mixture-Rasch model. Psychometrika, 70, 481–496.
Rijmen, F., De Boeck, P., & van der Maas, H.L.J. (2005). An IRT model with a parameter-driven process for change. Psychometrika, 70, 651–669.
Rost, J. (1990). Rasch models in latent classes: an integration of two approaches to item analysis. Applied Psychological Measurement, 14, 271–282.
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464.
Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: multilevel, longitudinal and structural equation models. Boca Raton: Chapman & Hall.
Steinley, D., & McDonald, R.P. (2007). Examining factor score distributions to determine the nature of latent spaces. Multivariate Behavioral Research, 42, 133–156.
Sugiura, N. (1978). Further analysis of the data by Akaike’s information criterion of model fitting. Suri-Kagaku (Mathematic Science), 153, 12–18 (in Japanese).
Vermunt, J.K. (2003). Multilevel latent class models. Sociological Methodology, 33, 213–239.
Vermunt, J.K. (2008). Latent class and finite mixture models for multilevel data sets. Statistical Methods in Medical Research, 17, 33–51.
Vermunt, J.K., & Magidson, J. (2005). Technical guide for Latent GOLD 4.0: basic and advanced. Belmont: Statistical Innovations.
von Davier, M., Xu, X., & Carstensen, C.H. (2011). Measuring growth in a longitudinal large-scale assessment with a general latent variable model. Psychometrika, 76, 318–336.
von Davier, M., & Yamamoto, K. (2004). Partially observed mixtures of IRT models: an extension of the generalized partial-credit model. Applied Psychological Measurement, 28, 389–406.
Yang, C. (2006). Evaluating latent class analyses in qualitative phenotype identification. Computational Statistics & Data Analysis, 50, 1090–1104.
Yu, H.-T. (2007). Multilevel latent Markov models for nested longitudinal discrete data. Unpublished doctoral dissertation. University of Illinois at Urbana-Champaign.
Acknowledgements
The authors wish to thank Dr. Paul De Boeck (Ohio State University) for his valuable comments on previous drafts of this paper. We also would like to thank three anonymous reviewers and the editor, Dr. Maydeu-Olivares, for their helpful comments and suggestions on previous versions of this paper.
The research reported in this paper was supported by a grant from the U.S. Department of Education, Institute of Education Sciences, PR Number H324A090179. Any opinions, findings, or conclusions are those of the authors and do not necessarily reflect the views of the supporting agency.
Author information
Authors and Affiliations
Corresponding author
Appendix: Error Codes and Examples from a Fractions Computation Test
Appendix: Error Codes and Examples from a Fractions Computation Test
• Combining (C): Student combines numerators and combines denominators, consistently applying the same operation to numerator and denominator.
Examples: \(\frac{1}{3} + \frac{1}{3} = \frac{2}{6}\), \(\frac{7}{8} - \frac{1}{4} = \frac{6}{4}\)
• Add All (AA): Student separately adds together all the components of the fractions.
Example: \(\frac{3}{4} + \frac{2}{5} = 14\)
• Select Denominator (SD): Student selects one of the denominators listed in the problem and makes no attempt to make equivalent fraction. Denominator given in the answer must be present in the problem.
Examples: \(\frac{1}{2} + \frac{3}{16} = \frac{4}{16}\)
• Adding Components (AC): Students adds the numerator and denominator of each individual fraction together and those two sums are represented in the answer.
Example: \(\frac{1}{2} + \frac{3}{16} = \frac{3}{19}\)
• Equivalent Fraction Error (EQ): Student makes an error when attempting to represent an equivalent fraction.
• Computation Error (CE): Student makes an arithmetic error.
• Wrong Operation (WO): Student adds given a subtraction problem or subtracts given an addition problem.
Example: \(3\frac{5}{8} + 2\frac{1}{2} = 6\frac{1}{8}\)
• Other (O): Student makes error other than those listed above.
• Large–small (L/S): Student subtracts smaller from larger fraction out of order. Applies only to Fraction Computation Test items 18 and 20. Or student subtracts smaller part of fraction from larger part of fraction out of order when combined with (C) error.
Examples: \(7\frac{1}{4} - 3\frac{1}{2} = 4\frac{1}{4}\), \(4 - 2\frac {3}{4} = 2\frac{3}{4}\)
• Renaming (RN): Student makes a mistake when renaming a whole number as a mixed number; the student fails to borrow correctly from a whole number.
• No response (NR): Student leaves problem blank.
Rights and permissions
About this article
Cite this article
Cho, SJ., Cohen, A.S. & Bottge, B. Detecting Intervention Effects Using a Multilevel Latent Transition Analysis with a Mixture IRT Model. Psychometrika 78, 576–600 (2013). https://doi.org/10.1007/s11336-012-9314-0
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-012-9314-0