The Reliability of Diagnosing Broad and Narrow Skills in Middle School Mathematics with the Multicomponent Latent Trait Model

Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 140)

Abstract

The multicomponent latent trait model for diagnosis (MLTM-D;Embretson and Yang, Psychometrika 78:14–36, 2013) is a conjunctive item response model that is hierarchically organized to include broad and narrow skills. A two-stage adaptive testing procedure was applied to diagnose skill mastery in middle school mathematics and then analyzed with MLTM-D. Strong support for the reliability of diagnosing both broad and narrow skills was obtained from both stages of testing using decision confidence indices.

Keywords

Diagnostic models Item response theory Multidimensional models Decision confidence reliability 
This is a preview of subscription content, log in to check access.

References

  1. Du Toit, M. (2003). IRT from SSI. Lincolnwood, IL: Scientific Software International, Inc.Google Scholar
  2. Embretson, S. E. (2015). The multicomponent latent trait model for diagnosis: Applications to heterogeneous test domains. Invited paper for Applied Psychological Measurement (pp. 6–30).Google Scholar
  3. Embretson, S. E., & Yang, X. (2013). A multicomponent latent trait model for diagnosis. Psychometrika, 78, 14–36.MathSciNetCrossRefMATHGoogle Scholar
  4. Embretson, S. E., Morrison, K., & Jun, H. (2014). Implementation of a multistage adaptive testing system for diagnosing mathematical deficits in middle school. Report IES1001A-2014 for Institute of Educational Sciences Grant R305A100234. Atlanta, GA: Cognitive Measurement Laboratory, Georgia Institute of Technology.Google Scholar
  5. Fischer, G. H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359–374.CrossRefGoogle Scholar
  6. Hensen, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.MathSciNetCrossRefGoogle Scholar
  7. Leighton, J. P., & Gierl, M. J. (2007). Cognitive diagnostic assessment for education. New York: Cambridge University Press.CrossRefGoogle Scholar
  8. Lewis, C., & Sheehan, K. (1990). Using Bayesian decision theory to design a computerized mastery test. Applied Psychological Measurement, 14, 367–386.CrossRefGoogle Scholar
  9. Rudner, L. M. (2005). Expected classification accuracy. Practical Assessment Research & Evaluation, 10, 1–4.Google Scholar
  10. Rupp, A. A., Templin, J., & Hensen, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York: Guilford Press.Google Scholar
  11. Sinharay, S. (2010). How often do subscores have added value? Results from operational and simulated data. Journal of Educational Measurement, 47, 150–174.CrossRefGoogle Scholar
  12. von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287–307.MathSciNetCrossRefGoogle Scholar
  13. Wainer, H., Wang, X. A., Skorupski, W. P., & Bradlow, E. T. (2005). A Bayesian method for evaluating passing scores: The PPoP curve. Journal of Educational Measurement, 42,271–281.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Susan Embretson
    • 1
  • Kristin Morrison
    • 1
  • Hea Won Jun
    • 1
  1. 1.Georgia Institute of TechnologyAtlantaUSA

Personalised recommendations