Simulation-Extrapolation with Latent Heteroskedastic Error Variance

Psychometrika volume 82, pages717–736(2017)Cite this article

Abstract

This article considers the application of the simulation-extrapolation (SIMEX) method for measurement error correction when the error variance is a function of the latent variable being measured. Heteroskedasticity of this form arises in educational and psychological applications with ability estimates from item response theory models. We conclude that there is no simple solution for applying SIMEX that generally will yield consistent estimators in this setting. However, we demonstrate that several approximate SIMEX methods can provide useful estimators, leading to recommendations for analysts dealing with this form of error in settings where SIMEX may be the most practical option.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

References

  1. Akram, K., Erickson, F., & Meyer, R. (2013). Issues in the estimation of student growth percentiles. Paper presented at the annual meeting of the Association for Education Finance and Policy, New Orleans, LA.

  2. Battauz, M., & Bellio, R. (2011). Structural modeling of measurement error in generalized linear models with Rasch measures as covariates. Psychometrika, 76(1), 40–56.

    Article  Google Scholar 

  3. Battauz, M., Bellio, R., & Gori, E. (2008). Reducing measurement error in student achievement estimation. Psychometrika, 73(2), 289–302.

    Article  Google Scholar 

  4. Betebenner, D. (2009). Norm- and criterion-referenced student growth. Educational Measurement: Issues and Practice, 28(4), 42–51.

    Article  Google Scholar 

  5. Bollen, K. (1989). Structural equations with latent variables. New York: Wiley.

    Google Scholar 

  6. Buonaccorsi, J. (2010). Measurement error: Models, methods, and applications. Boca Raton, FL: Chapman and Hall/CRC Interdisciplinary Statistics.

    Google Scholar 

  7. Carroll, R., Ruppert, D., Stefanski, L., & Crainiceanu, C. (2006). Measurement error in nonlinear models: A modern perspective (2nd ed.). London: Chapman and Hall.

    Google Scholar 

  8. Carroll, R., & Wang, Y. (2008). Nonparametric variance estimation in the analysis of microarray data: A measurement error approach. Biometrika, 95(2), 437–449.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Cook, J., & Stefanski, L. (1994). Simulation-extrapolation estimation in parametric measurement error models. Journal of the American Statistical Association, 89(428), 1314–1328.

    Article  Google Scholar 

  10. Dawid, A. P. (1979). Conditional independence in statistical theory. Journal of the Royal Statistical Society, Series B, 41(1), 1–31.

    Google Scholar 

  11. Devanarayan, V., & Stefanski, L. (2002). Empirical simulation extrapolation for measurement error models with replicate measurements. Statistics and Probability Letters, 59, 219–225.

    Article  Google Scholar 

  12. Fox, J.-P., & Glas, C. (2003). Bayesian modeling of measurement error in predictor variables using item response theory. Psychometrika, 68(2), 169–191.

    Article  Google Scholar 

  13. Fuller, W. (2006). Measurement error models (2nd ed.). New York: Wiley.

    Google Scholar 

  14. Junker, B., Schofield, L., & Taylor, L. (2012). The use of cognitive ability measures as explanatory variables in regression analysis. IZA Journal of Labor Economics, 1(4), 1–19.

    Google Scholar 

  15. Kolen, M., & Brennan, R. (2004). Test equating, scaling, and linking: Methods and practice (2nd ed.). New York: Springer.

    Google Scholar 

  16. Laird, N. (1978). Nonparametric maximum likelihood estimation of a mixing distribution. Journal of the American Statistical Association, 73(364), 215–232.

    Article  Google Scholar 

  17. Lockwood, J. R., & Castellano, K. E. (2015). Alternative statistical frameworks for student growth percentile estimation. Statistics and Public Policy, 2(1), 1–8.

    Article  Google Scholar 

  18. Lockwood, J. R., & Castellano, K. E. (2016). Estimating true student growth percentile distributions using latent regression multidimensional IRT models. Educational and Psychological Measurement. doi:10.1177/0013164416659686.

  19. Lockwood, J. R., & McCaffrey, D. F. (2014a). Correcting for test score measurement error in ANCOVA models for estimating treatment effects. Journal of Educational and Behavioral Statistics, 39(1), 22–52.

    Article  Google Scholar 

  20. Lockwood, J. R., & McCaffrey, D. F. (2014b). Should nonlinear functions of test scores be used as covariates in a regression model? In R. Lissitz (Ed.), Value-added modeling and growth modeling with particular application to teacher and school effectiveness. Charlotte, NC: Information Age Publishing.

    Google Scholar 

  21. Lockwood, J. R., & McCaffrey, D. F. (2015). Simulation-extrapolation for estimating means and causal effects with mismeasured covariates. Observational Studies, 1, 241–290.

    Google Scholar 

  22. Lord, F. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  23. Lord, F. (1984). Standard errors of measurement at different ability levels. Journal of Educational Measurement, 21(3), 239–243.

    Article  Google Scholar 

  24. Maryland State Department of Education. (2014). Maryland high school assessment and modified high school assessment 2013 technical report. Princeton, NJ: Educational Testing Service.

  25. McCaffrey, D. F., Castellano, K. E., & Lockwood, J. R. (2015). The impact of measurement error on the accuracy of individual and aggregate SGP. Educational Measurement: Issues and Practice, 34(1), 15–21.

    Article  Google Scholar 

  26. McCaffrey, D. F., Lockwood, J. R., & Setodji, C. (2013). Inverse probability weighting with error-prone covariates. Biometrika, 100(3), 671–680.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Monroe, S., & Cai, L. (2015). Examining the reliability of student growth percentiles using multidimensional IRT. Educational Measurement: Issues and Practice, 34(4), 21–30.

    Article  Google Scholar 

  28. Novick, S., & Stefanski, L. (2002). Corrected score estimation via complex variable simulation extrapolation. Journal of the American Statistical Association, 97(458), 472–481.

    Article  Google Scholar 

  29. Pearson. (2014). New York State Testing Program 2014: English Language Arts and Mathematics Grades 3–8. http://www.p12.nysed.gov/assessment/reports/2014/gr38cc-tr14.

  30. R Core Team. (2016). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.

  31. Rabe-Hesketh, S., Pickles, A., & Skrondal, A. (2003). Correcting for covariate measurement error in logistic regression using nonparametric maximum likelihood estimation. Statistical Modelling, 3, 215–232.

    Article  Google Scholar 

  32. Shang, Y., Van Iwaarden, A., & Betebenner, D. (2015). Covariate measurement error correction for student growth percentiles using the SIMEX method. Educational Measurement: Issues and Practice, 34(1), 4–14.

    Article  Google Scholar 

  33. Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal and structural equation models. Boca Raton, FL: Chapman and Hall/CRC Press.

    Google Scholar 

  34. Stefanski, L., & Cook, J. (1995). Simulation-extrapolation: The measurement error jackknife. Journal of the American Statistical Association, 90(432), 1247–1256.

    Article  Google Scholar 

  35. Steiner, P., Cook, T., & Shadish, W. (2011). On the importance of reliable covariate measurement in selection bias adjustments using propensity scores. Journal of Educational and Behavioral Statistics, 36(2), 213–236.

    Article  Google Scholar 

  36. Stuart, E. (2007). Estimating causal effects using school-level data sets. Educational Researcher, 36(4), 187–198.

    Article  Google Scholar 

  37. van der Linden, W., & Hambleton, R. (1997). Handbook of modern item response theory. New York, NY: Springer.

    Google Scholar 

  38. Wang, Y., Ma, Y., & Carroll, R. (2009). Variance estimation in the analysis of microarray data. Journal of the Royal Statistical Society: Series B, 71(2), 425–445.

    Article  Google Scholar 

  39. Warm, T. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54(3), 427–450.

    Article  Google Scholar 

  40. Yen, W. M. (1984). Obtaining maximum likelihood trait estimates from number-correct scores for the three-parameter logistic model. Journal of Educational Measurement, 21(2), 93–111.

    Article  Google Scholar 

Download references

Author information

Affiliations

  1. Educational Testing Service, Princeton, NJ, USA

    J. R. Lockwood & Daniel F. McCaffrey

Authors
  1. J. R. Lockwood
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel F. McCaffrey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. R. Lockwood.

Additional information

The research reported here was supported in part by the Institute of Education Sciences, US Department of Education, through Grant R305D140032 to ETS. The opinions expressed are those of the authors and do not represent views of the Institute or the US Department of Education. We thank Shelby Haberman, Hongwen Guo, Rebecca Zwick, the Editor, an Associate Editor, and three anonymous referees for helpful comments on earlier drafts.

Electronic supplementary material

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lockwood, J.R., McCaffrey, D.F. Simulation-Extrapolation with Latent Heteroskedastic Error Variance. Psychometrika 82, 717–736 (2017). https://doi.org/10.1007/s11336-017-9556-y

Download citation

Keywords

  • achievement test scores
  • measurement error
  • item response theory
  • covariate adjustment
  • nonlinear models