Advertisement

Detection of Differential Item Functioning via the Credible Intervals and Odds Ratios Methods

  • Ya-Hui Su
  • Henghsiu TsaiEmail author
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 265)

Abstract

Differential item functioning (DIF) analysis is an essential procedure for educational and psychological tests to identify items that exhibit varying degrees of DIF. DIF means that the assumption of measurement invariance is violated, and then test scores are incomparable for individuals of the same ability level from different groups, which substantially threatens test validity. In this paper, we investigated the credible intervals (CI) and odds ratios (OR) methods to detect uniform DIF within the framework of the Rasch model through a series of simulations. The results showed that the CI method performed better than the OR method to identify DIF items under the balanced DIF conditions. However, the CI method yielded inflated false positive rates under the unbalanced DIF conditions. The effectiveness of these two approaches was illustrated with an empirical example.

Keywords

Credible interval Odds ratio DIF Markov chain Monte Carlo IRT 

Notes

Acknowledgements

The research was supported by Academia Sinica and the Ministry of Science and Technology of the Republic of China under grant number MOST 106-2118-M-001-003-MY2. The authors would like to thank Ms. Yi-Jhen Wu for her helpful comments and suggestions.

References

  1. Agresti, A. (2002). Categorical data analysis (2nd ed.). New York, NY: Wiley.CrossRefGoogle Scholar
  2. Chang, J., Tsai, H., Su, Y.-H., & Lin, E. M. H. (2016). A three-parameter speeded item response model: estimation and application. In L. A. van der Ark, D. M. Bolt, W.-C. Wang, J. A. Douglas, & M. Wiberg (Eds.), Quantitative psychology research (Vol. 167, pp. 27–38). Switzerland: Springer.  https://doi.org/10.1007/978-3-319-38759-8_3.CrossRefGoogle Scholar
  3. Chang, Y.-W., Tsai, R.-C., & Hsu, N.-J. (2014). A speeded item response model: Leave the harder till later. Psychometrika, 79, 255–274.  https://doi.org/10.1007/s11336-013-9336-2.MathSciNetCrossRefzbMATHGoogle Scholar
  4. Clauser, B., Mazor, K., & Hambleton, R. K. (1993). The effects of purification of matching criterion on the identification of DIF using the Mantel-Haenszel procedure. Applied Measurement in Education, 6, 269–279.  https://doi.org/10.1207/s15324818ame0604_2.CrossRefGoogle Scholar
  5. Cohen, A. S., Kim, S. H., & Wollack, J. A. (1996). An investigation of the likelihood ratio test for detection of differential item functioning. Applied Psychological Measurement, 20, 15–26.  https://doi.org/10.1177/014662169602000102.CrossRefGoogle Scholar
  6. Frederickx, S., Tuerlinckx, F., de Boeck, P., & Magis, D. (2010). RIM: A random item mixture model to detect differential item functioning. Journal of Educational Measurement, 47, 432–457.  https://doi.org/10.1111/j.1745-3984.2010.00122.x.CrossRefGoogle Scholar
  7. French, B. F., & Maller, S. J. (2007). Iterative purification and effect size use with logistic regression for differential item functioning detection. Educational and Psychological Measurement, 67, 373–393.  https://doi.org/10.1177/0013164406294781.MathSciNetCrossRefGoogle Scholar
  8. Glas, C. A. W. (1998). Detection of differential item functioning using Lagrange multiplier tests. Statistica Sinica, 8, 647–667.MathSciNetzbMATHGoogle Scholar
  9. Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129–145). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  10. Jin, K. -Y., Chen, H. -F., & Wang, W. -C. (2018). Using odds ratios to detect differential item functioning. Applied Psychological Measurement, 42, 613–629.  https://doi.org/10.1177/0146621618762738.CrossRefGoogle Scholar
  11. Kopf, J., Zeileis, A., & Strobl, C. (2015). Anchor selection strategies for DIF analysis: Review, assessment, and new approaches. Educational and Psychological Measurement, 75, 22–56.  https://doi.org/10.1177/0013164414529792.CrossRefGoogle Scholar
  12. Lord, F. M. (1980). Application of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  13. Magis, D., Béland, S., Tuerlinckx, F., & De Boeck, P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42, 847–862.  https://doi.org/10.3758/brm.42.3.847.CrossRefGoogle Scholar
  14. Narayanon, P., & Swaminathan, H. (1996). Identification of items that show nonuniform DIF. Applied Psychological Measurement, 20, 257–274.  https://doi.org/10.1177/014662169602000306.CrossRefGoogle Scholar
  15. Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53, 495–502.  https://doi.org/10.1007/bf02294403.MathSciNetCrossRefzbMATHGoogle Scholar
  16. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Educational Research.Google Scholar
  17. Rogers, H. J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17, 105–116.  https://doi.org/10.1177/014662169301700201.CrossRefGoogle Scholar
  18. Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159–194.  https://doi.org/10.1007/bf02294572.CrossRefzbMATHGoogle Scholar
  19. Su, Y.-H., Chang, J., & Tsai, H. (2018). Using credible intervals to detect differential item functioning in IRT Models. In M. Wiberg, S. Culpepper, R. Janssen, J. González, & D. Molenaar (Eds.), Quantitative psychology research (Vol. 233, pp. 297–304). Switzerland: Springer.  https://doi.org/10.1007/978-3-319-77249-3_25.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of PsychologyNational Chung Cheng UniversityChiayi CountyTaiwan
  2. 2.Institute of Statistical Science, Academia SinicaTaipeiTaiwan

Personalised recommendations