A new procedure for detection of crossing DIF

Abstract

The purpose of this paper is to present a hypothesis testing and estimation procedure, Crossing SIBTEST, for detecting crossing DIF. Crossing DIF exists when the difference in the probabilities of a correct answer for the two examinee groups changes signs as ability level is varied. In item response theory terms, crossing DIF is indicated by two crossing item characteristic curves. Our new procedure, denoted as Crossing SIBTEST, first estimates the matching subtest score at which crossing occurs using least squares regression analysis. A Crossing SIBTEST statistic then is used to test the hypothesis of crossing DIF. The performance of Crossing SIBTEST is evaluated in this study.

This is a preview of subscription content, log in to check access.

References

  1. Bennett, R. E., Rock, D. A., & Kaplan, B. A. (1987). SAT differential item performance for nine handicapped groups.Journal of Educational Measurement, 24(1), 56–64.

    Google Scholar 

  2. Bock, R. D. (1975).Multivariate statistical methods. New York: McGraw-Hill.

    Google Scholar 

  3. Dorans, N. J., & Kulick, E. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the scholastic aptitude test.Journal of Educational Measurement, 23, 355–368.

    Google Scholar 

  4. Douglas, J., Roussos, L., & Stout, W. F. (in press). Item bundle DIF hypothesis testing: identifying suspect bundles and assessing their DIF.Journal of Educational Measurement.

  5. Drasgow, F. (1987). A study of measurement bias of two standard psychological tests.Journal of Applied Psychology, 72, 19–30.

    Google Scholar 

  6. Edgington, E. S. (1987).Randomization tests. New York, NY: Maecel Dekker.

    Google Scholar 

  7. Ellis, B. (1989). Differential item functioning: Implications for test translations.Journal of Applied Psychology, 74(6), 912–921.

    Google Scholar 

  8. Hambleton, R. K., & Rogers, H. J. (1989). Detecting potentially biased test items: Comparison of IRT area and Mantel-Haenszel methods.Applied Measurement in Education, 2(4), 313–334.

    Google Scholar 

  9. Holland, P. W., & Thayer, D. T. (1988). Differential item functioning and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.),Test validity (pp. 129–145). Hillsadle, NJ: Lawrence Erlbaum.

    Google Scholar 

  10. Li, H., & Stout, W. (1993, June). A new procedure for detection of crossing bias/DIF. Paper presented at the 58th Annual Meeting of Psychometric Society, Berkeley, CA.

  11. Li, H., & Stout, W. (1994, April). Detecting crossing item bias/DIF: Comparison of logistic regression and crossing SIBTEST procedures. Paper presented at the 1994 AERA Annual Meeting, New Orleans, LA.

  12. Mellenbergh, G. J. (1982). Contingency table models for assessing item bias.Journal of Educational Statistics, 7(2), 105–118.

    Google Scholar 

  13. Mislevy, R. J., & Bock, R. D. (1984). Item operating characteristics of the Armed Services Aptitute Battery (ASVAB), Form 8A (Tech. Rep. N00014-83-C-0283). Washington, DC: Office of Naval Research.

    Google Scholar 

  14. Nandakumar, R. (1992). Simultaneous DIF amplification and cancellation: Shealy-Stout's test for DIF.Journal of Educational Measurement, 30, 293–311.

    Google Scholar 

  15. Oshima, T., & Miller, D. (1991, April). Multidimensionality and item bias in item response theory. Presented at 1991 AERA Annual Meeting, Chicago, IL.

  16. Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation.Psychometrika, 56, 611–630.

    Google Scholar 

  17. Ramsay, J. O. (1993). TESTGRAF: A program for the graphical analysis of multiple choice test and questionnaire data. TESTGRAF user's guide.

  18. Roussos, L., & Stout, W. F. (1991). BIASDEM: A program for simulating models of unidirectional and crossing DIF. Unpublished manuscript.

  19. Roussos, L., & Stout, W. F. (in press). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance.Journal of Educational Measurement.

  20. Shealy, R., & Stout, W. (1993a). An item response theory model for test bias. In P. W. Holland & H. Wainer (Eds.),Differential item functioning (pp. 197–239). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  21. Shealy, R., & Stout, W. (1993b). A model-based standardization approach that separate true bias/DIF from group ability differences and detects test bias/DIF as well as item bias/DIF.Psychometrika, 58, 159–194.

    Google Scholar 

  22. Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures.Journal of Educational Measurement, 27(4), 361–370.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to William Stout.

Additional information

This research was partially supported by a grant from the Law School Admission Council and by National Science Foundation Mathematics Grant NSF-DMS-94-04327. The research reported here is collaborative in every respect and the order of authorship is alphabetical. The authors thank Jeff Douglas and Louis Roussos for their useful comments and discussions.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Li, H., Stout, W. A new procedure for detection of crossing DIF. Psychometrika 61, 647–677 (1996). https://doi.org/10.1007/BF02294041

Download citation

Key words

  • DIF
  • bias
  • unidirectional DIF
  • crossing DIF
  • multidimensional IRT
  • randomization tests
  • SIBTEST
  • Mantel-Haenszel
  • logistic regression procedure crossing
  • SIBTEST