Advertisement

Psychometrika

, Volume 61, Issue 4, pp 647–677 | Cite as

A new procedure for detection of crossing DIF

  • Hsin-Hung Li
  • William Stout
Article

Abstract

The purpose of this paper is to present a hypothesis testing and estimation procedure, Crossing SIBTEST, for detecting crossing DIF. Crossing DIF exists when the difference in the probabilities of a correct answer for the two examinee groups changes signs as ability level is varied. In item response theory terms, crossing DIF is indicated by two crossing item characteristic curves. Our new procedure, denoted as Crossing SIBTEST, first estimates the matching subtest score at which crossing occurs using least squares regression analysis. A Crossing SIBTEST statistic then is used to test the hypothesis of crossing DIF. The performance of Crossing SIBTEST is evaluated in this study.

Key words

DIF bias unidirectional DIF crossing DIF multidimensional IRT randomization tests SIBTEST Mantel-Haenszel logistic regression procedure crossing SIBTEST 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bennett, R. E., Rock, D. A., & Kaplan, B. A. (1987). SAT differential item performance for nine handicapped groups.Journal of Educational Measurement, 24(1), 56–64.Google Scholar
  2. Bock, R. D. (1975).Multivariate statistical methods. New York: McGraw-Hill.Google Scholar
  3. Dorans, N. J., & Kulick, E. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the scholastic aptitude test.Journal of Educational Measurement, 23, 355–368.Google Scholar
  4. Douglas, J., Roussos, L., & Stout, W. F. (in press). Item bundle DIF hypothesis testing: identifying suspect bundles and assessing their DIF.Journal of Educational Measurement.Google Scholar
  5. Drasgow, F. (1987). A study of measurement bias of two standard psychological tests.Journal of Applied Psychology, 72, 19–30.Google Scholar
  6. Edgington, E. S. (1987).Randomization tests. New York, NY: Maecel Dekker.Google Scholar
  7. Ellis, B. (1989). Differential item functioning: Implications for test translations.Journal of Applied Psychology, 74(6), 912–921.Google Scholar
  8. Hambleton, R. K., & Rogers, H. J. (1989). Detecting potentially biased test items: Comparison of IRT area and Mantel-Haenszel methods.Applied Measurement in Education, 2(4), 313–334.Google Scholar
  9. Holland, P. W., & Thayer, D. T. (1988). Differential item functioning and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.),Test validity (pp. 129–145). Hillsadle, NJ: Lawrence Erlbaum.Google Scholar
  10. Li, H., & Stout, W. (1993, June). A new procedure for detection of crossing bias/DIF. Paper presented at the 58th Annual Meeting of Psychometric Society, Berkeley, CA.Google Scholar
  11. Li, H., & Stout, W. (1994, April). Detecting crossing item bias/DIF: Comparison of logistic regression and crossing SIBTEST procedures. Paper presented at the 1994 AERA Annual Meeting, New Orleans, LA.Google Scholar
  12. Mellenbergh, G. J. (1982). Contingency table models for assessing item bias.Journal of Educational Statistics, 7(2), 105–118.Google Scholar
  13. Mislevy, R. J., & Bock, R. D. (1984). Item operating characteristics of the Armed Services Aptitute Battery (ASVAB), Form 8A (Tech. Rep. N00014-83-C-0283). Washington, DC: Office of Naval Research.Google Scholar
  14. Nandakumar, R. (1992). Simultaneous DIF amplification and cancellation: Shealy-Stout's test for DIF.Journal of Educational Measurement, 30, 293–311.Google Scholar
  15. Oshima, T., & Miller, D. (1991, April). Multidimensionality and item bias in item response theory. Presented at 1991 AERA Annual Meeting, Chicago, IL.Google Scholar
  16. Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation.Psychometrika, 56, 611–630.Google Scholar
  17. Ramsay, J. O. (1993). TESTGRAF: A program for the graphical analysis of multiple choice test and questionnaire data. TESTGRAF user's guide.Google Scholar
  18. Roussos, L., & Stout, W. F. (1991). BIASDEM: A program for simulating models of unidirectional and crossing DIF. Unpublished manuscript.Google Scholar
  19. Roussos, L., & Stout, W. F. (in press). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance.Journal of Educational Measurement.Google Scholar
  20. Shealy, R., & Stout, W. (1993a). An item response theory model for test bias. In P. W. Holland & H. Wainer (Eds.),Differential item functioning (pp. 197–239). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  21. Shealy, R., & Stout, W. (1993b). A model-based standardization approach that separate true bias/DIF from group ability differences and detects test bias/DIF as well as item bias/DIF.Psychometrika, 58, 159–194.Google Scholar
  22. Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures.Journal of Educational Measurement, 27(4), 361–370.Google Scholar

Copyright information

© The Psychometric Society 1996

Authors and Affiliations

  • Hsin-Hung Li
    • 1
  • William Stout
    • 1
  1. 1.Department of StatisticsUniversity of Illinois at Urbana-ChampaignChampaign

Personalised recommendations