Quality of Life Research

, 16:69

A comparison of three sets of criteria for determining the presence of differential item functioning using ordinal logistic regression

  • Paul K. Crane
  • Laura E. Gibbons
  • Katja Ocepek-Welikson
  • Karon Cook
  • David Cella
  • Kaavya Narasimhalu
  • Ron D. Hays
  • Jeanne A. Teresi
Original Paper

DOI: 10.1007/s11136-007-9185-5

Cite this article as:
Crane, P.K., Gibbons, L.E., Ocepek-Welikson, K. et al. Qual Life Res (2007) 16: 69. doi:10.1007/s11136-007-9185-5

Abstract

Background

Several techniques have been developed to detect differential item functioning (DIF), including ordinal logistic regression (OLR). This study compared different criteria for determining whether items have DIF using OLR.

Objectives

To compare and contrast findings from three different sets of criteria for detecting DIF using OLR. General distress and physical functioning items were evaluated for DIF related to five covariates: age, marital status, gender, race, and Hispanic origin.

Research design

Cross-sectional study.

Subjects

1,714 patients with cancer or HIV/AIDS.

Measures

A total of 23 items addressing physical functioning and 15 items addressing general distress were selected from a pool of 154 items from four different health-related quality of life questionnaires.

Results

The three sets of criteria produced qualitatively and quantitatively different results. Criteria based on statistical significance alone detected DIF in almost all the items, while alternative criteria based on magnitude detected DIF in far fewer items. Accounting for DIF by using demographic-group specific item parameters had negligible effects on individual scores, except for race.

Conclusions

Specific criteria chosen to determine whether items have DIF have an impact on the findings. Criteria based entirely on statistical significance may detect small differences that are clinically negligible.

Keywords

Differential item functioningOrdinal logistic regressionTest biasItem response theoryPsychometrics

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  • Paul K. Crane
    • 1
  • Laura E. Gibbons
    • 1
  • Katja Ocepek-Welikson
    • 2
  • Karon Cook
    • 3
    • 4
  • David Cella
    • 5
    • 6
  • Kaavya Narasimhalu
    • 1
  • Ron D. Hays
    • 7
    • 8
  • Jeanne A. Teresi
    • 9
    • 10
    • 11
  1. 1.Department of Internal Medicine, Harborview Medical CenterUniversity of WashingtonSeattleUSA
  2. 2.Research DivisionHebrew Home for the Aged at RiverdaleRiverdaleUSA
  3. 3.Department of Rehabilitation MedicineUniversity of WashingtonSeattleUSA
  4. 4.HoustonUSA
  5. 5.Psychiatry and Behavioral Science, Institute for Healthcare StudiesNorthwestern UniversityEvanstonUSA
  6. 6.Center on Outcomes, Research and EducationEvanston Northwestern HealthcareEvanstonUSA
  7. 7.Health Services and MedicineUCLALos AngelesUSA
  8. 8.RANDSanta MonicaUSA
  9. 9.Columbia University Stroud Center and Faculty of MedicineNew York State Psychiatric InstituteNew YorkUSA
  10. 10.Research DivisionHebrew Home for the Aged at RiverdaleBronxUSA
  11. 11.Stroud Center for the Quality of LifeNew YorkUSA