Skip to main content

Advertisement

Log in

Rapid detection of differential item functioning in assessments of health-related quality of life: The Functional Assessment of Cancer Therapy

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Reason for study

Differential item functioning (DIF) occurs when a test item functions differently in different groups when controlling for the level of the underlying construct measured by the test. DIF assessment is a first step in the evaluation of test bias. We sought to demonstrate a rapid hybrid approach to DIF detection by determining the presence and scale-level impact of DIF related to eight covariates in four domains measured by the Functional Assessment of Cancer Therapy (FACT).

Major findings

The number of items found with DIF in each domain depended on the criterion chosen to define the presence of DIF. With a few exceptions, scale-level differential functioning was similar regardless of the criteria chosen. For physical well-being, there was relevant scale-level differential functioning related only to race. For social and family well-being, there was relevant scale-level differential functioning related to each of the covariates. For emotional well-being, there was relevant scale-level differential functioning related to ethnicity, language, and race. For functional well-being, there was relevant scale-level differential functioning related to ethnicity, race, education, and self- vs. interviewer-administration.

Principal conclusions

Our rapid hybrid approach to DIF detection may be broadly applicable in other studies of health-related quality of life.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Abbreviations

DIF:

differential item functioning. DIF occurs when item has different statistical properties in different groups when controlling for the underlying trait or ability measured by the test

FACT:

Functional Assessment of Cancer Therapy; FACT-G is the FACT-General. This is a widely used assessment system for functioning and well-being. It assesses five domains, four of which are analyzed in this paper: physical well-being (PWB), social and family well-being (SFWB), emotional well-being (EWB), and functional well-being (FWB)

HRQL:

health-related quality of life

IRT:

item response theory. This is a technique for analyzing item-level test data based on the premise that item responses are a function of the relationship between an underlying latent trait and characteristics of the item

References

  1. Camilli G, Shepard LA (1994) Methods for Identifying Biased Test Items. Sage, Thousand Oaks

    Google Scholar 

  2. Millsap RE, Everson HT (1993) Methodology review: statistical approaches for assessing measurement bias. Appl Psychol Measurement 17:297–334

    Article  Google Scholar 

  3. Holland PW, Wainer H (eds) (1993) Differential Item Functioning. Erlbaum, Hillsdale, NJ

    Google Scholar 

  4. Martin M, Blaisdell B, Kwong JW, Bjorner JB (2004) The Short-Form Headache Impact Test (HIT-6) was psychometrically equivalent in nine languages. J Clin Epidemiol 57:1271–1278

    Article  PubMed  Google Scholar 

  5. Roorda LD, Jones CA, Waltz M et al. (2004) Satisfactory cross cultural equivalence of the Dutch WOMAC in patients with hip osteoarthritis waiting for arthroplasty. Ann Rheum Dis 63:36–42

    Article  CAS  PubMed  Google Scholar 

  6. Prieto L, Novick D, Sacristan JA, Edgell ET, Alonso J. A Rasch model analysis to test the cross-cultural validity of the EuroQoL-5D in the Schizophrenia Outpatient Health Outcomes Study. Acta Psychiatr Scand Suppl 2003; 24–29

  7. Moorer P, Suurmeije Th P, Foets M, Molenaar IW (2001) Psychometric properties of the RAND-36 among three chronic diseases (multiple sclerosis, rheumatic diseases and COPD) in The Netherlands. Qual Life Res 10:637–645

    Article  CAS  PubMed  Google Scholar 

  8. Bjorner JB, Kreiner S, Ware JE, Damsgaard MT, Bech P (1998) Differential item functioning in the Danish translation of the SF-36. J Clin Epidemiol 51:1189–1202

    Article  CAS  PubMed  Google Scholar 

  9. Groenvold M, Bjorner JB, Klee MC, Kreiner S (1995) Test for item bias in a quality of life questionnaire. J Clin Epidemiol 48: 805–816

    Article  CAS  PubMed  Google Scholar 

  10. Hahn EA, Holzner B, Kemmler G, Sperner-Unterweger B, Hudgens SA, Cella D (2005) Cross-cultural evaluation of health status using item response theory: FACT-B comparisons between Austrian and U.S. patients with breast cancer. Eval Health Prof 28: 233-59

    Article  PubMed  Google Scholar 

  11. Kutlay S, Kucukdeveci AA, Gonul D, Tennant A (2003) Adaptation and validation of the Turkish version of the Rheumatoid Arthritis Quality of Life Scale. Rheumatol Int 23: 21–66

    PubMed  Google Scholar 

  12. Petersen MA, Groenvold M, Bjorner JB et al. (2003) Use of differential item functioning analysis to assess the equivalence of translations of a questionnaire. Qual Life Res 12: 373–85

    Article  PubMed  Google Scholar 

  13. Tennant A, Penta M, Tesio L et al. (2004) Assessing and adjusting for cross-cultural validity of impairment and activity limitation scales through differential item functioning within the framework of the Rasch model: the PRO-ESOR project. Med Care 42: I37–48

    Article  PubMed  Google Scholar 

  14. Pagano IS, Gotay CC (2005) Ethnic differential item functioning in the assessment of quality of life in cancer patients. Health Qual Life Outcomes 3: 60

    Article  PubMed  Google Scholar 

  15. Dancer LS, Anderson AJ, Derlin RL (1994) Use of log-linear models for assessing differential item functioning in a measure of psychological functioning. J Consult Clin Psychol 62: 710–7

    Article  CAS  PubMed  Google Scholar 

  16. Fleishman JA, Spector WD, Altman BM (2002) Impact of differential item functioning on age and gender differences in functional disability. J Gerontol B Psychol Sci Soc Sci 57: S275–84

    PubMed  Google Scholar 

  17. Hagell P, Whalley D, McKenna SP, Lindvall O (2003) Health status measurement in Parkinson’s disease: validity of the PDQ-39 and Nottingham Health Profile. Mov Disord 18: 773–83

    Article  PubMed  Google Scholar 

  18. Fleishman JA, Lawrence WF (2003) Demographic variation in SF-12 scores: true differences or differential item functioning? Med Care 41: III75-III86

    PubMed  Google Scholar 

  19. Wu AW, Jacobson DL, Berzon RA et al. (1997) The effect of mode of administration on medical outcomes study health ratings and EuroQol scores in AIDS. Qual Life Res 6: 3–10

    CAS  PubMed  Google Scholar 

  20. Garcia-Losa M, Unda M, Badia X et al. (2001) Effect of mode of administration on I-PSS scores in a large BPH patient population. Eur Urol 40: 451–457

    Article  CAS  PubMed  Google Scholar 

  21. Bozlu M, Doruk E, Akbay E et al. (2002) Effect of administration mode (patient vs physician) and patient’s educational level on the Turkish version of the International Prostate Symptom Score. Int J Urol 9: 417–21

    Article  PubMed  Google Scholar 

  22. Weinberger M, Oddone EZ, Samsa GP, Landsman PB (1996) Are health-related quality-of-life measures affected by the mode of administration?. J Clin Epidemiol 49: 135–40

    Article  CAS  PubMed  Google Scholar 

  23. Crane PK, van Belle G, Larson EB (2004) Test bias in a cognitive test: differential item functioning in the CASI. Stat Med 23:241–56

    Article  PubMed  Google Scholar 

  24. Crane PK, Gibbons LE, Jolley L, van Belle G (2006) DIF analysis with ordinal logistic regression: DIFdetect. Med Care (in press)

  25. Wagle AC, Berrios GE, Ho L (1999) The cognitive failures questionnaire in psychiatry. Compr Psychiatry 40: 478–484

    Article  CAS  PubMed  Google Scholar 

  26. Cella D (1994) Manual for the Functional Assessment of Cancer Therapy (FACT) and Functional Assessment of HIV Infection (FAHI) Scales (Version 3). Rush-Presbyterian-St. Luke’s Medical Center, Chicago

    Google Scholar 

  27. Cella DF, Tulsky DS, Gray G et al. (1993) The Functional Assessment of Cancer Therapy scale: development and validation of the general measure. J Clin Oncol 11: 570–579

    CAS  PubMed  Google Scholar 

  28. Cella DF, Bonomi AE. The Functional Assessment of Cancer Therapy (FACT) and Functional Assessment of HIV Infection (FAHI) quality of life measurement system. In: Spilker B (ed.), Quality of Life and Pharmacoeconomics in Clinical Trials. NY: Lippincott-Raven, 1996.

  29. Bonomi AE, Cella DF, Hahn EA et al. (1996) Multilingual translation of the Functional Assessment of Cancer Therapy (FACT) quality of life measurement system. Qual Life Res 5: 309–320

    Article  CAS  PubMed  Google Scholar 

  30. Cella D, Hernandez L, Bonomi AE et al. (1998) Spanish language translation and initial validation of the functional assessment of cancer therapy quality-of-life instrument. Med Care 36: 1407–18

    Article  CAS  PubMed  Google Scholar 

  31. Cella DF (1997) Manual for the Functional Assessment of Chronic Illness Therapy (FACIT) Scales (Version 4). Center on Outcomes, Research, and Education (CORE), Evanston Northwestern Healthcare and Northwestern University, Evanston, IL

    Google Scholar 

  32. Ward WL, Hahn EA, Mo F, Hernandez L, Tulsky DS, Cella D (1999) Reliability and validity of the Functional Assessment of Cancer Therapy-Colorectal (FACT-C) quality of life instrument. Qual Life Res 8: 181–195

    Article  CAS  PubMed  Google Scholar 

  33. Woodcock RW, Johnson MB (1990) Woodcock–Johnson Psycho-educational Battery. DLM Teaching Resources, Allen, TX

    Google Scholar 

  34. Muraki E, Bock D. PARSCALE for Windows, version 4.1. Chicago: Scientific Software International, 2003.

  35. Samejima F. Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph 1969; No. 17.

  36. Samejima F. Graded response model. In: van der Linden WJ, Hambleton RK (eds.), Handbook of Modern Item Response Theory. NY: Springer, 1997

  37. StataCorp. Stata statistical software: release 8.0.

  38. McCullagh P, Nelder JA (1989) Generalized linear models. Chapman and Hall, London

    Google Scholar 

  39. Maldonado G, Greenland S (1993) Simulation study of confounder-selection strategies. Am J Epidemiol 138: 923–936

    CAS  PubMed  Google Scholar 

  40. Crane PK, Hart DL, Gibbons LE, Cook KF (2006) A 37-item shoulder functional status item pool had negligible differential item functioning. J Clin Epidemiol 59: 478–484

    Article  PubMed  Google Scholar 

  41. Reise SP, Widaman KF, Pugh RH (2002) Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychol Bull 114: 552–566

    Article  Google Scholar 

  42. Cella D, Eton DT, Lai JS, Peterman AH, Merkel DE (2002) Combining anchor and distribution-based methods to derive minimal clinically important differences on the Functional Assessment of Cancer Therapy (FACT) anemia and fatigue scales. J Pain Symptom Manage 24: 547–561

    Article  PubMed  Google Scholar 

  43. Eton DT, Cella D, Yost KJ et al. (2004) A combination of distribution- and anchor-based approaches determined minimally important differences (MIDs) for four endpoints in a breast cancer scale. J Clin Epidemiol 57:898–910

    Article  PubMed  Google Scholar 

  44. Yost KJ, Sorensen MV, Hahn EA, Glendenning GA, Gnanasakthy A, Cella D (2005) Using multiple anchor- and distribution-based estimates to evaluate clinically meaningful change on the Functional Assessment of Cancer Therapy-Biologic Response Modifiers (FACT-BRM) instrument. Value Health 8: 117–27

    Article  PubMed  Google Scholar 

  45. Shealy RT, Stout WF. An item response theory model for test bias and differential test functioning. In: Holland PW, Wainer H (eds.), Differential item functioning. Hillsdale, N.J.: Erlbaum, 1993

  46. Williams DR (2005) The health of US racial and ethnic populations. J Gerontol B Psychol Sci Soc Sci 60(Spec No 2): 53–62

    PubMed  Google Scholar 

  47. Williams DR, Jackson PB (2005) Social sources of racial disparities in health. Health Aff (Millwood) 24: 325–334

    Article  Google Scholar 

  48. Smedley BD, Stith AY, Nelson AR. Unequal treatment: confronting racial and ethnic disparities in health care. In: Smedley BD, Stith AY, Nelson AR (eds), Washington, DC.: National Academy Press, 2002

  49. Wan GJ, Counte MA, Cella DF, Hernandez L, McGuire DB, Deasay S, Shiomoto G, Hahn EA (1999) The impact of socio-cultural and clinical factors on health-related quality of life reports among Hispanic and African-American cancer patients. J Outcome Meas 3: 200–15

    CAS  PubMed  Google Scholar 

Download references

Acknowledgements

Drs. Crane and Gibbons and Ms. Narasimhalu were supported by grant K08 022232 from the National Institute on Aging. Drs. Cella and Lai were supported by R01 61679 from the National Cancer Institute. A portion of this work was presented at the 2005 International Society for Quality of Life Research in San Francisco.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Paul K. Crane.

Appendix: Item content for the FACT-G version 3

Appendix: Item content for the FACT-G version 3

General instructions: “Below is a list of statements that other people with your illness have said are important. By circling one number per line, please indicate how true each statement has been for you during the past 7 days.”

For each item, responses are 0, “not at all”; 1, “a little bit”; 2, “somewhat”; 3, “quite a bit”; 4, “very much”, except as noted (items 8, 16, 19, 26, and 34).

PHYSICAL WELL-BEING

During the past 7 days:

  1. 1.

    I have a lack of energy

  2. 2.

    I have nausea

  3. 3.

    Because of my physical condition, I have trouble meeting the needs of my family

  4. 4.

    I have pain

  5. 5.

    I am bothered by side effects of treatments

  6. 6.

    I feel sick

  7. 7.

    I am forced to spend time in bed

  8. 8.

    Looking at the above 7 questions, how much would you say your PHYSICAL WELL-BEING affects your quality of life (responses from 0, “Not at all”, to 10, “Very much so”)*

SOCIAL/FAMILY WELL-BEING

During the past 7 days:

  1. 9.

    I feel distant from my friends

  2. 10.

    I get emotional support from my family

  3. 11.

    I get support from my friends and neighbors

  4. 12.

    My family has accepted my illness

  5. 13.

    Family communication about my illness is poor

  6. 14.

    I feel close to my partner (or the person who is my main support)

  7. 15.

    Have you been sexually active in the past year? If yes: I am satisfied with my sex life

  8. 16.

    Looking at the above 7 questions, how much would you say your SOCIAL/FAMILY WELL-BEING affects your quality of life (responses from 0, “Not at all”, to 10, “Very much so”)*

RELATIONSHIP WITH DOCTOR

During the past 7 days:

  1. 17.

    I have confidence in my doctor(s)*

  2. 18.

    My doctor is available to answer my questions*

  3. 19.

    Looking at the above 2 questions, how much would you say your RELATIONSHIP WITH THE DOCTOR affects your quality of life (responses from 0, “Not at all”, to 10, “Very much so”)*

EMOTIONAL WELL-BEING

During the past 7 days:

  1. 20.

    I feel sad

  2. 21.

    I am proud

  3. 22.

    I am losing hope in the fight against my illness

  4. 23.

    I feel nervous

  5. 24.

    I worry about dying

  6. 25.

    I worry that my condition will get worse

  7. 26.

    Looking at the above 6 questions, how much would you say your EMOTIONAL WELL-BEING affects your quality of life (responses from 0, “Not at all”, to 10, “Very much so”)*

FUNCTIONAL WELL-BEING

During the past 7 days:

  1. 27.

    I am able to work (include work in home)

  2. 28.

    My work (include work in home) is fulfilling

  3. 29.

    I am able to enjoy life

  4. 30.

    I have accepted my illness

  5. 31.

    I am sleeping well

  6. 32.

    I am enjoying the things I usually do for fun

  7. 33.

    I am content with the quality of my life right now

  8. 34.

    Looking at the above 7 questions, how much would you say your FUNCTIONAL WELL-BEING affects your quality of life? (responses from 0, “Not at all”, to 10, “Very much so”)*

* Item not included in these analyses

Rights and permissions

Reprints and permissions

About this article

Cite this article

Crane, P., Gibbons, L., Narasimhalu, K. et al. Rapid detection of differential item functioning in assessments of health-related quality of life: The Functional Assessment of Cancer Therapy. Qual Life Res 16, 101–114 (2007). https://doi.org/10.1007/s11136-006-0035-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-006-0035-7

Keywords

Navigation