Skip to main content

Advertisement

SpringerLink
Log in

Differential item functioning to validate setting of delivery compatibility in PROMIS-global health

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Patient-reported outcomes measures (PROMs) such as PROMIS are increasingly utilized in healthcare to assess patient perception and functional status, but the effect of delivery setting remains to be fully investigated. To our knowledge, no current study establishes the absence of differential item functioning (DIF) across delivery setting for these PROMIS- Global Health (PROMIS-GH) measures among orthopedic patients. We sought to investigate the correlation of PROMIS-GH scores across in-clinic versus remote delivery by evaluating DIF within the Global Physical Health (GPH) and Global Mental Health (GMH) items. We hypothesize that the setting of delivery of the GPH and GMH domains of PROMIS-GH will not impact the results of the measure, allowing direct comparison between the two delivery settings.

Methods

Five thousand and seven hundred and eighty-five complete PROMIS-Global Health measures were analyzed retrospectively using the ‘Lordif’ package on the R platform. DIF was measured for GPH and GMH domains across setting of response (in-clinic vs remote) during the pre-operative period, immediate post-operative period, and 1-year post-operative period using Monte Carlo estimation. McFadden pseudo-R2 thresholds (> 0.02) were used to assess the magnitude of DIF for individual PROMIS items.

Results

No GPH or GMH items contained in the PROMIS-GH instrument yielded DIF across in-clinic vs remote delivery setting during the pre-operative, immediate post-operative, or 1-year post-operative window.

Conclusion

The GPH and GMH domains within the PROMIS-GH instrument may be delivered in the clinic or remotely with comparable accuracy. This cross-delivery setting validation analysis may aid to improve the quality of patient care by allowing mixed platform PROMIS-GH data tailored to individual patient circumstance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Code availability

Analyses were performed in R.

References

  1. Lavallee, D. C., Chenok, K. E., Love, R. M., Petersen, C., Holve, E., Segal, C. D., & Franklin, P. D. (2016). Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Aff (Millwood), 35(4), 575–582.

    Article  Google Scholar 

  2. Black, N. (2013). Patient reported outcome measures could help transform healthcare. BMJ, 346, f167.

    Article  PubMed  Google Scholar 

  3. Brodke, D. J., Saltzman, C. L., & Brodke, D. S. (2016). PROMIS for orthopaedic outcomes measurement. Journal of American Academy of Orthopaedic Surgeons, 24(11), 744–749.

    Article  Google Scholar 

  4. Fries, J. F., Witter, J., Rose, M., Cella, D., Khanna, D., & Morgan-DeWitt, E. (2014). Item response theory, computerized adaptive testing, and PROMIS: Assessment of physical function. Journal of Rheumatology, 41(1), 153–158.

    Article  PubMed  Google Scholar 

  5. Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., Thissen, D., Revicki, D. A., Weiss, D. J., Hambleton, R. K., Liu, H., Gershon, R., Reise, S. P., Lai, J.-S., Cella, D., OBOTPC Group. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: plans for the patient-reported outcomes measurement information system (PROMIS). Medical Care, 45(5), S22–S31.

    Article  PubMed  Google Scholar 

  6. Crins, M. H. P., Terwee, C. B., Ogreden, O., Schuller, W., Dekker, P., Flens, G., Rohrich, D. C., & Roorda, L. D. (2019). Differential item functioning of the PROMIS physical function, pain interference, and pain behavior item banks across patients with different musculoskeletal disorders and persons from the general population. Quality of Life Research, 28(5), 1231–1243.

    Article  PubMed  Google Scholar 

  7. Cook, K. F., Bamer, A. M., Amtmann, D., Molton, I. R., & Jensen, M. P. (2012). Six patient-reported outcome measurement information system short form measures have negligible age- or diagnosis-related differential item functioning in individuals with disabilities. Archives of Physical Medicine and Rehabilitation, 93(7), 1289–1291.

    Article  PubMed  Google Scholar 

  8. Goel, A., & Gross, A. (2019). Differential item functioning in the cognitive screener used in the longitudinal aging study in India. International Psychogeriatrics, 31(9), 1–11.

    Article  Google Scholar 

  9. Lamers, S.M., Glas, C.A., Westerhof, G.J., & Bohlmeijer, E.T. (2012). Longitudinal evaluation of the mental health continuum-short form (MHC-SF). European journal of psychological assessment.

  10. Naumann, A., Hochweber, J., & Hartig, J. (2014). Modeling instructional sensitivity using a longitudinal multilevel differential item functioning approach. Journal of Educational Measurement, 51(4), 381–399.

    Article  Google Scholar 

  11. Wang, M. (2016). Longitudinal differential item functioning detection using bifactor models and the Wald test. University of Kansas.

  12. Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., Amtmann, D., Bode, R., Buysse, D., Choi, S., Cook, K., Devellis, R., DeWalt, D., Fries, J. F., Gershon, R., Hahn, E. A., Lai, J. S., Pilkonis, P., Revicki, D., … PC Group. (2010). The patient-reported outcomes measurement information system (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Gregory, J. J., Werth, P. M., Reilly, C. A., & Jevsevar, D. S. (2021). Cross-specialty PROMIS-global health differential item functioning. Quality of Life Research. https://doi.org/10.1007/s11136-021-02812-6

    Article  PubMed  Google Scholar 

  14. Bliven, B. D., Kaufman, S. E., & Spertus, J. A. (2001). Electronic collection of health-related quality of life data: Validity, time benefits, and patient preference. Quality of Life Research, 10(1), 15–21.

    Article  CAS  PubMed  Google Scholar 

  15. Campbell, N., Ali, F., Finlay, A. Y., & Salek, S. S. (2015). Equivalence of electronic and paper-based patient-reported outcome measures. Quality of Life Research, 24(8), 1949–1961.

    Article  PubMed  Google Scholar 

  16. Hartkopf, A. D., Graf, J., Simoes, E., Keilmann, L., Sickenberger, N., Gass, P., Wallwiener, D., Matthies, L., Taran, F.-A., Lux, M. P., Wallwiener, S., Belleville, E., Sohn, C., Fasching, P. A., Schneeweiss, A., Brucker, S. Y., & Wallwiener, M. (2017). Electronic-Based Patient-Reported Outcomes: Willingness, Needs, and Barriers in Adjuvant and Metastatic Breast Cancer Patients. JMIR Cancer, 3(2), e11.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Jensen, R. E., Snyder, C. F., Abernethy, A. P., Basch, E., Potosky, A. L., Roberts, A. C., Loeffler, D. R., & Reeve, B. B. (2014). Review of electronic patient-reported outcomes systems used in cancer clinical care. Journal of Oncology Practice, 10(4), e215–e222.

    Article  PubMed  Google Scholar 

  18. Malhotra, K., Buraimoh, O., Thornton, J., Cullen, N., Singh, D., & Goldberg, A. J. (2016). Electronic capture of patient-reported and clinician-reported outcome measures in an elective orthopaedic setting: a retrospective cohort analysis. British Medical Journal Open, 6(6), e011975.

    Google Scholar 

  19. Schwartzberg, L. (2016). Electronic patient-reported outcomes: the time is ripe for integration into patient care and clinical research. American Society of Clinical Oncology Educational Book, 36, e89–e96.

    Article  Google Scholar 

  20. Bjorner, J. B., Rose, M., Gandek, B., Stone, A. A., Junghaenel, D. U., & Ware, J. E., Jr. (2014). Method of administration of PROMIS scales did not significantly impact score level, reliability, or validity. Journal of Clinical Epidemiology, 67(1), 108–113.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Gwaltney, C. J., Shields, A. L., & Shiffman, S. (2008). Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: A meta-analytic review. Value Health, 11(2), 322–333.

    Article  PubMed  Google Scholar 

  22. Rutherford, C., Costa, D., Mercieca-Bebber, R., Rice, H., Gabb, L., & King, M. (2016). Mode of administration does not cause bias in patient-reported outcome results: A meta-analysis. Quality of Life Research, 25(3), 559–574.

    Article  PubMed  Google Scholar 

  23. Cook, C. (2010). Mode of administration bias. Journal of Manual & Manipulative Therapy, 18(2), 61–63.

    Article  Google Scholar 

  24. Delgado-Rodríguez, M., & Llorca, J. (2004). Bias. Journal of Epidemiology and Community Health., 58(8), 635.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Grootendorst, P. V., Feeny, D. H., & Furlong, W. (1997). Does it matter whom and how you ask? Inter- and intra-rater agreement in the Ontario Health Survey. Journal of Clinical Epidemiology, 50(2), 127–135.

    Article  CAS  PubMed  Google Scholar 

  26. Khoury, M. J., & Flanders, W. D. (1995). Bias in using family history as a risk factor in case-control studies of disease. Epidemiology, 6(5), 511–519.

    Article  CAS  PubMed  Google Scholar 

  27. Hammarstedt, J. E., Redmond, J. M., Gupta, A., Dunne, K. F., Vemula, S. P., & Domb, B. G. (2017). Survey mode influence on patient-reported outcome scores in orthopaedic surgery: Telephone results may be positively biased. Knee Surgery, Sports Traumatology, Arthroscopy, 25(1), 50–54.

    Article  PubMed  Google Scholar 

  28. Hutchings, A., Grosse Frie, K., Neuburger, J., van der Meulen, J., & Black, N. (2013). Late response to patient-reported outcome questionnaires after surgery was associated with worse outcome. Journal of Clinical Epidemiology, 66(2), 218–225.

    Article  PubMed  Google Scholar 

  29. Manual, P.G.S., (2017) GLOBAL HEALTH - A brief guide to the PROMIS Global Health instruments. Patient-Reported Outcomes Measurement Information System (PROMIS), pp. 1–18.

  30. Liu, H., Cella, D., Gershon, R., Shen, J., Morales, L. S., Riley, W., & Hays, R. D. (2010). Representativeness of the patient-reported outcomes measurement information system internet panel. Journal of Clinical Epidemiology, 63(11), 1169–1178.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of statistical Software, 48(1), 1–29.

    Google Scholar 

  32. Van der Ark, L. A. (2012). New developments in Mokken scale analysis in R. Journal of Statistical Software, 48(5), 1–27.

    Article  Google Scholar 

  33. Choi, S.W., L.E. Gibbons, & P.K. Crane. (2011). lordif: An R package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and monte carlo simulations. 39(8), 30.

  34. Crane, P. K., Gibbons, L. E., Jolley, L., & van Belle, G. (2006). Differential item functioning analysis with ordinal logistic regression techniques: DIFdetect and difwithpar. Medical Care, 44(11), S115–S123.

    Article  PubMed  Google Scholar 

  35. Hays, R. D., Bjorner, J. B., Revicki, D. A., Spritzer, K. L., & Cella, D. (2009). Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. Quality of Life Research, 18(7), 873–880.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Lapin, B. R., Thompson, N. R., Schuster, A., & Katzan, I. L. (2019). Patient versus proxy response on global health scales: No meaningful DIFference. Quality of Life Research, 28(6), 1585–1594.

    Article  PubMed  Google Scholar 

  37. Cramér, H., Mathematical Methods of Statistics (PMS-9), Volume 9. 2016: Princeton university press.

  38. Mokken, R.J. (2011). A theory and procedure of scale analysis. De Gruyter Mouton.

  39. Molenaar, I.W. (1990) A weighted Loevinger H-coefficient extending Mokken scaling to multicategory items. Psychologische Instituten der Rijksuniversiteit Groningen.

  40. Carlozzi, N. E., Boileau, N. R., Roche, M. W., Ready, R. E., Perlmutter, J. S., Chou, K. L., Barton, S. K., McCormack, M. K., Stout, J. C., Cella, D., Miner, J. A., & Paulsen, J. S. (2020). Responsiveness to change over time and test-retest reliability of the PROMIS and Neuro-QoL mental health measures in persons with Huntington disease (HD). Quality of Life Research, 29(12), 3419–3439.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological methods & research, 21(2), 230–258.

    Article  Google Scholar 

  42. Jöreskog, K.G. & D. Sörbom. (1993). LISREL 8: Structural equation modeling with the SIMPLIS command language. Scientific Software International.

  43. Lin, O. S., Schembre, D. B., Ayub, K., Gluck, M., McCormick, S. E., Patterson, D. J., Cantone, N., Soon, M. S., & Kozarek, R. A. (2007). Patient satisfaction scores for endoscopic procedures: Impact of a survey-collection method. Gastrointestinal Endoscopy, 65(6), 775–781.

    Article  PubMed  Google Scholar 

  44. Feveile, H., Olsen, O., & Hogh, A. (2007). A randomized trial of mailed questionnaires versus telephone interviews: Response patterns in a survey. BMC Medical Research Methodology, 7(1), 27.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Höher, J., Bach, T., Münster, A., Bouillon, B., & Tiling, T. (1997). Does the mode of data collection change results in a subjective knee score? Self-administration versus interview. American Journal of Sports Medicine, 25(5), 642–647.

    Article  PubMed  Google Scholar 

  46. Ludemann, R., Watson, D. I., & Jamieson, G. G. (2003). Influence of follow-up methodology and completeness on apparent clinical outcome of fundoplication. American Journal of Surgery, 186(2), 143–147.

    Article  PubMed  Google Scholar 

  47. Plante, C., Jacques, L., Chevalier, S., & Fournier, M. (2012). Comparability of Internet and telephone data in a survey on the respiratory health of children. Canadian Respiratory Journal, 19(1), 13–18.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Powers, J. R., Mishra, G., & Young, A. F. (2005). Differences in mail and telephone responses to self-rated health: Use of multiple imputation in correcting for response bias. Australian and New Zealand Journal of Public Health, 29(2), 149–154.

    Article  CAS  PubMed  Google Scholar 

  49. Rhodes, T., Girman, C. J., Jacobsen, S. J., Guess, H. A., Hanson, K. A., Oesterling, J. E., & Lieber, M. M. (1995). Does the mode of questionnaire administration affect the reporting of urinary symptoms? Urology, 46(3), 341–345.

    Article  CAS  PubMed  Google Scholar 

  50. Harewood, G. C., Yacavone, R. F., Locke, G. R., 3rd., & Wiersema, M. J. (2001). Prospective comparison of endoscopy patient satisfaction surveys: e-mail versus standard mail versus telephone. American Journal of Gastroenterology, 96(12), 3312–3317.

    Article  CAS  PubMed  Google Scholar 

Download references

Funding

This Project had no external funding.

Author information

Authors and Affiliations

  1. Department of Orthopaedics, Dartmouth-Hitchcock Medical Center, Lebanon, NH, 03766, USA

    Dylan J. Parker, Paul M. Werth, David D. Christensen & David S. Jevsevar

  2. Department of Orthopaedics, Geisel School of Medicine at Dartmouth, Hanover, NH, 03755, USA

    Paul M. Werth, David D. Christensen & David S. Jevsevar

Authors
  1. Dylan J. Parker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul M. Werth
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David D. Christensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David S. Jevsevar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dylan J. Parker.

Ethics declarations

Conflict of interest

No potential conflict of interests to report.

Ethical approval

Study approved by Dartmouth College by the Institution’s Ethics Board, approval number STUDY02000132.

Consent to participate

This study was submitted for IRB approval and was deemed exempt from requiring patient consent.

Consent for publication

This study was submitted for IRB approval and was deemed exempt from requiring patient consent.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 340 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Parker, D.J., Werth, P.M., Christensen, D.D. et al. Differential item functioning to validate setting of delivery compatibility in PROMIS-global health. Qual Life Res 31, 2189–2200 (2022). https://doi.org/10.1007/s11136-022-03084-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-022-03084-4

Keywords

Search

Navigation