Skip to main content

Advertisement

SpringerLink
Log in

An analysis of missing items in real-world electronic patient reported outcomes data: implications for clinical care

  • Original Article
  • Published:
Supportive Care in Cancer Aims and scope Submit manuscript

Abstract

Purpose

Utilization of electronic patient-reported outcomes (ePROs) in the clinic can improve quality of life and prolong survival in cancer care. However, there remain unanswered questions regarding trends in missing data and the potential effect on real-time patient care.

Methods

This study utilized a prospectively collected dataset of ePROs from oncology clinics that administered the Patient Care Monitor 2.0 (PCM), a validated symptoms survey assessing 78 items for men, and 86 for women. We tabulated the frequency of missing items, by item and domain (emotional, functional and physical symptom-related), and examined these by age, gender, education, race and marital status.

Results

Within 20,986 encounters, there were responses to at least 1 PCM item from 6933 unique patients. The highest frequency of missing answers occurred for: “attend a paid job” (10.7%), “reduced sexual enjoyment” (3.8%), and “run” (3.7%). By domain, 12.3% of functional, 8.4% of physical symptom-related, and 1.6% of emotional constructs contained at least one missing item. For functional and physical symptom-related items, missingness was most common in patients >60 years old.

Conclusion

The frequency of missingness was highest for functional items, like attending a paid job, suggesting that some respondents (e.g., retirees without a paid job) skipped questions that were less applicable to them. More universal issues for cancer patients, such as emotional well-being, had much lower frequencies of missingness. This suggests differential item completion that warrants further study to understand the inherent drivers. Identifying causes of missingness could improve the clinical utility of ePROs and highlight opportunities to personalize care.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. LeBlanc TW, Abernethy AP (2017) Patient-reported outcomes in cancer care - hearing the patient voice at greater volume. Nat Rev Clin Oncol 14(12):763–772. https://doi.org/10.1038/nrclinonc.2017.153

    Article  PubMed  Google Scholar 

  2. Abernethy AP, Herndon JE 2nd, Wheeler JL, Day JM, Hood L, Patwardhan M, Shaw H, Lyerly HK (2009) Feasibility and acceptability to patients of a longitudinal system for evaluating cancer-related symptoms and quality of life: pilot study of an e/tablet data-collection system in academic oncology. J Pain Symptom Manag 37(6):1027–1038. https://doi.org/10.1016/j.jpainsymman.2008.07.011

    Article  Google Scholar 

  3. Pakhomov SV, Jacobsen SJ, Chute CG, Roger VL (2008) Agreement between patient-reported symptoms and their documentation in the medical record. Am J Manag Care 14(8):530–539

    PubMed  PubMed Central  Google Scholar 

  4. Fromme EK, Eilers KM, Mori M, Hsieh YC, Beer TM (2004) How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the quality-of-life questionnaire C30. J Clin Oncol 22(17):3485–3490. https://doi.org/10.1200/jco.2004.03.025

    Article  PubMed  Google Scholar 

  5. Basch E, Jia X, Heller G, Barz A, Sit L, Fruscione M, Appawu M, Iasonos A, Atkinson T, Goldfarb S, Culkin A, Kris MG, Schrag D (2009) Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes. J Natl Cancer Inst 101(23):1624–1632. https://doi.org/10.1093/jnci/djp386

    Article  PubMed  PubMed Central  Google Scholar 

  6. Atkinson TM, Li Y, Coffey CW, Sit L, Shaw M, Lavene D, Bennett AV, Fruscione M, Rogak L, Hay J, Gonen M, Schrag D, Basch E (2012) Reliability of adverse symptom event reporting by clinicians. Qual Life Res 21(7):1159–1164. https://doi.org/10.1007/s11136-011-0031-4

    Article  PubMed  Google Scholar 

  7. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, Rogak L, Bennett AV, Dueck AC, Atkinson TM, Chou JF, Dulko D, Sit L, Barz A, Novotny P, Fruscione M, Sloan JA, Schrag D (2016) Symptom monitoring with patient-reported outcomes during routine Cancer treatment: a randomized controlled trial. J Clin Oncol 34(6):557–565. https://doi.org/10.1200/JCO.2015.63.0830

    Article  CAS  PubMed  Google Scholar 

  8. Basch E, Deal AM, Dueck AC, Scher HI, Kris MG, Hudis C, Schrag D (2017) Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine Cancer treatment. JAMA 318(2):197–198. https://doi.org/10.1001/jama.2017.7156

    Article  PubMed  PubMed Central  Google Scholar 

  9. Sperti E, Di Maio M (2017) Outcomes research: integrating PROs into the clinic - overall survival benefit or not, it's worth the trouble. Nat Rev Clin Oncol 14(9):529–530. https://doi.org/10.1038/nrclinonc.2017.109

    Article  PubMed  Google Scholar 

  10. Bennett AV, Jensen RE, Basch E (2012) Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J Clin 62(5):337–347. https://doi.org/10.3322/caac.21150

    Article  PubMed  Google Scholar 

  11. de Leeuw E (2001) Reducing Missing Data in Surveys: An Overview of Methods, vol 35. https://doi.org/10.1023/A:1010395805406

  12. Colsher PL, Wallace RB (1989) Data quality and age: health and psychobehavioral correlates of item nonresponse and inconsistent responses. J Gerontol 44(2):P45–P52

    Article  CAS  Google Scholar 

  13. Huisman M (1999) Item nonresponse: occurrence, causes, and imputation of missing answers to test items

  14. Hutchings A, Neuburger J, Grosse Frie K, Black N, van der Meulen J (2012) Factors associated with non-response in routine use of patient reported outcome measures after elective surgery in England. Health Qual Life Outcomes 10:34. https://doi.org/10.1186/1477-7525-10-34

    Article  PubMed  PubMed Central  Google Scholar 

  15. Kjaer MN, Mortensen CB, Hjortrup PB, Rygard SL, Andersen I, Perner A (2018) Factors associated with non-response at health-related quality of life follow-up in a septic shock trial. Acta Anaesthesiol Scand 62(3):357–366. https://doi.org/10.1111/aas.13056

    Article  CAS  PubMed  Google Scholar 

  16. Wysham NG, Wolf SP, Samsa G, Abernethy AP, LeBlanc TW (2017) Integration of electronic patient-reported outcomes into routine Cancer care: an analysis of factors affecting data completeness. JCO Clin Cancer Inform 1:1–10. https://doi.org/10.1200/cci.16.00043

    Article  PubMed  Google Scholar 

  17. Abernethy AP, Herndon JE 2nd, Wheeler JL, Patwardhan M, Shaw H, Lyerly HK, Weinfurt K (2008) Improving health care efficiency and quality using tablet personal computers to collect research-quality, patient-reported data. Health Serv Res 43(6):1975–1991. https://doi.org/10.1111/j.1475-6773.2008.00887.x

    Article  PubMed  PubMed Central  Google Scholar 

  18. Abernethy AP, Zafar SY, Uronis H, Wheeler JL, Coan A, Rowe K, Shelby RA, Fowler R, Herndon JE 2nd (2010) Validation of the patient care monitor (version 2.0): a review of system assessment instrument for cancer patients. J Pain Symptom Manag 40(4):545–558. https://doi.org/10.1016/j.jpainsymman.2010.01.017

    Article  Google Scholar 

  19. Fortner B, Okon T, Schwartzberg L, Tauer K, Houts AC (2003) The Cancer care monitor: psychometric content evaluation and pilot testing of a computer administered system for symptom screening and quality of life in adult cancer patients. J Pain Symptom Manag 26(6):1077–1092. https://doi.org/10.1016/j.jpainsymman.2003.04.003

    Article  Google Scholar 

  20. Samsa GP, Wolf S, LeBlanc TW, Abernethy AP (2016) An exploratory factor analysis of the scale structure of the patient care monitor version 2.0. J Pain Symptom Manag 51(4):776–783 e772. https://doi.org/10.1016/j.jpainsymman.2015.11.013

    Article  Google Scholar 

  21. Dupont A, Wheeler J, Herndon J, Coan A, Zafar S, Hood L, Patwardhan M, Shaw H, Lyerly H, Abernethy A (2009) Use of tablet personal computers for sensitive patient-reported information. J Support Oncol 7(3):91–97

    PubMed  Google Scholar 

  22. Analysis of Correlated Binary Data. In: Statistical Methods for Rates and Proportions. pp 440–490. doi:https://doi.org/10.1002/0471445428.ch15

  23. LeBlanc TW, Kamal AH (2017) Assessing psychological toxicity and patient-reported distress as the sixth vital sign in cancer care and clinical trials. AMA J Ethics 19(5):460–466

    Article  Google Scholar 

  24. Reese JB, Shelby RA, Abernethy AP (2011) Sexual concerns in lung cancer patients: an examination of predictors and moderating effects of age and gender. Support Care Cancer 19(1):161–165. https://doi.org/10.1007/s00520-010-1000-0

    Article  PubMed  Google Scholar 

  25. Reese JB, Shelby RA, Keefe FJ, Porter LS, Abernethy AP (2010) Sexual concerns in cancer patients: a comparison of GI and breast cancer patients. Support Care Cancer 18(9):1179–1189. https://doi.org/10.1007/s00520-009-0738-8

    Article  PubMed  Google Scholar 

  26. Revicki D, Cella D (1997) Health status assessment for the twenty-first century: item response theory, item banking and computer adaptive testing. Qual Life Res 6(6):595–600

    Article  CAS  Google Scholar 

  27. Smith AB, Hanbury A, Retzler J (2019) Item banking and computer-adaptive testing in clinical trials: standing in sight of the PROMISed land. Contemp Clin Trials Commun 13:100306

    Article  Google Scholar 

  28. Papuga M, Dasilva C, McIntyre A, Mitten D, Kates S, Baumhauer J (2018) Large-scale clinical implementation of PROMIS computer adaptive testing with direct incorporation into the electronic medical record. Health Syst 7(1):1–12

    Article  CAS  Google Scholar 

  29. Tyser AR, Beckmann J, Franklin JD, Cheng C, Hon SD, Wang A, Hung M (2014) Evaluation of the PROMIS physical function computer adaptive test in the upper extremity. J Hand Surg 39(10):2047–2051.e2044. https://doi.org/10.1016/j.jhsa.2014.06.130

    Article  Google Scholar 

  30. Wagner LI, Schink J, Bass M, Patel S, Diaz MV, Rothrock N, Pearman T, Gershon R, Penedo FJ, Rosen S (2015) Bringing PROMIS to practice: brief and precise symptom screening in ambulatory cancer care. Cancer 121(6):927–934

    Article  Google Scholar 

Download references

Acknowledgements

We acknowledge, with great appreciation, Dr. Amy P. Abernethy, who led the creation and implementation of this ePRO system into our clinics, and has passed down this dataset so that we can continue learning from our patients, and from her remarkable efforts.

Author information

Authors and Affiliations

  1. Duke University School of Medicine, Durham, NC, 27710, USA

    Heather A. Rosett, Jesse D. Troy & Thomas W. LeBlanc

  2. Duke Cancer Institute, Durham, NC, 27710, USA

    Susan C. Locke, Kris W. Herring & Jesse D. Troy

  3. Duke University Medical Center, Durham, NC, 27708, USA

    Steven P. Wolf

  4. Duke Department of Biostatistics and Bioinformatics, Durham, NC, USA

    Gregory P. Samsa

Authors
  1. Heather A. Rosett
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Susan C. Locke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Steven P. Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kris W. Herring
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gregory P. Samsa
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jesse D. Troy
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Thomas W. LeBlanc
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas W. LeBlanc.

Ethics declarations

Research support

The Duke BERD Methods Core’s support of this project was made possible (in part) by Grant Number UL1TR002553 from the National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health (NIH), and NIH Roadmap for Medical Research. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of NCATS or NIH. This work was also supported in part by grant number P30CA014236 from the National Cancer Institute at the NIH.

Disclaimers

Heather Rosett: None to disclose.

Susan C. Locke:

Consulting - Robin Care, Inc.

Consulting - Predigen, Inc.

Steven P. Wolf: None to disclose.

Kris W. Herring: None to disclose.

Gregory P. Samsa: None to disclose.

Jesse D. Troy:

Honorarium for serving on DSMB for Ghbamida Cell.

Consulting services to The EMMES Corporation, the Community Data Roundtable, AegisCN, Cohortias International.

Thomas W. LeBlanc:

I am currently, or have recently been, a consultant for AstraZeneca, CareVive, Flatiron, Helsinn, Otsuka, Pfizer, and Seattle Genetics. I have served on recent advisory boards for AbbVie, Agios, Amgen, Daiichi-Sankyo, Heron, Medtronic, and Otsuka. I have received honoraria from Agios and Celgene, for non-branded speaking engagements. I have received recent research funding from the American Cancer Society, AstraZeneca, Duke University, Jazz Pharmaceuticals, the National Institute of Nursing Research / National Institutes of Health, and Seattle Genetics.

Data

We have full control of all primary data dn agree to allow the journal to review these data if requested.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rosett, H.A., Locke, S.C., Wolf, S.P. et al. An analysis of missing items in real-world electronic patient reported outcomes data: implications for clinical care. Support Care Cancer 28, 5099–5107 (2020). https://doi.org/10.1007/s00520-020-05338-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00520-020-05338-8

Keywords

Search

Navigation