Skip to main content

Do reminder emails and past due notifications improve patient completion and institutional data submission for patient-reported outcome measures?

Quality of Life Research volume 30, pages 81–89 (2021)Cite this article

Abstract

Purpose

NRG Oncology, part of the National Cancer Institute’s National Clinical Trials Network, took efforts to increase patient-reported outcome measures (PROMs) completion and institutional data submission rates within clinical trials. Lack of completion diminishes power to draw conclusions and can be a waste of resources. It is hypothesized that trials with automatic email reminders and past due notifications will have PROM forms submitted more timely with higher patient completion.

Methods

Automatic emails sent to the research associate were added to selected NRG Oncology trials. Comparisons between trials with and without automatic emails were analyzed using Chi-square tests with respect to patient completion and timeliness of form submission rates. Multivariable analyses were conducted using repeated measures generalized estimating equations. If PROMs were not completed, a form providing the reason why was submitted and counted towards form submission.

Results

For both disease sites, form submission was significantly higher within 1 month of the form’s due date for the studies with automatic emails vs. those without (prostate: 79.7% vs. 75.7%, p < 0.001; breast: 59.2% vs. 31.3%, p < 0.001). No significant differences in patient completion were observed between the breast trials. The prostate trial with automatic emails had significantly higher patient completion but this result was not confirmed in the multivariable analysis.

Conclusions

Although patient completion rates were higher on trials with automatic emails, there may be confounding factors requiring future study. The automatic emails appeared to have increased the timeliness of form submission, thus supporting their continued use on NRG Oncology trials.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

References

  1. Yang, L. Y., Manhas, D. S., Howard, A. F., & Olson, R. A. (2018). Patient-reported outcome use in oncology: A systematic review of the impact on patient-clinician communication. Supportive Care in Cancer, 26(1), 41–60.

    CAS  Article  Google Scholar 

  2. Stover, A. M., Tompkins Stricker, C., Hammelef, K., et al. (2019). Using stakeholder engagement to overcome barriers to implementing patient-reported outcomes (PROs) in cancer care delivery: Approaches from 3 prospective studies. Medical Care, 57(Suppl 5 Suppl 1), S92–S99.

    Article  Google Scholar 

  3. Snyder, C. F., Aaronson, N. K., Choucair, A. K., et al. (2012). Implementing patient-reported outcomes assessment in clinical practice: A review of the options and considerations. Quality of Life Research, 21(8), 1305–1314.

    Article  Google Scholar 

  4. Atherton, P. J., Burger, K. N., Pederson, L. D., Kaggal, S., & Sloan, J. A. (2016). Patient reported outcomes questionnaire compliance in cancer cooperative group trials (Alliance N0992). Clinical Trials, 13(6), 612–620.

    Article  Google Scholar 

  5. Gilbert, M. R., Dignam, J. J., Armstrong, T. S., Wefel, J. S., Blumenthal, D. T., Vogelbaum, M. A., et al. (2014). Randomized clinical trial of bevacizumab for newly diagnosed glioblastoma. New England Journal of Medicine, 370(8), 699–708.

    CAS  Article  Google Scholar 

  6. Klopp, A. H., Yeung, A. R., Deshmukh, S., Gil, K. M., Wenzel, L., Westin, S. N., et al. (2018). Patient-reported toxicity during pelvic intensity-modulated radiation therapy: NRG oncology-RTOG 1203. JCO, 36(24), 2538–2544.

    CAS  Article  Google Scholar 

  7. Laack, N., Pugh, S. L., Brown, P., Fox, S., Wefel, J., Meyers, C., et al. (2019). Quality of life in patients receiving memantine for the prevention of cognitive dysfunction during whole-brain radiotherapy (WBRT): A randomized, double-blind, placebo-controlled trial. Neuro-Oncology Practice, 6(4), 274–282.

    Article  Google Scholar 

  8. Lukka, H., Pugh, S. L., Bruner, D. W., Bahary, J. P., Lawton, C., Efstathiou, J., et al. (2018). Patient reported outcomes in NRG Oncology RTOG 0938, evaluating two ultrahypofractionated regimens for prostate cancer. International Journal of Radiation Oncology*Biology*Physics, 102(2), 287–295.

    Article  Google Scholar 

  9. Movsas, B., Hu, C., Sloan, J., et al. (2016). Quality of life analysis of a radiation dose-escalation study of patients with non-small-cell lung cancer: A secondary analysis of the Radiation Therapy Oncology Group 0617 randomized clinical trial. JAMA Oncology, 2(3), 359–367.

    Article  Google Scholar 

  10. Wyatt, G., Pugh, S. L., Wong, R. K. W., Sagar, S. M., Lele, S. B., Koyfman, S. A., et al. (2016). Xerostomia health-related quality of life: NRG oncology/RTOG 0537. Quality of Life Research, 25(9), 2323–2333.

    Article  Google Scholar 

  11. Renovanz, M., Hechtner, M., Kohlmann, K., Janko, M., Nadji-Ohl, M., Singer, S., et al. (2018). Compliance with patient-reported outcome assessment in glioma patients: Predictors for drop out. NOP, 5(2), 129–138.

    Article  Google Scholar 

  12. Bae, K., Bruner, D. W., Baek, S., Movsas, B., Corn, B. W., & Dignam, J. J. (2011). Patterns of missing mini mental status exam (MMSE) in radiation therapy oncology group (RTOG) brain cancer trials. Journal of Neuro-Oncology, 105(2), 383–395.

    CAS  Article  Google Scholar 

  13. Sloan, J. A., Berk, L., Roscoe, J., et al. (2007). Integrating patient-reported outcomes into cancer symptom management clinical trials supported by the National Cancer Institute-sponsored clinical trials networks. Journal of Clinical Oncology, 25(32), 5070–5077.

    Article  Google Scholar 

  14. Gamper, E. M., Nerich, V., Sztankay, M., et al. (2017). Evaluation of noncompletion bias and long-term adherence in a 10-year patient-reported outcome monitoring program in clinical routine. Value Health, 20(4), 610–617.

    Article  Google Scholar 

  15. Turner-Bowker, D. M., Hao, Y., Foley, C., Galipeau, N., Mazar, I., Krohe, M., et al. (2016). The use of patient-reported outcomes in advanced breast cancer clinical trials: A review of the published literature. Current Medical Research and Opinion, 32(10), 1709–1717.

    Article  Google Scholar 

  16. Roydhouse, J. K., Gutman, R., Bhatnagar, V., Kluetz, P. G., Sridhara, R., & Mishra-Kalyani, P. S. (2019). Analyzing patient-reported outcome data when completion differs between arms in open-label trials: An application of principal stratification. Pharmacoepidemiology and Drug Safety, 28(10), 1386–1394.

    Article  Google Scholar 

  17. Brown, P. D., Pugh, S. L., Laack, N. N., Wefel, J. S., Khuntia, D., Meyers, C., et al. (2013). Memantine for the prevention of cognitive dysfunction in patients receiving whole-brain radiotherapy: A randomized, double-blind, placebo-controlled trial. Neuro-Oncology, 15(10), 1429–1437.

    CAS  Article  Google Scholar 

  18. Basch, E. (2019). High compliance rates with patient-reported outcomes in oncology trials submitted to the US Food and Drug Administration. Journal of the National Cancer Institute, 111(5), 437–439.

    Article  Google Scholar 

  19. Chafe, S., Moughan, J., McCormick, B., et al. (2013). Late toxicity and patient self-assessment of breast appearance/satisfaction on RTOG 0319: A phase 2 trial of 3-dimensional conformal radiation therapy-accelerated partial breast irradiation following lumpectomy for stages I and II breast cancer. International Journal of Radiation Oncology Biology Physics, 86(5), 854–859.

    Article  Google Scholar 

  20. Buysse, D. J., et al. (1989). The Pittsburgh Sleep Quality Index: A new instrument for psychiatric practice and research. Psychiatry Research, 28(2), 193–213.

    CAS  Article  Google Scholar 

  21. Godin, G., Jobin, J., & Bouillon, J. (1986). Assessment of leisure time exercise behavior by self-report: A concurrent validity study. Canadian Journal of Public Health Revue Canadienne de Sante Publique, 77(5), 359–362.

    CAS  PubMed  Google Scholar 

  22. Gionet, N. J., & Godin, G. (1989). Self-reported exercise behavior of employees: A validity study. Journal of Occupational Medicine, 31(12), 969–973.

    CAS  Article  Google Scholar 

  23. Ameringer, S., Elswick, R. K., Jr., Menzies, V., et al. (2016). Psychometric evaluation of the patient-reported outcomes measurement information system fatigue-short form across diverse populations. Nursing Research, 65(4), 279–289.

    Article  Google Scholar 

  24. EuroQol Group. (1990). EuroQol–a new facility for the measurement of health-related quality of life. Health Policy, 16(3), 199–208.

    Article  Google Scholar 

  25. Land, S. R., Ritter, M. W., Costantino, J. P., et al. (2007). Compliance with patient-reported outcomes in multicenter clinical trials: Methodologic and practical approaches. Journal of Clinical Oncology, 25(32), 5113–5120.

    Article  Google Scholar 

  26. Mercieca-Bebber, R., Friedlander, M., Calvert, M., et al. (2017). A systematic evaluation of compliance and reporting of patient-reported outcome endpoints in ovarian cancer randomised controlled trials: Implications for generalisability and clinical practice. Journal of Patient-Reported Outcomes, 1(1), 5.

    Article  Google Scholar 

  27. Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New York, NY: Wiley.

    Book  Google Scholar 

  28. Fairclough, D. (2010). Design and analysis of quality of life studies in clinical trials (2nd ed.). Boca Raton, FL: Chapman & Hall/CRC.

    Book  Google Scholar 

  29. Gwaltney, C. J., Shields, A. L., & Shiffman, S. (2008). Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: A meta-analytic review. Value Health, 11(2), 322–333.

    Article  Google Scholar 

  30. Brown, P. D., Gondi, V., Pugh, S. L., Tome, W. A., Wefel, J. S., Armstrong, T. S., et al. (2020). Hippocampal avoidance during whole-brain radiotherapy plus memantine for patients with brain metastases: Phase III trial NRG oncology CC001. Journal of Clinical Oncology, 38(10), 1019–1029.

    Article  Google Scholar 

  31. Michalski, J. M., Moughan, J., Purdy, J., Bosch, W., Bruner, D. W., Bahary, J. P., et al. (2018). Effect of standard vs dose-escalated radiation therapy for patients with intermediate-risk prostate cancer: The NRG oncology RTOG 0126 randomized clinical trial. JAMA Oncology, 4(6), e180039.

    Article  Google Scholar 

  32. Rabinovitch, R., Moughan, J., Vicini, F., Pass, H., Wong, J., Chafe, S., et al. (2016). Long-term update of NRG oncology RTOG 0319: A phase 1 and 2 trial to evaluate 3-dimensional conformal radiation therapy confined to the region of the lumpectomy cavity for stage I and II breast carcinoma. International Journal of Radiation Oncology*Biology*Physics, 96(5), 1054–1059.

    Article  Google Scholar 

  33. Brown, P. D., Decker, P. A., Rummans, T. A., Clark, M. M., Frost, M. H., Ballman, K. V., et al. (2008). A prospective study of quality of life in adults with newly diagnosed high-grade gliomas: Comparison of patient and caregiver ratings of quality of life. American Journal of Clinical Oncology, 31(2), 163–168.

    Article  Google Scholar 

Download references

Funding

This project was support by grants U10CA180868 (NRG Oncology Operations), U10CA180822 (NRG Oncology SDMC), UG1CA189867 (NCORP) from the National Cancer Institute (NCI) and CURE from the state of Pennsylvania. The Department specifically disclaims responsiblity for any analyses, interpretations, or conclusions.

Author information

Affiliations

  1. NRG Oncology Statistics and Data Management Center, American College of Radiology, Philadelphia, PA, USA

    Stephanie L. Pugh, Joseph P. Rodgers, Jennifer Moughan & Roseann Bonanni

  2. NRG Oncology Operations Center, American College of Radiology, Philadelphia, PA, USA

    Jaskaran Boparai

  3. University of Kansas, Kansas City, KS, USA

    Ronald C. Chen

  4. NRG Oncology Statistics and Data Management Center, University of Chicago, Chicago, IL, USA

    James J. Dignam

  5. Emory University Hospital/Winship Cancer Institute, Atlanta, GA, USA

    Deborah W. Bruner

Authors
  1. Stephanie L. Pugh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joseph P. Rodgers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jennifer Moughan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roseann Bonanni
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jaskaran Boparai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ronald C. Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. James J. Dignam
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Deborah W. Bruner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephanie L. Pugh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 14 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pugh, S.L., Rodgers, J.P., Moughan, J. et al. Do reminder emails and past due notifications improve patient completion and institutional data submission for patient-reported outcome measures?. Qual Life Res 30, 81–89 (2021). https://doi.org/10.1007/s11136-020-02613-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-020-02613-3

Keywords

  • Quality of life
  • Neurocognitive testing
  • Patient-reported outcomes
  • Patient compliance
  • Delinquent data
  • Form submission