Skip to main content

Advertisement

SpringerLink
Log in

Improving an electronic system for measuring PROs in routine oncology practice

  • Published:
Journal of Cancer Survivorship Aims and scope Submit manuscript

Abstract

Purpose

Our aim was to study how patients and their clinicians evaluated the usability of PatientViewpoint, a webtool designed to allow patient-reported outcome (PRO) measures to be used in clinical practice.

Methods

As part of a two-round quality improvement study, breast and prostate cancer patients and their medical and radiation oncology clinicians completed semi-structured interviews about their use of PatientViewpoint. The patient interview addressed different phases of the PRO completion workflow: reminders, completing the survey, and viewing the results. The clinician interviews asked about use of PatientViewpoint, integration with the clinical workflow, barriers to use, and helpful and desired features. Responses were recorded, categorized, and reviewed. After both rounds of interviews, modifications were made to PatientViewpoint.

Results

Across the two rounds, 42 unique patients (n = 19 in round 1, n = 23 in round 2) and 12 clinicians (all in both rounds) completed interviews. For patients, median age was 65, 81 % were white, 69 % were college graduates, 80 % had performance status of 0, 69 % had loco-regional disease, and 81 % were regular computer users. In the quality improvement interviews, patients identified numerous strengths of the system, including its ability to flag issues for discussion with their provider. Comments included confusion about how scores were presented and that the value of the system was diminished if the doctor did not look at the results. Requests included tailoring questions to be applicable to the individual and providing more explanation about the score meaning, including having higher scores consistently indicating either better or worse status. Clinicians also provided primarily positive feedback about the system, finding it helpful in some cases, and confirmatory in others. Their primary concern was with impact on their workflow.

Conclusions

Systematically collected feedback from patients and clinicians was useful to identify ways to improve a system to incorporate PRO measures into oncology practice. The findings and evaluation methods should be useful to others in efforts to integrate PRO assessments into ambulatory care.

Implications for Cancer Survivors

Systems to routinely collect patient-reported information can be incorporated into oncology practices and provide useful information that promote patient and clinician partnership to improve the quality of care.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Wu AW, Jensen RE, Salzberg C, Snyder C. Advances in the use of patient reported outcome measures in electronic health records. Including case studies. 2013. PCORI national workshop to advance the use of PRO measures in electronic health records.

  2. Snyder CF, Aaronson NK. Use of patient-reported outcomes in clinical practice. Lancet. 2009;374(9687):369–70.

    Article  PubMed  Google Scholar 

  3. Wu AW, Bradford AN, Velanovich V, Sprangers MA, Brundage M, Snyder C. Clinician’s checklist for reading and using an article about patient-reported outcomes. Mayo Clin Proc. 2014;89(5):653–61.

    Article  PubMed  Google Scholar 

  4. Snyder CF, Aaronson NK, Choucair AK, Elliott TE, Greenhalgh J, Halyard MY, et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res. 2012;21(8):1305–14.

    Article  PubMed  Google Scholar 

  5. Acquadro C, Berzon R, Dubois D, Leidy NK, Marquis P, Revicki D, et al. Incorporating the patient’s perspective into drug development and communication: an ad hoc task force report of the patient-reported outcomes (PRO) harmonization group meeting at the Food and Drug Administration, February 16, 2001. Value Health. 2003;6(5):522–31.

    Article  PubMed  Google Scholar 

  6. Snyder CF, Wu AW, Miller RS, Jensen RE, Bantug ET, Wolff AC. The role of informatics in promoting patient-centered care. Cancer J. 2011;17(4):211–8.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Basch E, Abernethy AP. Supporting clinical practice decisions with real-time patient-reported outcomes. J Clin Oncol. 2011;29:954–6.

    Article  PubMed  Google Scholar 

  8. Wu AW, Kharrazi H, Boulware LE, Snyder CF. Measure once, cut twice—adding patient-reported outcome measures to the electronic health record for comparative effectiveness research. J Clin Epidemiol. 2013;66(8 Suppl):S12–20.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Detmar SB, Aaronson NK. Quality of life assessment in daily clinical oncology practice: a feasibility study. Eur J Cancer. 1998;34:1181–6.

    Article  CAS  PubMed  Google Scholar 

  10. Detmar SB, Muller MJ, Schornagel JH, et al. Health-related quality-of-life assessments and patient-physician communications. A randomized clinical trial. JAMA. 2002;288:3027–34.

    Article  PubMed  Google Scholar 

  11. Velikova G, Booth L, Smith AB, Brown PM, Lynch P, Brown JM, et al. Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol. 2004;22:714–24.

    Article  PubMed  Google Scholar 

  12. Rose M, Bezjak A. Logistics of collecting patient-reported outcomes (PROs) in clinical practice: an overview and practical examples. Qual Life Res. 2009;18:125–36.

    Article  PubMed  Google Scholar 

  13. Aaronson N, Elliott T, Greenhalgh J, Halyard M, Hess R, Miller D, Reeve B, Santana M, Snyder C. User’s guide to implementing patient-reported outcomes assessment in clinical practice. Version 2:2015; ISOQOL. http://www.isoqol.org/UserFiles/2015UsersGuide-Version2.pdf. Accessed 16 Feb 2015.

  14. Jones JB, Snyder CF, Wu AW. Issues in the design of Internet-based systems for collecting patient-reported outcomes. Qual Life Res. 2007;16:1407–17.

    Article  PubMed  Google Scholar 

  15. Jensen RE, Snyder CF, Abernethy AP, Basch E, Potosky AL, Roberts AC, et al. Review of electronic patient-reported outcomes systems used in cancer clinical care. J Oncol Pract. 2014;10(4):e215–22.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Snyder CF, Jensen R, Courtin SO, et al. PatientViewpoint: a website for patient-reported outcomes assessment. Qual Life Res. 2009;18:793–800.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Web-based system helps doctors, patients communicate. https://www.youtube.com/watch?v=S-r4ykaUhfU. Accessed 16 Feb 2015.

  18. Snyder CF, Blackford AL, Wolff AC, Carducci MA, Herman JM, Wu AW, et al. Feasibility and value of PatientViewpoint: a web system for patient-reported outcomes assessment in clinical practice. Psychooncology. 2013;22(4):895–901.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Snyder CF, Herman JM, White SM, Luber BS, Blackford AL, Carducci MA, et al. When using patient-reported outcomes in clinical practice, the measure matters: a randomized controlled trial. J Oncol Pract. 2014;10(5):e299–306.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Langley GJ, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The improvement guide: a practical approach to enhancing organizational performance, 2nd Edition. 2009. Hoboken: Jossey-Bass. ISBN 978-0-470-19241-2.

  21. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85(5):365–76.

    Article  CAS  PubMed  Google Scholar 

  22. Sanson-Fisher R, Girgis A, Boyes A, et al. The unmet supportive care needs of patients with cancer. Cancer. 2000;88:226–37.

    Article  CAS  PubMed  Google Scholar 

  23. Cella D, Gershon R, Lai JS, et al. The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment. Qual Life Res. 2007;16 Suppl 1:133–41.

    Article  PubMed  Google Scholar 

  24. WA Shewhart. Statistical method from the viewpoint of quality control. 1939. New York: Dover. ISBN 0-486-65232-7.

  25. Cleghorn GD, Headrick LA. The PDSA cycle at the core of learning in health professions education. Jt Comm J Qual Improv. 1996;22(3):206–12.

    CAS  PubMed  Google Scholar 

  26. Jensen RE, Rothrock NE, DeWitt EM, Spiegel B, Tucker CA, Crane HM, et al. The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care. Med Care. 2015;53(2):153–9.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Varni JW, Magnus B, Stucky BD, Liu Y, Quinn H, Thissen D, et al. Psychometric properties of the PROMIS ® pediatric scales: precision, stability, and comparison of different scoring and administration options. Qual Life Res. 2014;23(4):1233–43.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–13.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Wagner LI, Schink J, Bass M, Patel S, Diaz MV, Rothrock N, et al. Bringing PROMIS to practice: brief and precise symptom screening in ambulatory cancer care. Cancer. 2015;121(6):927–34.

    Article  PubMed  PubMed Central  Google Scholar 

  30. eClinicalWorks improving healthcare together www.eClinicalWorks.com Accessed 1 May 2015.

  31. NEXTGEN® Healthcare www.nextgen.com Accessed 1 May 2015.

  32. Nazi KM, Hogan TP, Wagner TH, et al. Embracing a health services research perspective on personal health records: lessons learned from the VA My HealtheVet system. J Gen Intern Med. 2010;251 Suppl 1:62–7.

    Article  Google Scholar 

  33. Goldzweig CL, Towfigh AA, Paige NM, Orshansky G, Haggstrom DA, Beroes JM, Miake-Lye IM, Shekelle PG. Systematic review: secure messaging between providers and patients, and patients’ access to their own medical record. Evidence on health outcomes, satisfaction, efficiency and attitudes. VA-ESP Project #05-226, 2012. http://www.hsrd.research.va.gov/publications/esp/myhealthevet.pdf Accessed 15 May 2015.

  34. Kruse CS, Bolton K, Freriks G. The effect of patient portals on quality outcomes and its implications to meaningful use: a systematic review. J Med Internet Res. 2015;17(2):e44.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Brundage MD, Smith K, Little EA, Bantug ET, Snyder CF, PRO Data Presentation Stakeholder Advisory Board. Communicating patient-reported outcome scores using graphic formats: results from a mixed-methods evaluation. Qual Life Res. 2015;24:2457–72.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The PatientViewpoint Scientific Advisory Board includes Neil Aaronson, PhD (Netherlands Cancer Institute, Amsterdam, Netherlands); Michael Brundage, MD, MSc (Queen’s University, Kingston, ON, Canada); Carolyn Gotay, PhD (University of British Columbia, Vancouver, BC, Canada); Michele Halyard, MD (Mayo Clinic, Scottsdale, AZ, USA); Denise Hynes, RN, MPH, PhD (Edward Hines, Jr. VA Hospital, Hines, IL, USA, and University of Illinois, Chicago, IL, USA); J.B. Jones, MBA, PhD (Geisinger Health System, Danville, PA, USA); Susan Yount (Northwestern University, Chicago, IL, USA); and Galina Velikova, BMBS(MD), PhD (St James’s Institute of Oncology, Leeds, UK). We also want to thank Michelle Campbell, Ray Hamann, and colleagues for their technical development of PatientViewpoint. Finally, we are most grateful to the patients and clinicians who participated in this study.

Author information

Authors and Affiliations

  1. Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA

    Albert W. Wu & Claire F. Snyder

  2. Division of General Internal Medicine, Johns Hopkins School of Medicine, Baltimore, MD, USA

    Albert W. Wu & Claire F. Snyder

  3. Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins, Baltimore, MD, USA

    Sharon M. White, Amanda L. Blackford, Antonio C. Wolff, Michael A. Carducci, Joseph M. Herman & Claire F. Snyder

  4. Center for Health Services & Outcomes Research, 624 N. Broadway, Room 653, Baltimore, MD, 21205, USA

    Albert W. Wu

Authors
  1. Albert W. Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sharon M. White
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amanda L. Blackford
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Antonio C. Wolff
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael A. Carducci
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Joseph M. Herman
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Claire F. Snyder
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

PatientViewpoint Scientific Advisory Board

Corresponding author

Correspondence to Albert W. Wu.

Ethics declarations

Financial support

PatientViewpoint’s development was supported by the National Cancer Institute (1R21CA134805-01A1; 1R21CA113223-01). The American Cancer Society provided the support for the randomized controlled trial (MRSG-08-011-01-CPPB). The funding sources had no role in study design, data collection, analysis, interpretation, writing, or decision to submit the manuscript for publication.

Conflict of interest

The authors have no other relevant conflicts of interest.

Appendix

Appendix

Continuous quality improvement (QI) interview script (PATIENTS)

We would like for you to provide some feedback about how the PatientViewpoint system works for you and could be improved. The information that you give us today will contribute to the ongoing improvement of the PatientViewpoint website. Please provide your honest assessment.

Can you show me how you go through the process of completing the survey using the PatientViewpoint website.

  1. I.

    Email reminders

    Have you received the email reminders?

    Are the email reminders useful? Do you use the reminders to connect to the survey?

    Is there anything in particular that you like about the reminders?

    Is there anything that you think could be better with the email reminders?

  1. II.

    Completing the survey

    Show me how you log onto the PatientViewpoint website and complete the survey.

    What aspects of the process do you find helpful?

    What problems have you encountered when completing the process?

  1. III.

    Viewing the results

    What do you like about the presentation of the results?

    Is there anything in the results section that is hard to understand?

    What do you think could be done to improve the presentation of the survey results?

  1. IV.

    Other suggestions and comments

Continuous quality improvement (QI) interview script (CLINICIANS)

We are interested in your feedback about how the PatientViewpoint system works and how it might be improved. The information you give to us today will contribute to the ongoing improvement of our web system. Please provide your honest assessment.

  1. 1.

    Have you used PVP in a patient encounter in the past month?

  2. 2.

    Tell me about a patient encounter where you used PatientViewpoint:

  3. 3.

    Tell me about a patient encounter where you had the opportunity to use PatientViewpoint but DID NOT:

  4. 4.

    Tell me about your most recent visit with your patient Mr. /Ms. (insert name of most recently seen patient enrolled in the study). What happened? What was done?

  5. 5.

    Tell me how the PatientViewpoint web system fits in with what you do in your clinic. When do you view the survey results for your patients? What is your preferred way to view the results (website, EPR or printout)?

Now I would like for you to walk me through how you use the website and the information it provides during the patient encounter.

Notes and observations

Which aspects of the system do you find informative/helpful?

What barriers do you perceive in using the system?

Are there any other features that you would like PatientViewpoint to have?

Do you have any other comments?

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, A.W., White, S.M., Blackford, A.L. et al. Improving an electronic system for measuring PROs in routine oncology practice. J Cancer Surviv 10, 573–582 (2016). https://doi.org/10.1007/s11764-015-0503-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11764-015-0503-6

Keywords

Search

Navigation