Introduction

There has been growing interest in the assessment of patient-reported outcomes (PROs) over the last 40 years, and the use of PROs in clinical and health services research is common [1]. PROs have been defined as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else.” [2] The incorporation of PROs in clinical practice can serve numerous purposes [3] including: (1) describing a patient’s overall state; (2) screening for incipient disease and undetected disability [4, 5]; (3) monitoring disease progression and response to treatment; (4) assessing patient-centered needs; (5) formulating treatment plans consistent with patient preferences [6,7,8]; (6) improving physician-patient communication [9,10,11]; (7) providing patient-based data for quality initiatives [12,13,14]; and (8) standardizing interactions between healthcare providers and patients [3]. While the use of PROs in clinical practice can help in all of these areas [1, 15], critical questions remain about how patient outcome data should be collected, shared, and used to improve the quality of care and patient health outcomes [16]. Some reports have emerged regarding the use of PROs in routine clinical practice across different conditions [1, 17,18,19,20,21,22], but the incorporation of these tools in oncology clinical practice has been slower than adoption in research [23,24,25,26].

More recently, routine use of PROs in oncology practice has been identified as a priority area by the President’s Cancer Panel [27] as well as by national oncology societies such as the American Society of Clinical Oncology (ASCO) [28]. There is increasing interest in bringing “the patient’s perspective” to cancer decision making which is demonstrated by a number of key initiatives of PRO application in oncology research and regulatory decisions [29]. However, little evidence has been generated with regards to clinical-practice implementation [30].

The interest in implementation of PROs specifically in oncology care is exemplified by the number of recent reviews on PRO clinical applications and their impact on health outcomes [31,32,33,34]. All of the recent oncology reviews provide some insights on gaps in existing evidence of PRO use in clinical practice related to both challenges in implementation and PRO use impact. For example, Howell identified that more attention needs to be paid to complexity of implementation and interpretation [32]. King and colleagues [33] found a scarcity of studies reporting data on actions and medical decisions [33]. Two others—Chen [31] and Kotronoulas [34]—examined PRO intervention evidence and identified weak signals specific to changes in patient management and improved health outcomes [31, 34]. While these reviews identified important evidence gaps, none of them used an existing implementation framework to organize findings or focused on a review of publications reporting on the actual implementation of PRO in real-world settings beyond the context of a feasibility study or intervention trial. The current review makes a unique contribution to the field, by summarizing currently existing evidence using an implementation framework based on the user’s guide for the implementation of PROs in clinical practice recently developed by the International Society for Quality of Life Research (ISOQOL) [35]. The guide includes recommendations for the following implementation elements: (1) identifying the goals for collecting PROs in clinical practice and which key patient outcomes or barriers need attention; (2) considering group of patients and the care settings; (3) determining which questionnaire(s) to use (e.g., whether to use generic or disease-specific questionnaires, profile or preference-based measures, single or multi-item scales, and static or dynamic questionnaires); (4) choosing how often a patient should complete the questionnaires and whether it should be one-time completion or repeated, tied to clinic visits, or a way to monitor patients between visits; (5) deciding how the PRO will be administered and scored; (6) identifying interpretation benchmarks for the PRO score and how scores requiring follow-up will be determined; (7) developing strategies for when the PRO results will be presented and discussed with the patient (such as during or after the visit), how the results will be presented (e.g., numeric, graphical, one-time results or trends over time), and who will see the PRO score reports; (8) determining what will be done to respond to issues identified by the PROs and follow-up; and (9) evaluating the impact and value of the PRO interventions on the practice and patient [35]. While previous publications have discussed various considerations and potential applications of PROs in clinical practice [36,37,38], the ISOQOL PRO Implementation Guide is most recent and provides specific implementation guidance developed by subject matter experts and endorsed by a professional organization.

The objective of this systematic literature review (SLR) was to explore and summarize the existing evidence of PRO use in oncology clinical practice. We address two key aims in the review: (1) summarize available evidence of PRO use in clinical practice using a framework based on the ISOQOL PRO Implementation Guide [35]; (2) describe reports of real-world implementation of PRO measures with oncology patients. Real-world implementation of PROs can provide evidence regarding the usage and potential benefits of PRO adoption derived from real-world clinical settings. For the purposes of this review, real-world implementation studies were defined as those reporting the process of ongoing standardized PRO administration and related clinical actions to manage patient care conducted in a routine clinical practice beyond the scope of a specific research study.

Methods

Study design

We conducted a SLR in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [39, 40]. The protocol was developed following PRISMA guidelines and the Guidance on the Conduct of Narrative Synthesis in Systematic Reviews [41].

Data sources

The literature search was conducted in two databases: MEDLINE and Embase. Blocks of medical subject heading (MeSH) terms were used to identify the most relevant articles and conference papers that describe PRO implementation in oncology clinical practice.

Search strategy

The SLR search strategy was developed in consultation with a professional librarian and used a two-step approach for identifying studies. The first step included a search strategy and approach based on a previously published systematic review of use of PROs in oncology care [31]. As part of the first step, two additional, closely-related systematic reviews were identified [32, 33]. References from these oncology literature reviews were examined, and studies meeting the selection criteria were incorporated into the review [32, 33]. As time has elapsed since the publication of these reviews, they had a narrower focus and used different search strategies and search terms, a second search was conducted to replicate and update the earlier Howell [32] and King [33] reviews to further ensure a comprehensive review of recent publications from the end date of published reviews. MeSH terms and free-text keyword groups (e.g., “neoplasm,” “PRO measure,” “clinical practice,” and “treatment”) were used in different combinations. These updated searches also included specific PROs as search terms, minimizing the risk of missing relevant articles that used these measure, but may have resulted in overrepresentation of these specific measures in the final results. Terminology adjustments were made according to the requirements of each database. Both searches were supplemented by a hand search of references of relevant articles. Appendix A shows the full search strategy.

Selection criteria

Articles were included in the review if they met the following inclusion criteria: (1) cancer focus; (2) articles published in the past 10 years from 2006 to 2016 (inclusive) and abstracts from meetings held in 2015–2016 to ensure review of the current state of the field; (3) published in English; (4) title, abstract, or article contained information pertaining to the measurement of treatment satisfaction, process of care, treatment adherence, treatment decision-making, patient activation, PROs of health-related quality of life (HRQoL), symptoms, or function; and (5) the study design was a randomized controlled trial or an observational study in a clinical-practice setting or a report of PRO implementation in clinical practice.

Exclusion criteria were: (1) articles focused on non-cancer populations; (2) measures did not pertain to clinical outcomes or PROs associated with cancer treatment; (3) basic science studies (e.g., molecular biomarkers, neuroimaging drug formulation); (4) study designs not relevant including study protocols, case studies, case reports, case series, editorials, reviews, commentary, news, or study protocols; and (5) non-English language.

Data screening and abstraction

All abstracts were reviewed using DistillerSR® [42]—a systematic literature review reporting software—to assist with the organization, extraction, and categorization of all literature. Abstract and article screening was performed by three trained reviewers in a two-step process. During the Level 1 review, in order to standardize the review process, a calibration exercise was conducted by the reviewers for all abstracts and titles to assess eligibility for inclusion in the full-text review. Full-text articles that met the inclusion criteria were retrieved. If a determination of eligibility was not possible from the abstract, the full-text article was reviewed. During Level 2, full-text articles were reviewed again for eligibility. Disagreements on eligibility of screened publications at both levels were resolved through discussion with reviewers and final adjudication of unresolved disagreements by the first author of this paper (M.A). For eligible articles, the data was abstracted into a detailed source table that included data fields on study country, study type, cancer type, study objectives, sample size, study duration, study inclusion/exclusion criteria, PRO intervention characteristics, PRO reporting characteristics, study endpoints, assessment timepoints, PRO study results, and limitations/contextualization. The data abstracted into the detailed source table was validated by a second independent senior reviewer to ensure the accuracy of data abstraction. The detailed source table was used to organize information in summary tables that were developed during data analysis and based on the ISOQOL PRO Implementation Guide Framework.

Data analysis

We used the ISOQOL PRO Implementation Guide [35] as the basis for developing a framework to address our first research aim (Table 1). Categories corresponding to each of the ISOQOL PRO Implementation Guide recommendations were created, and information from the articles was extracted into summary tables from the original detailed literature source table. This was done to explore relationships in the data and to establish if all recommended information was included in reports of PRO use in oncology clinical practice. The framework was used to explore the use of PROs in clinical care and their relationship to outcomes in the context of the ISOQOL PRO Implementation Guide. To inform the second aim of this review—to describe reports of real-world implementation of PRO measures settings with oncology patients—we examined the characteristics of all real-world implementation reports.

Table 1 ISOQOL PRO Implementation Guide Framework for SLR Data Synthesis
Full size table

Results

As shown in the PRISMA diagram (Fig. 1), a total of 5754 records were identified, and they yielded 4427 unique publications (following removal of duplicates). A total of 117 full-text articles were retrieved for a second round of screening. Of these, 36 articles met the inclusion criteria and are presented herein (Table 2). The largest number of studies were from the United States (US) (n = 16, 44%) followed by the United Kingdom (UK) (n = 6, 17%), Germany (n = 3, 8%), Canada (n = 2, 6%), The Netherlands (n = 2, 6%), other individual European Union (EU) countries (n = 6, 17%), and Australia (n = 1, 3%).

Fig. 1
figure 1

PRISMA Flow Chart

Full size image
Table 2 Summary Table with Key Elements of Included Studies
Full size table

Results: Available evidence on reporting of ISOQOL PRO implementation guide categories

Data from all 36 reports of PRO measures used in clinical settings were summarized according to the ISOQOL PRO Implementation Guide Framework [35]. .Publications included research studies (intervention research (n = 19, 58%), feasibility research (n = 10, 28%), combination intervention and feasibility (n = 3, 8%), real-world implementation reports (n = 3, 8%); and an intervention for quality improvement (n = 1, 3%) We initially summarized findings related to design considerations of PRO integration in clinical practice within the specified ISOQOL categories (goals for collecting PROs, assessment details, PRO selection, and mode of administration) followed by evidence in categories related to reporting and use of PRO results (reporting of PRO results, PRO score interpretation, plans for addressing issues identified by the PRO, and evaluation of PRO impact on clinical practice). Table 2 presents a summary of key elements from each paper included in the review.

Goals for collecting PROs

As part of this synthesis step, we examined the stated clinical-practice goals for the PRO data collection. Planned use of PROs included monitoring symptoms (n = 13, 36%), improving quality of care (n = 10, 28%), enhancing patient-provider communication (n = 9, 25%), and delivering patient-centered care (n = 9, 25%) (Table 3).

Table 3 Reported Goals of PRO Inclusion in Clinical Care
Full size table

Patients, setting, and timing of assessments

All studies provided relevant details on the type of patients included, study setting, and timing of assessment (before visit, during visit, after visit, at home). Most studies were conducted with adult populations (94%) in outpatient settings (92%). PROs were administered most often at the clinic immediately before seeing the doctor (36%) or during a visit (33%). The type of cancer patient varied with a majority of studies including three or more cancer types (69%), and only seven studies (19%) with a single cancer type.

PROs selected for use in the studies

A total of 46 PRO measures were used across the 36 studies; 33 of the PRO measures were rarely used and were included in only one or two studies suggesting wide variability in measures used. Studies predominantly reported measuring symptoms (n = 15, 42%) or cancer-specific HRQoL (n = 13, 33%) outcomes. The most widely-used measure was the EORTC QLQ-C30 (n = 10,28%) followed by the Hamilton Anxiety and Depression Scale (HADS) (n = 5, 14%).

PRO mode of administration

All but one of the studies reported the mode of PRO administration. Electronic administration was the most common mode (n = 31, 69%) followed by paper-and-pencil (n = 12, 27%). Two studies used interactive voice response (IVR) (n = 2, 5%).

Reporting of PRO results

The summary of information on PRO results indicated that the preferred format of results presentation was an electronic summary report (n = 19, 56%) or a printed copy of PRO results (n = 7, 19%) while e-mails/telephones (n = 5, 14%) were used less often. Results were most often presented only to the clinical team (n = 30, 83%); only three studies (11%) presented the PRO results to both the clinician and the patient, and one study presented results to patients only.

PRO score interpretation

A large proportion of studies (n = 17, 47%) failed to report information on how to interpret PRO scores. Eleven studies (31%) provided PRO scores alone with no interpretation guidance, and six articles (17%) did not report any information on PRO score interpretation. Fifteen studies (42%) provided scores in the form of a graphical display which may aid in score interpretation. About half of the papers reviewed (n = 19, 53%) reported PRO scores along with some information on threshold values, cut-off scores, or severity levels. Only three studies (8%) provided reference groups or norms information for the selected PRO.

Plans for addressing issues identified by the PRO

The majority of studies did not provide any instructions on follow-up steps when PRO scores raised areas of concern. In 13 studies (36%), no instructions were given on next steps or patient-management action items based on PRO results; eight studies (22%) did not include sufficient information on whether PRO results were addressed. The plans for addressing issues identified by the PROs were often related to discussing the identified issues with the provider (n = 10, 28%) with single studies also suggesting specialist referrals, reporting adverse events (AEs), and/or providing educational materials to patients.

Evaluation of PRO impact on clinical practice

Only 19 studies (53%) included results of PRO intervention on patient outcomes. The outcomes for which most studies reported evidence of PRO intervention effect included patient reported symptoms, functioining or quality of life scores (n = 13) and patient-provider communication (n = 8) (Table 4). Eleven of the 19 studies (58%) reported significant PRO intervention effects for all reported endpoints, and five studies (26%) had mixed results and reported significant PRO intervention effects for some—but not all—of the assessed outcomes. Only three of the 19 studies (16%) reported no intervention effect (Table 2).

Table 4 Intervention Study Outcomes and Evidence of PRO Intervention Effect
Full size table

Results: Real-world implementation of PRO measures with oncology patients

Our review identified only three reports of real-world implementation of PRO measures in clinical practice which we defined as the ongoing administration of a standardized PRO and related clinical actions to manage patient care in routine clinical practice beyond the scope of a specific research study.

The first study [43] used retrospective chart review to investigate the relationship between standardized symptom screening and clinical actions to manage symptoms using the Edmonton Symptom Assessment Scale (ESAS) [44]. The ESAS was included in routine clinic visits though self-reporting via an electronic touch-screen kiosk. The ESAS measures the severity (scale of 0–10; 0 = none, 10 = worst) of nine common cancer physical and psychological symptoms (pain, shortness of breath, nausea, anxiety, depression, tiredness, drowsiness, appetite, and well-being). ESAS symptoms were categorized into four severity categories: none (0 score), mild (1–3 score), moderate (4–6 score), and severe (7–10 score) where scores of > 4 indicate clinically-significant symptom issues. Symptom-related actions included relevant drugs being prescribed, medication dosage titration, or a test, treatment, or referral being made. Pain and shortness of breath were documented in 52% and 30% of charts; a related action occurred in 17% and 4% of charts, respectively. However, the frequency of relevant clinical actions was not proportionate to the documented symptom severity [43].

Trautmann and colleagues [45] described the development, implementation, completeness, and first results of an electronic, real-time assessment program for the collection of PROs in a tertiary referral cancer center in Germany. The EORTC QLQ-C30 [46], National Comprehensive Cancer Network Distress Thermometer (DT) [47], and the Hornheider Screening Instrument (HIS) of need for psycho-oncological support [48] were measured. Nutritional status was assessed using the Short-Form Mini Nutritional Assessment (MNA) [49], and pain was assessed using the Brief Pain Inventory (BPI) [50, 51]. A traffic-light system was applied for visualized score interpretation using published cutoff values or means/standard deviations (SDs). A green light indicated scores below a critical clinical importance thresholds based on means and standard deviations of reference population of cancer patients thereby indicating no need for clinical action; a red light indicated scores above a critical cut-off that indicated need for further action. Overall, 67% of patients provided complete information on 12 PROs. Rates of approach and participation varied between the different departments with the highest completion rates in patients presenting for oncological surgical consultation. The number of patients approached to complete PROs increased from 17% to 56% over three months. The percentage of patients completing the PROs increased from 70% to 92% over three months. The majority of patients (62%) reported a score of five or higher on the NCCN Distress scale indicating moderate to high burden; 53% of the patients had a score of four or higher on the HSI indicating a need for psycho-oncological support. Very few participants reported on pain outcomes as EORTC QLQ-C30 pain intensity, and impairment scores were only documented in patients reporting moderate to severe pain. Findings revealed that physician usage of PRO during the clinical consultation was limited. Limiting factors reported by physicians were the lack of knowledge of the PRO reporting system and perceived irrelevancy of some of the assessed PRO data. Rates of clinical action were not reported in the study.

The authors acknowledged a number of obstacles in the study—even though there was an increased number of patients recruited, the usage of PROs in the patient-physician interaction was limited due to physician turnover and lack of completion time provided to patient prior to consultation. The authors concluded that PRO assessments should be more carefully selected to be more clearly of benefit to the health care provider and patient. Additionally, sustaining the implementation and interpretation of PROs should be constantly reinforced with clinicians [45].

Wagner and colleagues [52] assessed cancer-related symptoms with electronic health record (EHR) integration to communicate assessment results to clinical teams in real time. PROMIS computer adaptive tests (CATs) use a computer algorithm developed with item response theory to administer the items. The psychosocial assessment was adapted from the National Comprehensive Cancer Network Distress Thermometer and Problem Checklist [53]. Over the course of three years, 636 patients completed a total of 1493 assessments with 636 patients completing the assessment at least once (301 twice, 184 three times, and 129 four times). Most patients (90.1%) completed the assessment at home rather than at the clinic (9.3%). Severe PROMIS symptom scores (≥70 or 75 depending on symptom) triggered a message to the oncology team. PROMIS T-score clinical severity thresholds (normal, mild, moderate, or severe) have been previously determined with a standard setting exercise that converged clinician expert ratings and patient self-reported severity scores [54]. Overall, one-third of the patients reported current psychosocial health needs. The authors consider that this study demonstrates that precise measurement of symptoms can be implemented while maintaining the brevity required for clinical implementation. EEHR integration also facilitated automated triage for psychosocial and supportive care [52].

Discussion

Evidence on reporting of ISOQOL PRO implementation guide categories

In order to be able to follow the ISOQOL PRO Implementation Guide, researchers need to have a body of evidence to guide choices on recommended categories.The first aim of our review was to examine existing information on recommended implementation based on published information from oncology clinical settings. While no studies in our review directly referenced the ISOQOL PRO Implementation Guide [35], publications on PRO use in oncology clinical practices were well aligned in their reporting of most recommended implementation elements. However, a gap exists in the description of PRO interpretation guidelines and attendant patient-management recommendations necessary to improve PRO outcomes.

Most studies adequately described the planned goal for PRO data collection, study setting, and selection of PRO and mode of administration. Electronic administration, which allows for flexible integration of PROs in clinical care, was used by the majority of studies, but most were not formally integrated into the electronic health record. In addition, formal integration into the EHR may require intensive resources and stakeholder buy-in that have been lacking perhaps due to limited evidence of the improvement in patient outcomes or lack of financial alignment (or incentives).

While the EORTC QLQ-C30 was the most commonly-used measure across studies, a wide variety of PRO measures was reported suggesting there is little consensus on core domains or the “best” PRO to use or consideration whether a measure is developed for clinical trial or clinical practice use. Such variability may be an implementation barrier for PROs in everyday oncology practice—the burden to individual organizations associated with selecting PRO measures, developing assessment guidelines for the selected PROs, and interpreting PRO results may discourage adoption in routine clinical care.

The main gap in evidence identified by this review was the sparsity of interpretation guidelines for PRO results provided to care providers. While most of the reviewed studies provided PRO scores, fewer added interpretation guidelines to these scores or provided follow-up instructions or procedures in case a problem was identified by the PRO. This is an important gap. Without clarity on the meaning, significance, and interpretation of collected PRO data, how can clinical actions be effected to result in improved health care processes and outcomes?

Real-world implementation of PRO measures with oncology patients

Based on the review of the published literature, the use of PRO measures in routine cancer clinical practice outside the context of feasibility or research intervention studies is seldom reported. Only three reports of routine implementation of PROs in clinical settings were identified by the current review and provided limited information for our first key aim The multi-stage process that is required for developing, introducing, testing, integrating, and monitoring PROs in EHR systems has been achieved in only a few US medical centers [55]. Numerous barriers to implementation have been discussed in the literature including: (1) perception among clinicians that PRO completion consumes valuable time during the patient visit; (2) EHR systems have limited abiity to deliver PROs in user-friendly formats for patients; and (3) clinical ecosystem workflow demands challenge full implementation and integration into clinical practice [56].

While our review suggests that PRO implementation in real-world settings outside of research context is scarce, it is possible that PRO implementation in real-world oncology clinical practice may be underreported in the research literature. Implementation efforts may be viewed more as quality-improvement efforts that are building on existing evidence but not always viewed as generating evidence warranting dissemination [23]. Therefore, it is plausible that PRO implementation may be more widespread than indicated by existing peer-reviewed publications. In a recent article, Basch and colleagues [55] noted that a handful of institutions have successfully integrated systematic PRO collection into routine clinical practice; however, no published data have been generated from these institutions related to real-world PRO implementation in oncology clinical care.

Comparison to other systematic literature reviews

Some of our findings are consistent with the results from earlier literature reviews evaluating different aspects of PROs in the context of oncology care (Table 5). We confirmed earlier findings that there is evidence for the effectiveness of PROs on improving provider-patient communication and increased discussion of mental health issues [23, 31,32,33,34]. The EORTC QLQ-C30 was also found to be the most commonly-evaluated PRO in oncology clinical practice settings [32]. The wide variability of PRO measures used has also been noted [55] and continues to be a challenge as confirmed by our findings. Several of the earlier reviews also pointed out the need for increased attention in providing guidance for PRO implementation in oncology clinical practice [33, 34, 57]. Since the completion of our review window, several additional reviews have appeared that focus on some aspect of PRO use in oncology clinical care such as mechanisms through which PROs facilitate increase in patient-physician communication [58] and use of PROs specifically in treating lung cancer [59]. The unique contributions of the current review remains, as no other review focused on separately examining PRO real-world implementation reports or used the ISOQOL PRO Implementation Guide as the framework in analyzing the identified articles. The use of this framework has allowed us to identify specific gaps in the PRO implementation cycle that need to be addressed to encourage use in clinical practice―mainly, the insufficient focus on developing and providing clear PRO score interpretation guidelines and patient-management action plans related to PRO results.

Table 5 Summary of Earlier Relevant SLR of Implementation of PRO in Cancer Clinical Care
Full size table

Strengths and limitations

The strengths of this review include the compliance with PRISMA guidelines, development of a comprehensive two-step search strategy, and review of results in the context of a framework built on the ISOQOL PRO Implementation Guide [35]. The results of the review help further the conversation on PRO implementation in oncology clinical practice by identifying gaps in guidance on interpretation of PRO results and action-oriented patient management based on PRO results. The review also has some limitations including the relatively small number of databases included in the review.

Conclusion

The existing evidence of PRO implementation in real-world clinical care in the published literature is very limited. It is unclear whether implementation efforts are not being studied, not being reported in peer-reviewed journals, simply being published in the grey literature, or not taking place at all.

While publication on PRO real-world implementation is uncommon, a good number of publications on PRO feasibility and/or PRO use in research in oncology clinical care exists as evidenced by our work and earlier literature reviews. This paper also aimed to organize findings of published studies in a framework informed by the ISOQOL PRO Implementation Guide [35]. Results suggested that, with the notable exception of PRO score interpretation and action strategies for PRO-identified problems, most studies report information suggested by the ISOQOL PRO Implementation Guide.

Based on the findings from our review, we offer two insights to help enable more widespread PRO implementation in routine clinical practice. First, adequate interpretation guidelines are needed for PRO results to be acted upon in clinical practice. Second, exploration should be conducted into how to best address issues raised by PRO results—particularly when the identified needs of patients extend beyond the expertise or training found in a routine oncology clinical practice such as depression or lack of social support. In the absence of available information on these key elements, implementation of PROs in clinical practice is unlikely to bridge the gap between perceived usefulness by researchers and routine uptake in oncology practice by clinicians.