Introduction

There is an increasing emphasis on improving the quality of cancer treatment side effects and toxicity management, alongside the need to collect real-world pharmacovigilance data to improve health outcomes and quality of life [1, 2]. This can be achieved through systematically collecting and evaluating chemotherapy-related adverse events from a patient-centered perspective [3,4,5]. Unlike healthcare utilization outcomes (e.g., length of hospital stays, morbidity/mortality) [6, 7], subjective and patient-reported toxicities may not be captured in electronic health records. Thus, routine prospective patient-reported adverse event data (outside the context of a clinical trial) is appealing.

Chemotherapy related symptomatic side effects are prevalent in a substantial proportion of patients receiving treatment, with the most prevalent being fatigue (59–91%), decreased appetite (42–62%), nausea (28–60%), vomiting (17–26%), and oral mucositis (17–40%) [4, 8,9,10,11]. Distressing symptoms during treatment interfere with daily activities and may adversely affect quality of life [12,13,14,15]. Moreover, objective Common Terminology Criteria for Adverse Events (CTCAE) toxicity reporting by clinicians may not always align with patient reports [3, 4, 16,17,18,19,20]. Thus, data capture using patient-reported outcomes (PRO) is becoming a standard of care in many cancer organizations; however, generic PRO measures such as the Edmonton Symptom Assessment System may lack precision in detecting treatment-specific toxicities [21]. The US National Cancer Institute’s PRO version of the CTCAE is a library of 124 self-report items representing 78 symptomatic toxicities [22, 23]. For each symptom, up to three items capture the attributes of frequency, severity, and interference with daily activities. The standard recall period is for the past 7 days. As an adjunct to traditional survival outcomes, inclusion of PROs in routine clinical practice are posited to improve patient’s health-related quality of life (HRQOL) through enhanced communication and feedback to clinicians for therapeutic side effect management, though such positive effects have not been consistently observed [24, 25]. PRO-CTCAE was developed for use in cancer clinical trials; however, interest in using PRO-CTCAE in routine clinical cancer care has not been previously examined.

Traditionally, treatment toxicities elicited by healthcare providers tend to grade symptom severity lower, compared to patient self-assessments [16,17,18,19,20, 26, 27]. Use of PRO-CTCAE may result in more precise and reliable data about symptomatic adverse events. Electronic PRO data collected through tablets, mobile phones, or web-based applications can be used to auto-populate clinical notes, trigger notifications to healthcare providers when symptoms are clinically significant, and allow for the systematic collection of data for quality monitoring and pharmacoepidemiologic research (including pharmacovigilance). However, patient acceptability and feasibility remain important questions when introducing new methods of routine data collection, especially the collection of data designed to be used dually for clinical management of patients and as a performance metric for improvement in health outcomes and care quality [28, 29].

A recent systematic review suggested that well-implemented PRO tools, particularly when there is systematic attention to provider-feedback and alerts for severe symptoms, may improve patient–provider communication and patient satisfaction, though salutary effects of PROs on patient-centered and health service outcomes are not observed consistently in randomized trials [30]. There is also growing evidence that PRO tools improve the monitoring of treatment response and the detection of unrecognized problems [31, 32]. However, there is a gap in our current knowledge with respect to the effects on care processes and health outcomes of routine collection of PRO-CTCAE as part of cancer care delivery [25, 33,34,35].

Systematic collection of chemotherapy toxicities can either come from the healthcare provider or directly from the patient. Since healthcare providers have increasing time pressures, the likelihood of success in introducing additional training to conduct directed toxicity interviews seems low [36]. In contrast, PROs in a patient-centered setting could systematically capture and improve documentation of chemotoxicities, potentially streamlining aspects of the clinical encounter; however, there are also concerns that improved collection of toxicity burden can lead to increased burden of managing these toxicities. Nonetheless, a fundamental question is whether PRO tools such as the PRO-CTCAE are even needed or acceptable to patients. Perhaps historical methods (dictations, written progress notes, and review of systems checklists) used by healthcare providers already provide rigorous symptomatic toxicity data.

Our aims for this study are (1) to examine the adequateness of toxicity documentation by historical control methods versus when PRO-CTCAE is used; (2) to determine whether patients receiving anticancer treatment are satisfied with the current management of their symptomatic toxicities, given the healthcare provider-centered data collection processes currently in use; (3) to compare the adequateness of toxicity documentation when using retrospective electronic health record for data collection, as compared to the systematic and prospective application of the PRO-CTCAE tool; (4) to examine the prevalence and severity distribution of the major chemotherapy toxicities; and (5) to determine the feasibility and acceptability of collecting symptom reports electronically using PRO-CTCAE administered in clinic before chemotherapy visits.

Methods

Approval was obtained from the University Health Network Research Ethics Board. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study. At the Princess Margaret Cancer Centre (Toronto, Ontario, CA, USA), a comprehensive cancer center, adult outpatients over the age of 18 years old, with solid or hematologic malignancies of any stage and who had received chemotherapy in the past month, were eligible to complete PRO-CTCAE online questionnaires administered on touchscreen tablets in the chemotherapy day suites and clinic waiting rooms. Ineligible patients included those with significant cognitive impairments and those without an acceptable level of English comprehension.

When patients were seen in either the clinic or the chemotherapy daycare unit for routine chemotherapy administration, an English-language, touchscreen tablet survey collected information about patient demographics, along with a health utility assessment through the three-level version of the EuroQol 5-Dimensions instrument (EQ-5D-3L) [37], and PRO-CTCAE elements of the five most common and prevalent chemotherapy toxicities (Table 2). EQ-5D-derived health utilities utilized Canadian conversion values to represent general health status, with lower scores indicating worse HRQOL [38, 39].

For the PRO-CTCAE portion, patients were asked to quantify the frequency, severity, and interference with daily activities domains associated with 12 symptomatic toxicities (total of 32 PRO-CTCAE items), using a five-point Likert scale (“never” to “almost constantly” for frequency, “none” to “very severe” for severity, and “not at all” to “very much” for interference), with a 7-day recall period. Significant toxicities were considered a high (point 4) or very high (point 5) Likert scale rating in any of the three domains, as such ratings are likely to result in clinical interventions.

There were several phases of this study (Fig. 1). In the pilot phase, the focus was on (1) proper implementation of the procedures in clinic (which followed an abbreviated CIHR knowledge-to-action framework to identify barriers, facilitators, obtain stakeholder counsel, and identify potential solutions); (2) assessment of patient acceptability (through a short survey of Likert items). Subsequently, in phase I of the study, the focus was on assessing and confirming the appropriateness of toxicity data being collected by PRO-CTCAE, and assessing the current management of any such toxicities by healthcare providers who had no knowledge of results of PRO-CTCAE. Patients in phase I were told that information from these surveys would not be given to any clinician or healthcare provider, as this represented the pre-intervention period, when we were documenting current practices. There was originally a phase II study whereby PRO-CTCAE results would have been provided immediately to their healthcare providers; improvements in patient care and satisfaction would have been measured relative to phase I. However, phase II was abandoned when phase I demonstrated that patient care and satisfaction was found to be already excellent, even before PRO-CTCAE results were incorporated into the patient management at our institution. Replacing phase II was a retrospective assessment of proper documentation of toxicities in the patient chart record; this was to address a potential secondary reason for incorporating PRO-CTCAE into routine clinical practice, that is, to improve the rate of proper documentation of toxicities, given that proper management of treatment toxicities is an important metric of quality of cancer care.

Fig. 1
figure 1

Consort diagram: summary of recruitment statistics for enrollment into the study

Full size image

In phase I, approximately 3 days subsequent to their clinic or day hospital visit (median 3 days; range 1–12), patients were contacted by a member of the non-clinical research team and asked to complete a short telephone survey. The surveys asked whether healthcare providers had discussed toxicities with the patient during their last clinical visit, whether any clinical management changes were instituted as a result of these discussions, and whether patients were satisfied with the outcome of these most recent discussions (Fig. 2). Satisfaction was also collected on a 5-point Likert scale, with “1” being very satisfied with care, and “5” being very dissatisfied, and dichotomized 1–2 (satisfied) vs 3–5 (neutral or dissatisfied) for analysis.

Fig. 2
figure 2

Flowchart demonstrating the tablet-based survey and follow-up phone survey processes for n = 119 phase I patients. Prior to using this flowchart, patient toxicities and grades of toxicities were confirmed by the phone coordinator. Boxed items are questions that were asked to patients. HCP healthcare provider, CT chemotoxicities, rec recommendation

Full size image

Phase II Retrospective Assessment of Documentation of Toxicities in the Patient Chart Record

Data abstraction from the electronic health records confirmed the administration of chemotherapy in the past month, along with any documented symptoms-related toxicities in the electronic pharmacy, nursing, chemotherapy clinic, electronic chemotherapy order entry system, and physician notes within that timeframe. The 497 eligible patients recruited to the study were included in the data abstraction. Chemotoxicity symptoms (to be conservative, any symptom that could be attributed to treatment toxicity was attributed to treatment rather than disease) were recorded along with any corresponding severity rating in its original wording and format. For patients who had been hospitalized, handwritten inpatient notes were also evaluated. For patients enrolled in clinical trials, clinical trial charts were not reviewed, as these were generally not allowed to be used outside the context of the original study. However, all trial primary source data had to be available in the primary electronic charts.

All analyses were performed using SASv9.3.

Results

Participants

Of 671 eligible patients, 74% (n = 497) consented and completed PRO-CTCAE and EQ 5D (Fig. 1). Patient clinico-demographic characteristics are shown in Table 1. A diverse range of tumor types and age at diagnosis were represented in this patient population. The median age of recruited patients was 54, with two-thirds being female. PRO-CTCAE was administered to 497: 153 in the pilot phase and 344 in the subsequent phase I portion. Of these 344 patients, 195 reported significant toxicities (defined as having reported at least one score 4 or 5 on any PRO-CTCAE question corresponding to grade 3 or 4 on the CTCAE scales, which are the two highest scores of toxicity for each question); these patients were to have a follow-up phone call. Among these 195 patients, 119 received a phone call and successfully completed the phone survey confirming toxicities and management of such toxicities by healthcare providers who were blinded to PRO measures. No interventions were offered during the follow-up phone call, as most chemotoxicities were addressed satisfactorily by in-clinic physician intervention. Major reasons why 76 patients were unable to complete the follow-up phone call survey were three unsuccessful call attempts (n = 35), more than 2 weeks had elapsed since PRO survey completion (n = 25), missing/wrong phone number (n = 12), and other reasons (n = 4). The reasons why three phone call attempts were unsuccessful for some patients were mainly logistical. Some possible reasons include patients coincidentally not being at home at the time of the call, missed picking up their cell phone and were unable to call back as a result of outgoing hospital numbers being marked as private, or hospitalization that may be cancer/chemotoxicity-related.

Table 1 Summary of socio-demographic variables from our patient population
Full size table

Did Healthcare Providers Inquire About and Discuss Significant Toxicity Issues with Patients?

In the subset of 119 patients who had significant toxicities and who were surveyed over the phone, 91% (95% CI 82–99%) were satisfied with their chemotoxicity management regimen. The most prevalent moderate to severe symptomatic chemotoxicities were fatigue (85%; n = 101), decreased appetite (62%; n = 74), pain (52%; n = 62), nausea (50%; n = 60), and difficulty tasting food or drink (47%; n = 56). Only 10% (n = 12) of patients reported significant toxicities that their healthcare providers were not aware of. A majority of the patients (n = 107; mean 90%, 95% CI 84–96%) reported that their providers were aware of and had discussed all significant chemotoxicities with them. A large majority of these respondents (n = 83; mean 70%, 95% CI 61–79%) also received a change in management as a result of these discussions. Although 30% (n = 36) of the respondents did not receive a change in management after discussing their symptomatic toxicities with their provider, only 17% (n = 6) of such patients reported that they would have wanted a change in symptom management.

How Complete is the Documentation of Toxicities?

As we collected toxicity data in both the pilot and phase I portion of this study, we were able to compare our PRO-CTCAE data with chart documentation in all 497 patients; 51% reported at least one significant toxicity of the “severe” or “very severe” grading. In only 34% of all charts reviewed, chemotoxicities were mentioned in the electronic outpatient clinic charts, and mostly through outpatient physician notes. Documentation format for chemotoxicities varied greatly among providers, and various method of severity scoring were employed. Most providers used qualitative descriptors (e.g., “significant”, “some”, “slight”, “a little bit”, and “fair bit”) in place of a numerical grading system such as the CTCAE version 4. Quantitative severity measures (e.g., CTCAE) were rarely used (under 10% of the time).

In contrast, the presence of any chemotoxicities was documented in only 13/497 patients (3%) in the computerized order entry system: seven patients had specific side effects documented, while another six patients had “patient tolerated treatment well” and similar descriptions in place of a numerical grading system. Adverse events mentioned included creatinine increase (n = 1), hypersalivation (n = 1), rash (n = 1), nausea (n = 2), fatigue (n = 1), constipation (n = 2), and diarrhea (n = 2).

How Prevalent Were Symptomatic Adverse Events as Captured by PRO-CTCAE?

The PRO tool was able to document a high prevalence of the most common symptoms (which we attributed conservatively to chemotoxicities), as well as their severities. In all 497 patients surveyed, 51% reported at least one grade 3 or higher toxicity (i.e., answered “4” or “5” on the Likert scales). Among 344 phase I patients, a slightly higher 195 patients or 57% reported at least one significant grade 3 or higher toxicity in the 7 days prior to PRO-CTCAE assessment. In the subgroup of 119 patients with at least one significant toxicity confirmed by telephone, the most prevalent frequencies of significant chemotherapeutic toxicities were fatigue (67%), decreased appetite (40%), pain (34%), nausea (29%), and aching joints (29%) (Fig. 3; Table 2).

Fig. 3
figure 3

Prevalence of individual chemotoxicities (at various stage of treatment) as captured by PRO-CTCAE. Each bar represents the frequency of a level (see legend). Numbers reported above each bar represent the number of individuals reporting that level for each symptom. Bars to the right of the solid black line indicate significant toxicity. Percentage of significant and non-significant sums are represented in the percentage boxes on each side of the graphs. "Never/None/Not at all" is reported as "grade 1" in the text, and the extreme "almost constantly/very severe/very much" is reported as "grade 5". Some symptoms are missing either frequently, severity, or interference, and this was because the PRO-CTCAE items determined that these questions were not as relevant to the symptom assessment

Full size image
Table 2 Prevalence of significant toxicities (at various stages of treatment) as captured by PRO-CTCAE, among all individuals reporting at least one significant toxicity and who had received a follow-up phone call
Full size table

Do Patients Find It Acceptable to Use This Electronic Version of the PRO-CTCAE to Document Their Chemotherapy Adverse Events?

Patients undergoing the feasibility pilot (n = 153) generally had no problems using the touchscreen tablet-administered PRO tool (Table 3). We identified very few barriers, and had no significant problems with implementation. The most frequent barrier that arose was internet disconnection, but these were resolved with refreshment of the application screen and reconnection with the Wi-Fi. Over 80% of patients reported that questions included in the survey were appropriate and relevant, and were happy to complete the survey using tablet touchscreen technology (Table 3). From qualitative patient assessments, major reasons for not wanting to complete surveys regularly included inconvenience and time requirements in the face of competing requests for other surveys, feeling unwell, and lack of perceived benefit to patient; the last reason was expected since we had told every patient that we would not be relaying this information back to the healthcare treating team of physicians, nurses, and pharmacists as part of this pilot phase.

Table 3 Patient acceptability of the electronic PRO-CTCAE tool
Full size table

Discussion

Outside the context of a clinical trial, current documentation of chemotoxicities at our institution was poor, and comparison of the data collected in a systematic manner suggested that clinician documentation does not fully reflect symptoms from the patients’ perspective. Although patients reported to be overwhelmingly satisfied with their current clinical care, they continue to experience chemotoxicity-related symptoms that are inadequately documented, thus highlighting the limitations in using patient satisfaction as a surrogate marker for measuring quality of care and health outcomes. The PRO-CTCAE appears to be an easy and effective way of integrating patient self-assessments into current electronic health record systems, to help improve the lack of documentation for chemotoxicities while facilitating its management. PRO results can easily inform healthcare providers of toxicities and elicit early interventions or assessments when necessary, with the future potential to expand into real-time toxicities monitoring for patients at home. With real-time toxicities monitoring, the tool can help mitigate recall bias for patients who have difficulty recalling the frequency and severity of their symptoms between visits [40, 41]. The tool may also help improve communication between patients and physicians, by reducing the effects of potential barriers such as perceived status differences, language, and patient not feeling well. There is recent data demonstrating a positive impact on patients’ HRQOL when a PRO tool was administered for symptoms monitoring on a regular basis both in clinic and at home [42].

This tool can also help facilitate interprofessional communication and collaboration, by actively involving an interdisciplinary team of physicians, nurses, and pharmacists in the monitoring of adverse drug reactions through assessment of survey results. When used in real time, instead of the current pilot method, PRO-CTCAE may be able to track the onset and progression/amelioration of adverse drug reactions with respect to the timing of drug administration and toxicities management. This offers a proactive reporting model that helps to mitigate the gross underreporting, latency, and inconsistency that come with the traditional, more passive reporting systems as demonstrated in this study. In the setting of quality assessment studies, PRO-CTCAE-collected adverse reaction information can eliminate the need for retrospective data abstraction, which requires significant resources to conduct and often faces numerous methodological issues [43, 44].

There are a number of limitations for this analysis. Firstly, this study was carried out in a single institution and the findings may not reflect all oncology care settings. Secondly, the act of collecting PRO-CTCAE data may have influenced subsequent ratings of satisfaction by participants, whereby participants may have subconsciously considered the collection of such data (with no reporting to their healthcare providers) as an intervention. By completing the PRO questionnaire prior to their clinic visit, patients may have also been prompted to discuss their chemotoxicities with physicians when they would not have otherwise. Chart reviews were conducted using electronic records; however, a limited number of paper charts were reviewed as well and no additional toxicity data were found. Furthermore, the follow-up call was conducted by a research coordinator from the same institution; thus, patients may have responded more positively than had the call been made by an impartial third party. Thirdly, as with any study that relies on chart review to reflect what took place in the clinical encounter, it is possible that these findings under- or overestimate the true occurrence of discussions of symptomatic toxicities. Providers may omit detail in their documentation because of time pressures, or fail to accurately recall the specific issues addressed during the patient’s visit. Fourthly, acceptability was measured on the basis of patients’ perception of whether or not they would be willing to complete the PRO questionnaire, rather than prospectively collected rates of completion; the fact that no intervention was tied to the acceptability may have resulted in fewer patients willing to accept the tool. Lastly, we did not extensively explore, through either focus groups or in-depth interviews, the reasons why some patients did not want to complete the PRO-CTCAE on a regular basis; there is a need to perform such analyses across multiple institutions in the future.

In summary, although patients were satisfied with the management of their chemotoxicities, documentation for such issues was poor, making retrospective data collection for pharmacovigilance or clinical management assessments difficult. PRO-CTCAE, a tool generally utilized in research, was found to significantly improve the data collection of such chemotoxicities, and was easy for patients to use in a routine clinical setting. This can lead to earlier detection and prevention of chemotoxicities, potentially limiting its progression to more severe stages that can hinder treatment and patients’ HRQOL [45,46,47,−48]. In addition, the PRO tool can be adapted to detect and report side effects related to newly marketed medications, and flag significant toxicities sooner by allowing easy access to more reliable toxicity data.