Introduction

The importance of the patients’ perspective on their well-being is increasingly recognized. Patient-reported outcome (PRO) measures are a valid method to provide valuable insight into patients’ experiences. PROs usually ask patients to self-report general well-being, symptoms, and functional status [2] revealing a patient-centered view on their subjective experiences [3].

By integrating PROs in the clinical routine, communication and engagement between healthcare providers and their patients is facilitated [4, 5]. If physicians have access to PROs, they can identify a higher number of symptoms [6]. In addition, PROs have shown to allow for more efficient use of clinic time [7]. Overall, the integration of PROs has resulted in better treatment quality [8, 9] and patients’ compliance [10]. Consequently, the collection and use of patient-reported information are extremely valuable.

PROs have been increasingly implemented in clinical practice as well as in research [11]. Many PRO measures have originally been completed on paper. However, there are several issues associated with paper versions of PROs. Frequent problems arise from incomplete questionnaires, making it difficult for healthcare professionals to accurately use data [12]. In addition, errors can occur due to manual scoring and data entry [13].

Subsequently, PRO measures have been altered to allow for electronic administration [14] offering a replacement to the traditional use of paper by using, for example, computers, tablets, or smartphones.

Studies have shown that electronic data capture offers several advantages: It reduces the number of data entry errors [15, 16] as well as the amount of missing data [17, 18]. Data are automatically calculated and transferred to a central database, strengthening the accuracy and efficiency of data collection [19]. Furthermore, the end user has immediate access to the data through the centralized database.

In addition, data capture can be improved since respondents can neither create their own response option nor have the opportunity to provide ambiguous responses [15, 17]. However, patients with less technical experiences may encounter difficulties when operating technical devices, such as tablets that were used in this study.

As the quality of data collection has improved and the use of data has become easier through electronic assessment, health care teams increasingly prefer electronic data capture over paper [20].

This has resulted in higher demand for research on the equivalence between electronic and paper-based PRO measures [14]. To target support and understand which mode of PRO-administration is most useful, we need to understand patients experience with the electronic assessment.

The purpose of this study was to [1] implement an electronic assessment of PROs in routine inpatient cancer care and test its feasibility. Furthermore, [2] we assessed whether the implementation of an electronic version of the standardized questionnaires results in equivalent responses to the paper version. In addition, [3] we examined the completion rate between paper-based and electronic assessment.

Methods

Ethical approval

The study was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of the Chamber of Physicians in Berlin, Germany (Eth-48/16). All patients provided written informed consent to participate in the study.

Study design

We evaluated the feasibility of an electronic PRO assessment tool in inpatient oncology care by conducting a multicenter, randomized, controlled trail. Patients were recruited between July 2017 and February 2019 while being admitted to four participating oncology wards (Helios Klinikum Emil von Behring, two centers at Helios Klinikum Berlin Buch, Helios Klinikum Bad Saarow) for planned anticancer treatment including chemotherapy, radiotherapy, or immunotherapy.

For the primary analysis, we randomized patients into three groups. As the current paper reports a secondary analysis, we analyzed two subgroups of the study: group A (now designated as the intervention arm) received a tablet version, while group B (control arm) completed the questionnaire on paper. PRO results from the third arm were graphically displayed and presented to their treating physicians for them to explore before their next patient encounter. However, this will be discussed in a further paper. Randomization was carried out using computerized routine by a staff member not further involved in the study.

Both groups received a set of standardized PRO questionnaires. While group B was given a paper version, group A completed the PROs in an electronic survey using an Apple iPad. Participants completed measurements independently but were allowed to ask for assistance.

There were three different points of measurement: T0: admission, T1: 1 week after admission (if applicable), and T2: discharge. Following T2, a feasibility questionnaire was administered to arm A to assess their experience using the electronic tool. If patients remained in the hospital less than 1 week, T1 did not take place.

A summary of the study design is depicted in the following consort diagram.

Questionnaires

The composition of PROs used for the study was developed in a multi-professional expert team and consisted of different instruments used at different assessment points (see Fig. 1). Standardized questionnaires with a total amount of 102 items were applied.

Fig. 1
figure 1

Consort diagram

Full size image

EORTC QLQ-C30. The Quality of Life Questionnaire C30 (QLQ-C30) [21] developed by the European Organization for Research and Treatment of Cancer (EORTC) assesses the global health status, five functional scales (physical, role, emotional, cognitive, social), and nine common symptoms in cancer patients. The QLQ-C30 consists of 30 items that relate to the state of health and well-being with scores ranging from 0 to 100. Higher symptom scores indicate higher symptom burden; however, higher scores in the global health status and the functional scales imply a better functioning.

IN-PATSAT32. The EORTC cancer inpatient satisfaction with care measure (IN-PATSAT 32) [22] holds 32-items assessing patients’ satisfaction with care by physicians and nurses. Higher values indicate greater satisfaction in the respective area.

SDM-Q-9. The German translation of the Shared Decision-making Questionnaire (SDM-Q-9) [23] assesses the extent of participatory decision-making. Patients indicate on a scale how appropriate individual elements of participation were at their last doctor’s visit. Higher values indicate a higher degree of participation.

PRO-CTCAE. Patient-reported adverse events are measured using the Common Terminology Criteria for Adverse Events (CTCAE) [24] that has also been adapted for patients to self-report (PRO-CTCAE) [25]. Patients rate the frequency, severity, and impairment of symptoms. For the present work, we only evaluated the severity. Modules were created that are suitable for the patients’ entities and treatments.

Feasibility questionnaire (shown in supplement). Feasibility is assessing the patients’ acceptability for the electronic assessment. The feasibility questionnaire presents 10 questions that were adapted from Ashley et al. [1] and applying goal attainment scaling [26].

Questions were answered on a scale ranging from “1” (illustrating non achievement) to “5” (illustrating achieving goal). Additionally, there were three items being answered with “yes” or “no”: “Did a staff member help you how to use the questionnaire today?,” “Did you need help from a staff member while answering the questions?,” and “Would you have preferred to answer the questions with pencil and paper?.” The feasibility questionnaire was administered to the intervention group (group A).

Participants

Patients (aged 18 years or older) diagnosed with hematological or oncological cancer were eligible to enter the study. Hematological cancers relate to malignant hematological neoplasm, while we categorized any solid tumors as oncological. Eligibility was restricted to patients with a planned hospital stay for ≥ 3 days to undergo anticancer therapy.

Statistical analysis

Scales of questionnaires were calculated according to the respective scoring manuals. For categorical variables, absolute frequencies and percentage are presented. For comparisons of the distributions of categorical variables between groups, the chi-squared-test [27] or Fisher’s exact test [28] (in case of counts of 5 or lower) were used. For continuous variables, arithmetic mean and standard deviation are presented. Continuous parameters between groups were compared using the Wilcoxon-Mann-Whitney-U-test [29, 30]. Overall significance level was 10% two-sided. Statistical analyses were conducted using SPSS version 27 and R version 4.0.1. Statistical analyses were pre-specified in a detailed statistical analysis plan.

Results

Between July 2017 and February 2019, a total of n = 125 patients admitted for inpatient care were included in this study. Due to n = 12 dropouts (6 in group A, 6 in group B), the target sample resulted in n = 113 patients (100%). Patients were randomly assigned to two groups: group A consisted of 56 patients (100%) and group B of 57 patients (100%).

After 1 week, 34% of patients in group A and 35% in group B, that remained in the hospital participated in T1. The median duration of hospital stay was 6 days in group A and 7 days in group B. All patients included in the study participated at time of discharge (T2).

Study population

Demographic data for patients is shown in Table 1.

Table 1 Characteristics of the study populations (n = 113)
Full size table

On average, patients in group A were 61.7 (SD ± 12.5) years and patients in group B were 65.7 (SD ± 14.4) years old. More than half in group A (62%, n = 35) and 49% (n = 28) in group B were male. The majority (82% in group A and 75% in group B) were educated with an apprenticeship or university degree. More than half in group A (55%) as well as 45% in group B were employed before diagnosis. Most patients (66% in group A and 61% in group B) were treated for oncological disease.

Stage IV tumor stadium applied to 28% in group A and 35% in group B.

Completion rate

The following Table 2 shows completion rates for admission (T0), 1 week after admission (T1) and discharge (T2). We evaluated differences between the surveys as well as among the questionnaires.

Table 2 Completion rates
Full size table

For the PRO-CTCAE and the SDM-Q-9, completion rates at T2 were higher with the electronic assessment versus paper-based. However, the QLQ-C30 showed better completion rates for paper-based assessment at all assessment points. Significant differences between ePRO and paper-based assessment could be found: 64% at T0 (group A) versus 93% at T0 (group B), chi-squared test, p = 0.0005. As well as 59% at T2 (group A) versus 82% at T2 (group B), chi-squared-test, p = 0.011.

There was no difference regarding the 100% completion rate for the IN PATSAT32.

For the PRO-CTCAE, the completion rate was higher at admission (T0) than at discharge (T2).

The PRO-CTCAE was more often fully completed than the other questionnaires.

Differences in PROs at T0 and T2

Results regarding QoL and symptom burden at admission (T0) are reported in Tables 3 and 4. High concordances were noted between the paper and the electronic version. Throughout all items, we did not observe significant differences between paper and electronic assessment of PROs. This indicates that electronic and paper-based assessment collect comparable information.

Table 3 Quality of life at admission (T0) and at discharge (T2)
Full size table
Table 4 Symptom burden at (T0) and at discharge (T2)
Full size table

Feasibility

Feasibility results are reported in Table 5. Almost 79% participants reported not needing support for answering the questionnaire indicating that the electronic assessment was broadly acceptable for participants. However, 51.8% required help to operate the questionnaire.

Table 5 Feasibility (n = 56)
Full size table

Respondents needing help presented a relatively high satisfaction with the support they received (mean 3.75, SD 1.31). This question was added to all patients, even to those not actively asking for help. We did assume our staff to give an introduction in the study rationale, procedure, and device handling. High satisfaction with help can therefore also be interpreted with the staff supporting our patients satisfactorily if any assistance was needed. Furthermore, the patients commonly reported that they found it relatively easy finding time for filling out the questionnaire (mean 4.15, SD 1.05).

The majority (mean 4.24, SD .99) reported no difficulties handling the electronic assessment.

Patients were generally satisfied with the completion of the questionnaire (mean 3.78, SD .88). More than 59% participants found the number of questions asked to be adequate and the majority would have continued answering more questions with the system (mean 3.59, SD 1.60).

The majority (84%) would have not preferred to complete the questionnaires in a paper version indicating a preference for the electronic assessment.

Discussion

As previously discussed, self-assessment and external assessment of patients’ symptoms tend to diverge [31,32,33] and patients’ subjective view can never be reliably represented except from the patient itself. PROs and their efficient assessment are therefore of major importance.

However, as previously suggested [34], incorporating ePRO measures into existing workloads was a significant barrier.

Feasibility

We found that the electronic tool was feasible as most patients reported no difficulties handling the electronic assessment indicating that the tool was broadly acceptable for patients.

However, there was a difference between operating the questionnaire and answering the questions itself. While 78.6% reported not needing help for answering the questions, more than half (51.8%) did require help to operate the questionnaire. Possibly, this is related to the age of the patients, as older age is associated with lower computer and internet use [35]. In the long term, the internet will become established almost universally. We still consider the tool feasible as the ePRO assessment was well received with the majority presenting a relatively high satisfaction with the tool. Furthermore, patients would not prefer to complete the questionnaires in a paper version, which indicates a preference for the electronic assessment.

Those findings suggest good feasibility, suggesting that the electronic capture of PROs provides a reliable replacement for the paper form. This is consistent with other research findings comparing electronic and paper assessment of PROs [17, 18] and showing good concordance between the surveys across a wide variety of diseases [36].

Multiple benefits are associated with ePRO implementation. If ePROs were implemented in routine care, data could be efficiently stored in one location and was immediately available for the healthcare professionals to review in the data base. In addition to these improvements, our results have shown that patients are satisfied with the electronic capture. This is consistent with previous studies comparing electronic forms to paper forms [37,38,39,40]. As already stated, our patients showed little problems handling the electronic tool which aligns with previous studies that suggest high acceptability among patients using tablet based assessment tools [41, 42].

In addition, electronic completion may be easier for patients with limited manual dexterity [43].

Electronic surveys are also perceived as more anonymous than paper-based surveys [44], potentially leading to greater honesty on the patients side.

While there are advantages being associated with electronic assessment of PROs, it should be kept in mind that studies, that require an electronic device, risk excluding patients whose insight and experiences with technology are limited.

Completion rate

We included four questionnaires with a total of 102 items in the study. For the PRO-CTCAE and the SDM-Q-9, completion rates at T2 were higher with the electronic assessment versus paper-based. However, the QLQ-C30 showed better completion rates for paper-based assessment at all assessment points. This shows that neither the electronic nor the paper-based assessment consistently resulted in better completion rates.

For the QLQ-C30, nearly half of those in group A (47%) had no entries in the questionnaire at T1. This is quite surprising, especially since at all other timepoints a minimum of 85% patients in both groups reached at least 96% completion rate. We were not able to completely clarify the reason for this deviation. Most likely it was due to a technical issue based on the study staff not appropriately administering the survey. This shows that even with electronic administration, manual input can still be necessary if problems occur.

Overall, a high number of questionnaires were not fully completed. This leads to the question whether an unsuitable amount of items was chosen for this study. Considering the fact that patients undertook the survey more than once, it is thinkable that the number of items was too high. This can be underlined by a previous study that indicated a negative correlation between the completion rate and survey length [45]. The burden of symptoms could also have an impact on questionnaire completion, as described in further studies [46, 47]. If patients are unwell, this might lead to difficulties completing PROs, regardless if they are electronic or paper based.

If PROs are seen to be useful for clinical care, then patients are more likely to overcome barriers to achieve completion.

Limitations

There were some limitations that should be considered when interpreting these findings. The main limitation of this study was that the feasibility questionnaire was only given to the intervention group that used the electronic assessment. Therefore, we cannot report on patients’ experiences with the paper form.

We also assessed differences within the PROs. However, the questionnaires were completed only either in the electronic or the paper version. Respondents were not exposed to both surveys, making direct comparison somewhat difficult.

The relatively short median duration of hospitalization (6 days in group A and 7 days in group B) could have affected the survey completion rates. A longer stay, so that all patients could have participated in T1, would have increased the informative value about differences between electronic and paper-based assessment.

Conclusion

Our study concludes that the assessment of electronic PRO measures is feasible in routine inpatient cancer care. The majority (84%) would have not preferred to complete the questionnaires in a paper version indicating a preference for the electronic assessment. Patients were therefore receptive to the electronic assessment and showed little problems handling the tool. Our findings support that electronic assessment is a valuable replacement to paper versions. The results of this study can allow for better understanding of the complexity of ePRO implementation and be helpful in creating strategies for further application.