Introduction

In the Netherlands, approximately 120,000 patients are diagnosed with cancer every year [1]. These patients often undergo a long and intensive healthcare trajectory from the moment of diagnosis through to long-term follow-up. Patients’ opinions about the care provided are an important quality measure as they can offer different perspectives and provide insights into aspects of healthcare that professionals are unaware of. Consequently, there is increasing attention given to the use of patient-reported experience measures (PREMs) in seeking to improve quality as part of value-based healthcare (VBHC) [2,3,4]. In the Netherlands, most tertiary hospitals measure patients’ experiences with the Picker Institute’s patient experience questionnaire [5]. This experience measure is based on “Picker’s principles of patient-centered care” and determines generic experiences of the care process using topics such as experience with “waiting time” and “friendliness of personnel” [5]. These generic PREMs are convenient for general quality improvements at the hospital level or for integrated care. Since the delivered care and logistic processes will differ between departments within a hospital (for example in “the use of volunteers on the diagnostic day” or “the use of prediction models to inform patients in decision making”), patients’ experiences with such care-specific processes are useful for improving quality at the department level [3].

Although there are potential advantages, the effectiveness of routine PREM assessments in clinical practice in enhancing healthcare services is uncertain as the evidence is inconclusive [6,7,8,9]. These inconsistent findings may be due to heterogeneity in the PREMs used. Healthcare is dynamic and contains a wide range of aspects that contribute to quality of care. To further enhance the healthcare process within a department, it is important to respond to opportunities for improvement identified through the PREM data. At a certain moment, when the items in the PREM are fully satisfied, it is no longer possible to further improve on these items and it is no longer useful to assess them. A more dynamic approach would offer the opportunity to switch to including other items covering another or new aspects of the healthcare process in seeking continuous improvement.

Based on this idea, we have developed a PREM-item bank for improving cancer-specific VBHC. This PREM-item bank complements the generic PREM (Picker) by providing opportunities to add specific or detailed questions about care experiences. This PREM item bank can be dynamically used to measure patients’ experiences with their care at the outpatient clinic.

Methods

In a co-creation process with patients, we took the following steps to create the PREM-item bank: (1) a literature search to identify existing PREMs, (2) the identification of themes through focus group discussions in an approach based on patient journeys, (3) combining themes from the literature and themes from the qualitative analysis, (4) mapping by healthcare professionals of items within themes, (5) prioritizing the items seen as most important by patients and by healthcare professionals, and (6) rewriting items in a language understandable for most people (level B1) (Fig. 1).

Fig. 1
figure 1

Development of the PREM-item bank. PREM, patient-reported experience measure; B1 language level, intermediate level

Full size image

1. Explorative literature search

An explorative literature search was performed using the PubMed, Embase, and Medline databases in August 2019 to obtain details of available generic and cancer-specific PREMs. For the search strategy, the words “patient reported experience measure” OR “experience measure” AND “cancer” were used. Articles were included when they described a PREM validated in the Dutch language that contained items assessing experience with the healthcare process covering both emotional and cognitive evaluation of the delivered care. We included generic and cancer-specific PREMs. Exclusion criteria were studies describing patient-reported outcomes measures (PROMs) and other questionnaires that did not include any patient-reported experience measures.

2a. Patient journey: focus group

Together with patients and healthcare professionals, we gathered relevant themes for the PREM-item bank. We asked (ex-) patients (n = 8) and healthcare professionals (n = 3) to participate in a focus group meeting during which a “patient journey of outpatient clinic experiences” was evaluated (Figs. 2 and 3). From our previous experience with focus groups and qualitative research, we know that a group size of 6–10 participants is ideal for ensuring a trusted environment and that everyone has a chance to speak, while also obtaining diverse input from different individuals. Creating a patient journey is a method to obtain the patient’s perspective on all (possible) activities and contact moments between patients and healthcare professionals. We focused on two moments in the outpatient clinic in this full patient journey: the diagnostic phase and the follow-up phase, as these specific contact periods contribute to how the total provided cancer care is experienced. The diagnostic phase was defined as the time between the moment of referral through to the consultation in which treatment options are discussed. Contact moments were grouped within four phases: referral, registration procedure, contact with healthcare professionals on intake to the hospital, and consultation about treatment options. The follow-up phase started with the first regular follow-up visit after treatment and ended at the last follow-up visit, five years after treatment. Contact moments were arranged in three phases: before the follow-up visit, during the registration procedure, and later contact with several healthcare professionals. Each possible contact moments was visualized and discussed separately. Patients were asked if they thought there were missing aspects in the proposed patient journey. Following this, all the steps were evaluated using the following questions: “What were positive experiences with the provided care?”, “What were negative experiences with the provided care?”, “What did you miss in the provided care?”, “Do you have suggestions to improve our care?”, and “Who or what was indispensable in the provided care?”. Data saturation was reached when no new themes were opted.

Fig. 2
figure 2

Contact moments in the diagnostic phase of the patient journey for HNC patients. HNC, head and neck cancer

Full size image
Fig. 3
figure 3

Contact moments in the follow-up phase of the patient journey for HNC patients. HNC, head and neck cancer; PROMs, patient-reported outcome measure; ePRO structure, electronic patient-reported outcome structure

Full size image

2b. Qualitative data analysis

An inductive approach for coding the data was followed by three researchers (MO, KD, and HW) who independently performed thematic analysis on the data. Consensus on the themes present was achieved through discussion, with verification by a fourth researcher (KH).

3. Combining themes from the literature and from the qualitative analysis

Three researchers (MO, KD, and HW) compared the themes identified from the literature with the themes from the focus group and formed eight new themes that covered different aspects of the intake or diagnostic phase and of the follow-up phase.

4. Mapping items within themes

After identifying suitable PREMs in step 1, all the individual items included in the PREMs were listed. Duplicates were removed. Items provided by the focus group that were not covered by the list of existing PREM questions were added. Following this, all the items were independently mapped on to the formulated themes by two senior researchers (MO and KD) and one junior researcher (HW). Consensus was achieved through later discussion.

5. Prioritizing

In order to select the most important items for each theme, a prioritization exercise was completed by both patients (n = 7) and 11 healthcare professionals (e.g., head and neck surgeons, psychologists, and oncology nurses). For themes with a fairly limited number of items (1, 2, 5, 7, and 8), a top-five list was compiled, while for more extensive themes (3, 4, and 6), a top-ten was generated. Items were weighted based on their ordering (i.e., first place = 5 and second place = 4). Final prioritization was based on the square root of the frequency of occurrence of an item multiplied by the weight of that item: \(\sqrt{\left(\textrm{frequency}\ \textrm{of}\ \textrm{appearance}\ \textrm{x}\ \textrm{weight}\right)}\). The outcome was adjusted for group size such that both groups (patients and experts) had an equal influence on the outcome. Both healthcare professionals and patients were asked whether items were relevant, understandable, and whether important items were missing.

6. Qualitative analysis open comments

After the prioritization, three researchers (MO, KD, and HW) reviewed the open comments in which participants could formulate missing items. After discussion, the researchers reached a consensus on the items to be added when they had been mentioned multiple times and fit within the themes formulated from the previously conducted focus groups.

7. Rewriting items in Dutch language level B1 (intermediate level)

Next, all the items were rewritten using B1 language, i.e., put in a form that patients with limited health literacy could understand. Following the method applied with the Picker questionnaire, questions were partly repeated in the answer options to make them easier to understand [5]. The reformulated items were again sent to the participating patients to check for understandability.

Results

1. Explorative literature search

Two previously validated generic PREMs (Consumer Quality Index (CQI) [10], Picker questionnaire [11]) plus five cancer-specific PREMs (FACE-Q head and neck [12], PREM VBHC head and neck [13], EORTC-INFO25 [14], and EORTC-COMU26 [15]) were included in the review. In addition, a PREM concerning the diagnostic day at the outpatient clinic constructed by the Department of Head and Neck Oncology of Erasmus Medical Center was included [16]. After removing duplications, 141 items were identified from the above instruments (Table 1). Additionally, six themes were derived from the included PREMs: (1) organization, (2) treatment by healthcare providers, (3) expertise of healthcare providers, (4) patient empowerment, (5) end of treatment process, and (6) outpatient clinic.

Table 1 Included patient-reported experience measures (PREMs)
Full size table

2. Qualitative analyses of patient journey: focus group

In the focus group, the patient journey was discussed with head and neck cancer patients and healthcare professionals. Two key points in the full patient journey were visualized (the diagnostic stage and the follow-up phase), and additional contact moments were added. Six themes were derived from this qualitative analysis: (1) “Communication,” (2) “Information,” (3) “Shared decision-making,” (4) “Provided care,” (5) “Environment,” and (6) “Technology.” The topics considered important by the focus-group participants in the first phase are included in Fig. 2 and in the second phase in Fig. 3.

3. Combining themes from literature and themes from qualitative analysis

After comparing our themes resulting from the focus group with those derived from the literature, we proposed eight final themes: (1) organization, (2) competence of healthcare professionals, (3) communication, (4) information and services, (5) patient empowerment, (6) continuity of provided care, (7) environment, and (8) technology (Table 2). Further, during the qualitative analysis, 19 additional items were formulated that were not covered in the 141 items that were identified in the existing PREMs.

Table 2 Final PREM-item bank
Full size table

4 & 5. Mapping and prioritizing

Subsequently, 160 items were mapped within the eight themes. Following this, 7 (ex-)patients and 11 healthcare professionals participated in prioritizing the items within each theme. The top-five items selected by the two groups, from each of the smaller themes (1, 2, 5, 7, and 8), and the top-ten from the larger themes (3, 4, and 6) were included in the item-bank (n = 57). In most cases, the prioritizations of the (ex-)patients and healthcare professionals were similar: 72% of the selected items were in the top five/ten of both groups, and 25% of the selected items were in the top five/ten of one group and in the top 11/16 respectively of the other group (Δ < 6 places). Only two items were selected in which the prioritization of (ex-)patients and healthcare professionals differed widely: “Were you satisfied with the communication between you and the specialist nurse?” (theme 3) was put in fifth place by the (ex-)patients but only 24th by the healthcare professionals. Further, “Did you receive information about the effects of smoking and alcohol on your illness?” (theme 4) was placed 11th and 22nd, respectively.

6. Qualitative analysis open comments

Eventually, the open comments were evaluated and discussed. An additional 19 items were added to the PREM-item bank after consensus of all three researchers. In total, 76 items were included in the final PREM-item bank (Table 2).

Discussion

To the best of our knowledge, there is currently no patient-reported experience measure (PREM) item bank applicable to cancer care at an outpatient clinic. In this mixed-method study, we proposed a PREM-item bank that can be used to dynamically evaluate the provided care at a cancer outpatient clinic. Through the discussion of a patient journey with patients and professionals, which assessed the key contact moments of care during the intake and diagnostic phase and the follow-up phase, eight themes were derived. Together with patients and healthcare professionals, the most important items from existing PREMs were chosen and missing relevant items were added. Eventually, an item bank with 76 items was formed (Table 2).

Since the introduction of value-based healthcare (VBHC) in 2006, there has been increasing interest in the patient’s perspective on healthcare [17, 18]. Both the perspective of patients on their own functioning (PROMs) and their experiences with the provided care (PREMs) are seen as important instruments for quality improvement [3]. As the World Health Organization (WHO) notes, person-centered care is a key component of providing high-quality healthcare [19]. Furthermore, patient-reported experiences are associated with higher patient safety and clinical effectiveness [7, 20].

To date, most PREMs that are used are generic so that results can be compared between departments and hospitals (e.g., the patient experience monitor (PEM) [5]). Although the PEM is used in all academic university hospitals in the Netherlands, it can be difficult to pinpoint concrete actions to improve the care quality in specific departments based on the results of this generic PREM. Some of the evaluated items are outside the control of a department, such as how parking is experienced. The review by Gleeson et al. similarly noted the problem of a lack of specificity in national surveys, and that staff often thought that such data were not applicable or relevant to their daily work [8].

Furthermore, there are problems with the PEM in that there appears to be a ceiling effect: the overall experience is often rated as good (score of 8 out of 10) in all participating university hospitals [21]. This makes it difficult to further improve healthcare based on these already good scores. While we acknowledge the importance of this generic measurement for benchmarking experiences with the provided care, our PREM-item bank can complement this generic measurement with a department-specific measurement, in which the care delivered by a specific department can be evaluated. Each department has its own care pathways and may provide care in ways that are not seen in other departments (e.g., the use of volunteers on the first day).

In particular, the use of VBHC during consultations and remote care is relatively a new aspect of healthcare, and the ways these are provided often differ between departments. For patients to actively take part in VBHC, they have to contribute by completing questionnaires or using a web application. Since we can only request this of patients, it is particularly important that we continuously assess their experiences. On one hand, we want to identify areas where we can further improve the care we provide while, on the other hand, we want to know what patients already value. From an earlier study in which we used a department-specific PREM, we learned that patients did not see completing these PROMs as a burden, and, further, that filling in and discussing PROMs during consultations enhanced patient empowerment [13]. These appreciated aspects can be used to convince other stakeholders that VBHC leads to quality improvements from the patient’s perspective [22]. Although quality improvements based on PREM data have often been described, only a few studies report a quantitative impact as a result of using PREM-data [3]. It is our hypothesis that with the introduction of this dynamic manner of collecting department-specific PREM-data, identifying potential quality improvements will be more straightforward and applicable for a specific care pathway. When items are given the highest possible rating or it is not possible to further improve an aspect, PREM items could be changed in order to focus on another aspect of care. In this way, patients’ experiences will be able to optimally contribute to improving the quality of various aspects of healthcare. In the future, we intend to evaluate the impact of using aggregated PREM-data on patient-reported outcomes and experiences.

Strengths and limitations

A strength of this study is its mixed-method design. Furthermore, inputs from both patients and healthcare professionals were taken into account when forming the PREM-item bank. Both groups prioritized similar items that they considered important to include in the PREM-item bank. This item bank could be used at any oncology department worldwide, and additional items can be added to enable the constantly evolving healthcare to be structurally evaluated. The questions were phrased in an easily understandable way so that patients with low health literacy could participate. A limitation is that our study only included head and neck cancer patients and healthcare professionals who were working in this area. Nevertheless, to make this item bank applicable to all cancer departments, we included both generic and cancer-specific PREMs based on our explorative literature search. Another limitation is that the literature search was explorative, limited to known PREMs, meaning that some relevant PREMs might have been overlooked. Furthermore, only PREMs validated in the Dutch language were included. In this phase, the item bank is not psychometrically validated, and we added non-validated items from our focus group analysis. However, this is the first step and validation will be performed in the upcoming paper.

Future perspectives

In order to evaluate the VBHC offered in various oncology departments, we intend, in co-creation with other departments and patients, to develop a PREM-VBHC that will be structurally embedded in the provided care. We aim for the associated questionnaire to be limited to approximately 15 fixed items and 5 dynamic items. In this way, oncology departments can be compared, while department-specific care can be dynamically evaluated. The PREM-VBHC will be part of our electronic patient-reported outcome system so it will not increase the workload of healthcare professionals during their daily practice, which has been reported as a barrier to the use of PREMs [22]. The PREM-VBHC results will not be used during consultations as this could affect patients’ honesty [22]. Rather, the aggregated data will be discussed during regular meetings with all involved healthcare professionals following a plan-do-study-act cycle. Ideally, the aggregated PREM data will be complemented with aggregated PROM results. Although the use of aggregated PROMs is still in its infancy, their combination with PREMs for quality improvement seems promising [23]. The PREM-item bank can additionally be used as a basis when a specific topic needs to be addressed (e.g., acute care).

Conclusions

In collaboration with patients and healthcare professionals, we have developed a bank of patient-reported experience measure (PREM) items to evaluate patients’ experiences while receiving cancer care in an outpatient clinic. This PREM-item bank is the first step to dynamically assess the quality of cancer care provided in an outpatient setting. The questions were formulated in straightforward language so that patients with limited health literacy could understand them. We will use the PREM-item bank to develop a PREM that is purposefully designed to evaluate the value-based healthcare that is offered in various oncology departments in our institute and could be used elsewhere.