Plain English summary

What is the key problem/issue/question this manuscript addresses?

With growing numbers of patients living longer following cancer treatment, new models of follow-up care are needed. Using technology, symptoms can be monitored remotely as part of long term follow-up, but this requires careful planning to ensure it meets patient needs.

Why is this study needed?

It is important to explore the views of patients and healthcare professionals so they can contribute to the design of any new follow-up process.

What is the main point of your study?

Before piloting a web-based system, we conducted a survey amongst patients and healthcare professionals to gather their views about the symptom-related questions we should ask, and how often patients should be answering these in order to manage symptoms/side effects and detect recurrence.

Provide a brief overview of your results and what they mean.

Patients and healthcare professionals agreed on the key symptoms to monitor, and the importance of having an opportunity to report and request support for any holistic needs following treatment. Differences of opinion were highlighted in how often symptoms should be reported. This work has informed a web-based symptom reporting system which is being piloted within a community follow-up pathway.

Introduction

Ovarian cancer survival is improving, leading to an increase in patients on routine follow-up and growing pressure on clinics as incidence rates are expected to rise further [1]. Ovarian cancer follow-up is not protocol-driven and evidence-based best practice is yet to be established [2]. Current practice involves monitoring symptoms and serum biomarkers (CA125) through routine appointments to detect recurrence, which is likely to occur within 5 years [3]. Evidence is scarce on the value of routine follow-up for survival or quality of life [4]. Nama et al. [5] suggested that follow-up could utilise patient-reported outcome measures (PROMs) as symptoms are often present at the point of relapse. Research has also indicated that patients delay help-seeking for symptoms until their next appointment [6], and some gynaecological patients have high anxiety before/during appointments [7]. More sustainable models of follow-up and long-term symptom tracking are clearly needed to ensure care provision is commensurate with patient need, rather than a one-size-fits-all service which may detract resources from those who need them most [8].

In recent years pilot studies have explored nurse-led telephone methods [9,10,11,12]. Most have been small qualitative studies, but they illustrate positive experiences amongst early endometrial or stage I-IV ovarian cancer patients. Cox [10] notes the convenience of telephone follow-up in facilitating psychosocial support and the reassurance of continued blood tests.

Recent proliferation of modern technology means the internet offers another follow-up method [13]. Some patients prefer face-to-face appointments, but a recent UK-based randomised controlled trial illustrates the feasibility and patient value of online symptom self-reporting when clinicians are engaged [14]. Furthermore, web-mediated follow-up (supplementary, weekly) vs. routine follow-up has demonstrated increased survival/earlier relapse detection [15].

To explore the feasibility of an electronic PROMs (ePROMs) follow-up pathway, the ‘electronic Patient self-Reported outcomes to Improve cancer Management and patient Experiences’ (ePRIME) study proposed an online intervention for ovarian patients following treatment. The intervention included an ePROMs symptom questionnaire and a blood test to monitor CA125 (performed at GP, local hospital or cancer centre). The clinical team interpret these results, instead of the patient attending hospital-based appointments. Prior to piloting this pathway, decisions about the intervention content and logistics required consultation, which is an important step of any intervention development [16]. This paper describes the process undertaken to reach agreement from healthcare professionals (HCPs) and patients on the core symptoms for monitoring relapse and life after treatment, the most appropriate symptom measure/items, and the frequency of completion to inform the ePROM intervention.

Methods

A Delphi consultation methodology was chosen as it offers an iterative consensus-based approach to reach agreement [17]; group responses are fed back to participants allowing them to re-assess their views within multiple rounds. A ‘modified’ Delphi survey [18] was undertaken which allows the initial round to be informed from group discussions/interviews or a literature review, and is less prescriptive on the number of rounds required. The survey was conducted in two iterative rounds over a 10 month period (Round 1 August 2016–February 2017, Round 2 February–April 2017, but one patient completed in June 2017).

Participants

All doctors and clinical nurse specialists (CNS) working in the gynae medical oncology teams at the four participating hospitals were invited to complete the online survey (https://www.onlinesurveys.ac.uk/) via an email, including a unique username/password for access. Paper copies were provided on request.

Patients who were between 6 and 36 months post-treatment and attending routine ovarian cancer follow-up were consecutively invited as they could offer opinions on the proposed online intervention having experienced the follow-up process. Purposive sampling aimed to get a mixture of patient ages and time/experience on follow-up. Patients were approached by their clinical team, and if willing were provided with a study information sheet. Following verbal agreement patients were given a paper survey and a username/password if they preferred to complete the online version.

All participants were reminded that the survey was anonymous, and the survey return was taken as formal consent. One email/telephone reminder was undertaken.

The demographic details collected included patient age, time post-treatment, and whether this was their first diagnosis. HCPs age, gender, role, and years worked in speciality were collected.

Procedure and Delphi survey content

Ethical approval was obtained from NHS Leeds West Research Ethics Committee (ref: 16/YH/0239).

Firstly, the key symptoms to monitor during ovarian cancer follow-up were explored through informal discussions with HCPs at each hospital. A literature review was conducted to identify and select six potential PROMs for this clinical group (see Additional file 1: Appendix 1 for details of the review and the measures).

Figure 1 outlines the survey topics and an abridged version of each round is presented in Additional file 1: Appendix 2.

Fig. 1
figure 1

Schematic showing questions areas on the Delphi panel in each round

Full size image

Round 1

Relevant symptom items from each of the six PROMs (Additional file 1: Appendix 1, Table A1) were selected and grouped, rather than presenting the full measure (see Additional file 1: Appendix 2 for an example). Then participants rated how well the overall set of items captured the key symptoms to monitor on a 7-point Likert scale (1 = not at all, 7 = very much). The survey also explored the frequency of completion in a follow-up setting, the timeframe for patients to reflect upon (e.g. during the past X weeks), whether symptom duration/frequency/change and holistic needs should be captured.

Round 2

The results from round 1 (aggregated percentages from HCPs, patients, overall) were presented, and further questions asked where opinion was diverse or new aspects had emerged since round 1.

Statistical analysis

There is no universal agreement in Delphi methodology on the percentage that constitutes ‘consensus’, with recommendations ranging from 51% to 80%, but Humphrey-Murto et al. (2017) suggested 70% is a typical cut-off [19] and therefore this was defined a priori as the proposed target level. A lack of consensus was defined as percentages below 70% in round 2.

The proportion of each response was also explored across the participant groups (patients, HCPs, overall). For items on a Likert scale (e.g. round 1-PROMs rating 1–7), mean scores were calculated for each measure for each group/overall, and the highest scoring PROMs were presented in round 2.

Descriptive analysis was performed using Microsoft Excel 2013.

Results

Round 1

Forty-three participants (28 patients/15 HCPs) were approached, and 30 responded (69.8% response rate), including 18/28 patients (64.3%) and 12/15 HCPs (80%). Table 1 illustrates the demographic characteristics in both rounds. Half the patients (n = 9) were 6–12 months post-treatment, whilst others were 1–3 years post-treatment, and for most patients it was their first diagnosis (n = 14). The HCPs included at least one consultant and CNS from each hospital, but more from the largest hospital (site 1).

Table 1 Demographic details of the Delphi participants in each round
Full size table

The key symptoms agreed were: abdominal pain/discomfort, abdominal swelling/bloating, nausea, vomiting, appetite loss, change in bowel habit, urinary symptoms, shortness of breath, fatigue, swollen legs and unexpected weight change. Table 2 summarises the percentage agreement for the questions in each round.

Table 2 Results from each round, percentage of agreement for each area and the consequent decisions made
Full size table

Overall 93.3% felt it was important to monitor the duration and change in symptoms, and 86.7% agreed that symptom frequency should be explored. Eighty percent felt ongoing chemotherapy toxicity and psychological wellbeing were important.

Consensus was not achieved for how frequently patients should complete the online questionnaire, the timeframe/period that patients reflected upon (… past X weeks), and the specific holistic needs question (e.g. travel insurance, emotional). Similarly, the choice of which PROMs to use did not reach consensus, but two options (MDASI-OC; FACT-O) were removed as these were endorsed the least. HCPs also scored the PRO-CTCAE low but it was retained into round 2 due to patient endorsement.

Round 2

15/18 patients wished to be contacted for round 2, and 11/15 completed (73.3%). All HCPs were re-contacted regardless if they had completed round 1. One nurse was no longer working in the speciality, but 8/14 HCPs completed (57.1%), including 5 consultants and 3 CNS (5/8 were from site 1, with one representative from the other 3 sites, see Table 1). Overall 19/29 responded (65.5%).

Three new areas were consulted (Table 2), namely: the provision of self-management advice (62.5% HCPs yes, but 45.5% patients unsure), asking about medication use (84.2% overall), and weight changes (89.5% overall).

The frequency of completion fell short of reaching agreement with 68.4% endorsing 3-monthly reports. However, the acceptability of including a checklist of holistic needs (e.g. emotional, financial, sexual, work/employment, family issues) was agreed by 100% HCPs and 81.8% patients.

Sufficient consensus was not reached on the timeframe (… past X weeks) or the preferred PROMs to use. Given the time constraints a third round could not be undertaken. Discussions were held within the project steering committee, and as around half (47.4% overall) chose ‘ … past 2 weeks’ this was selected. PROMs opinions varied widely for three measures across the participant groups (PRAE 42.1% - 5 patients/3 HCPs; MOST 26.3% - 1 patient/4 HCPs; PRO-CTCAE 21.1% - 4 patients) (Table 2). Further consultation selected the PRAE as this measure was endorsed most overall, particularly by patients who were prioritised as the intended online system users.

Figure 2 visually presents the symptom monitoring intervention.

Fig. 2
figure 2

Overview of the final symptom monitoring intervention

Full size image

Discussion

This work achieved its aim of exploring the key symptoms to monitor during ovarian cancer follow-up, and gathering HCP/patient views of the frequency/proposed content of an online follow-up intervention. The value of remote online methods to facilitate follow-up has become increasingly relevant during/following the COVID-19 pandemic where the reduction of face-to-face hospital-based appointments was essential. This has led to calls for rapid introduction of electronic PROMs in cancer clinical practice [20] and the continued importance of providing support for cancer survivors during/following the pandemic [21].

For most areas, 70% consensus was achieved within two rounds, but some major differences were observed within and between groups. We hope this report provides transparency of the process [16] to enable others to learn from our experiences.

Most HCPs felt 3-monthly was an appropriate frequency (in line with clinical practice), but patients were divided between monthly and 3-monthly reports. For the ePRIME pilot 3-monthly was chosen but the system is open-access should patients wish to complete earlier. This work highlights the importance and value of consulting both patient and HCPs equally when developing interventions [22, 23].

Furthermore, rather than a purely remote system whereby patients report online without clinical contact, free-text comments and discussions with both groups led to the decision that a telephone-based appointment would allow a CNS to formally review online completions and blood results. Therefore, this format is being evaluated in the ongoing pilot.

Patients and HCPs placed great value on reporting/reviewing holistic needs, adding to the evidence that holistic needs assessment is integral to delivering person-centred care [24]. Within the ePRIME system we specifically ask about non-medical holistic needs, and we would urge others to continue prioritising this important area in research and clinical practice, especially if follow-up care is being delivered remotely [23].

The overall survey response rates in both rounds were ~ 65%, and non-response/attrition reasons were not collected. Selection bias is possible as non-responders may have contrasting views of the proposed intervention. Furthermore, those who dropped out may be different, for example less older patients (66+ years) completed round 2. However individuals were consecutively approached, there was a variety of demographics and hospital sites represented across both rounds.

Consensus was not reached on the choice of which PROMs to use, with views divided across patients and HCPs. However, as discussed the level of percentage agreement is disputed in the literature, and the stability of responses through rounds may be more valuable [17]. To our knowledge there is no official guidance for how to proceed in such circumstances. In hindsight a third round or a formal consensus meeting inviting all participants may have been beneficial [22], but with limited time/resources this was resolved through discussion within the project steering committee [25]. It may be that Delphi methodology is not the most appropriate method of choosing between similar validated measures.

Conclusions

It is imperative that intervention development and the selection of PROMs for trials/research is explicitly described to ensure transparency about the decision-making and confirm the appropriateness and content validity of the intervention [16]. The views of patients and HCPs have informed the development of the ePRIME follow-up intervention which is being piloted in two hospitals in the North of England. If successful, the model of remote follow-up pathway and the ePROM system could be adapted to other cancer sites.