Introduction

Gallstones are a common problem, affecting 10–15% of adults over the age of 18 years, with around 60,000 cholecystectomies being performed in England alone [1]. Symptomatic gallstone disease can pose a significant health and societal burden, impacting physical well-being, and an individual’s ability to work [2]. Despite the clear benefits of surgery and the potential economic savings resulting from preventing further morbidity from gallstones, laparoscopic cholecystectomy is not without risks, some of these being associated with a significant reduction in quality of life [3, 4].

Patient-reported outcomes (PROs) are used as a subjective measure for a patient’s health-related quality of life (HRQoL), at a set point in time, and from the patient’s perspective [5]. The use of validated questionnaires or patient-reported outcome measures (PROMs) has been mandatory practice in the UK National Health Service’s (NHS) national PROMs programme since 2009 [6, 7]. HRQoL data collected for patients undergoing hip and knee replacement surgery, groin hernia repair, and varicose vein surgery have been used to enable providers, commissioners, and stakeholders to make informed changes to service delivery which can then be used to improve clinical outcomes following intervention or surgery [6, 8]. The collection of these data often involves postal surveys which are time-consuming and costly [9, 10]. However, recent studies, including a meta-analysis, have demonstrated no major differences or limitations in using electronically collected PROMs (ePROMs) compared with more traditional paper PROMs [10,11,12,13].

Given current deficiencies in the NHS budget, and a recent government drive to a paperless NHS, ePROMs offer a functional solution for the routine use of PROMs and incorporation into Digital Health Records [14, 15].

The aim of this feasibility study was to pilot the use of a digital platform to facilitate the routine collection of pre- and post-operative ePROMs in participants undergoing elective laparoscopic cholecystectomy to calculate a change in HRQoL. Secondary objectives were to validate the use of existing PROs for our population by appraising their methodological quality.

Methods

PRO selection

Several PROs relevant to patients undergoing laparoscopic cholecystectomy have previously been identified and appraised [16,17,18]. However, due to the commercial nature of many trademarked PROs and the cost of licensing, we were limited in the selection available for this study. We took guidance from existing literature on the reporting of HRQoL studies and selected both disease-specific and generic PROs and planned for PRO validation within the study [19, 20].

Due to the inclusion of two PROs, and to prevent question fatigue, the 6-item Otago gallstones condition-specific questionnaire (CSQ) [21] was identified as the disease-specific PRO, and the RAND 36-Item health survey - version 1.0 (SF36) [22], as the generic PRO. Each of the items in both the CSQ and SF36 has one of 5 potential answers based on a Likert scale. Each answer has an allocated score: for CSQ 0 = best health, and 100 = worst health [21], and for SF36 0 = worst health, and 100 = best health [22]. Further domain scores can also be calculated for each PRO by combining and averaging specific items.

Variables

Participant and hospital variables collected included details on age, sex, Charlson comorbidity index [23], body mass index (BMI), and total length of hospital stay. Data were also collected on 30-day outcomes, specifically post-operative complications, re-attendances and readmission to hospital, a return to theatre, and mortality.

For comparison with our local population, data were also collected on all other patients undergoing laparoscopic cholecystectomy in the study time who opted not to participate in our PROMs study.

Outcomes

The main outcome measures included generic and disease-specific quality of life. PROs were offered to participants preoperatively, and post-operatively at 30 days, 3 months, and 6 months. Domain scores were calculated for each completed PRO, with post-operative results compared directly with baseline (preoperative) results.

The validity of PROs as ePROs was assessed by appraising their methodological quality within the study [16, 24]. Response rates, reliability, internal consistency, floor and ceiling effects, and responsiveness were all assessed within the study sample. Cross-correlation between CSQ and SF36 was also performed.

Statistical analysis

Data were analysed using GraphPad Prism® version 8.3.0. (GraphPad Software LLC, San Diego, CA, USA). Nonparametric tests were used to compare differences between groups as results were not normally distributed. Fisher’s exact or Chi-square tests were used for categorical variables and the Mann–Whitney U test for continuous variables. Spearman’s r was taken as a measure of correlation and reliability between scales, and Cronbach’s alpha for internal consistency assessment. Differences were considered significant at p < 0.05.

Ethics and reporting

Study participants were recruited to participate in this study on referral to hospital for management of symptomatic gallstones. On receiving study information either postally or in person, each participant indicated their consent to participate digitally. The study was performed as part of a larger PhD project sponsored by Nottingham University Hospitals NHS Trust, through a collaboration with EIDO Healthcare Limited and the Royal College of Surgeons of England. The study proposal was appraised by the confidentiality advisory group: 16/CAG/0045, with public and patient involvement, and research ethics committee approval: 16/SW/0088. It was registered with ClinicalTrials.gov: NCT02810860. The study was conducted and reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for observational studies [25].

Results

Participant recruitment and study demographics

A total of 255 eligible patients who were invited to participate in the study and use our digital interface completed the preoperative PROMs surveys between 6 September 2016 and 16 April 2018. Only 200 of these went on to undergo a laparoscopic cholecystectomy. These participants completed their preoperative survey a median of 86.5 days (IQR: 51–141.3) before laparoscopic cholecystectomy. The remaining participants were managed non-operatively because of patient choice, incorrect initial diagnosis, or being medically unfit for surgery. Only a small proportion of the 200 participants who completed their preoperative PROMs questionnaire completed the post-operative surveys at 30 days, 3 months, and 6 months (30.5, 27.0, 19.0%, respectively) (Fig. 1).

Fig. 1
figure 1

Participant recruitment—STROBE flow chart

Full size image

None of the 255 participants who completed the preoperative survey asked to withdraw from the study. Most participants, who opted not to complete a post-operative survey, maintained contact with the research team via a support email and described the complexity of accessing our ePROMs interface as the main reason for non-participation.

Participant demographics are described in Table 1. Participants in the study group were statistically younger than our local population. There were no other significant differences between the two groups.

Table 1 Participant demographics
Full size table

30-Day outcomes

Thirty-day outcomes were collected for all participants (Table 2). Most attendances for complications [61.5% (16 of 26) 30-day complications] were managed in an ambulatory setting and did not require overnight admission. Admission for treatment other than surgery or radiological drainage included intravenous hydration or medications (antibiotics, anti-emetics, analgesia), wound management, or a period of observation.

Table 2 30-day post-operative outcomes
Full size table

Patient-reported outcome measures

Quality-of-life domain scores were calculated from item scores for both CSQ (Table 3) and SF36 (Table 4). Compared with baseline preoperative values, statistically significant improvements in quality of life were seen in most domains in both CSQ and SF36.

Table 3 Disease-specific quality-of-life scores (CSQ)
Full size table
Table 4 Generic quality-of-life scores (SF36)
Full size table

The change in HRQoL before and after surgery is mapped in Fig. 2. These radar plots demonstrate that all participants experienced an improvement in HRQoL, in all domains, between preoperative and post-operative time points as demonstrated by no crossover between blue (solid) and green (dashed) plot lines.

Fig. 2
figure 2

a Change in health measured by CSQ. Visual representation of change in disease-specific HRQoL between each time point. Preoperative scores are represented in green, and post-operative scores are represented in blue. The central point of the radar chart represents best possible health b Change in health measured by RAND SF36. Visual representation of change in generic HRQoL between each time point. Preoperative scores are represented in green, and post-operative scores are represented in blue. The outer most edge of the radar chart represents best possible health.

Full size image

Completion rates

The digital nature of our ePROMs platform meant that surveys were either completed in their entirety or not all and there were no missing items within individual returned surveys.

Reliability

Test–retest correlations were performed between post-operative groups using Spearman’s coefficient of correlation on PROMs taken at 30 days and 3 months, and at 3 and 6 months post-operatively, and showed moderate-to-strong reliability (Table 5).

Table 5 Assessment of reliability
Full size table

Internal consistency

Homogeneity within PROs or internal consistency was good or excellent (Table 6).

Table 6 Assessment of internal consistency
Full size table

Floor and ceiling effects

To measure the ability of the PROs to identify population variation, floor and ceiling effects were assessed in both HRQoL instruments. Whilst floor and ceiling effects were seen in both CSQ and SF36 domain measurements, there were few instances exceeding > 15% (Table 7).

Table 7 Assessment of floor and ceiling effects
Full size table

Responsiveness and effect sizes

The ability to capture statistically significant changes in HRQoL over time was assessed by calculating effect sizes (Table 8). Effect sizes were classified as < 0.3, 0.5, or > 0.8, as small, medium, or large. Scores for CSQ were converted to a positive scale much like SF36 for the purposes of analysis.

Table 8 Assessment of responsiveness
Full size table

Cross-correlation between CSQ and SF36

Assessment of cross-correlation between the disease-specific and generic PROs showed moderate-to-strong reliability across the domains of ‘Physical Functioning’, ‘Systemic Functioning’, ‘Emotional Functioning’, ‘Social Functioning’, and ‘Overall health’ (Supplementary Figs. 1–5 and Supplementary Table 1).

Discussion

This novel feasibility study demonstrates the potential benefits of using a digital platform to collect real-time quality-of-life information utilising ePROMs. Despite initial problems in implementing our digital platform, and difficulties with user access, the process was deemed acceptable by most participants. However, these difficulties did lead to a significant attrition rate in the completion of post-operative questionnaires and resultant reporting bias.

Participant recruitment

Participants described the remote process of initial contact and invitation to the study positively and were pleased to have a study helpline for the digital platform. Whilst a significant number of participants subsequently dropped out of the study, discussion with those who contacted the helpline described this as mainly due to difficulties with Website onboarding due to the multiple security measures which were requested by our local information governance team (email, secure password, two type-sensitive memorable words). Of the 80 participants who contacted the user helpline, 64 described difficulties with either site registration or log-in (requiring support on between 1 and 6 occasions).

Participants subsequently recruited and retained in the study were found to be younger and have fewer comorbidities than those from a similar cohort that underwent laparoscopic cholecystectomy during the same time frame but out-with the study. This generational difference in our study group was in keeping with other studies utilising ePROMs [13].

All other study demographics demonstrated no significant differences between the two groups, indicating that the study group was representative of our local population.

Disease-specific PROMs

Statistically significant improvements in disease-specific HRQoL were seen across all survey items, and all quality-of-life domains following laparoscopic cholecystectomy when compared with baseline preoperative values. Marked changes with over twofold improvements in quality-of-life scores were evident at 30 days post-operatively in all domains.

Continued improvement was seen in emotional functioning and social functioning at 6 months post-operatively, whereas physical functioning, systemic functioning, and overall health either peaked or plateaued at 3 months post-operatively. Post-operative HRQoL did not fall to preoperative levels in any of the domains demonstrating that the continued benefit laparoscopic cholecystectomy had on participants.

Generic PROMs

Post-operative improvements were seen in all domains for generic HRQoL measures after laparoscopic cholecystectomy. This finding was in keeping with other non-UK-based studies on HRQoL [26, 27]. Continued significant improvement in quality of life was seen at 30 days, 3 months, and 6 months for fatigue, social functioning, and pain, whereas significant sustained improvement was seen at 3 and 6 months for physical functioning, and role limitations due to physical health. Emotional well-being and role functioning due to emotional problems showed peak improvement at 3 months post-operatively, whereas general health measurement peaked at 30 days. As demonstrated with disease-specific PROMs and other historic studies, this confirms the continued benefit laparoscopic cholecystectomy had on participants [26, 27].

Validation assessments

Despite a poor overall return rate of surveys post-operatively, all returned surveys were 100% complete. This all or nothing process is more frequent with digital surveys where the ability to skip questions can be removed, or users can be prevented from progressing. Other than high response rates, the additional benefits are that missing or spurious data points do not need to be accounted for, and there are lower chances of imputation error [8]. This level of acceptability and ease of ePROM dissemination are in keeping with existing evidence which has shown similar benefits when compared to more traditional PROM dissemination methods [10, 28, 29].

Assessments to validate CSQ and SF36 for use as ePROs in our study population demonstrated strong reliability scores for CSQ across all domains, with good internal consistency scores. Comparatively, SF36 demonstrated strong reliability scores with good-to-excellent measures of internal consistency. These results were similar to those described by the original CSQ research group [21], and the Oxford PROM Group review [17], suggesting that the PROs were accurate in measuring what they set out to and that they were stable in their measurements across the different time points.

The low percentages for floor and ceiling effects in both CSQ and SF36 surveys, and the medium-to-large effect sizes, also confirmed sufficient scale variance and responsiveness suggesting suitability to continue using them as ePROs and measure the wide variability of change in HRQoL.

The moderate-to-strong cross-correlation between surveys in this study demonstrates that future platforms may be justified in offering patients a single HRQoL survey to reduce survey fatigue.

Strengths and weaknesses

The benefit of using a digital platform meant that the PROs were easily disseminated to participants for completion at the correct time points, with an easy system in place to facilitate reminders for non-responders. The digital modality also meant that participants had access to complete HRQoL surveys in real time at any time and place, and from any digital medium which allowed Internet access. This proved especially convenient for participants who had already returned to work.

Results from completed ePROMs were also immediately available with no concerns about managing missing results, imputation errors, or researcher bias which may occur when collecting PROMs by more traditional means [8].

Although the results from this study suggest significant improvements in quality of life following laparoscopic cholecystectomy in our patient population, the study is significantly underpowered to be able to draw any reliable conclusions given the limited number of patients who completed both pre- and post-operative questionnaires. It is not possible to assume that all participants who suffered complications completed post-operative surveys and, therefore, this reporting bias could have impacted the overall results.

Despite initial problems with study set-up and participant recruitment, a large proportion of participants still engaged actively with the study process with positive interactions and acceptability demonstrating a place for digital mediums in delivering ePROMs. Although the use of ePROMs removes many of the logistic and cost issues associated with more traditional paper or telephone versions, access to ePROMs systems needs to be more inclusive and usable for both patients and clinicians [5, 30, 31]. Whilst further developmental costs may be necessary, this is fundamental to improve survey response rates and reduce recruitment bias.

Future work and recommendations

This feasibility study has helped to provide the groundwork and real-world data necessary to plan for a powered randomised controlled trial across more than one region. Further studies should also aim to actively involve patient advisory groups in study planning to develop study protocols and help test any digital medium prior to study commencement. The inclusion of focus groups within new studies would also be beneficial in recording patient attitudes, in particular reasons for non-participation.