Background

The Abdominal Aortic Aneurysm (AAA) Quality Improvement Programme (AAAQIP) was introduced in 2009 with the explicit remit of halving national mortality following elective repair by 2014. Latest published outcomes show the dramatic effect the AAAQIP has had, with mortality figures superior to those initially targeted at 2.4 % (open and endovascular outcomes combined) (Potgieter et al. 2012). As care improves for patients undergoing surgery, mortality becomes less common and therefore becomes a less sensitive measure of differences in the quality of care (Patila et al. 2006). Morbidity following AAA repair is significantly higher than mortality, with an estimated prevalence of 28 % following open repair and 12 % following EVAR (Giles et al. 2010). Despite this, morbidity is not robustly captured in the AAAQIP outcome dataset. Introducing standardised morbidity reporting would represent a logical next step to ensuring transparency of outcomes, whilst providing a more sensitive measure for differences in quality of care.

Capturing postoperative complications is highly relevant given their strong association with adverse outcomes for both patient and institution including prolonged hospital stay, poor functional ability on hospital discharge, reduced life-expectancy, reduced quality of life and increased hospital resource utilisation (Khuri et al. 2005; Head et al. 2008). Despite this, there is no standardised tool used to reliably capture morbidity following AAA repair. Beyond being able to identify complications postoperatively, a robust recording system would also facilitate quality assurance of hospital care and determine the effects of new surgical interventions.

The Postoperative Morbidity Survey (POMS) is a nine-domain tool that prospectively describes in-hospital morbidity following major surgery (Bennett-Guerrero et al. 1999). The POMS is able to detect a range of complication severities—from minor to life threatening—and classify morbidity by organ system, thereby representing a particularly useful scoring system. The original research from which the POMS tool was derived consisted of a heterogeneous population of surgical patients, including a minor cohort undergoing vascular surgery. Subsequent studies demonstrate the POMS has good inter-rater reliability, high patient satisfaction, accurately describe all postoperative complications (Grocott et al. 2007; Davies et al. 2013) and is associated with length of stay (LOS) (Grocott et al. 2007; Davies et al. 2013; Wakeling et al. 2005; Ackland et al. 2007; Ackland et al. 2010; Snowden et al. 2010; Ackland et al. 2011; Sanders et al. 2011; Ausania et al. 2012; Jones et al. 2013; Moonesinghe et al. 2014; West et al. 2014; Lee et al. 2014).

To date, the POMS has not been specifically evaluated in vascular surgery patients. Moreover, the POMS has not been validated against the primary guiding principle for its design—that it should capture morbidity substantial enough to delay discharge from hospital (Bennett-Guerrero et al. 1999). Rather, previous studies have used POMS scores on an arbitrary day post-surgery, for example, day 3 or 5 (Grocott et al. 2007; Davies et al. 2013). The aim of the present study therefore was to validate the POMS following elective AAA repair, both open and endovascular (EVAR), using repeated daily measurement in each patient up to the point of discharge or competing event (death). Consistent with the key guiding principle of the POMS, the primary objective was to estimate the instantaneous relative risk of being discharged from hospital, at any time-point, with any POMS-recorded morbidity (POMS score 1–9) versus no morbidity. A secondary objective was to assess the POMS against the second guiding principle in its development; that it should be a simple and rapid patient-screening tool.

Methods

This study was a four-centre, prospective observational cohort study that recruited patients from the James Cook University Hospital, Middlesbrough; York Hospital; University Hospital of North Durham and the Northern General Hospital, Sheffield, between July 2012 and March 2013. All patients ≥18 years of age and with capacity to give informed consent, undergoing elective infrarenal AAA repair, were eligible for inclusion. Exclusion criteria were the following: inability to provide informed consent, surgery for suprarenal thoracic or non-elective aneurysms and non-operative management.

A study investigator approached consecutive patients with an information sheet at the preoperative assessment or vascular surgical clinics, and written informed consent was obtained on admission to hospital for the surgical procedure. The study was approved by the West Midlands—Coventry & Warwickshire Research Ethics Committee (reference 12/WM/0114) and locally in recruiting centres (South Tees Hospitals Research and Development Department, York Foundation Trust Research and Development Unit, County Durham and Darlington Centre for Clinical Research and Innovation and the Sheffield Teaching Hospitals Clinical Research Office).

Study procedure

Patients underwent preoperative assessment and multidisciplinary team discussion based on the AAAQIP care pathway as part of their usual care. Baseline data were collected from preoperative assessment prior to open or endovascular AAA repair. Anaesthetic and surgical techniques were used at the discretion of attending clinicians.

Following surgery, the POMS (Table 1) was recorded daily, by trained research staff, until hospital discharge or death. A modification was made a priori, to reflect changes in clinical practice away from intravenous opioid analgesics towards oral formulations (Pergolizzi et al. 2012). The medical notes were reviewed on a daily basis with any postoperative morbidity not identified by the POMS recorded as free text. Postoperative patient management was otherwise performed per routine clinical practice at the discretion of the attending clinical teams, who were blinded to the POMS data. If patients remained in hospital with no morbidity, the reason for continued admission was recorded as free text. Critical care and overall hospital length of stay were recorded in addition to in-hospital mortality.

Table 1 The Postoperative Morbidity Survey (Bennett-Guerrero et al. 1999)
Full size table

Sample size and statistical analysis

Sample size and power estimations were conducted using R® software (Qiu et al. 2012). We predicted that at least 60 patients could be recruited over 9 months. This sample size provides 85 % power to detect a clinically relevant hazard ratio of ~0.4 reflecting a substantially lower instantaneous relative risk of being discharged at any time with any POMS-recorded morbidity versus no morbidity (with 2P = 0.05, an expected discharge rate of 95 %, a predicted proportion of the sample with POMS morbidity at discharge of 25 % and a correlation coefficient of 0.1 between the primary exposure (POMS) and the other covariates in the model (see below).)

We analysed the data using a simple multistate Cox regression model to derive the cause-specific association between morbidity and hospital discharge (alive), accounting for the competing risk of in-hospital death (Beyersmann et al. 2012). For each day for each patient, ‘discharged’ (yes/no) was the event, with the elapsed number of days post-surgery as the time variable. The POMS score (yes = 1–9, no = zero) was a binary time-dependent internal covariate. Analysis was conducted using Stata® software (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP). The hazard ratio (95 % confidence interval) for the effect of any POMS morbidity vs. no morbidity was estimated, controlling for type of repair (open vs. EVAR), patient age and size of aneurysm. These three variables were selected, a priori, as each may be predicted to affect length of stay independent of POMS morbidity (Schouten et al. 2006; Lee et al. 2004). Sex was not included as a covariate as the sample was 95 % male. The hazard ratio reflects the instantaneous relative risk of alive discharge for a state of morbidity vs. no morbidity.

One patient was discharged 8 days post-surgery but due to administrative error had no POMS scores recorded after day 5. To include this patient’s event data in the analysis, we carried forward the day 5 morbidity (in the gastrointestinal domain) for days 6–8—a conservative approach in this context.

With respect to the validity of POMS as conceptualised in the current study, there are two types of ‘error’ versus the reference method of the discharge decision by the vascular team. First, a patient might attain a zero score for POMS on a particular day but not be discharged within 24 h of that time-point. Second, a patient might be discharged despite having recorded POMS morbidity. We present a summary of both of these categories in the ‘Results’ section.

We also conducted a sensitivity analysis to examine the effect that patients who were not discharged within 24 h of achieving a zero score had on the validity of the POMS (degree of attenuation of the observed hazard ratio). To conduct this analysis, we first manually edited the data to reflect a scenario in which all patients who reached a zero for POMS had been discharged immediately (i.e. within 24 h).

Results

Participant flow

Out of the 104 potential participants screened for inclusion, 81 met the inclusion criteria and were approached by an investigator, of which 64 were recruited into the study. Reasons for non-enrolment were non-operative management (n = 8), surgery performed outside recruitment window (n = 5), refusal (n = 3) and cognitive dysfunction precluding informed consent (n = 1). All 64 patients completed follow-up and were included in the analysis.

Sample characteristics

Patient and perioperative characteristics are described in Table 2. Prevalence of daily presence of POMS morbidity and the count of the number of domains affected are presented in Table 3. Data are presented up to day 6, when approximately 50 % of the overall sample had been discharged. One patient (1.6 % of the total sample) died on the ninth postoperative day after an open AAA repair from multi-organ failure. One patient had postoperative limb ischaemia suspected clinically, resulting in further investigation, but did not meet any of the existing POMS domain criteria.

Table 2 Patient and perioperative characteristics
Full size table
Table 3 Prevalence of any POMS-recorded morbidity (yes/no) and POMS count for those patients remaining in hospital at that time-point
Full size table

Validity of the POMS

Compliance with POMS completion was 99.5 %, with just three missing days for one patient from a total of 608 possible observations (see ‘Methods’). The hazard ratio for discharge with any POMS-recorded morbidity versus no morbidity was 0.130 (95 % confidence interval 0.070 to 0.243). The median time-to-discharge was 13 days after recording any POMS morbidity vs. 2 days after scoring zero for POMS morbidity.

Twenty-nine patients remained in hospital beyond 24 h of first attaining a zero score for POMS. For 10 of these patients, nothing was documented in the patient notes that might shed light on reasons for any delay. For the other 19 patients, the notes suggested that 4 were awaiting the results of diagnostic tests, 7 had evidence of social/non-medical reasons that delayed discharge, 3 remained in hospital for medical reasons, and 5 developed morbidity very shortly after first reaching zero POMS morbidity: 2 with an infective complication, 2 with pain, and 1 with atrial fibrillation. The sensitivity analysis assuming that all of these 29 patients had been discharged when they first reached a zero score for POMS revealed a hazard ratio for morbidity vs. no morbidity of 0.058 (0.030 to 0.114).

Nineteen patients were discharged with a POMS score of >0 (morbidity in one or more POMS domains). The majority of these (n = 17) had recorded morbidity in a single POMS domain: 10 for Infectious, 4 for Pain, and one each for Gastrointestinal, Cardiovascular and Pulmonary. Two patients had recorded morbidity in two POMS domains—Cardiovascular plus Pain and Infectious plus Pain.

Discussion

Validity

This is the first study to validate the POMS specifically in patients undergoing abdominal aortic aneurysm repair. The observed hazard ratio reveals that the chances of being discharged by usual clinical decision-making at any time-point with any POMS-recorded morbidity are approximately 1/8 those of being discharged in the absence of POMS morbidity—a very large effect size. There was also a very large difference for the median time-to-discharge in the morbidity vs. no morbidity states. Taken together, these findings provide strong evidence for the validity of the POMS against its primary guiding principle—to only capture morbidity that is of a type and severity to delay discharge from hospital. Following abdominal aortic aneurysm repair the POMS therefore has good, albeit not perfect, validity.

The factor primarily responsible for the non-perfect validity of the POMS in this context is the 19 patients who were discharged with a POMS score >0. The majority of these patients had POMS-recorded morbidity in a single domain—most frequently ‘Infectious’, followed by ‘Pain’. If these 19 patients had instead not been discharged (censored data, i.e. ‘event’ did not occur) then the hazard ratio for morbidity vs. no morbidity would have been essentially zero, as the POMS morbidity covariate at the point of discharge would have been constant across the sample. The sensitivity analysis revealed that the 29 patients who were not discharged within 24 h of first reaching a POMS score of zero attenuated the observed hazard ratio by >50 % (0.130 vs. 0.058). This implies that if all 29 had in fact been discharged within 24 h of first reaching zero, the chances of being discharged with a POMS score >0 compared to a POMS score =0 would decrease significantly from 1/8 (the value we observed) to 1/17. This difference can be explained by imperfections in the healthcare system (delayed recognition of recovery from perioperative morbidity) or by the presence of factors preventing discharge that were not identified by either the POMS or the patient notes—see below for a critique of the POMS.

With respect to delayed discharge, hospital length of stay is affected by social and organisational factors unrelated to the patients’ physiological recovery from surgery (Bennett-Guerrero et al. 1999; Grocott et al. 2007). Grocott et al. reported that 161 out of 200 patients (81 %) remained in hospital with no identifiable morbidity following major surgery (Grocott et al. 2007). In contrast, the current study found fewer than half of the patients stayed in hospital beyond the day on which POMS first identified zero morbidity. This disparity could be due to differences between the study samples, differing discharge practices between institutions (de Jong et al. 2006) or the emphasis on throughput and efficiency in the NHS in recent years (Paton et al. 2014), resulting in more prompt discharge once postoperative morbidity has resolved.

Potential use

Our strategy of collecting daily POMS count facilitates objective ‘morbidity tracking’ and discharge planning for individual patients in the postoperative period. Considering the former, progressive improvements in POMS count can be used as an indicator of patient recovery leading towards appropriately targeted discharge planning once no morbidity is recorded. Conversely, an upward trend in POMS count should trigger appropriate senior level review and reconsideration of ‘fitness for discharge’. In this respect, we propose POMS to be an adjunct, rather than replacement, for other regularly recorded physiological scoring systems, e.g. National Early Warning Score (NEWS) (Royal College of Physicians of London 2012). The organ-specific structure of the POMS may guide clinicians making diagnostic and treatment decisions in the postoperative period, over and above the non-specific information from NEWS. Using two complementary systems enables early identification of postoperative complications, when patients can be more easily rescued. This could be achieved without undue time-burden on attending healthcare professionals as we expand on below.

The concept of enhanced discharge decision-making is particularly pertinent when considering the aforementioned pressures to increase efficiency and decrease hospital length of stay. This situation creates a significant risk of discharging patients before they are truly ready, resulting in increased readmissions (Kaboli et al. 2012) and resultant financial penalties for NHS trusts (Department of Health 2013). The objective nature of POMS provides a safeguard against this particularly with daily tracking. Standardising the level of recovery at which patients are discharged through use of the POMS could therefore prevent both premature and late discharge. In addition, the objective nature enables nurse initiated discharge and reduction in burden on senior medical staff.

Contrary to widespread belief, we believe that it is indeed permissible to sum POMS domains into a total score. The misconception is based on the low–moderate internal consistency reported for the nine POMS domains, which has led researchers into believing that the items may not be summed into a unidimensional scale (Grocott et al. 2007; Davies et al. 2013; Moonesinghe et al. 2014). However, the POMS is clearly composed of so-called ‘causal indicators’ rather than ‘effect indicators’ (Streiner 2003). With scales composed of effect indicators, the items are manifestations of an underlying construct. For example, in a scale consisting of multiple items to measure the construct of depression, scores for all items would be expected to increase or decrease together if depression got worse or better, respectively. For such scales, therefore, unidimensionality (evaluated via confirmatory factor analysis) and high internal consistency are indeed important. However, for indexes composed of causal indicators, where the items taken together define the construct—in this case postoperative morbidity—internal consistency is not relevant. Indeed, the low–moderate correlation between the POMS domains is a direct consequence of its design, focusing on multiple physiological systems. In short, these domains are sometimes largely independent (e.g. ‘Pain’ and ‘Renal’) and other times correlated (e.g. ‘Wound’ and ‘Infectious’). This differentiation between a scale and an index is critical; with due caution, the nine domains of the POMS may be summed into an index to reflect the magnitude of postoperative morbidity. This index should be validated in larger studies prior to further use.

Critique of the POMS

A guiding principle in the development of the POMS was that data collection should be low-burden. Feedback from the research team confirmed this to be true and compliance with POMS completion was excellent in the current study. Indeed, the simplicity of the POMS is one of its core strengths. It has previously been shown to have good reliability even when used by researchers previously unfamiliar with its use (Davies et al. 2013). With definitions that are understandable and simple to answer by the clinical team caring for patients, it could easily be incorporated into routine postoperative monitoring. We chose to keep the existing structure of POMS to enable this goal. In comparison, the recently developed and validated cardiac-POMS (Sanders et al. 2012) changed and added definitions to develop a 13-domain survey suited to patients recovering from cardiac surgery. However, our data do suggest one change to the POMS to make it suitable for patients undergoing vascular surgery: the addition of limb ischaemia to the cardiovascular domain definition.

The POMS may not capture all relevant reasons why patients remain in hospital following surgery. General deconditioning and reduced mobility can require a period of rehabilitation, delaying hospital discharge without falling into one of the POMS domains. In the study by Grocott et al. (Grocott et al. 2007), 41 out of 200 patients remained in hospital on day 8 due to postoperative mobility problems. No formal assessment of reduced mobility was used in that study or in ours. Had one been used, it may have revealed some of the unexplained delayed discharges. The newly developed cardiac-POMS added a separate domain for assisted ambulation (Sanders et al. 2012). Impaired mobility can arguably be considered as postoperative morbidity that is currently unrecognised by the POMS and could be addressed in future research with a similar modification. However, any such change would have to be carefully evaluated, as adding further domains would increase the complexity of the tool, making it more burdensome to use.

Several alternative instruments have been developed to measure morbidity following surgery, although many have not gained widespread popularity (Clavien et al. 1992; Pomposelli et al. 1997; Gawande et al. 1999; Pillai et al. 1999; Dindo et al. 2004; Slankamenac et al. 2013; Strasberg et al. 2009). The Clavien-Dindo classification of postoperative complications (Dindo et al. 2004) is perhaps the most commonly used alternative system. It retrospectively classifies complications by type of treatment required to manage them. Some of the problems encountered with the system include as follows difficulties in categorising some cases (Clavien et al. 2009), differing morbidity may be classified similarly and the same complication may be classified differently depending on the treatment given (Rassweiler et al. 2012)—although this criticism can also be levelled at the POMS. It is unclear whether the original description and validation study of the Clavien-Dindo classification (Dindo et al. 2004) or its subsequent analysis (Clavien et al. 2009) specifically assessed its use in vascular surgery. Although it has subsequently been used in several publications in the vascular literature (Pol et al. 2011; Desai et al. 2011; Arya et al. 2015; Visser et al. 2014; Patel et al. 2015; Gunawansa et al. 2011), it was not used by the AAAQIP (AAAQIP 2011). The National Surgical Quality Improvement Program has been widely adopted in the USA, but the definitions of the postoperative complications are not publically available (https://www.facs.org/quality-programs/acs-nsqip). A high-profile pilot scheme is soon to be launched across the UK in a combined venture between the Health Services Research Centre and National Institute of Academic Anaesthesia. The Perioperative Quality Improvement Programme aims to capture postoperative complications, mortality and patient reported outcome in a structured fashion utilising using a range of validated instruments (Moonesinghe and Grocott 2015). This venture should provide further much needed clarity around the optimal strategy for recording postoperative complications.

Strengths and limitations

The key strength of the current study is that it is the first to validate the POMS according to its primary guiding design principle—to only capture morbidity substantial enough to delay discharge from hospital. This required careful blinding of the clinical teams to the POMS scores and daily POMS recording up until the point of discharge or competing event (a range of over 3 months). A limitation is the relatively small sample. However, this sample size, together with multiple measures per patient, afforded adequate precision to estimate the effect of any morbidity versus no morbidity. Our design is statistically very efficient, as the expected event rate (discharge) is very high (>95 %) which results in a smaller sample size requirement versus studies in which the outcome occurs less frequently. Moreover, our sample is representative of this patient population, with age and length of stay for EVAR and open repair patients equivalent to national data (Potgieter et al. 2012). Furthermore, the multi-centre design of our study adds to its external validity. The institutions include those traditionally described as both teaching hospitals and district general hospitals, reflecting current practice in the UK.

Another limitation is the lack of observation following hospital discharge. Nineteen patients were discharged with POMS morbidity recorded. Given the experience of previous researchers, where patients remained in hospital with no postoperative morbidity (Grocott et al. 2007), we did not consider the alternative situation when planning the study. It may have been informative to have followed these participants after hospital discharge to see whether they had an increased rate of re-admission or use of primary care.

From a wider-reaching perspective, robust postoperative morbidity capture is highly relevant. The development of early postoperative complications is of greater prognostic significance than preoperative comorbidity in determining long-term survival (Khuri et al. 2005). In this setting, although the POMS describes only in-hospital morbidity, a recent study demonstrated its association with increased mortality for up to 2 years following surgery (Moonesinghe et al. 2014). In addition, objective postoperative morbidity data would add to the existing mortality and length of stay information used to benchmark institutions comparatively. Furthermore, daily POMS recording as part of perioperative care pathways might highlight areas for improvement and be used to audit any resultant change in practice. The POMS could therefore act as an informative tool both locally and nationally to drive quality improvement and institution benchmarking.

Conclusions

This study has demonstrated the POMS to be a valid tool to measure short-term postoperative morbidity in patients undergoing elective infrarenal AAA repair. The data suggest a minor modification to ensure complete capture of morbidity. Daily POMS measurement is recommended to fully capture morbidity and allow robust analysis. The survey could be a valuable outcome measure for use in quality improvement programmes and future research, or as an adjunct to clinical decision-making in the postoperative period.