Introduction

The burden of Emergency Surgery (ES) disease is substantial and has been steadily increasing in the USA over the past few decades. In the USA alone, ES-related conditions accounted for more than 27 million admissions between 2001 and 2010 [1]. Almost a third of these patients required one or more operations during the hospital admission [1]. ES has been consistently shown to be an independent risk factor for perioperative morbidity and mortality even when controlling for patient comorbidities, surgical procedure type, and the facility type at which care is provided [2–6].

Assessing the inherent risk of postoperative morbidity and mortality in ES is crucial for appropriate counseling of the patient in extremis needing surgical intervention. In addition, adequate risk adjustment is indispensable for any efforts aimed at benchmarking and improving the quality of care of ES. Despite the widespread consensus that patients undergoing ES are at greater risk of morbidity and mortality, most risk-adjusting studies have exclusively focused on patient demographics, comorbidities, the underlying disease or condition, and the nature of the surgical procedure needed, with little attention given to the impact of the acuity of disease at presentation (as manifested, for example, by physiological derangements) on patient outcomes. In comparison, many physiological derangements have been acknowledged as markers of the severity of condition in several medical and intensive care scoring systems [7–12]. Risk calculators such as the Universal American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) Surgical Risk Calculator and the procedure specific calculators such as the pancreatectomy or colorectal surgery risk calculators describe mortality and morbidity predictions for patients undergoing surgery based on preoperative patient demographics, comorbidities, and procedure type [13–15]. The American Society of Anesthesiologists (ASA) classification and the Surgical Risk Scale (SRS), additional examples of risk-scoring systems, all predict outcomes for surgical patients without accounting for the possible physiologic abnormalities present at the time of preoperative evaluation, a limitation of these tools especially for ES patients [16, 17]. The commonly utilized Elixhauser and Charlson Comorbidity Indices rely on administrative data to assess patient comorbidities using ICD-9 and ICD-10 diagnosis codes, rather than clinical markers of a patient’s condition [18, 19].

Our research group has recently developed and validated a comprehensive 22 demographic, comorbidity and laboratory variables risk calculator, the Emergency Surgery Acuity Score (ESAS), that helps predict a priori the risk of postoperative mortality in patients undergoing ES [14]. However, an objective system to specifically assess the acuity of the emergent condition and its resultant physiological derangements, based on clinical data, independently from comorbidities and demographics, is still lacking.

In this manuscript, we sought to derive and validate a user-friendly tool that specifically measures the acuity of an ES patient’s condition by assessing the physiological derangements at the time of initial presentation, the Physiological Emergency Surgery Acuity Score (PESAS).

Methods

To derive PESAS and measure its ability to predict postoperative mortality, we utilized the ACS-NSQIP 2011 database. To validate PESAS, we used the 2012 ACS-NSQIP database. The ACS-NSQIP is a prospective database that collects over 150 preoperative and intraoperative clinical variables, as well as 30-day postoperative morbidity and mortality outcomes for patients undergoing both inpatient and outpatient surgical procedures. The ACS-NSQIP structure and methodology (i.e., data collection, sampling, variables collected, outcomes tracked, analyses performed) have been well described and repeatedly validated in the surgical literature [20–23].

Patient population

For each of the derivation and validation phases, all patients undergoing ES as defined by the ACS-NSQIP variable “Emergency case” were identified. ACS-NSQIP defines “Emergency case” as one that is “performed as soon as possible and no later than 12 h after the patient has been admitted to the hospital or after the onset of related preoperative symptomatology” [21]. Cases simultaneously categorized as “Elective Surgery” were excluded.

Preoperative variables

All preoperative variables collected in the ACS-NSQIP data fields were identified, including demographics (e.g., age, race, gender), comorbidities (e.g., chronic obstructive pulmonary disease, hypertension, ascites), functional status, and preoperative laboratory variables (e.g., sodium, albumin, and white blood cell (WBC) count). Demographic factors and comorbid conditions were dichotomized using ACS-NSQIP definitions of normal and abnormal. Laboratory values were divided into low, normal, and high (where applicable) using clinically relevant cut-offs. White blood cell count (WBC) was further divided into ≤4.5, >4.5 and ≤11, >11 & ≤15, >15 & ≤25, and >25 × 103/mm3 based on clinical significance as has been previously described [11]. A few other variables were slightly restructured. For example, albumin was defined as low if <3.0 U/L. Missing data were coded as missing and not imputed. Only variables with at least 55% capture in the emergency surgery subset of patients were used for statistical analyses. Every patient included in the score derivation therefore had each of the variables recorded. Variables that incorporated multiple preoperative characteristics such as ASA classification and sepsis were excluded to prevent collinearity and erroneous inclusion of the variables included in these broader categories because their contribution may have already been accounted for in the broader variables (for example, WBC count and sepsis).

Outcomes

The primary outcome of interest assessed was 30-day mortality using the ACS-NSQIP variable “YRDEATH.” This variable reports whether the patient is alive 30 days postoperatively regardless of admission status.

Derivation of the score

The 2011 ACS-NSQIP dataset was used to derive PESAS, using the following methodology:

First, univariate analyses of 30-day mortality were performed using all available preoperative variables (e.g., patient demographics, comorbidities, preoperative laboratory variables). Second, variables with a p value of <0.2 were included in the multiple logistic regression models created to identify independent predictors of 30-day mortality in ES. Third, stepwise logistic regression models were constructed to identify independent predictors for 30-day mortality. Both forward and backward regressions were performed with a p-value of <0.05 as the cutoff for statistical significance. Fourth, stepwise logistic regressions models were reconstructed for 30-day mortality excluding all identified comorbidities/demographical variables while retaining markers of acuity, defined as physiologic derangements occurring in acute illness representing end-organ dysfunction (e.g., laboratory variables, evidence of acute organ failure). Fifth, based on the relative impact of each identified predictor (i.e., odds ratio), using weighted averages, a novel score was derived. The coefficients, i.e., odds ratio (OR) were divided by the lowest common denominator and rounded off to the nearest integer to develop a score that would be easy to utilize. Sixth, the receiver operating curve (ROC) was examined to measure the score’s c-statistic. The c-statistic has been used as a measure of model success in multiple score development efforts [11, 15, 24, 25].

Validation of the score

The coefficients of the score derived from the 2011 data were applied to the 2012 data for validation. The score c-statistic was measured again, thus assessing its ability to predict mortality at 30 days in a different independent dataset.

Statistical analyses

All data analyses were performed in STATA version 13.1, as described above.

Institutional Review Board (IRB)

IRB approval is waived for ACS-NSQIP studies utilizing the de-identified national database at our institution.

Results

Derivation of the score

Of a total of 280,801 NSQIP patients in the 2011 dataset, 24,702 ES patients were included in our analyses, including 2272 patients who died within 30 postoperative days (9.2%). More than half the patients were women, 53% were 60 years or older, and 70% were white. The detailed demographics, comorbidities, and laboratory characteristics of our derivation cohort are included in Table 1.

Table 1 Emergency surgery (ES) patient characteristics in the ACS-NSQIP 2011 (score development) and ACS-NSQIP 2012 (score validation) datasets
Full size table

Initial multiple logistic regression analyses identified 22 independent predictors of mortality. These included three demographic variables, 10 comorbidities, and nine laboratory variables. After exclusion of demographic and comorbidity variables, 10 variables were determined to be independent markers of acuity of the patient’s condition at the time of presentation (Table 2). These variables represent physiologic derangements that reveal end-organ dysfunction, for example, elevated creatinine, which represents renal dysfunction, and elevated bilirubin, which represents hepatic dysfunction. Based on the relative impact of these 10 predictors, recalculated in a second round of multiple logistic regression analyses, and using weighted averages, PESAS score was derived with a range between 0 and 15 points (Table 3). Multiple iterations of the score to simplify the coefficients yielded unchanged ROCs.

Table 2 Multivariate analysis with odds ratios for the 10 acuity predictors of mortality in emergency surgery patients in the 2011 ACS-NSQIP derivation dataset
Full size table
Table 3 Development of the physiologic emergency surgery acuity score using regression coefficients of the acuity variables significant for mortality
Full size table

The c-statistic of PESAS for mortality was measured at 0.80 with a 95% confidence interval of 0.7923–0.8121. The observed probability of 30-day mortality gradually and consistently increased from 1.7% at a score of 0 to 40.6% at a score of 8 and 100% at a score of 15 (Fig. 1).

Fig. 1
figure 1

PESAS: observed mortality rates per PESAS score points

Full size image

Validation of the score

The 2012 validation dataset included 26,412 patients, with an overall mortality rate of 7.7%. The detailed characteristics of the validation cohort patients are included in Table 1. The c-statistic of PESAS was unchanged at 0.79 when applied to the 2012 ACS-NSQIP dataset (Fig. 2). The 95% confidence interval for the 2012 validation ROC (0.7801–0.8025) included the c-statistic of PESAS in the derivation cohort. The ROC curves were computed for both 2011 and 2012 datasets.

Fig. 2
figure 2

Comparison of receiver operating characteristic (ROC) curves using the development (a) and the validation (b) datasets. a ACS NSQIP 2011 physiologic emergency surgery acuity score (PESAS) (0.80); and b ACS-NSQIP 2012 dataset (0.79)

Full size image

Discussion

We have therefore derived and validated a novel tool that specifically assesses the degree of physiological derangement and the acuity of the patient’s disease at the time of presentation for ES, the Physiological Emergency Surgery Acuity Score or PESAS. This abbreviated score complements the more comprehensive score previously described by our group for estimating mortality for the ES patient in that it remains statistically rigorous for approximating mortality for the ES patient but is more user-friendly for the clinician at the bedside [26]. We believe that PESAS is: (1) user-friendly, (2) robust (high ROC), and (3) predicts mortality in a stepwise progression.

In previous studies, ES patients have been consistently shown to have worse outcomes compared to patients undergoing non-emergency general surgery even when controlling for preoperative variables and procedure type [2, 3]. A large retrospective analysis of the ACS-NSQIP database from 2008–2012 revealed that the overall mortality was sixfold higher for patients undergoing emergency versus non-emergency general surgery [2]. The difference between ES and non-ES decreases but does not disappear when accounting for preoperative patient variables such as age, race, and gender. The acuity of disease at presentation, the inability to optimize preoperative status before surgery, and the inherent time sensitivity implying an immediate, imminent and real risk of complications with or without surgery all potentially account for this increased risk of death following ES. PESAS aims specifically to measure these less tangible factors, most importantly the physiological derangements resulting from the acuity of disease that dictate the urgent nature of the surgical intervention. PESAS can therefore serve as a valuable adjunct to administrative data derived comorbidity calculations such as the Elixhauser and Charlson indices.

Despite the elimination of demographic and comorbidity variables from the models and the score that we derived, PESAS retained a high discriminatory power for ES patients, compared to other, generic surgical risk assessment tools [2, 27]. Such discriminatory power demonstrates that the physiological derangements of ES patients are important predictors of mortality independent of demographic factors such as age and comorbid conditions.

Previously existing tools such as Universal ACS-NSQIP Surgical Risk Calculator, Surgical Risk Score (SRS), Physiological and Operative Severity Score for the enumeration of Mortality and Morbidity (POSSUM), and the Portsmouth predictor equation (p-POSSUM) all provide valuable information for patients undergoing operations. However, they have significant limitations when applied to the ES patient. For example, the SRS was validated on a cohort with a very low observed mortality compared to the ES population, and the Universal ACS-NSQIP Surgical Risk Calculator allows for surgeon-based crude and subjective adjustment to the calculated risk, based on the best judgment of the individual surgeon [13, 16]. The POSSUM has been shown to over-predict mortality by at least twofold and is quite complex to use [16, 28, 29]. The p-POSSUM requires determination of intraoperative variables, and therefore cannot be utilized preoperatively [29]. The ESAS, introduced recently by our research group, is a comprehensive risk calculator tool that predicts mortality very well in ES patients, but is complex and necessitates the assessment of 22 variables, and hence is more suitable for benchmarking and quality assessment. PESAS, on the other hand, measures acuity of presentation and is more friendly for bedside use, or in situations when a patient is unable to provide their medical history [26]. Besides ESAS, none of the scores mentioned above has been designed with the ES patient in mind, nor have they been tested rigorously in this high-risk patient population. From a statistical performance perspective, the PESAS has a higher or comparable c-statistic at 0.80 to that of widely accepted medical and surgical scoring systems [12–14, 16, 28–34]. The only large study evaluating the use of these general existing scoring systems in ES patients was a single institution study of 2349 patients and found similar or worse c-statistic performance [35].

The need for risk adjustment and grading models specifically for ES is highlighted by the current efforts to develop such benchmarking tools [36–38]. The currently existing risk adjustment systems offer disease specific grades, require a combination of clinical, radiographic, endoscopic, and pathologic findings to determine a grade, and do not account for patient preoperative physiologic derangements or indicators of disease acuity [36–39]. We therefore offer PESAS not only as a validated acuity assessment tool, but also as one that accurately predicts postoperative mortality progressively across its range of scores, based on easily available preoperative and objectively measured physiological markers. At the bedside, this can facilitate counseling and informed consent preoperatively for the ES patient or family [40].

From an institutional or national benchmarking perspective, PESAS will allow to differentiate between two theoretical patients that have the same age, gender, and comorbidities, who present with the same diagnosis (e.g., perforated diverticulitis), but with different levels of physiological derangements or acuity of disease. Such differentiation is crucial for fair benchmarking and to avoid penalizing the acute care surgeon that takes care of the sickest patients.

Our study has a few limitations. First, while the ACS-NSQIP is a large database with rich clinical information, the ES subset from the 2011 dataset contained a few variables that had lower than 55% capture. Those variables were excluded from our multivariate regressions, but perhaps may be significant if they were better captured and included. Second, there is a small loss of discrimination of PESAS compared to ESAS as a result of eliminating the demographic and comorbid conditions that are also independent predictors of mortality for the ES patients. Third, the tool does not, at this time, assess for morbidity. Assessing the ability of PESAS to predict major morbidity and complications is our research team’s clear next step.

Conclusion

We have therefore developed and validated a novel score, PESAS, that accurately predicts postoperative mortality in ES patients based on physiologic derangements at the time of presentation. Such a score could prove immensely useful for: (1) preoperative patient and family counseling; (2) identification of patients needing close postoperative monitoring; and (3) risk adjustment in any efforts aimed at benchmarking the quality of ES. Further, it is user-friendly and relies solely on objective and easily obtainable clinical data.