Background

Accurate prognostic predictors are crucial for patients admitted to an intensive care unit (ICU). Prognostic scoring systems are useful for clinical management such as predicting a survival rate, making decisions, and facilitating explanation of disease severity, by clinical physicians. Patients with cirrhosis admitted to an ICU frequently have disappointing outcomes despite intensive medical support, and these patients are particular targets for prognostic evaluation.

Various systems for scoring severity and predicting prognosis have been developed and applied for decades. The Acute Physiology and Chronic Health Evaluation III (APACHE III) score [1], one of the widely used scoring systems, is known for its accuracy in predicting mortality. However, the APACHE III scoring system was initially developed for various diseases and not exclusively for liver-related diseases. By contrast, the Sequential Organ Failure Assessment (SOFA) score [2], another widely used scoring system, is superior to the APACHE III scoring system for assessing specific organ dysfunction including cirrhosis [3, 4]. Our previous study demonstrated that the APACHE III and SOFA scores were both independently associated with a hospital mortality rate and demonstrated high discriminatory power for predicting mortality in patients with cirrhosis [5]. However, few studies have performed detailed independent comparisons between APACHE III and SOFA scores [6]. In the present case-control study, we matched APACHE III and SOFA scores and compared the different clinical characteristics and outcomes of patients with cirrhosis with those of their matched noncirrhotic controls.

Methods

Study design

The Chang Gung Medical Foundation Institutional Review Board approved the present study and waived the need for informed consent, because patient privacy was not breached during the study, and the study did not interfere with clinical decisions related to patient care (approval No. 98-3658A3). All data in our study were anonymised. This retrospective case-control study was conducted in a tertiary-care hospital. The enrolled patients were recruited from a database of critically ill patients admitted to medical ICUs between January 2006 and December 2009. For the APACHE III-matched group (174 patients), each patient with cirrhosis was matched 1:1 to a control patient without cirrhosis by using the criteria of APACHE III ± 3 points. For the SOFA-matched group (110 patients), each patient with cirrhosis was matched 1:1 to a control patient without cirrhosis by using the criteria of SOFA ± 1 point [7, 8]. The outcomes of interest were the length of stay in an ICU, length of stay in a hospital, and hospital mortality rate.

Study population and data collection

All patients admitted to medical ICUs between January 2006 and December 2009 with APACHE III and SOFA scores available were eligible for inclusion. Exclusion criteria were age < 18 years, length of stay in a hospital or an ICU of < 24 hours, patients with chronic uraemia and undergoing renal replacement therapy, and hospital readmission. Data were recorded regarding patient demographics, reason for ICU admission, clinical and laboratory variables, APACHE III and SOFA scores, the risk of renal failure, injury to the kidney, failure of kidney function, loss of kidney function, end-stage renal failure (RIFLE) classification [9], the length of stay in an ICU and a hospital, and hospital mortality. Data on the length of stay included those from patients with hospital mortality.

Definitions

Cirrhosis was diagnosed based on liver histology or a combination of physical presentation, biochemical data, and ultrasonographic findings. Illness severity was assessed according to the APACHE III and the SOFA scores, which were defined and calculated as described previously [1, 2]. Acute kidney injury (AKI) was defined using the RIFLE criteria, and patients were scored as RIFLE-R or higher severity. Baseline serum creatinine (SCr) concentration was the first value measured during hospitalisation. The Modification of Diet in Renal Disease formula was used to estimate baseline SCr concentration in patients whose previous SCr concentration was unavailable. The criteria resulting in the most severe RIFLE classification were used [9]. A simple model for assessing mortality was developed as follows: non-AKI (0 points), RIFLE-R (1 point), RIFLE-I (2 points), and RIFLE-F (3 points) on Day 1 of ICU admission [10, 11].

The lowest physiological and biochemical values on Day 1 of ICU admission were recorded. In sedated or paralysed patients, neurological scoring was not performed and was not classified as neurological failure. In patients who were intubated but not sedated, the best verbal response was determined according to clinical judgment.

Statistical analysis

Descriptive statistical results were expressed as mean ± standard error (SE). In primary analysis, the patients with cirrhosis were compared with the patients without cirrhosis. All variables were tested for normal distribution using the Kolmogorov–Smirnov test. Student’s t-test was used to compare the means of continuous variables and normally distributed data, whereas the Mann–Whitney U test was used for all other comparisons. Categorical data were tested using the χ2 test or Fisher’s exact test.

Calibration was assessed using the Hosmer–Lemeshow goodness-of-fit test (C statistic) to compare the number of observed and predicted deaths in various risk groups for the entire range of death probabilities. Discrimination was assessed by determining area under the receiver operating characteristic curve (AUROC). Areas under 2 receiver operating characteristic curves were compared by applying a nonparametric approach. The AUROC analysis was also conducted to estimate the cut-off values, sensitivity, specificity, overall correctness, and positive and negative predictive values. Finally, cut-off points were calculated by determining the best Youden’s index (sensitivity + specificity −1).

Cumulative survival curves over time were generated by applying the Kaplan–Meier approach and compared using the log rank test. All statistical tests were 2-tailed; P < .05 was considered statistically significant. Data were analysed using SPSS Version 13.0 for Windows (SPSS, Inc., Chicago, IL, USA).

Results

Patient characteristics

A total of 336 critically ill patients admitted to the medical ICU between January 2006 and December 2009 were enrolled in the present study. Table 1 lists the reasons for ICU admission. The demographic data, clinical characteristics, and outcomes of the 2 score-matched groups are depicted in Tables 2 and 3, respectively.

Table 1 Reasons for ICU admission
Full size table
Table 2 APACHE III-matched patient demographic data and clinical characteristics according to cirrhosis and non-cirrhosis
Full size table
Table 3 SOFA-matched patient demographic data and clinical characteristics according to cirrhosis and non-cirrhosis
Full size table

In the APACHE III-matched group (Table 2), the critically ill patients with cirrhosis had a lower arterial HCO3 − level (P = .002), a lower haemoglobin level (P = .024), a lower platelet count (P < .001), and a higher serum bilirubin level (P < .001) on Day 1 of ICU admission compared with the patients without cirrhosis. The patients without cirrhosis with the same score were older and experienced more shock episodes than the patients with cirrhosis did. The hospital mortality rate was significantly higher in the patients with cirrhosis than in the patients without cirrhosis (P = .026). Renal function and a PaO2/FiO2 ratio were similar between the 2 groups. Among APACHE III-matched patients, the patients with cirrhosis had significantly higher SOFA scores than those of the patients without cirrhosis (P < .001).

In the SOFA-matched group (Table 3), the patients with cirrhosis had a lower platelet count (P < .001), a higher Glasgow coma scale (GCS), a more stable haemodynamic status, more favourable renal function, a higher bilirubin level, and a lower albumin level compared with the patients without cirrhosis. The patients without cirrhosis experienced more shock episodes than did the patients with cirrhosis. No significant difference in the hospital mortality rate was observed between the patients with and without cirrhosis.

In both the APACHE III and SOFA-matched groups, the patients with cirrhosis had a shorter overall length of stay in an ICU than that of the patients without cirrhosis. This was attributed to the higher hospital mortality rate in the patients with cirrhosis. In both the APACHE III and SOFA-matched groups, the patients without cirrhosis had a significantly higher rate of shock than the patients with cirrhosis did. This was because of the main reason that ICU admission of the patients with cirrhosis was mainly due to hepatic failure; the reasons for ICU admission of the patients without cirrhosis were GI bleeding and sepsis. (Data not shown here)

Mortality and severity of illness scoring systems

Prediction abilities of the APACHE III, SOFA, and RIFLE scoring systems were compared; Table 4 lists the calibration and discrimination of the models. In the APACHE III-matched group, the SOFA scoring system demonstrated the highest prediction ability (AUROC = 0.810 ± 0.056) among all 3 systems. All the 3 scoring systems predicted mortality more precisely in the patients with cirrhosis than in those without cirrhosis. In the SOFA-matched group, the APACHE III scoring system was the most accurate predictor among all 3 systems. In the patients with cirrhosis, the SOFA scoring system demonstrated the highest prediction ability. To determine the selected cut-off points for predicting in-hospital mortality, the sensitivity, specificity, and overall accuracy of prediction were determined (Table 5). In the APACHE III-matched group, the SOFA scoring system had the best Youden’s index among the patients with cirrhosis, whereas the APACHE III scoring system had the best Youden’s index among the total population. In the SOFA-matched group, the APACHE III scoring system had the best Youden’s index among both the patients with cirrhosis and the total population. The RIFLE scoring system demonstrated the highest specificity for prognostic prediction in the patients with cirrhosis of both the SOFA-matched and APACHE III-matched groups.

Table 4 Calibration and discrimination for the scoring methods in predicting hospital mortality
Full size table
Table 5 Subsequent hospital mortality predicted after ICU admission
Full size table

Figure 1A and B show the cumulative survival rates in the patients with and without cirrhosis in the APACHE III-matched and SOFA-matched groups, respectively. The cumulative survival rates showed that patients with cirrhosis in the APACHE III-matched group had significantly higher mortality rates than the patients without cirrhosis did, whereas no significant difference was detected between the patients with and without cirrhosis in the SOFA-matched group. In both the APACHE III-matched and SOFA-matched groups, the cumulative survival rates significantly differed when an underlying AKI was considered (Figure 2A and B).

Figure 1
figure 1

The cumulative survival rates for cirrhotic and non-cirrhotic patients in the APACHE III-matched (1A, 174 patients, p < 0.05) and SOFA-matched (1B, 110 patients, p -value: not significant) groups, respectively.

Full size image
Figure 2
figure 2

The cumulative survival rates for acute kidney injury (AKI) and non-AKI patients in the APACHE III-matched (2A, 174 patients, p < 0.05) and SOFA-matched (2B, 110 patients, p < 0.05) groups, respectively.

Full size image

Discussion

Based on our research, this is the first study to compare the usefulness of different scoring systems for outcome prediction in patients admitted to an ICU with and without cirrhosis by using a score-matched method. In the present study, several clinical characteristics and outcomes of critically ill patients with cirrhosis were compared with those of critically ill patients without cirrhosis with matched APACHE III or SOFA scores. The poor outcome of patients with cirrhosis was consistent with the results of previous studies [12–14]. Recent studies have supported the efficacy of the SOFA scoring system for assessing the extent of organ dysfunction in various groups including critically ill patients with cirrhosis [3–5].

The APACHE III score consisted of the acute physiology score, age, and chronic health problem scores, which are widely used for predicting clinical outcomes currently. Among the patients with cirrhosis, bilirubin levels and incidences of cirrhosis were higher than those among the patients without cirrhosis. Results of the present study suggested that among patients with the same APACHE III score, patients with cirrhosis were older with a higher risk of acidaemia and shock than those without cirrhosis. However, renal function and consciousness status were nonsignificantly different between the 2 groups. The requirement for mechanical ventilation was not included in the APACHE III score, and no difference in the PaO2/FiO2 ratio was observed.

Although the SOFA score includes a fewer number of items and does not assess age and comorbid conditions, it enhances its simplicity and demonstrates high discriminatory power for predicting critically ill patients with cirrhosis. The patients with cirrhosis in the SOFA-matched group showed higher GCS and lower SCr concentrations on Day 1 of ICU admission, whereas these differences were nonsignificant in the APACHE III-matched group. Because patients with cirrhosis typically have higher serum bilirubin levels and lower platelet counts (caused by de novo liver disease), which contribute to a higher SOFA score, than their counterparts do, the patients with cirrhosis with similar SOFA scores may have relatively improved characteristics in the other 5 organic fields. This phenomenon is consistent with our findings showing that the patients with cirrhosis had a more stable neurologic status and renal function than did the patients without cirrhosis in the SOFA-matched group.

APACHE III, a prognostic model, predicts mortality. Prognostic scoring models such as APACHE III assume that mortality is affected by physiological disturbances that occur early in the course of illness, whereas organ dysfunction-scoring systems such as SOFA allow determination of organ dysfunction at the time of admission and at regular intervals throughout the stay in an ICU, thus allowing for the assessment of changes in organ function. The SOFA score is an organ dysfunction score that quantifies the burden of organ dysfunction. Although the SOFA score was originally used to describe morbidity, it was also used in mortality prediction. The accuracy of mortality predictions may be improved with repeated measurements by using organ dysfunction scoring systems such as SOFA.

As shown in Table 4, in the APACHE III-matched group, the SOFA score demonstrated a higher discrimination ability in the patients with cirrhosis than in the patients without cirrhosis (AUROC = 0.810 ± 0.056 vs 0.624 ± 0.060). The SOFA score is simpler for assessment than the APACHE III score by clinicians. Meanwhile, the SOFA score allows for sequential measurements and more accurately reflects the dynamic aspects of disease processes and may provide information of higher quality on the mortality risk. Therefore, the SOFA score is a superior and easier-to-implement model for predicting mortality in the patients with cirrhosis, with a cut-off value of 10 points, providing the optimal overall correctness. In addition, both the SOFA and APACHE III scores are comparable in the patients without cirrhosis.

When the patients with and without cirrhosis were matched by using SOFA scores, difference in APACHE III scores between the 2 groups was nonsignificant in the present study. The RIFLE scoring system, however, showed superior results in patients with cirrhosis to those of the noncirrhotic controls (P = .041) in the SOFA-matched group. The patients with cirrhosis tend to have malnutrition, low muscle mass, and impaired synthesis of creatinine. Therefore, the RIFLE scoring system, which is based on the serum creatinine and urine output, may lead to underestimation of AKI severity and overall illness.

Despite the encouraging results of the present study, several potential limitations should be considered. First, this was a retrospective study performed at a single tertiary-care medical centre, which limits generalisation of the findings. Second, the patients without cirrhosis mainly composed of patients with sepsis and those with a low proportion of patients with other diseases such as cardiovascular disease or acute respiratory distress syndrome. The specificity of the subgroup of the patients with cirrhosis may limit the generalisation. Third, the patient population comprised a high proportion of patients with hepatitis B virus infection. Therefore, this study has limited applicability to typical North American and European patients with hepatitis C virus infection or those with alcohol dependence. Finally, the sample size was insufficient for matching SOFA and APACHE III scores among the patients with and without cirrhosis. Therefore, we cannot draw definitive conclusions regarding the relatively poor short-term prognosis of the patients admitted to the ICU with cirrhosis compared with that of the patients admitted to the ICU without cirrhosis.

Conclusions

Our results provide additional evidence that SOFA scores differ significantly between patients with and without cirrhosis matched according to APACHE III scores. The score-matched analytical data showed that the predictive accuracy of SOFA is superior to that of APACHE III in evaluating critically ill patients with cirrhosis. We also demonstrated that the mean arterial pressure, GCS, and RIFLE classification play critical roles in determining prognosis in this subset of patients. When considering cost-effectiveness and ease of implementation, the SOFA scale is recommended for evaluating short-term prognosis in critically ill patients with cirrhosis.