Journal of Anesthesia

, Volume 27, Issue 4, pp 541–549 | Cite as

An in-hospital mortality equation for mechanically ventilated patients in intensive care units

  • Takeshi Umegaki
  • Masaji Nishimura
  • Kimitaka Tajimi
  • Kiyohide Fushimi
  • Hiroshi Ikai
  • Yuichi Imanaka
Original Article

Abstract

Objective

To develop an equation model of in-hospital mortality for mechanically ventilated patients in adult intensive care using administrative data for the purpose of retrospective performance comparison among intensive care units (ICUs).

Design

Two models were developed using the split-half method, in which one test dataset and two validation datasets were used to develop and validate the prediction model, respectively. Nine candidate variables (demographics: age; gender; clinical factors hospital admission course; primary diagnosis; reason for ICU entry; Charlson score; number of organ failures; procedures and therapies administered at any time during ICU admission: renal replacement therapy; pressors/vasoconstrictors) were used for developing the equation model.

Setting

In acute-care teaching hospitals in Japan: 282 ICUs in 2008, 310 ICUs in 2009, and 364 ICUs in 2010.

Participants

Mechanically ventilated adult patients discharged from an ICU from July 1 to December 31 in 2008, 2009, and 2010. Main Outcome Measures: The test dataset consisted of 5,807 patients in 2008, and the validation datasets consisted of 10,610 patients in 2009 and 7,576 patients in 2010. Two models were developed: Model 1 (using independent variables of demographics and clinical factors), Model 2 (using procedures and therapies administered at any time during ICU admission in addition to the variables in Model 1). Using the test dataset, 8 variables (except for gender) were included in multiple logistic regression analysis with in-hospital mortality as the dependent variable, and the mortality prediction equation was constructed. Coefficients from the equation were then tested in the validation model.

Results

Hosmer–Lemeshow χ2 are values for the test dataset in Model 1 and Model 2, and were 11.9 (P = 0.15) and 15.6 (P = 0.05), respectively; C-statistics for the test dataset in Model 1and Model 2 were 0.70 and 0.78, respectively. In-hospital mortality prediction for the validation datasets showed low and moderate accuracy in Model 1 and Model 2, respectively.

Conclusions

Model 2 may potentially serve as an alternative model for predicting mortality in mechanically ventilated patients, who have so far required physiological data for the accurate prediction of outcomes. Model 2 may facilitate the comparative evaluation of in-hospital mortality in multicenter analyses based on administrative data for mechanically ventilated patients.

Keywords

Mechanical ventilation In-hospital mortality Prediction model Intensive care units 

Introduction

Growing concerns about the quality of care and patient safety have increased the importance of monitoring intensive care units (ICUs) in health care organizations. In response to increasing demands to improve the quality of care, performance measures for intensive care have been developed [1, 2, 3]. As patient mortality is a major health care outcome, many studies have included this measure as a quality outcome indicator. However, mortality rates vary among ICUs due to differences in patient case mix and disease severity [4, 5, 6]. Several risk-adjustment models [1] have been developed to compare ICU mortality rates among institutions; these models include the Acute Physiology and Chronic Health Evaluation (APACHE) system, the Mortality Prediction Model (MPM), and the Simplified Acute Physiology Score (SAPS). Additionally, Render et al. [7] have proposed an automated ICU risk-adjustment tool, and the Critical Care Outcome Prediction Equation (COPE) model was developed as a hospital mortality prediction model using only administrative data [8].

There is an increasing demand for performance measurement in ICU benchmarking in health care organizations. Several indicators for ICU performance have been developed [9, 10, 11], which include mechanical ventilation-associated indicators such as the prevention of ventilator-associated pneumonia, protocol-driven ventilator weaning, daily sedation interruption policy, low tidal volume ventilation in acute lung injury/adult respiratory distress syndrome (ALI/ARDS), ventilator-associated pneumonia rate, average days on mechanical ventilation, and mortality (crude and severity-adjusted). Therefore, mechanically ventilated patients have been shown to be an important target for ICU performance evaluation.

The Diagnosis Procedure Combination (DPC) system in Japan was introduced in 2004, and has since become the standard method of payment in the health care financial system. Administrative data in this system include records of patient information and daily medical care. From these data, the types of all tests, medications, and procedures, as well as the use of intensive or special care and nursing services, are itemized on a daily basis. Procedures such as mechanical ventilation, renal replacement therapy, and the use of vasoactive agents are available from DPC data, and have been reported to be closely associated with mortality [11, 12, 13, 14, 15, 16]. However, the utilization rates and patterns of these procedures vary among intensivists, and the inclusion of data concerning these procedures may therefore support the accurate prediction of mortality.

The aims of this study were to use administrative data to develop an in-hospital mortality equation that includes patient demographics and clinical factors, and procedures administered during the ICU admission as independent variables; and to examine the viability of the equation in conducting retrospective evaluations of ICUs.

Methods

Data sources and case selection criteria

All data for the study were extracted from the Japanese DPC database, which was collected by the DPC Research Group. Data were obtained from July 1st to December 31st for each of the 3 years from 2008 to 2010. Of the participant hospitals that comprise the database, we included 437 acute care teaching hospitals with ICUs (including surgical ICUs, medical ICUs, and surgical-medical ICUs); we obtained data from 282 ICUs in 2008, 310 ICUs in 2009, and 364 ICUs in 2010. The initial study population included all patients aged ≥20 years treated in an ICU at any of the sample hospitals. We identified the time of ICU entry and the dates of ICU stay based on specific codes in the administrative data. The data did not indicate whether a patient had been previously hospitalized in another ICU, but critical care patients are rarely transferred from one center to another in Japan. We therefore assumed that patients entering the ICU had not been transferred from another ICU.

Because the focus of this analysis was to develop accurate mortality prediction equations on mechanically ventilated patients undergoing critical care interventions, we analyzed patients who required mechanical ventilation ≥2 days after ICU entry, which was identified from the corresponding codes. Non-invasive positive pressure ventilation was not included in the analysis.

Development of the prediction model and potential risk factors

We utilized a split-half approach to prediction model development, using data from 2008 as the test dataset and data from 2009 and 2010 as the validation datasets. The primary measure used was in-hospital mortality. The in-hospital mortality prediction equation was constructed using the test dataset and evaluated using the validation datasets. Coefficients obtained from the test dataset were applied to cases in the validation datasets to calculate the predicted mortality.

Model development was based on up to 9 variables (Table 1). Age was used as a continuous variable. To determine the reason for ICU entry, patients who underwent surgery on the first ICU day were considered to be surgical cases. In these cases, patients who had both emergency hospital admission and underwent surgery on the day of hospital admission or the following day were defined as emergency surgical cases, whereas those who did not undergo emergency surgery were defined as elective surgical cases. All other patients were considered to be medical cases. To define admission categories, items in the administrative database pertaining to the course of admission were used. The emergency admission category indicates hospital admission after transport by ambulance or an unexpected admission. Organ failure was identified according to the study conducted by Angus et al. [17]. The DPC system in Japan utilizes the International Classification of Diseases, 10th Revision (ICD-10) coding, rather than ICD-9 CM or ICD-10 AM, and ICD-10 AM codes were therefore translated to ICD-10 codes for identification of primary diagnosis and organ failure (Table 2). The Charlson score is a clinical comorbidity index that predicts the 10-year survival of a patient based on a range of comorbid conditions (e.g., heart disease, cancer, and liver disease). The score can be calculated from ICD codes available from administrative data, and increasing scores have been shown to have strong associations with mortality [18, 19].
Table 1

Candidate variables used to develop the in-hospital mortality prediction equation

Type

Candidate variables

Categories

Demographics

(1) Age (years)

Continuous variable

(2) Gender

Male, female

Clinical factors

(3) Hospital admission course

Scheduleda, emergency

(4) Primary diagnosis on admission

See Table 2

(5) Reason for entering ICU

After elective surgerya, after emergency surgery, medical disease

(6) Charlson score

0a, 1, 2, 3, ≥4

(7) Number of organ failures (except for respiratory failure)

0a, ≥1

Procedures administered at any time during ICU admission

(8) Renal replacement therapy

Yes = 1, No = 0

(9) Pressor/vasoconstrictor

Yes = 1, No = 0

aReference category for hospital mortality prediction

Table 2

Recoding of ICD-10 AM and ICD-9 CM codes to ICD-10 codes

Diagnostic category

ICD-10 AM

ICD-10

Diagnostic category

ICD-9 CM

ICD-10

Hemopoietic malignancy

C80–99

C81–96

Cardiovascular dysfunction

  

Penetrating trauma

T15–19

T15–19

Shock without trauma

785.5

A419

Other central nervous system disease

G9

G9

  

A483

Cardiac arrest

I46

I46

  

R570

Aplastic anemia

D6

D60–61

  

R571

Protozoal sepsis

B50–64

B50–64

  

R578

Hemorrhagic shock

R57–58

R57–58

  

R579

Secondary malignancy

C76–79

C76–80

Hypotension

458

I959

Stroke or cerebrovascular accident

I63–64

I63–64

Neurologic dysfunction

  

Interstitial lung disease

J8

J8

Encephalopathy

348.3

F058

Liver disease

K7

K7

  

G934

Bacterial sepsis

A4

A4

  

G938

Lung malignancy

C3

C3

  

I672

Intracranial hemorrhage

I60–62

I60–62

  

I674

Anemia

D5

D5

  

I678

Central nervous system malignancy

C69–72

C69–72

   

Pulmonary vascular

I26–28

I26–28

  

K729

Fungal sepsis

B30–49

B35–49

  

K868

Renal failure

N1

N17–N19

  

G948

Ischemic bowel

K55

K55

Transient organic psychosis

293

F069

Gastrointestinal investigation

R1

R1

Anoxic brain damage

348.1

G931

Environmental disease

T66–79

T66–78

Hematologic dysfunction

  

Breast cancer

C5

C5

Secondary thrombocytopenia

287.4

D695

Malignancy, other

D37–49

D37–48

Thrombocytopenia, unspecified

287.5

D696

Pneumonia

J1

J12–18

Other/unspecified coagulation defect

286.9

D65

Pneumoconiosis

J60–79

J60–J70

Defibrination syndrome

286.6

D65

Head injury

S0

S0

Hepatic dysfunction

  

Pancreatic cancer

C22–26

C25

Acute and subacute necrosis of liver

570

K729

Type 2 diabetes

E11

E11

Hepatic infarction

573.4

K763

Cardiac arrhythmias

I49

I47–49

Renal dysfunction

  

Fluid and electrolyte disorders

E86–88

E86–88

Acute renal failure

584

N179

Enteritis or colitis

K50–52

K50–52

   

Other intestinal disease

K63

K63

   

Respiratory failure

J95–99

J96

   

Lower limb trauma

S7

S7

   

Other cerebrovascular disease

I65–69

I65–69

   

Chronic obstructive pulmonary disease

J40–44

J40–44

   

Malabsorption

K9

K90

   

Drug poisoning

T36–50

T36–50

   

Epilepsy

G4

G40

   

Cardiac failure

I22–25

I50

   

Myocardial ischemia

I20

I20–25

   

All other diagnosesa

     

aReference category for hospital mortality prediction

Renal replacement therapy and pressors/vasoconstrictors were included in the candidate variables due to their reported association with 28-day hospital mortality in a previous study [20]. Renal replacement therapy included continuous renal replacement therapy, intermittent renal replacement therapy, plasma absorption, and plasma exchange, but excluded peritoneal dialysis due to its rare utilization for ICU patients. Pressors/vasoconstrictors included dopamine, dobutamine, norepinephrine, epinephrine, and vasopressin, but excluded the use of epinephrine in cardiopulmonary resuscitation. We were unable to distinguish whether dopamine was given as a low (renal) dose or for cardiovascular support, but found no evidence to support the possibility that low-dose dopamine was used [21]. We therefore assumed that dopamine was used for cardiovascular support.

Relationships between the individual candidate variables and in-hospital mortality were analyzed with χ2 tests or Student t tests using the test dataset, depending on the type of data. After exclusion of variables with P > 0.25, the remaining variables were subjected to multiple logistic regression analyses (stepwise backward selection method). The model was constructed using variables with P < 0.05, and the C-statistics were calculated. Two models were developed: Model 1 included independent variables of demographics and clinical factors, and Model 2 included independent variables of procedures and therapies administered at any time during ICU admission (in addition to the variables in Model 1).

Prediction model performance

Calibration of the model was evaluated using the Hosmer–Lemeshow χ2 test. A well-calibrated model has a low χ2 value (<15.5; df = 8) and a high P value (>0.05). The discrimination of the model was assessed by the C-statistics, for which a value 0.9–1.0 was determined to represent high accuracy, 0.7–0.9 moderate accuracy, and 0.5–0.7 low accuracy.

Prediction model validation

The mortality prediction equation was cross-validated using the validation datasets to demonstrate the predictive validity of the prediction equation obtained from multiple logistic regression analysis of the test dataset. Coefficients derived from analysis of the test dataset were applied to the validation dataset in order to calculate predicted mortality. The performance of the equation was tested using the C-statistics [95 % confidence interval (CI)] for the validation datasets from 2009 and 2010. All statistical analyses were performed using SPSS 18.0J (SPSS Inc., Chicago, IL, USA).

Results

The final samples for analysis from 2008, 2009 and 2010 comprised the following: the numbers of hospitals per year were 282, 310 and 364, respectively; numbers of ICU patients were 38,625, 71,243 and 49,230, respectively; numbers of ventilated patients were 5,807, 10,610, and 7,576, respectively. The incidence of mechanically ventilated patients in ICU entry was approximately 15 % for all 3 years. The details of patient characteristics for the sample are shown in Table 3. Preliminary univariate analyses showed significant differences between survivors and non-survivors in all patient characteristics except for gender in the three study years. In 2008 (test dataset), intracranial haemorrhage was the most frequent condition in the diagnostic category for non-survivors (10.4 %), while myocardial ischemia was the most frequent condition for survivors (13.2 %). All patient characteristics (excluding gender) were thus included as independent variables in the consequent multiple logistic analysis.
Table 3

Patient characteristics in 2008, 2009, and 2010

 

2008

2009

2010

 

Survivors (n = 4,002)

Non-survivors (n = 1,805)

P value

Survivors (n = 7,056)

Non-survivors (n = 3,554)

P value

Survivors (n = 5,234)

Non-survivors (n = 2,342)

P value

(1) Age (years)

68.0 (13.6)

71.8 (12.8)

<0.01**

68.8 (13.7)

72.5 (13.3)

<0.01**

67.9 (14.2)

72.0 (13.0)

<0.01**

(2) Gender (male)

62.5

62.9

0.76

61.6

62.6

0.29

62.9

62.7

0.91

(3) Hospital admission course (emergency)

64.8

84.3

<0.01**

68.2

84.7

<0.01**

57.5

72.3

<0.01**

(4) Primary diagnosis on admission

       

 Hemopoietic malignancy

0.2

1.2

<0.01**

0.3

1.8

<0.01**

0.2

1.5

<0.01**

 Penetrating trauma

0.1

0.1

0.1

0.1

0

0.1

 Other central nervous system disease

0.6

1.6

0.4

1.1

0.3

1.3

 Cardiac arrest

0

0

0

0

0

0

 Aplastic anemia

0

0.1

0

0.1

0

0.1

 Protozoal sepsis

0

0.1

0

0

0.1

0.2

 Hemorrhagic shock

0.3

0.2

0.3

0.6

0.2

0.3

 Secondary malignancy

0.6

0.7

0.4

0.4

0.5

0.7

 Stroke or cerebrovascular accident

1.5

2.5

1.5

2.6

1.5

2.4

 Interstitial lung disease

1.2

2.4

1.2

3.6

1.1

3.2

 Liver disease

0.1

0.3

0.2

0.8

0.2

0.6

 Bacterial sepsis

1.5

2.9

1.7

2.9

1.5

3.5

 Lung malignancy

2.9

1.7

0.9

1.5

1.1

1.4

 Intracranial hemorrhage

5.1

10.4

5.6

9.8

5.4

8.5

 Anemia

0

0.1

0.1

0.1

0.1

0.1

 Central nervous system malignancy

0.1

0.2

0.1

0.1

0.2

0.3

 Pulmonary vascular

0.3

0.2

0.4

0.5

0.6

0.3

 Fungal sepsis

0

0.1

0.1

0.2

0

0.3

 Renal failure

0.9

1.3

0.8

1.3

0.6

1.7

 Ischemic bowel

0.3

0.7

0.3

0.4

0.2

0.3

 Gastrointestinal investigation

0.2

0.3

0.2

0.2

0.1

0.3

 Environmental disease

0.2

0.5

0.1

0.3

0.5

0.6

 Breast cancer

0.2

0.4

0.3

0.2

0.3

0.3

 Malignancy-other

0.4

0.6

0.4

0.5

0.7

0.7

 Pneumonia

2.4

4.8

2.5

4.8

2.4

4.5

 Pneumoconiosis

1

2.3

1.5

2.1

1.4

2

 Head injury

1.4

3

1.3

2.1

1.3

2.6

 Pancreatic cancer

0.2

0.1

0.4

0.4

0.3

0.6

 Type 2 diabetes

0.1

0.1

0.2

0.2

0.1

0.2

 Cardiac arrhythmias

0.8

1.1

0.8

0.8

1

0.9

 Fluid and electrolyte disorders

0.2

0.3

0.2

0.5

0.4

0.5

 Enteritis or colitis

0

0.2

0.1

0.1

0.2

0

 Other intestinal disease

0.7

0.7

0.8

1

0.8

0.7

 Respiratory failure

1.6

2.9

2

2.6

1.9

2.2

 Lower limb trauma

0.3

0

0.5

0.7

0.4

0.6

 Other cerebrovascular disease

1.2

0.4

1.1

0.3

1.3

0.5

 Chronic obstructive pulmonary disease

0.4

0.3

0.4

0.3

0.6

0.5

 Malabsorption

0

0

0

0

0

0

 Drug poisoning

0.8

0.1

0.8

0.1

0.9

0

 Epilepsy

0.3

0.1

0.4

0.1

0.4

0.1

 Cardiac failure

9.1

6.1

9.9

6.7

10.1

6.2

 Myocardial ischemia

13.2

8.6

12.5

7.4

12.6

6.7

(5) Reason for entering ICU

        

 After emergency surgery

19.9

21.6

<0.01**

20

20.5

<0.01**

18.2

50.8

<0.01**

 Internal medical disease

53.7

72

55.2

72.9

18.4

71.4

(6) Charlson score

         

 1

29.2

26

<0.01**

31

26.9

<0.01**

31.4

25

<0.01**

 2

15.8

15.9

17.6

18.1

17.3

17.4

 3

6.4

9.1

7.6

8.7

7.7

9.9

 ≥4

3

4.9

3.6

4.9

3.6

6.6

(7) Number of organ failures (except for respiratory failure) ≥1

28.5

47.1

<0.01**

32.8

51.9

<0.01**

32.4

55.7

<0.01**

(8) Renal replacement therapy

10.9

28.1

<0.01**

10.3

28.6

<0.01**

10.3

32.4

<0.01**

(9) Pressors/vasoconstrictors

73.2

88.3

<0.01**

71.4

88.9

<0.01**

70

88.1

<0.01**

Continuous variables presented as mean (SD); categorical variables presented as percentage

** P < 0.01

Coefficients of the variables, odds ratios (OR), and 95 % CIs are shown in Table 4. Factors associated with a high risk of death in Models 1 and 2 were hemopoietic malignancy (OR = 4.92, 95 % CI = 2.15–11.24) and lung malignancy (OR = 6.00, 95 % CI = 3.41–10.55), respectively. The Hosmer–Lemeshow χ2 values (P values) in Models 1 and 2 were 11.9 (0.15) and 15.6 (0.05), respectively; the models’ C-statistics (95 % CI) in the test dataset were 0.70 (0.69–0.72) and 0.78 (0.77–0.79), respectively (Table 5). Applying the final equation to the validation dataset showed similar discrimination when compared with that of the test dataset.
Table 4

Variable coefficients used in the hospital mortality prediction models

Variable

Model 1

Model 2

B

OR (95 % CI)

B

OR (95 % CI)

(1) Age (years)

0.02

1.02 (1.02–1.03)

0.02

1.02 (1.02–1.03)

(2) Hospital admission course

    

 Emergency

0.33

1.39 (1.12–1.72)

0.48

1.61 (1.28–2.01)

(3) Primary diagnosis on admission

    

 Hemopoietic malignancy

1.59

4.92 (2.15–11.24)

1.78

5.90 (2.51–13.90)

 Other central nervous system disease

0.65

1.91 (1.08–3.36)

0.99

2.69 (1.49–4.85)

 Stroke or cerebrovascular accident

  

0.81

2.25 (1.46–3.45)

 Interstitial lung disease

  

0.77

2.16 (1.37–3.40)

 Bacterial sepsis

    

 Lung malignancy

0.96

2.62 (1.49–4.58)

1.79

6.00 (3.41–10.55)

 Intracranial hemorrhage

0.42

1.53 (1.20–1.95)

1.53

4.64 (3.61–5.96)

 Pneumonia

  

0.63

1.88 (1.34–2.64)

 Pneumoconiosis

  

0.84

2.32 (1.43–3.78)

 Head injury

0.46

1.58 (1.06–2.37)

1.45

4.26 (2.79–6.52)

 Respiratory failure

  

0.69

1.99 (1.32–2.99)

 Drug poisoning

−1.99

0.14 (0.03–0.58)

  

 Cardiac failure

−1.06

0.35 (0.27–0.44)

-0.65

0.52 (0.41–0.67)

 Myocardial ischemia

−0.59

0.55 (0.44–0.69)

-0.36

0.70 (0.56–0.86)

(4) Reason for entering ICU

    

 After emergency surgery

1.02

2.77 (2.03–3.79)

0.71

2.04 (1.47–2.83)

 Medical disease

1.32

3.76 (2.86–4.94)

1.25

3.50 (2.63–4.66)

(5) Charlson score

    

 3

0.46

1.58 (1.26–1.98)

0.36

1.44 (1.13–1.83)

 ≥4

0.61

1.84 (1.35–2.50)

0.44

1.56 (1.12–2.16)

(6) Number of organ failures (except for respiratory failure) ≥1 (%)

  

0.63

1.88 (1.64–2.15)

(7) Renal replacement therapy (%)

  

1.05

2.86 (2.42–3.36)

(8) Pressors/vasoconstrictors (%)

  

1.27

3.55 (2.95–4.26)

Constant

−3.49

 

−5.41

 

Predicted mortality risk = ey/(ey + 1), where y = B(1) × (1) + B(2) × (2) + B(3) × (3) + B(4) × (4) + B(5) × (5) + B(6) × (6) + B(7) × (7) + B(8) × (8) + constant. (1), (2), (3), (4), (5), (6), (7), and (8) = 1 if variables are applicable and 0 if variables are not applicable

B β coefficient, OR odds ratio, CI confidence interval

Table 5

Model discrimination for the prediction equation in test and validation datasets

Model

Dataset

C statistics

95 % CI

Model 1

2008 (test)

0.70

0.69–0.72

2009 (validation)

0.69

0.68–0.70

2010 (validation)

0.70

0.69–0.71

Model 2

2008 (test)

0.78

0.77–0.79

2009 (validation)

0.78

0.77–0.79

2010 (validation)

0.79

0.78–0.80

Discussion

In this study, we developed and validated in-hospital mortality prediction equations in mechanically ventilated patients using data from Japan; we showed moderate discrimination in a model using patient demographics, clinical factors, and procedures administered during the ICU admission (Model 2). Mechanically ventilated patients in the ICU are frequently the subjects of epidemiological studies [22, 23, 24]. The risk-adjustment methods previously employed in these studies frequently include APACHE, MPM and SAPS [25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36], but these methods are primarily dependent on organ scores that require physiological data. Ohno-Machado et al. [25] obtained C-statistics for mortality prediction models using APACHE-II, APACHE-III, MPM0, MPM24, MPM-II0, MPM-II24, SAPS, and SAPS-II; and found that these all had C-statistics ≥0.8, except for SAPS. In contrast to these models, Duke et al. [8] derived the COPE model using solely administrative data. This model has advantages in that it can predict mortality with relatively few variables from routinely-available data. The COPE model also includes mechanical ventilation as an intensive care therapy which has been shown to be strongly associated with hospital mortality. Therefore, for an analysis focusing on mechanically ventilated patients using DPC data, there was a need to develop a new robust mortality prediction tool that did not include the independent variable of mechanical ventilation. In addition, because these models were developed specifically using Japanese data, and because of the ability of the DPC database to identify the details of medical care for mechanically ventilated patients in a uniform format from numerous hospitals, this approach has the capacity to support comparative evaluations of ICU performance using multicenter analysis in Japan.

Model 2 from our analysis may serve as a possible alternative model due to displaying moderate accuracy in the C-statistics. If the prognoses of mechanically ventilated patients are to be required, the use of existing scoring systems using physiological data may be more useful. But the intended applications of this equation (Model 2) lie in the retrospective evaluations of ICU performance in a multicenter analysis based on the identical format of DPC data introduced in 2004 in Japan.

The prediction equation in this study has the following advantages over existing models: the variables used in our equation utilize information that can be routinely obtained from administrative data. These variables are submitted by doctors and nurses in a timely manner on a daily basis rather than at or after discharge, which ostensibly improves the reliability of the data. Also, the model uses only 8 variables, which facilitates its generalizability and application. However, there is also the risk of coding errors, especially in ICU patients [8].

There are several limitations in the present study. First, we did not compare our model with scoring systems using physiological data, since our data did not include severity scores. Therefore, we cannot determine the relative accuracy of the model compared with other systems. Second, the administrative data include information on a ‘calendar day’ basis, rather than an hourly basis, and therefore the first ICU day was defined by a calendar day. This provides no distinction regarding the use of renal replacement therapy and pressors/vasoconstrictors before or after ICU entry on the first ICU day. However, these resources are mostly used under monitoring in the ICU. Third, the indications for renal replacement therapy and pressors/vasoconstrictors varied among the hospitals, which may have resulted in therapeutic bias. Fourth, the administrative data do not indicate if renal replacement therapy was given for chronic or acute renal failure, or for a non-renal indication; or if pressors/vasoconstrictors were used to treat hypovolemic or septic shock. Fifth, different admission criteria among the ICUs could have produced a selection bias that affected mortality. Our model has a therapeutic bias similar to that of the COPE model, including the use of mechanical ventilation, renal replacement therapy, and pressors/vasoconstrictors. However, it is likely that there is little if any inappropriate application of these therapies due to ethical considerations. Finally, since our study sample included approximately 30 % of all ICUs in Japan, and did not include university hospitals and non-teaching hospitals, further verification and modification of the model may be required in a larger sample of patients and ICUs from a greater variety of hospital types.

The absence of physiological data is disadvantageous since diagnosis is not possible, but our model has the additional advantage in that it uses administrative data routinely collected for all patients with a high level of accuracy. Comparison of ICU performance using administrative data has applications for benchmarking and quality improvement, and our model establishes a method for the comparative evaluation of ICU performance.

Conclusions

The hospital mortality prediction equation for mechanically ventilated patients in intensive care proposed in this study is based solely on administrative data, and uses a relatively small number of variables that can be easily collected. In addition to the COPE model, Model 2 can be used to evaluate illness severity of mechanically ventilated patients based on administrative data and may be applicable to future critical care studies.

References

  1. 1.
    Kuzniewicz MW, Vasilevskis EE, Lane R, Dean ML, Trivedi NG, Rennie DJ, Clay T, Kotler PL, Dudley RA. Variation in ICU risk-adjusted mortality: impact of methods of assessment and potential confounders. Chest. 2008;133:1319–27.PubMedCrossRefGoogle Scholar
  2. 2.
    McMillan TR, Hyzy RC. Bringing quality improvement into the intensive care unit. Crit Care Med. 2007;35(Suppl):S59–65.PubMedCrossRefGoogle Scholar
  3. 3.
    Leary T, Ridley S, Burchett K, Kong A, Chrispin P, Wright M. Assessing critical care unit performance: a global measure using graphical analysis. Anaesthesia. 2002;57:751–5.PubMedCrossRefGoogle Scholar
  4. 4.
    Knaus WA, Draper EA, Wagner DP, Zimmerman JE. An evaluation of outcome from intensive care in major medical centers. Ann Intern Med. 1986;104:410–8.PubMedCrossRefGoogle Scholar
  5. 5.
    Knaus WA, Wagner DP, Zimmerman JE, Draper EA. Variations in hospital mortality and length-of-stay from intensive care. Ann Intern Med. 1993;118:753–61.PubMedCrossRefGoogle Scholar
  6. 6.
    Zimmerman JE, Shortell SM, Rousseau DM, Duffy J, Gillies RR, Knaus WA, Devers K, Wagner DP, Draper EA. Improving intensive care: observations based on organizational case studies in nine intensive care units: a prospective, multicenter study. Crit Care Med. 1993;21:1443–51.PubMedCrossRefGoogle Scholar
  7. 7.
    Render ML, Welsh DE, Kollef M, Lott JH 3rd, Hui S, Weinberger M, Tsevat J, Hayward RA, Hofer TP. Automated computerized intensive care unit severity of illness measure in the Department of Veterans Affairs: preliminary results. Crit Care Med. 2000;28:3540–6.PubMedCrossRefGoogle Scholar
  8. 8.
    Duke GJ, Santamaria J, Shann F, Stow P, Pilcher D, Ernest D, George C. Critical care outcome prediction equation (COPE) for adult intensive care. Crit Care Resusc. 2008;10:35–41.Google Scholar
  9. 9.
    Curtis JR, Cook DJ, Wall RJ, Wall RJ, Angus DC, Bion J, Kacmarek R, Kane-Gill SL, Kirchhoff KT, Levy M, Mitchell PH, Moreno R, Pronovost P, Puntillo K. Intensive care unit quality improvement: a ‘how-to’ guide for the interdisciplinary team. Crit Care Med. 2006;34:211–8.PubMedCrossRefGoogle Scholar
  10. 10.
    de Vos M, Graafmans W, Keesman E, Westert G, van der Voort PH. Quality measurement at intensive care units: which indicators should we use? J Crit Care. 2007;22:267–74.PubMedCrossRefGoogle Scholar
  11. 11.
    Terblanche M, Adhikari NK. The evolution of intensive care unit performance assessment. J Crit Care. 2006;21:19–22.PubMedCrossRefGoogle Scholar
  12. 12.
    Lin YF, Ko WJ, Chu TS, Chen YS, Wu VC, Chen YM, Wu MS, Chen YW, Tsai CW, Shiao CC, Li WY, Hu FC, Tsai PR, Tsai TJ, NSARF Study Group. The 90-day mortality and the subsequent renal recovery in critically ill surgical patients requiring acute renal replacement therapy. Am J Surg. 2009;198:325–32.PubMedCrossRefGoogle Scholar
  13. 13.
    Brar H, Olivier J, Lebrun C, Gabbard W, Fulop T, Schmidt D. Predictors of mortality in a cohort of intensive care unit patients with acute renal failure receiving continuous renal replacement therapy. Am J Med Sci. 2008;335:342–7.PubMedCrossRefGoogle Scholar
  14. 14.
    Soubrier S, Leroy O, Devos P, Nseir S, Georges H, d’Escrivan T, Guery B. Epidemiology and prognostic factors of critically ill patients treated with hemodiafiltration. J Crit Care. 2006;21:66–72.PubMedCrossRefGoogle Scholar
  15. 15.
    Pinsky MR. Hemodynamic evaluation and monitoring in the ICU. Chest. 2007;132:2020–9.PubMedCrossRefGoogle Scholar
  16. 16.
    Lundberg GD. Practice parameter for the use of fresh-frozen plasma, cryoprecipitate, and platelets. JAMA. 1994;271:777–81.CrossRefGoogle Scholar
  17. 17.
    Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29:1303–10.PubMedCrossRefGoogle Scholar
  18. 18.
    Deyo R, Cherkin D, Ciol M. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613–9.PubMedCrossRefGoogle Scholar
  19. 19.
    Sundararajan V, Henderson T, Perry C, Muggivan A, Quan H, Ghali WA. New ICD-10 version of the Charlson Comorbidity Index predicted in-hospital mortality. J Clin Epidemiol. 2004;57:1288–94.PubMedCrossRefGoogle Scholar
  20. 20.
    Umegaki T, Sekimoto M, Hayashida K, Imanaka Y. An outcome prediction model for adult intensive care. Crit Care Resusc. 2010;12:96–103.PubMedGoogle Scholar
  21. 21.
    Dunning J, Khasati N, Barnard J. Low dose (renal dose) dopamine in the critically ill patient. Interact Cardiovasc Thorac Surg. 2004;3:114–7.PubMedCrossRefGoogle Scholar
  22. 22.
    Kim IB, Fealy N, Baldwin I, Bellomo R. A pilot study of the epidemiology and associations of pulse pressure variation among non-cardiac surgery critically ill patients. Crit Care Resusc. 2011;13:156–61.PubMedGoogle Scholar
  23. 23.
    Lombardi R, Nin N, Lorente JA, Frutos-Vivar F, Ferquson ND, Hurtado J, Apeztequia C, Desmery P, Raymondos K, Tomicic V, Cakar N, González M, Elizalde J, Nightingale P, Abrouq F, Jibaja M, Arabi Y, Moreno R, Matamis D, Anzueto A, VENTILA Group. An assessment of the Acute Kidney Injury Network creatinine-based criteria in patients submitted to mechanical ventilation. Clin J Am Soc Nephrol. 2011;6:1547–55.PubMedCrossRefGoogle Scholar
  24. 24.
    Peñuelas O, Frutos-Vivar F, Fernández C, Anzueto A, Epstein SK, Apeztequía C, González M, Nin N, Raymondos K, Tomicic V, Desmery P, Arabi Y, Pelosi P, Kuiper M, Jibaja M, Matamis D, Ferquson ND, Ventila Group. Characteristics and outcomes of ventilated patients according to time to liberation from mechanical ventilation. Am J Resp Crit Care Med. 2011;184:430–7.PubMedCrossRefGoogle Scholar
  25. 25.
    Ohno-Machado L, Resnic FS, Matheny ME. Prognosis in critical care. Annu Rev Biomed Eng. 2006;8:567–99.PubMedCrossRefGoogle Scholar
  26. 26.
    Rosenberg AL. Recent innovations in intensive care unit risk-prediction models. Curr Opin Crit Care. 2002;8:321–30.PubMedCrossRefGoogle Scholar
  27. 27.
    Afessa B, Gajic O, Keegan MT. Severity of illness and organ failure assessment in adult intensive care units. Crit Care Clin. 2007;23:639–58.PubMedCrossRefGoogle Scholar
  28. 28.
    Cook DA. Methods to assess performance of models estimating risk of death in intensive care patients: a review. Anaesth Intensive Care. 2006;34:164–75.PubMedGoogle Scholar
  29. 29.
    Junger A, Engel J, Benson M, Hartmann B, Röhriq R, Hempelmann G. Risk predictors, scoring systems and prognostic models in anesthesia and intensive care. Part II. Intensive care. Anasthesiol Intensivmed Notfallmed Schmerzther. 2002;37:591–9.PubMedCrossRefGoogle Scholar
  30. 30.
    Schusterschitz N, Joannidis M. Predictive capacity of severity scoring systems in the ICU. Contrib Nephrol. 2007;156:92–100.PubMedCrossRefGoogle Scholar
  31. 31.
    Kramer AA. Predictive mortality models are not like fine wine. Crit Care. 2005;9:636–7.PubMedCrossRefGoogle Scholar
  32. 32.
    Chen FG, Khoo ST. Critical care medicine: a review of the outcome prediction in critical care. Ann Acad Med Singapore. 1993;22:360–4.PubMedGoogle Scholar
  33. 33.
    Harrison DA, D’Amico G, Singer M. Case mix, outcome, and activity for admissions to UK critical care units with severe acute pancreatitis: a secondary analysis of the ICNARC Case Mix Programme Database. Crit Care. 2007;11:S1.PubMedCrossRefGoogle Scholar
  34. 34.
    Strand K, Flaatten H. Severity scoring in the ICU: a review. Acta Anaesthesiol Scand. 2008;52:467–78.PubMedCrossRefGoogle Scholar
  35. 35.
    Capuzzo M, Moreno RP, Le Gall JR. Outcome prediction in critical care: the Simplified Acute Physiology Score models. Curr Opin Crit Care. 2008;14:485–90.PubMedCrossRefGoogle Scholar
  36. 36.
    Duke GJ, Piercy M, DiGiantomasso D, Green JV. Comparison of intensive care outcome prediction models based on admission scores with those based on 24-hour data. Anaesth Intensive Care. 2008;36:845–9.PubMedGoogle Scholar

Copyright information

© Japanese Society of Anesthesiologists 2013

Authors and Affiliations

  • Takeshi Umegaki
    • 1
  • Masaji Nishimura
    • 2
  • Kimitaka Tajimi
    • 3
  • Kiyohide Fushimi
    • 4
  • Hiroshi Ikai
    • 5
  • Yuichi Imanaka
    • 5
  1. 1.Department of AnesthesiologyKansai Medical UniversityOsakaJapan
  2. 2.Department of Emergency and Critical CareThe University of Tokushima Graduate SchoolTokushimaJapan
  3. 3.Emergency and Critical Care MedicineAkita University Graduate School of MedicineAkitaJapan
  4. 4.Department of Health Care InformaticsTokyo Medical and Dental University Graduate SchoolTokyoJapan
  5. 5.Department of Healthcare Economics and Quality ManagementKyoto University Graduate School of MedicineKyotoJapan

Personalised recommendations