Introduction

Intensive care medicine is a relatively young specialty, originating and growing in the second half of the twentieth century. It is believed that no more than 10–20% of all treatments and interventions performed in intensive care units (ICUs) have been proved effective by scientific evidence. Other issues are less relevant if the care being provided is ineffective or harmful; thus, evaluating the effectiveness and efficiency of our therapies must be a top priority in intensive care research.

Besides patient-related factors, such as age, chronic disease status and severity of illness at admission, several studies have provided evidence that the organizational structure of ICUs affects outcome [1]. For example, Bastos et al. [2] showed that the amount of technology in the ICU influences risk-adjusted outcome, at least in Brazil: the less technology available, the higher the risk-adjusted mortality. Furthermore, several studies have highlighted the importance of nursing staff. Amaravadi et al. [3] evaluated 35 centres for the effect of the nurse-to-patient ratio on outcome of patients undergoing esophagectomy: A ratio of <1:2 (i.e. one nurse caring for more than two patients) was significantly associated with an increased incidence of complications. The same investigators confirmed these results for patients undergoing abdominal aortic surgery [4, 5] and hepatectomy [6]. In the UK [7] a strong association was found between occupancy rate and outcome in critically ill infants: patients admitted at higher occupancy rates exhibited a dramatically increased risk-adjusted mortality.

These results suggest that differences in structures and treatments between centres may have a profound effect on outcome of critically ill patients. The aim of this study was thus to evaluate the association between structures, treatment and outcome in a large cohort of critically ill patients in Austrian ICUs.

Materials and methods

Database

Data were collected by the Austrian Center for Documentation and Quality Assurance in Intensive Care Medicine (ASDI), a nonprofit organization that has established an intensive care database and benchmarking project [8, 9]. The prospectively collected data included sociodemographic data, such as age, gender and chronic conditions; the reason for admission, which was recorded according to a predefined list of medical and surgical diagnoses [10]; severity of illness, as measured by the Simplified Acute Physiology Score II (SAPS II [11]); numbers and severity of organ dysfunction, as measured by the Logistic Organ Dysfunction system (LOD [12]); level of provided care, as measured by the Simplified Therapeutic Intervention Scoring System (TISS-28) [13]; length of ICU and hospital stay; and outcome data, including survival status at ICU and hospital discharge.

Data collection for the benchmarking project is continuous. The number of participating ICUs varies from year to year, because participants sometimes lack time and resources to assess important data. Forty-one ICUs were initially included in the data collection for this study, but 10 participated only temporarily and were removed from the database before analysis; thus, data on all patients consecutively admitted to 31 Austrian ICUs from 1 January 1998 to 31 December 2000 were included in this study. The ICUs were divided into three groups according to the size and function of the hospital: group A (community hospitals and specialized trauma centres) had <300 beds (n=13); group B (central referral hospitals) had 300–599 beds (n=10); and group C (teaching hospitals) had ≥600 beds (n=8). Twenty-four ICUs were in anaesthesiology departments and 7 were in medical departments. It is important to note that all Austrian ICUs use a closed-policy approach, i.e. each ICU has a designated medical director and intensive care physicians covering the ICU around the clock.

A total of 29,249 patients were admitted to the 31 ICUs during the study period. For patients who were admitted more than once (n=1374), only the first admission was included. Patients who were <18 years of age (n=648), those with records that lacked an entry in the field “hospital outcome” (n=303) and those without a valid SAPS II score (n=738) were excluded. Since no additional interventions were performed, the need for informed consent was waived by the institutional review board.

Outcome measures

To check for a possible effect of organizational structures on outcome, we used additional questionnaires to evaluate available resources in three areas: hospital services; ICU facilities; and ICU nursing staff. Hospital services included such resources as transportable X-ray, computed tomography, angiography, echocardiography, blood bank, in-house laboratory and operating rooms. The ICU facilities included number of available beds and per-bed availability of ICU devices necessary to provide organ support: mechanical ventilation; invasive haemodynamic monitoring; and renal replacement therapy. The ICU nursing staff was evaluated with the use of three indices. The nurse-to-patient ratio, i.e. the number of beds assigned to one nurse, was calculated according to a formula published by Moreno and Miranda [14]. The efficient use of nursing personnel was evaluated through the work-utilization ratio—the ratio of the produced workload to the available workload in the ICU [14]. Occupancy rate was calculated as the percentage of occupied beds per day [15].

To check for a possible effect of treatment on outcome, we identified ten procedures and interventions from the TISS-28 that are invasive and could possibly harm a patient: mechanical ventilation; single and multiple vasoactive medication; peripheral arterial catheterization; pulmonary artery catheterization; central venous line; renal replacement therapy; intracranial pressure measurement; single specific intervention in the ICU; and multiple specific interventions in the ICU. Although other TISS-28 items might also influence outcome, they are not regarded as generally invasive and/or comparable for all patients; e.g. a “diagnostic intervention outside the ICU” might harm an unstable cardiovascular or respiratory patient who needs a high level of organ support but be harmless for an extubated patient who is awaiting discharge to the normal ward.

Data quality

To assess the reliability of data collection, we sent an independent observer to each unit to obtain SAPS II data from the clinical charts of a random sample of patients. Variance-component analyses with the random factors “units”, “patients within units” and “observers within units” were performed (SAS, procedure varcomp) as described previously [8]. To assess the completeness of the documentation, we calculated the number of missing parameters for the SAPS II score.

The quality of the recorded data was satisfactory with respect to both completeness of records and interrater variability. The median number of missing parameters necessary for the calculation of the SAPS II was 0 (interquartile range 0–2). Interrater quality control indicated an overall excellent grade of agreement: for all tested variables, practically no deviations between the observers were detected, the contribution to the variability being less than 1%. Further details have been reported elsewhere [16].

Statistical analysis

Statistical analysis was performed using the SAS system (version 8e). For tests of statistical significance, the Kruskal-Wallis test was used to compare quantitative variables between hospitals of different size. The chi-square test and Fisher exact test were used when appropriate. A p value <0.05 was considered significant. Unless otherwise specified, descriptive results are expressed as median and first and third quartiles. Observed-to-expected (O/E) mortality ratios were calculated by dividing the number of observed deaths per group by the number of SAPS II-predicted deaths per group. The 95% confidence intervals (CIs) were calculated according to a formula described by Hosmer and Lemeshow [17].

We used linear regression analysis to evaluate the impact of nursing staff availability and hospital size (as a possible proxy for resource availability) on outcome. Each treatment centre was entered as a single observation (n=18 ICUs from groups B and C). The ICUs were weighted according to the number of patients they had admitted. Risk-adjusted hospital mortality per ICU (O/E ratio) was used as the dependent variable. Independent variables were nurse-to-patient ratio, work-utilization ratio and occupancy rate. Hospital size (groups B and C) was also entered. Other structural parameters for which no differences between groups and no association with outcome could be found were not included (see Results).

Logistic regression analysis was used to further explore the effect of invasive procedures on outcome in group B and group C patients. Each patient was entered as a single observation. To exclude case mix factors as possible confounders, we adjusted the model for age, gender, severity of illness and reason for admission. Vital status at hospital discharge (hospital mortality) was used as the dependent variable. Procedures and interventions from the TISS-28 that were previously identified as invasive were entered as independent variables, using binary coding to indicate whether a specific intervention was performed at least once during the ICU stay. In all regression analyses, a p value <0.05 was considered significant.

Results

Data from 26,186 patients admitted to the 31 ICUs during the study period were included in the analysis (Table 1). Most of the ICUs admitted both medical and surgical patients. In group B, more patients were admitted after medical procedures (and fewer after surgical procedures) than in the other two groups (Table 1).

Table 1 Patient characteristics
Full size table

Organizational structures

Group A (the smallest) hospitals had significantly fewer services available than group B and group C hospitals: they were less likely to have the capability to perform angiography, less likely to have a blood bank and less likely to have a central in-house laboratory (Table 2). There was no significant difference in services between group B and group C hospitals. No differences between the three groups were found with respect to equipment available in the ICU (Table 2).

Table 2 Availability of services and facilities in hospitals and ICUs in the three groups
Full size table

The nurse-to-patient ratio was not significantly different between the three groups (Table 3). The work-utilization ratio increased significantly from group A to group C (Table 3). Furthermore, ICUs located in group C hospitals exhibited a significantly higher occupancy rate (Table 3). No differences between the three groups were found with respect to availability of intensive care specialists and physicians in intensive care training (data not shown).

Table 3 Nurse-to-patient ratio, work-utilization ratio and occupancy rate in the three groups
Full size table

Patients and treatments

Patients admitted to ICUs in group A presented with fewer organ dysfunctions and a lower severity of illness at admission, compared with patients admitted to group B and group C ICUs. Despite a similar severity of illness between group B and group C patients, patients in group C received a significantly higher level of care (Table 1). This difference was due to the more frequent use of invasive monitoring and organ support measures. All TISS-28 items that were identified as being invasive and thus potentially harmful were applied to patients in group C ICUs significantly more often (odds ratios 1.2–13.1; all 95% CIs >1 [Table 4]). To exclude differences in case mix as the reason for the difference in use of invasive procedures, we recalculated odds ratios after adjusting for age, gender, severity of illness at admission and reason for admission. After these adjustments, 7 of the 10 odds ratios remained significant. Corresponding unadjusted and adjusted odds ratios are given in Table 4.

Table 4 Invasive TISS-28 interventions during the ICU stay in the three groups
Full size table

Group C patients also had significantly higher ICU and hospital mortality rates, compared with groups A and B. Even after adjustment for the severity of illness, hospital mortality remained significantly higher in group C: O/E ratios were 0.88 (0.84–0.92), 0.83 (0.80–0.87) and 0.99 (0.96–1.02) for groups A, B and C, respectively (Fig. 1). To check for an artefact (lack of prognostic accuracy of the SAPS II), we further evaluated O/E ratios using a customized model for the Austrian ICU population [18], which did not alter the results (data not shown).

Fig. 1
figure 1

Severity-adjusted mortality in hospital groups. The graph shows the risk-adjusted mortality and the 95% confidence interval for patients admitted to the ICUs in each group. A, B and C denote the groups as described in Materials and methods

Full size image

Multivariate analysis

In the multivariate linear regression analysis of the ICU-specific variables, neither the nursing staff factors (nurse-to-patient ratio, work-utilization ratio, occupancy rate) nor the hospital group were significantly associated with outcome (data not shown).

Logistic regression analysis (each patient individually entered) revealed that six items were significantly associated with mortality (Table 5): renal replacement therapy; intracranial pressure measurement; pulmonary artery catheterization; multiple vasoactive medication; central venous line; and multiple specific interventions in the ICU. Mechanical ventilation was not significantly associated with mortality but showed a tendency in that direction [P=0.07; odds ratio 1.18 (range 0.99–1.40)].

Table 5 Logistic regression analysis of the invasive interventions under study (group B and group C patients)
Full size table

Discussion

To our knowledge, this is the largest cohort study to date to evaluate the association between organizational structures, treatments and outcomes in critically ill patients. Our reason for dividing ICUs into groups of hospital size was to compare ICUs at a similar level of available resources and function. Group A hospitals deliver a narrow spectrum of medical care to the local community and have limited resources. Referral hospitals (group B) provide a broader spectrum of medical care and are thus better equipped. Teaching hospitals (group C) offer all medical services and are thus not only the largest but also the best equipped. One might therefore expect that ICUs in group C hospitals would be the most capable of providing a high quality of intensive care and therefore have the most positive impact on the outcome of their patients.

None of the ICU facilities or hospital services was associated with outcome in univariate analysis. This finding is not surprising in a central European country with a relative abundance of health care resources; however, what was surprising was that ICU patients in group C hospitals received significantly more diagnostic and therapeutic interventions per patient and per day. This was despite the fact that their severity of illness at admission was no different from that of patients in group B (Table 1); odds ratios for patients treated in group C ICUs to undergo invasive procedures were greatly increased for all ten interventions studied. Even after adjustment for age, gender, severity of illness and reason for admission, 7 of the 10 interventions were still applied significantly more often to patients in group C (Table 4).

Evaluation of the work-utilization ratio in our study revealed that the availability of nursing staff did not differ significantly between the three groups (Table 3). As a net result, a nurse in a group C ICU generated approximately 30% more TISS-28 points than a nurse in groups A or B.

To further explore these results, we used a case mix-adjusted multivariate approach. The results of the patient-centred analysis proved the association of the invasive procedures with outcome: six interventions were significantly associated with mortality (Table 5), and one intervention (mechanical ventilation) narrowly missed being significant. How can we explain these results?

Several recent studies have shown that interventions carry inherent risks which possibly influence outcome. For example, central venous lines (CVL), although often regarded as a simple, routine procedure, are known to cause a variety of potentially life-threatening complications [19]. These complications can be related to either the insertion or the use of CVL [20, 21]. Bacteraemia occurs in approximately 1% of cases (amounting to approximately 50,000 cases each year in the United States), with an attributable mortality of 10–20% [22, 23]. Pulmonary artery catheterization is an invasive measure with acknowledged risks [19]. The demonstration of a worse instead of a better outcome for patients treated with pulmonary artery catheterization in at least one study [24] has led to an ongoing discussion of the effectiveness and efficiency of this procedure [25, 26].

It was recently demonstrated that patients receiving renal replacement therapy (RRT) have significantly increased mortality rates when compared with matched control subjects [16]. This excess mortality might be attributable not only to acute renal failure, but also to the treatment itself: RRT has a profound effect on the antioxidant state of critically ill patients by depleting nutritional antioxidants and by generating radical oxygen species in the extracorporeal circuit [27, 28]. Moreover, the activation of several physiological cascade systems results in an inflammatory reaction that induces protein catabolism and compromises immunocompetence [29]. All these phenomena can increase morbidity and, therefore, mortality.

Mechanically ventilated patients have a high risk of developing complications, such as ventilator-associated pneumonia (VAP). A recent consensus conference estimated that the incidence of VAP, using a clinical definition, is approximately 7 cases per 1000 ventilator days [30]. Moreover, although the mortality attributable to VAP has not been firmly established, some investigators believe that these patients have a two to ten times higher risk of death compared with patients not on mechanical ventilation [31]. An international multicentre study found that survival in mechanically ventilated adult patients depended not only on patient-related factors but also on factors related to patient management and the development of complications [32]: successful noninvasive ventilation significantly reduced the risk of death (odds ratio 0.46).

A relationship between nursing workload and deteriorating ICU performance [3, 33] has been demonstrated in several studies. Nurses who are assigned a higher workload have less time for routine activities, and hygienic tasks often suffer under these conditions. A reduction in nursing staff to below critical levels is associated with an increased number of bloodstream infections from CVL [34]. Moreover, reduced nurse staffing is also associated with increased rates of pneumonia and septicaemia [3, 4, 5].

Although univariate analysis of our data showed work utilization ratio and occupancy rate to be significantly different between groups (and highest in group C), none of these parameters reached statistical significance in the multivariate analysis. There are several possible explanations for this: on one hand, our results could provide evidence that the inherent risk of the invasive procedures under study is more important than the increased nursing workload. On the other hand, it is very likely that the number of observations in the ICU-based linear regression analysis (n=18) was just too small to detect existing differences.

The question arises as to whether the increased severity-adjusted mortality in group C could be an artefact. SAPS II has been shown to lack prognostic accuracy when applied to populations other than the one from which it was derived [35, 36]. We have shown, however, that medical reasons for admission have a higher severity-adjusted mortality than surgical reasons for admission [18]. In our study, group B ICUs had a much higher incidence of medical admissions than did group C (Table 1). If case-mix differences influenced our O/E ratios, we would expect group B to have a higher severity-adjusted mortality than group C. Our data show exactly the opposite, which renders our results even more reliable.

In addition, multivariate analysis allowed us to adjust for a majority of what constitutes case mix: we were able to rule out the influence of age, gender, severity of illness at admission and reason for admission. We can therefore virtually exclude the possibility that our results were influenced by differences in case mix of patients between the three groups. Of course, we cannot fully exclude the possibility that other confounding variables might have influenced the difference in O/E ratios between groups. Nevertheless, it is important that the (patient-based) association between interventions and outcome remained highly significant throughout all analyses. Our results thus provide strong evidence that this association was at least partly responsible for the increased risk-adjusted mortality in group C patients.

What can we learn from these findings? They can be viewed from both an individual and an epidemiological perspective. In an attempt to achieve the best outcome for their patients (individual perspective), physicians will often use a strategy to reduce the “maximum risk” as far as possible and therefore use available resources to whatever extent they think is necessary. In large hospitals, more resources are available and (until now) fewer restrictions have been applied, especially in the care of critically ill patients. As our results suggest (epidemiological perspective), such a strategy might, however, increase the “average risk” and thus there might be a level of invasive treatment beyond which the associated risks outweigh the benefits. We acknowledge, of course, that in everyday practice it is difficult to apply such statistically derived results to an individual patient. Nevertheless, when decisions about invasive treatments are to be made, intensivists should always consider both the possible benefits associated with an intervention and the associated risks. Although our results cannot be used to answer the question, “How much treatment is optimal?,” they should encourage practising intensivists to carefully consider the issues we have raised.