Introduction

An adverse event (AE) can be defined broadly as any event which actually or potentially compromises patient care [1]. The incidence of AE among hospitalised patients varies according to the definition used and the method of reporting. Prospective epidemiological studies have quoted rates as high as 20–46% [2, 3, 4]. It has been estimated that AE contribute to between 44,000 and 98,000 hospital deaths annually in the United States [5]. Several authors have suggested that this death rate represents an overestimate [6]; however, there is consensus that the quantitative impact of AE on patient care is substantial [7, 8].

Analysis of AE in the hospital setting has traditionally been retrospective and has focussed upon individuals and actions preceding the event, for example, as occur during morbidity and mortality meetings. Such an approach is reactive rather than proactive, is prone to hindsight bias [9], and fails to acknowledge that medical care is delivered via a complex system, involving the interplay of individuals, work environment and wider organisational processes [10, 11, 12]. A systems approach represents an alternative method of AE analysis which takes these complexities into account [13]. This has been proven successful in non-medical environments, including the industrial, aeronautical and military [14, 15]. Here an AE is thought to occur at the end of a chain of events, whereby flawed management and organisational processes contribute to sub-optimal work conditions, encompassing aspects such as workload, supervision, communication, training and equipment [13, 16]; these represent latent system failures, which increase the likelihood of an active failure, or AE, occurring in two ways: firstly, by creating the environment for an AE to occur (inadequately checked and maintained equipment is more likely to break down); and secondly, by eroding the defence mechanisms inherent in the system (inadequately trained staff are less likely to compensate for unexpected equipment failure).

The intensive care unit represents a high complexity environment with great potential for patient harm following an AE. To date, literature on AE in this setting has been predominantly taxonomic [4, 17, 18, 19, 20, 21, 22, 23, 24, 25]. The relationship between latent system failures and AE occurrence in the intensive care unit remains ill-defined; however, an association has been clearly demonstrated between latent (workload-related) factors and disease-severity adjusted mortality [26]. We have conducted a prospective observational study in a regional paediatric intensive care unit with two aims: firstly, to investigate the association between a broad range of latent factors (temporal, workload, skill mix and supervision) and AE occurrence; and secondly, to investigate if interactions occur between clinically related factors to increase the risk.

Methods

The study was conducted prospectively from 1 April 2001 to 31 March 2002, encompassing 730 12-hour nursing shifts. All data utilised in the study were sourced from the nursing, medical and AE databases, which are collected as part of routine PICU management. The databases are in Microsoft Excel and Access (Microsoft, Redman, Wash.) format, password protected and updated by senior medical and nursing staff twice daily. The need for informed consent is waived by our local research ethics committee for studies which utilise data collected as part of routine management, and are analysed in an anonymous format which is not patient identifiable.

Unit demographics

Guy’s PICU is a 17-bed, multidisciplinary PICU, admitting 800–900 patients per year. The PICU occupies two floors, the lower floor receiving predominantly post-operative patients. Approximately 35% of the workload comprises cardiology and complex congenital cardiac surgery. Many of these patients are diagnosed antenatally from an in-house fetal medicine program. Over 80% of non-cardiac referrals originate from district general hospitals in the South-East Thames region (1,500,000 children), of which over 90% are transported via the Regional Retrieval Service, based in the PICU at Guy’s Hospital. The remaining non-cardiac admissions are from the wards.

Medical staff comprises five full-time PICU consultants, eight fellows and six residents. Fellows are SpR year 3 (post-graduate year 6) and above, the majority are PICU trainees, who rotate through the PICU for variable periods (from 6 months to 2 years). Residents rotate on a 6-month basis as part of general paediatric/anaesthesia training (seniority ranging from post-graduate year 3 upwards). During the daytime, each junior doctor is given a five- to seven-bed area of responsibility, which is overseen by the chief fellow and consultant. During the evenings, two residents and one fellow are in-house; however, the fellow is available to undertake retrievals, and may thus leave the hospital for variable periods. The nursing establishment includes 87.5 funded posts (6.4 whole-time equivalents per bed for approximately 13 beds), supported by health care assistants (3.3 whole-time equivalents). Ninety percent of nurses are registered children’s nurses, and 55% hold a PICU diploma. Over the study period the nursing vacancy rate varied between 22 and 28%. Shift vacancies are filled using non-hospital (agency) or in-house nurses (permanent staff members performing extra shifts); however, approximately 50% of agency staff were employed on a semi-permanent basis and well known to the PICU; many were ex-permanent staff members.

Adverse event definition, categorisation and reporting

An AE is defined as any event which actually or potentially compromises either patient care or the usual PICU administrative process. Events are categorised as being either patient-, unit- or management related (Table 1). Although all are reported under the AE system, only patient- and unit-related AE were included as outcomes of interest for the purpose of this study. Patient-related AE were further subdivided according to type, outcome (actual/near miss) and severity (severe/moderate; Table 1).

Table 1 Categorisation of adverse events
Full size table

Management-related AE were not classed as an outcome variable because they involve organisational processes that are beyond the control of the PICU; however, it is conceivable that a management-related AE may in turn contribute to a patient- or a unit-related AE, and thus was included as an independent rather than an outcome variable (see below).

Adverse event reporting has occurred in the PICU since 1993. Collation and analysis was formalised following the appointment of a research and audit nurse in 1995. Reporting is encouraged among all disciplines and grades, using a non-anonymised, non-punitive format. Initial reporting is via a standardised, paper format. The nurse and consultant in charge of the shift screen events as they are reported; those requiring urgent action are dealt with initially at this stage, the remainder are actioned by the research and audit nurse within 24–48 h. All AE are presented at a monthly, multidisciplinary PICU meeting attended by senior staff where final categorisation and action plans are agreed.

Definition of risk factors for the model

Potential risk factors included temporal, patient, staff composition and staff supervision factors.

Temporal factors included day (08:00 to 20:00 h) vs night shift, and weekend/bank holiday vs normal working week.

Patient factors included bed occupancy at the start of a shift, the number of admissions and discharges, and the average patient dependency during a shift. Patient dependency was calculated according to recommendations from the UK Paediatric Intensive Care Society [27]. The number of nurses required to care for a patient are defined as: 0.5 for non-ventilated or stable, chronically ventilated children; 1.0 for mechanically ventilated patients who are stable but need continuous supervision; 1.5 for those requiring more advanced organ support, usually in the setting of multi-organ failure (e.g. haemofiltration, high-frequency oscillatory ventilation, immediately after complex cardiac surgery); and 2.0 for patients requiring extracorporeal membrane oxygenation.

Factors relating to the composition of nursing staff included seniority, percentage of permanent staff and number of nurses in relation to patient dependency. Seniority was quantified by the percentage of F- and G-grade (senior) nurses on duty during a shift. A G-grade nurse (sister) is the more senior of the two and undertakes a significant amount of staff supervision, mentoring and management duties in addition to his/her clinical role. An F-grade refers to a senior staff nurse who has a high degree of clinical expertise but undertakes less managerial responsibility. Composition was analysed in two ways: firstly, by the percentage of a shift occupied by permanent nursing staff (this included permanent staff who were working extra, non-rostered shifts); and secondly, by the percentage of rostered permanent staff only. This differentiation was made to investigate any potential fatigue factor from working extra shifts. Number of nurses needed was calculated as: total nurses at start of shift minus total patient dependency at end of shift minus 2 charge nurses needed (1 only if the surgical PICU was closed); thus, a positive number represented a relative nursing excess, and a negative number a deficit.

Senior and middle-grade medical staff did not change during the study period; however, new junior staff (residents) rotated through the unit twice during the study period. We arbitrarily designated the 2-month period after new resident commencement (totalling 4 months during the study period) as a potential risk factor.

Nursing supervision was analysed according to two factors: firstly, we recorded whether the nurse in charge of a shift was of G-grade (highest) seniority; and secondly, we constructed a weighted, composite score to quantify factors which may compromise the ability of the nurse in charge to supervise patient care. Weighting was based on a time and motion study over 20 shifts performed prior to the study period, and comprised (weight given in parentheses): occurrence of a management event (1 per event); patient death on a shift (1 per event); both PICU floors open (1); lack of an unallocated nurse to cover meal breaks and troubleshoot (1.5); lack of both an unallocated nurse and a health care assistant (2); and requirement for nurse in charge to care for a patient (2.5). Thus, for example, a shift where both PICU floors are open, a patient death occurs and no supernumerary nurse is available would receive a composite supervision score of 3.5.

Statistics

The association between potential risk factors and outcome (patient- and unit-related AE, together with the various subcategories) was tested using a forward stepwise logistic regression model (SPSS, version 11.0). The significance criteria for stepwise inclusion and exclusion were p<0.05 and p>0.10, respectively. Potential interaction effects between clinically related variables (e.g. patient acuity and bed occupancy) were screened for in the first model. These effects were then tested formally using a second model, utilising the following procedure: firstly, only variables reaching significance for at least one of the outcomes in the first model were included in the second; secondly, those variables having a potential interaction effect (suggested clinically and by the first model) were paired; thirdly, continuous variables were transformed into categorical variables according to whether values fell above or below the median; and lastly, the two categorical variables were combined, and odds ratios (with 95% confidence intervals) for the various combinations (neither, either, both) were expressed. Because the purpose of this second model was screening for interaction effects, higher p values were chosen (p<0.10 for stepwise inclusion and p>0.15 for exclusion) [28]. Goodness of fit was assessed by the Hosmer-Lemeshow χ2 statistic, where p>0.05 implies satisfactory fit.

Results

There were 816 patient episodes over the study period, with very little seasonal fluctuation in terms of number of admissions or bed occupancy (Fig. 1). Admissions occurred throughout the 24 h, peaking during the late morning to early evening. Discharges (not including deaths) showed a similar peak, but were rare between midnight and 09:00 h. Patient demographics are given in Table 2. The median (interquartile) bed occupancy was 10 (8–11). Overall, the unit was well staffed from a nursing perspective: the median number of nurses at the start of each shift was slightly in excess of that needed relative to patient dependency (median 1.5 above), and the median percentage of senior nurses per shift was 28%.

Fig. 1
figure 1

Monthly admission rate and bed occupancy. Bars represent total admissions and median bed occupancy (error bars=interquartile range), respectively

Full size image
Table 2 Patient demographics (816 patient episodes)
Full size table

Overall, there were 284 AE occurring on 220 of 730 (30%) shifts. Forty-five (6%) shifts included more than one AE. There were 103 unit- and 181 patient-related AE; the latter occurred at a rate of 6.0 per 100 patient days. Patient-related AE included: drug error (55); intravenous/arterial line (37); equipment (32); patient injury (26); patient care (21); and accidental extubation (10). The number of patient-related AE categorised as serious was 83 (actual 49, near miss 34), and moderate was 98 (actual 85, near miss 13).

Results from the first logistic regression model, examining factors associated with the occurrence of an AE are given in Table 3. Variables identified as having an association with the occurrence of at least one category of AE included: day shift; percentage of a shift filled by rostered permanent staff; percentage of a shift filled by F- and G-grade nurses; presence of a G-grade nurse in charge; the composite supervision score; average patient dependency; number of occupied beds; number of admissions and discharges; and presence of new residents. Surprisingly, the latter two variables were associated with a decreased incidence of AE. From this model three potential interaction effects were identified: (a) patient workload, comprising bed occupancy and patient acuity; (b) nursing supervision, consisting of the presence of a G-grade nurse in charge with the composite supervision score; and (c) nursing coverage, combining percentage of a shift occupied by rostered permanent staff with the percentage of a shift occupied by F- and G-grade nurses. Results for the interaction effects model (adjusted for day shift and presence of new residents) are given in table 4. This shows an interaction effect between patient workload factors for both total and patient-related AE, and between nursing supervision factors for unit-related AE. There was no interaction between the two nursing composition factors; here the percentage of rostered permanent staff on a shift was more important than the percentage of F- and G-grade nurses on a shift.

Table 3 Multivariate analysis showing association between risk factors and adverse event type
Full size table

Only the logistic regression equation for accidental extubation in model 1 demonstrated poor fit (p value for χ2=0.04); elsewhere, the Hosmer-Lemeshow χ2 statistics were acceptable (p ranging from 0.11 to 0.99).

Discussion

Two principal findings emerge from this study: firstly, that an association exists between a broad range of latent risk factors and particular types of AE occurrence; and secondly, that clinically-related risk factors interact to increase this risk. Many of the associations will not be surprising to those with a knowledge of how an intensive care unit operates, although several deserve further mention.

There was a preponderance of most types of AE on the day shift. This phenomenon has been described by others, and is likely to be activity related [22, 25]. Certain activities, such as medication prescribing, and diagnostic and therapeutic investigations are more common at this time [22], which was reflected by the association of day shift with drug errors and equipment issues. Day shift aside, three thematic associations with AE emerged from our study: patient workload; staff supervision; and staff composition. The combination of higher-than-average bed occupancy with patient acuity impacted upon the total number of AE, in particular patient AE (Table 4). This finding has a parallel in an adult study, where risk-adjusted mortality increased during periods of excessive patient workload [26]. Although nursing supervisory issues figured in certain types of patient AE, the major effect was seen with unit AE, where lack of a senior nurse (G grade) in charge combined with a higher than average “hassle factor” produced a greater risk than either variable alone. The surprising association of new residents with a decreased risk of patient AE may also represent a supervisory phenomenon. Although not formally assessed in this study, it is our belief that new residents are less likely to be left to their own devices by senior medical and nursing staff, and are probably more likely to seek advice before making decisions. This is supported by a previous study demonstrating that the time of year when new residents are employed is not associated with increased risk of AE [23]; thus, our explanation for this finding is that the risk of AE is actually related to resident supervision, with unsupervised (“non-new”) residents being associated with a higher risk of AE rather than new residents being associated with a lower risk. Seen in this light, this finding is compatible with that demonstrated by Pollack, who highlighted an increase in risk-adjusted PICU mortality in teaching hospitals where care was provided by the most junior residents [29].

Table 4 Multivariate analysis examining interaction effects between clinically related risk factors
Full size table

Nursing composition was also an important factor, both in terms of the seniority (percentage of F and G grades on the shift) and the proportion of rostered, permanent staff on duty; however, these two variables did not interact (Table 4). Interestingly, the percentage of rostered permanent staff was associated with a decreased risk of actual, but not near miss AE (Table 3). This may mean that permanent staff members act as a defence mechanism, not preventing AE occurrence per se, but thwarting the progression of a near miss to an actual event. Fatigue may also be a factor when considering the contribution of permanent staff, as the beneficial association occurred only for the proportion of permanent rostered staff on duty (rather than including permanent staff working extra shifts).

Several limitations of this study exist. The outcome variable, AE lacks a precise and uniform definition in the medical literature [30], posing difficulty when making direct comparison with other published studies. Nonetheless, our definition is consistent with the majority of the larger studies [1, 12, 30], and is utilised as part of a clinical reporting system which predated the study by 8 years. Quantifying the strength of association between latent factors and AE is beset by two problems. Firstly, AE are a heterogeneous entity, and thus an association between a variable and a particular type of AE will not be demonstrated if AE are analysed as a single entity (total AE only). Thus, we also undertook AE subgroup analysis (Table 3), which in turn creates new problems. Subgroup analysis produces a decreased number of outcome events per variable, with a subsequent loss of statistical power and potential for overfitting. This effect could theoretically be minimised by increasing the study period; however, we felt that this was not justified as a longer period of data collection increases the likelihood of confounding factors influencing the outcome variable (such as staff and policy changes, new therapies, etc.). Secondly, it is likely that many other non-latent factors specific to the type of AE are important, e.g. the design of medication charts when considering drug errors. A final limitation is the potential for underreporting of AE [12]. The area affected here is likely that of the near-miss AE, particularly so for those with less severe consequences. This is suggested in our data from the discrepancy between the proportion of actual to near miss AE in the severe and moderate categories (serious: actual 59%, near miss 51%; moderate: actual 87%, near miss 13%). Underreporting can be minimised by utilisation of trained observers; however, the vast resources required for such an approach was beyond the scope of this study. But it is fair to state that the limitations described are shared by the majority of studies of this type and do not invalidate the results.

The contribution of latent factors to AE cannot be viewed in isolation, but instead is incorporated into a multimodal approach (including, for example, electronic prescribing) when considering strategies to both reduce the occurrence and consequences of AE [13, 30]. The findings from this study have influenced workforce planning, staff training and policy guidelines in our unit. Examples include restructuring of nursing and medical rotas, novel nursing recruitment and retention strategies, better awareness of sub-optimal nurse–patient ratios (particularly when planning for semi-elective admissions), and greater use of ancillary support staff to lessen the burden on the nurse in charge and the bedside nurses. This has been combined with incident-specific strategies such as redesigned fluid and prescription charts and protocols for IV line removal. We acknowledge that our findings may not be totally applicable to all other European PICUs due to differences in working practice (e.g. staff levels, nursing shift patterns); nonetheless, we suggest that they serve as a template for other units when considering strategies to reduce AE.