Introduction

Reports on paediatric patient safety [18, 41, 12, 25, 1] have made clear that children in intensive care units are at high risk of incidents and adverse events. Incidents are defined as events that could or did lead to harm in patients. As an illustration, voluntary incident reporting in Dutch neonatal intensive care units (NICUs) yielded 1.3 incidents per admission [31] and Agarwal et al. reported adverse events in almost two thirds of paediatric intensive care unit (PICU) patients in the USA [1]. Other studies have reported from 2.5 to 10 incidents and adverse events per admission [1, 32, 17, 26, 13, 16, 27]. Patient safety programmes [11, 15, 3, 14] suggest we first need to learn from the incidents before effective preventive measures can be taken.

Seeking answers to the following three questions is central here: What has actually occurred? What causes and circumstances contributed to allow the incident to occur? What measures and interventions could prevent recurrence? Root cause analysis may be able to answer these questions [3, 33, 29], notably Reason’s systems approach [23, 24]. Incidents usually occur not only because of human failure but also because there may be no or ineffective barriers to prevent an error from reaching an unfortunate endpoint. Reason’s approach focuses on the flaws in the system rather than on human failure because interventions to avoid the first are more effective [39].

The PICU and NICU in the Erasmus MC-Sophia Children’s Hospital have been pioneering patient safety projects here. These departments were the first to implement voluntary incident reporting [31] and incident analysis in the hospital [35]. The PICU later added adverse event registration [36] and scoring of nursing protocol violations [4]. In 2005, the entire PICU staff received crew resource management training from the Dutch Centre for Man and Aviation. The patient safety officers have adapted this training course to the medical setting and since then have been training new PICU employees and all NICU staff. Several interventions resulting from the team training course have been implemented on the units, such as the use of briefings, huddles and debriefings; a guideline for positions around the bedside of team members in acute situations; screensavers with reminders of rules on teamwork; and time out procedures before extra corporeal membrane oxygenation (ECMO) cannulation and decannulation, intubation, and surgical procedures on the intensive care unit (ICU) [20].

Other patient safety projects implemented in the PICU are as follows: critical incident analysis [38], prospective risk analysis [37], safety walk rounds [19], simulation training and safety-focused mortality and morbidity conferences. Several recommendations resulting from these activities have meanwhile been implemented, such as safe MRI scanning of premature patients [21] and safe drug ordering [7]. The other departments in our hospital have adopted voluntary incident reporting, but so far, few of the other patient safety management elements have spread to other units. We present the results of in-depth analysis of 17 critical incidents in our hospital, focusing on possible causal and contributing factors, recommendations stemming from the analyses, and the take up rates of the recommendations.

Methods

Setting

Erasmus MC-Sophia is a 215-bed university children’s hospital, including a 37-bed obstetrics department, a 28-bed paediatric ICU with a 6-bed high care (HC) unit and a 27-bed neonatal ICU with a 6-bed HC unit.

Design

The design of this study is a retrospective review of critical incident analyses in the period 2005–2010.

Patients

All patients admitted to the children’s hospital, including the obstetrics department, were eligible for incident analysis.

Patient safety management system

Since 2005, the PICU has developed and implemented a comprehensive safety management system. Key elements of this system are voluntary incident reporting, scoring of nursing protocol violations, registration of adverse events and crew resource management training.

In this latter 2-day mandatory multidisciplinary course, based on crew resource management training as developed in aviation, several topics relating to teamwork are discussed. Team composition is the same as on the work floor, that is, a number of nurses with one or two physicians, secretaries, managers and technicians. Issues dealt with on the first day are information management (Are we looking at this patient in the same way?), communication (How can we communicate effectively?), stress management (What are the effects of stress on our functioning as a team?) and group processes (What does it mean to be part of a team and what are the risks involved?). The second day is dedicated to leadership (Who takes the lead, and how to do that?), decision making (How are decisions taken?) and risk management (What are the risks on the unit for patients, family and employees, and how to reduce them?). Ample time is allotted for discussion, and the participants are stimulated to voice their opinions. Particular emphasis is laid on each one’s individual responsibility to help address the issues discussed during the course. Participants are expected to take part in a 1-day follow-up course once every 2 years in which the afternoons are dedicated to simulation training.

A physician (CvdS), a nurse manager (AvdB) and several rotating nurses investigate critical incidents on the request of the departmental heads of the intensive care units or other departments. The physician and nurse manager both have completed the patient safety officer executive development programme from the Institute for Healthcare Improvement [10]. The rotating nurses are not involved in the actual care of the individuals concerned and have attended a training course in root cause analysis [23].

The incidents analysed in this study had been reported to the heads of the departments because staff considered them severe; structured reporting of serious incidents was not yet in place at the time of the study. The departmental heads subsequently requested investigation because the incidents either had caused or could have caused serious harm. The patient safety officers of the PICU interviewed all staff involved with the incident to supplement the information provided in the incident report. If thought helpful, parents were interviewed as well. Both staff and parents were invited to suggest recommendations to prevent the incident from occurring again. Causal and contributing factors were identified by means of a cause-and-effect diagram or a causal tree analysis [38]. The investigating team then proposed recommendations based on the interviews and the identified causal and contributing factors, and the report was sent to the requester and all interviewed parents and employees.

For this study, we classified the identified factors according to the classification system devised by Vincent et al. [38]. The system distinguishes between factors related to seven categories: institutional context, organizational and management factors, work environment factors, team factors, provider factors, task factors, and patient factors. The recommendations were classified by the same system, and follow-up interviews with the responsible persons were held to check if the recommendations were carried out. Recommendations were considered to have been implemented if they were either taken up during the team training given by the authors (CvdS and AvdB) or if they had resulted in new or changed protocols and guidelines or had been realized by the management.

Results

In the study period 11,327 incidents on the PICU had been reported, of which 480 (4.2 %) were classified as potentially critical and thus eligible for analysis. In the NICU, 773 of 8,683 reported incidents (8.9 %) were classified as potentially causing serious harm and thus eligible for analysis. Data on the numbers of incidents and their severity in the other units are not available, as they did not yet employ a voluntary incident reporting system. Seventeen incidents were reported to the heads of the different departments because of the harm they caused, or could have caused, and were subsequently analysed as requested by the departmental heads. Eight of the 17 critical incidents occurred on the PICU, 6 on the neonatal ICU and 3 in other departments in Erasmus MC-Sophia. The eight incidents on the PICU had been reported in the voluntary incident reporting system, and six (75 %) had actually caused serious harm. The other two incidents were analysed because of the severity of the harm they could have caused.

Figure 1 gives an example of an incident reconstruction; Table 1 provides the causal and contributing factors identified in this example and the recommendations made.

Fig. 1
figure 1

Result of analysis of clonidine overdose, presented in the “Swiss Cheese” model developed by James Reason (19) (CPOE computerized physician order entry system)

Full size image
Table 1 Case example: clonidine overdose (30-fold) during 20 days
Full size table

Three additional, illustrative examples of incident analyses and the resulting factors and recommendations are detailed in Appendix 1. The examples were chosen because they represent the wide range of types of incidents that can occur in paediatric (intensive) care and because they show that incidents are not caused by a single person making a mistake but that multiple healthcare providers are involved. Also, these analyses are examples of how a system-based approach can result in recommendations not aimed at individual healthcare providers but at a broader range of contributing factors. The incidents were reconstructed from a median of 6 interviews (range 3–15; total 112). In two incident analyses, parents were interviewed. Time spent on the interviews, analysis and preparing a report amounted to a mean of 27 h per incident. The 17 incidents are listed in Table 2.

Table 2 Description of incidents, units where the incident occurred and patient outcomes
Full size table

In seven of the cases, the patient in question died during hospital stay; death was unanticipated in three of these cases, and this was the reason to instigate the investigation. One unanticipated death related to a failure to resuscitate a newborn with meconium aspiration syndrome; the second to unnoticed bleeding during the night shift in a patient with aplasia cutis; the third to foetal arrhythmia and hydrops foetalis. Causality between incident and death was established in two of the other four cases. In the first, a 1,000-fold overdose of alprostadil led to uncontrollable hypotension and pulmonary hypertension, followed by death 6 h post-incident. In the second, ECMO cannulation caused a tear in the right atrium followed by bleeding (see also Appendix 1). This was quickly discovered, and the tear was surgically repaired, but resuscitation failed. The other two patients died because life support was withdrawn, in one on account of an untreatable metabolic disorder. However, 12 h before life support was withdrawn, he had been given a 35-fold morphine overdose. The second was an extremely premature infant (26 weeks GA) who was severely burned when umbilical lines were inserted (the sterile cotton covering became soaked in blood and stuck to the skin, upon which the heat from the external heater caused a severe burn) and died 6 days postpartum due to problems related to prematurity (necrotizing enterocolitis and sepsis). In those two patients, a causal relation between death and incident was not established.

Six other patients suffered permanent or temporary harm, e.g. in the form of skin lesions or a chest drain inserted on the wrong side. One of these six patients needed longer hospitalization, and one suffered permanent severe neurological damage. Two incidents, near strangulation with a nasogastric tube during sleep and paracetamol overdose, caused no harm.

A median of 5 causal and contributing factors per incident were identified (range 2–10; total 85). Team factors (e.g. loss of information on shift handover or resident’s work not supervised by intensivist) and task-related factors (e.g. ambiguous protocols) were most frequent: both 19 out of 85 (22 %). Seventeen factors (20 %) related to individual providers, e.g. calculation errors or failing to adhere to protocol. Sixteen factors (19 %) related to work environment, such as a defective intravenous (IV) pump. Nine (11 %) were organizational factors, such as lack of medication dose check in the computerized physician order entry system. The remaining five factors related to patient features, such as turning during sleep and bleeding from aplasia cutis.

A total of 84 recommendations were formulated, i.e. a median of 5 per analysis (range 1–8). Twenty-nine of the recommendations (35 %) were actually implemented. All recommendations are detailed in Table 3, classified by type of factor they refer to and by either or not being implemented. Recommendations relating to organizational factors were least implemented, only 3 of 16 (19 %); recommendations pertaining to task factors most frequently, 13 of 30 (43 %), and these entailed adjustments to protocols and guidelines. The other recommendations that were implemented related to issues addressed in team training.

Table 3 Categories of recommendations and description of recommendations implemented and not implemented
Full size table

Another four recommendations that were implemented were technical adjustments, e.g. of the CPOE, that were within the sphere of influence of the patient safety officers.

Discussion

Our analyses provide useful information on the factors that caused or contributed to these serious incidents. The in-depth system-based approach enabled the investigators to formulate a large number of recommendations. One third of the recommendations were actually taken up.

The median number of 5 causal and contributing factors per incident is more than double that reported by others [22, 30]. As a possible explanation: most studies on the causes of incidents in paediatrics are based on anonymous reporting, thereby precluding the possibility to retrieve additional relevant information. We were able to gather more information as reports are not anonymous and everyone involved is interviewed. Furthermore, “latent factors” contributing to incidents, such as management decisions or organizational policies impacting patient safety, may have been left out of consideration, which was not the case in our study.

Only 6 % of the contributing factors in our study were ascribed to patient-related circumstances, versus 45 % in 462 paediatric incidents in the ICUSRS database [28] and 32 % in 2,075 incident reports in adult and paediatric ICUs analysed by Pronovost et al. [22]. Bagian et al. [2] found that incident analysis in general yielded much smaller proportions of patient factors than did focused review (10 vs. 43 %). Root cause analysis by Snijders et al. [30] also yielded a low percentage of patient-related factors (3 %) contributing to incidents in NICUs. In the present study, none of the patient-related factors could have been prevented, and there was no reason to recommend preventive measures.

In our study, only 20 % of the causal and contributing factors were related to performance of care providers. Frey et al. reported that provider performance assumedly played a role in 63 % of the incidents in NICUs and PICUs [8]. A study of incident analysis in Dutch NICUs concluded that provider performance played a role in 64 % of incidents [30]. Nevertheless, two other studies both reported that—like in our study—only 20 % of contributing factors were provider factors [22, 28]. Measures to counter failures of individual providers are hardly effective; e.g. recommending greater attention to medication preparation will not be permanently effective to prevent errors. Improvements on “systems” factors are more likely to be effective in this respect.

Team factors and task factors were the most frequent contributing factors found in our analyses. The team factors are clear examples of issues that could be addressed by team resource training, i.e. communication, situational awareness, and leadership and followership roles. Most of the employees interviewed had not yet received team resource training, with the exception of the involved PICU staff (other than the residents). This training course may prevent faulty teamwork if the team members are continually reminded of the principles of teamwork.

The differences in the types and numbers of contributing factors are likely explained by the different approaches to incident analysis. We believe that the more thorough and in-depth an analysis is performed, the more organizational, management and work environment-related factors will be uncovered. The less information is obtained from the involved healthcare workers and family/parents, the more likely the analysis will result in provider- and patient-related factors. For instance, a nurse might report she was distracted and therefore forgot to have a colleague perform a double check of the medication she was about to administer. But further questioning might reveal that she was distracted because she also had to answer the phone where a consultant was inquiring for a bed for a new patient to be admitted. The fact that she had to perform these two duties at the same time is caused by the management decision to have the nurse answer these phone calls while at the same time providing patient care.

Implementing recommended measures is known to be a difficult undertaking [33, 5, 40, 9]. In our hospital, one third of recommendations were acted upon, which seems to justify the use of reporting and analysing incidents. Recommendations aimed at organizational (redesign of processes) and work environment factors (equipment changes) are thought to have the highest likelihood of successful prevention of incidents; those aimed at providers (general education) and task factors (guideline development) have a lesser likelihood [33]. Three (of 16) recommendations aimed at organizational factors were implemented in our hospital. Examples of organizational recommendations not taken up included monitoring of vital signs on a medium-care unit and purchasing smart pump technology [34]. A number of reasons can be identified for the failed implementations of recommended improvements. First of all, in a large number of cases, adjusting or developing protocols/guidelines was not implemented because someone responsible for this task was lacking. Secondly, a number of recommendations were not specific or too “big”, such as “improve medical record keeping”, so they were also not acted upon. Third, as another possible explanation for the lack of implementing organizational changes, for instance “purchase software for prescribing medication safely”, this was not easily feasible in a single-care unit in a large university hospital. Fourth, some recommendations required action by external parties, such as manufacturers of ventilators, for the suggestions to improve the safe use of medical equipment. Another possible explanation is that acting upon the recommendations was left to the discretion of the departmental heads who might not feel “a sense of urgency”. A lesson to be learned from this is the necessity of appointing a responsible person for the implementation of the recommendations. Another issue that needs to be addressed is that implementing change requires expertise and specific knowledge [6]. Quite often, the involved employees (both nurses and physicians) have not been taught how to implement algorithms and procedures and therefore need support. On the other hand, non-compliance to guidelines and protocols remains a major barrier in implementing improvement measures [6]. Management and senior staff are the statutory stakeholders to address this problem and should facilitate and support employees trying to improve care. We recommend that both leaders and front line staff should be held accountable for initiating and developing improvement interventions. In Table 4, a number of recommendations for improving take-up are detailed, partly based on presentations held at the International Forum on Quality and Safety in 2013 and 2014 by the Institute for Healthcare Improvement (Boston, USA). Failure to act on these serious incidents can be very discouraging. Staff can easily perceive this as a sign that the management does not give priority to patient safety, thus hampering the development of a culture of learning from errors that is essential to improve patient safety. Furthermore, tracking the effect of a change is key to determining whether it was indeed an improvement. It will likely take years of careful monitoring of outcomes and adverse events before conclusions can be drawn on the effect and worthiness of incident analysis and safety management. A limitation in this report could be the small number of incidents analysed; however, the purpose of our report was to reveal how incident analysis provided insight into the vulnerabilities in the care and how these can be turned into recommendations. Apart from these 17 incidents, 12 other incidents in the PICU had actually caused serious harm. These had not been reported to the departmental head, and critical incident analysis had therefore not been performed. Thorough analysis of all reported critical incidents in the PICU (480 with potentially serious consequences) was also not feasible due to time constraints. There is no consensus in the literature how to determine which incidents should be analysed for the most benefit. In our experience, it is best to start with the incidents that have caused serious harm. However, analysing the ones where serious harm was (just) avoided might reveal important clues to potential preventive measures. It seems worthwhile to set up a structure that allows for fast detection of serious incidents and that enables management to respond quickly and request analysis. Analysing critical incidents should become routine procedure and not remain coincidental activities.

Table 4 Recommendations for successful implementation of recommendations
Full size table

Conclusion

In this report, we describe the results of 17 critical incident analyses conducted in an academic children’s hospital by a team of patient safety officers and front line staff familiar with the working conditions. This setup increases the likelihood of identifying all possible factors involved in an incident and thus allows articulating recommendations that might prevent similar incidents. One third of the recommendations were taken up, mostly those related to teamwork and protocol/guidelines. To determine whether critical incident analysis is worth the time investment, we need to reach a stage at which more incidents are analysed and more recommendations are implemented. Also, we need to monitor the outcomes of the care we provide, to assess the effectiveness of the implemented recommendations. Then, we can determine whether incident analysis improves patient safety and thus is worth the effort.