Background

The most likely place for serious safety events to occur is in hospitals’ emergency departments. To minimize such occurrences, simulation-based training of emergency care teams is valuable [1]. Many factors, such as high uncertainty, high cognitive load, high decision density, multiple interruptions, and a narrow window for decision-making, are inherent to the emergency department and may affect care. Team performance is affected by modifiable factors, such as teamwork quality, communication, overcrowding, environmental noise, inadequate supervision, lack of feedback, fatigue, sleep deprivation, multiple handovers of care, and even the department architecture [2].

During in situ simulation, professionals practice in an area where clinical care occurs [3]. Instead of training in a simulation center with incomplete teams and professionals that will not necessarily work together, in in situ simulation, the actual interprofessional care teams interact in their own environment. This simulation modality enables assessing system competence and detecting latent safety threats (LST), or the conditions that may risk patient safety [4].

Patterson et al. (2013) assessed the impact of simulation-based training on pediatric emergency department staff conducted in a simulation center; they found changes in safety attitudes of interprofessional team and significant reduction in serious safety events [5]. In two different follow-up studies by the same group of researchers, the findings were as follows: a 0.8 LST was found for each in situ simulation performed [4]; emergency department providers’ preferred in situ training over training in a simulation center; and high quality teamwork among professionals was developed because of frequent simulation training [6].

In procedural simulation, a task trainer is used to assist in the process of learning to complete a technical skill or a procedure. In just-in-place task training, skills are practiced in an area they are most likely needed, whereas in just-in-time task training, skills are practiced at moments close to the need of using them [7]. Both simulation-based techniques allow deliberate practice to prepare for low frequency, high acuity events. To practice in this manner likely allows higher skill retention and is a rational use of simulation resources, delivering training for those in most need of it [8,9,10].

We designed a hybrid training program after a serious safety event occurred (Table 1); a young adult nearly died during care due to technical difficulty in handling airway equipment and breaks in communication between interprofessional team members.

Table 1 Narrative of serious safety event that inspired need for in situ simulation training
Full size table

In this hybrid training program, in situ simulation is combined with just-in-time and just-in-place task training in an emergency department. To our knowledge, this is the first study to combine these simulation-based techniques in a single training of adult and pediatric emergency department providers.

In this study, we aimed to detect different types of LST in this high-risk unit. We attempted this by using a debriefing checklist of in situ simulation scenarios. Secondary objectives were to compare rates of LST detection per team leader profession, shift schedules, and number of participants in the scenarios. We hypothesized that a simulation-based training in actual care areas allows the detection of at least one LST per simulation scenario.

Methods

This prospective observational study assessed the first year of training from April 2015 to March 2016.

The training was conducted at the emergency bay and at one of the observation rooms of the Ibirapuera Advanced Unit of Hospital Israelita Albert Einstein in Sao Paulo, Brazil. This freestanding emergency department is physically separated by 7 km from our main hospital. It is a private secondary service, working non-stop with urgent and emergency care of adults and children. In 2013, it managed 64,891 patient consults, or an average of 5400 per month [11]. Before this training, a single in situ simulation scenario was conducted in 2013, resulting in the detection of four LST [12].

Analysis of the serious safety event described in Table 1 prompted establishment of interprofessional training. Since our facility is distant from the main hospital and the simulation center, it was difficult to schedule trainings for interprofessional teams; thus, the choice of in situ as the training strategy. Additionally, one of the study authors (TBC) had worked as visiting researcher at Cincinnati Children’s Hospital Medical Center, where he had contact with a well-established in situ simulation program [6]. During night shifts, staff was reduced and less experienced professionals were usually working. Management and senior providers thought this was a more vulnerable time, so it was decided to distribute trainings during all work shifts. Initial planning was to have weekly unexpected simulations [13], but logistical constrains of having to transport simulators and concerns with acceptability led to concentrating scenarios in this simulation week format.

The study population was 135 health care providers, including physicians, nurses, and nurse technicians working at the unit during the study period who could potentially be part of the teams caring for emergency cases. Even though the different work shifts were included in the training, simulations were scheduled independently of any individual professional’s work schedules. Therefore, the selection of participants for each scenario was by convenience. Our goal was to reach at least 80% of the providers during training (108 participants).

The training involved in situ simulation scenarios, in which a simulator was treated at an emergency bay by interprofessional teams working on that shift, followed by debriefing combined with stations of just-in-time and just-in-place self-directed task trainings. Training objectives were to increase safety awareness by detecting LST, to improve management of critically ill patients admitted to the emergency bays, and to reinforce crisis resource management (CRM) principles. In short, ultimately, the goal was to foster patient safety in the emergency department.

Four themes were defined as critical in our needs assessment, which was conducted by consulting the management and senior staff of the hospital and analyzing previous serious safety events at our institution’s emergency departments.

Three scenarios (pediatric, adult, and adolescent patients) were constructed for each theme and alternately repeated during simulation weeks, which occurred every 3 months (Table 2). All scenarios had technical objectives; the management of a specific emergency and objectives based on Gaba’s CRM key points [14].

Table 2 Themes and scenarios for each simulation week
Full size table

The patients in these scenarios could be treated by clinical or pediatric teams. We planned to conduct 14 scenarios in weeks 1 and 2, and 15 in weeks 3 and 4, totaling to 58 scenarios. All scenarios utilized simulators (SimMan 3G, SimBaby or SimNewB, Laerdal) and actual drugs and equipment available at the unit were used. Scenarios were scheduled at fixed hours, at times of few anticipated consults. They lasted for 10 min and were immediately followed by 10 more minutes of co-debriefing. Debriefing time was shorter than it would be at the simulation center since the emergency bay had to be ready for patients and participants were on regular work hours and expected to resume patient care immediately after scenarios. The co-debriefing was conducted by two facilitators (a physician and a nurse) that were part of the unit staff but were not on clinical hours. The following clear cancelation criteria were established to not risk patient care: medical supervisor decision, patient at the emergency bays or less than three available professionals. Based on literature, we expected a cancelation rate around 25% [4].

In the week prior to each simulation week, professionals received by email study materials related to the weekly theme, with institutional protocols and key publications. During simulation weeks, an observation room was converted into a training station, with task-trainers and step-by-step guides to procedural skills related to the theme chosen (Table 2). The professionals were encouraged to practice these skills in a self-directed manner.

Categories of LST were measured by analyzing the debriefing checklist filled out by the two facilitators after each scenario. The debriefing checklist was modeled after a similar instrument used by Patterson et al. [4], adapted to the Portuguese language (Additional file 1). Both facilitators completed the checklist during the scenario and the debriefing, and immediately after debriefing they met to compare notes and deliver a final joint version, which was analyzed by the study team.

Since the emergency department treated adult and pediatric patients, team leaders could be emergency physicians or pediatric emergency physicians. There was no fixed team compositions pre-established and scenarios occurred at different hours. Information on team leader, participants and time of simulation was collected to allow comparisons of LST detected per team leader profession, shift schedules, and number of participants in the scenarios.

The Hospital Israelita Albert Einstein’s ethics review board approved the study design, with approval number CAAE 54071816.8.0000.0071.

Statistical analysis

Categorical variables were described by absolute and relative frequencies; and numerical variables, by means and standard deviations, if they follow a normal distribution, or by medians and interquartile, if otherwise. Numerical variables also included minimum and maximum values. Since it did not present in a normal distribution, the number of LST was compared between groups using Mann-Whitney and Kruskal-Wallis tests.

The software package R version 3.2.2 (R Core Team, 2015) was used. The significance level adopted was 5%.

Results

One-hundred and fourteen professionals (84% of the population) participated in this study. Most participants were nurse technicians, which would be the Brazilian equivalent to a licensed practical nurse, representing 43.9% of the participants. The presence in self-training was verified, in its own worksheet, with 101 participants. Figure 1 (participant flow diagram) details characteristics of participants.

Fig. 1
figure 1

Participant flow diagram

Full size image

The number of scenario cancelations was nine, making the final total number of scenarios to 49 of the 58 proposed. The number of professionals in each scenario ranged from 3 to 36 participants, with a median of 11 (7.0; 13.0). This number included all debriefing participants, which included observers and not necessarily all professionals that treated the simulator. There was no correlation between the number of participants and detected LST, with a correlation coefficient of − 0.08 (p = 0.602). The number of scenarios that each professional participated ranged from 1 to 32, with a median of 3 (2.0, 5.25). Fifty-six LST were observed in 49 scenarios, with an average of 1.1 LST per scenario. There was no significant difference in the distribution of LST between categories of team leaders, themes, and shifts (Table 3).

Table 3 Latent safety threats per scenario theme, shift, and team leader
Full size table

The LST were divided into four categories, namely, medication (e.g., dilution errors, administration, and dosing) with 11 (19.6% of total) LST detected, equipment (e.g., difficulties with defibrillator, ventilator, and intraosseous access) with 23 (41.1%) LST detected, teamwork (e.g., absence of the leader, miscommunication, and unclear division of roles and responsibilities) with 12 (21.4%) LST detected, and others (e.g., laboratory-, personnel-, and radiology-related matters) with 10 (17.9%) LST detected. There was a higher proportion in equipment-related LST, with 41.1% of the total number of LST (p < 0.01). Table 4 shows LST by category.

Table 4 Latent safety threats by category
Full size table

Discussion

This study assessed the effect of combining in situ simulation scenarios with just-in-time and just-in-place self-directed training in an emergency department on detecting LST. The results are consistent with a previous study on in situ simulations at a pediatric emergency department [4] and review articles indicating that simulation is an effective way of detecting LST [15,16,17].

The rate of LST detection was slightly higher than expected (1.1 per scenario), with a higher proportion of equipment-related than medication and teamwork-related LST. There was no specific theme related to a higher detection of LST. Detecting LST also did not differ among shift schedules and team leader specialties. Some of the equipment needed in the scenarios, such as intraosseous needles [18] and video laryngoscope [19], are often not used by all emergency physicians. New types of equipment, such as pads with resuscitation feedback and a new pediatric ventilator, were also introduced to the unit closely to training, which might explain the higher proportion of equipment-related LST.

LST detected during training were communicated to the emergency department management, which in turn made efforts to mitigate LST found. Part of the strategy to respond to these threats was training-related, mitigated by availability of task trainers and equipment for procedural training during simulation weeks. This was especially important since many LST were related to unfamiliarity with equipment and difficulties in technical procedures. Pharmacists and pharmacist assistants were also included in in situ training after the first week, since medication issues were also often found. Some equipment was acquired in response to threats found, like ultrasound to assist central line catheterization. Other LST were related to system and organizational issues, and changes were made, like changing location of essential materials to ease access, acquiring a white board to annotate verbal orders in emergency situations, and making cognitive aids like table with doses of pediatric medications available. One major change in the emergency room dynamic was also adopted, with fixed emergency teams with pre-defined roles composed of six to eight professionals, to avoid overcrowding or lack of professionals in emergency situations.

This study has limitations. First, the single-center nature of this study makes the results not generalizable. Second, our main outcome, the detection of LST is only a Kirkpatrick level 3 outcome, with safety behavior being assessed during training which could lead to changes in the clinical environment. Although efforts were made to address LST found, we did not establish a formal way of measuring strategies to mitigate any LST found in the study. More important than the location of a simulation scenario is what this activity is trying to accomplish, as often healthcare simulation, and in situ simulation in particular, include a range of behaviors required for safe and effective clinical practice, which might dilute efforts and make it difficult to assess improvement [20]. The in situ program was considered a great diagnostic tool, but it lacked measurement of level 4 outcomes, which could show training-related improvement from clinical outcomes [21]. Another limitation was the short debriefing time, which might have prevented in-depth discussion of some of the LST detected. We classified our LST according to pre-established criteria, which might not describe them as well as a less strict categorization. We relied on our facilitator’s perception of LST detected during scenarios and debriefing and did not use video review to assess LST, which might have permitted a better understating of the nature of LST [22].

Although it was not anticipated as an outcome of this study, the number of days between serious safety events in our institution’s emergency departments has improved from every 122 days to the current rate of once every 365 days since the beginning of our in situ program. We cannot establish a clear causal relationship between training and this outcome, but this may have been a contributing factor to the improvement.

This first year of in situ simulation was considered very successful, initiating a culture change in our emergency department, breaking hierarchies, and allowing a better perception of safety issues. The in situ simulation program has been expanded since it now comprises five different emergency departments within our hospital system. Based on the results of this study, we changed the task training from exclusive self-directed to both self-training and instructor-led training within each simulation week. We also established a formal link between the simulation center and quality and patient safety departments to address the LST discovered during training. We intend to study this expanded program in the future, including measuring level 4 outcomes, such as reduction of serious safety events.

Conclusions

A simulation-based training program, which combined thematic weeks of scheduled in situ simulation scenarios with just-in-time and just-in place task training in an emergency department in Brazil, allowed a high rate of detecting LST regardless of the training theme.