Extracorporeal membrane oxygenation (ECMO), a form of temporary cardiopulmonary support, is one of the most intense and technologically complex therapies offered in medicine. It is a low-volume, high-risk procedure whose management requires quick and decisive application of knowledge as well as technical and behavioral prowess to solve multiple problems that may arise.

Currently, there is no standardized certification process for an ECMO specialist. The Extracorporeal Life Support Organization (ELSO) has maintained specific guidelines for designing ECMO programs and determining their use for training and educating. It is assumed that these guidelines are used as a structure for each ECMO center to develop its institution-specific guidelines and policies.

The “ECMO specialist” is defined as “the technical specialist trained to manage the ECMO system and the clinical needs of the patient on ECMO under the direction and supervision of a certified ECMO-trained physician”. Traditional training is primarily didactic in nature and usually, but not always, complemented with various hands-on training modules using a water-filled ECMO circuit [7], animal laboratory sessions, and if available, bedside training sessions with an experienced specialist as the preceptor.

Simulation courses with mannequins are available at a few institutions as supplemental training, but training with these devices is not considered mandatory for certification. Each specialist is evaluated by the preceptor or examiner for his or her skill and competence and undergoes a written or oral exam. Institutional certification is granted after completion of the training course and passing the examination.

As described in other studies, simulation-based training programs seem to be the preferred training due to the realistic simulation in real time and the clinical cues generated by the mannequin [6]. Moreover, in situ simulation at the point of care shows benefits because it facilitates integration into routine clinical schedules [15, 16]. In addition, simulation education can be used as a tool to identify potential threats within hospital systems and procedures [10, 14].

To initiate an ECMO program in our busy 10-bed dedicated pediatric cardiac intensive care unit (PCICU), we developed an ECMO training curriculum integrating simulation to train multidisciplinary health care professionals at the Le Bonheur Children’s Hospital in Memphis, Tennessee (LBCH).

This study examined the impact and efficacy of integrating a simulation-based training module for credentialing and competency testing of ECMO specialists. In situ simulation is applied to the highly specialized care environment of the PCICU. Emergencies in the PCICU involve complex physiology and require interactions among multiple care providers including physicians (cardiologists, cardiac intensivists, cardiac surgeons), nurses, respiratory therapists, and ECMO specialists. To optimize patient outcomes, providers must be able to perform specific technical and cognitive skills and to communicate and coordinate multiple tasks efficiently [1].

We hypothesized that multidisciplinary, simulation-based training improves knowledge, comfort, and confidence levels among participants. The first goal of this study was to assess whether simulation would improve technical and nontechnical proficiencies, leading to correct identification and optimal management of emergencies for a simulated ECMO patient. The second goal was to assess whether our ECMO specialists and physicians retained their skills by reassessing them within a 6- to 8-month period.

Methods

Course Design

The ECMO training curriculum was a multidisciplinary training program. The participants were physicians, registered nurses, nurse practitioners, and respiratory therapists. All cardiology fellows were required to participate. Two pediatric intensive care fellows and two neonatology fellows also chose to participate. Nurses, nurse practitioners, and respiratory therapists were chosen by the ECMO coordinators for their suitability to become ECMO specialists based on the following criteria. Except for one nurse, no other participants had previous training in ECMO. One nurse had “traditional ECMO training” without simulation training. Participation in the research aspects of the course was voluntary, but to acquire ECMO certification, training was mandatory.

The curriculum consisted of didactic lectures, hands-on experience on the ECMO circuit, and emergency ECMO scenarios performed on a mannequin. During the simulation scenarios, subjects replicated the composition of a PCICU clinical team. Extensive debriefing followed each scenario.

Pre- and postparticipation questionnaires were obtained electronically using Survey Monkey (Survey Monkey®, Palo Alto, CA) and used to determine basic demographic data, and questions regarding the participants’ perception of knowledge, ability, and confidence level were assessed with Likert scale responses (Table 1). We collected demographic data on gender, role/profession, unit of employment, years of clinical experience, and status of previous ECMO training.

Table 1 Likert scale for knowledge, ability and confidence level
Full size table

Identical surveys were used at four time points: before training, after training, at the reevaluation test (6–8 months after the initial test), and before and after the second simulation test. Subjects’ names were omitted, and a random identification number was assigned to track their compliance as well as to facilitate comparison with their own surveys.

By the end of the training, the subjects were required to pass a written test (90 % successful responses were required for passing) and successfully demonstrate competency on two randomly selected scenarios with a team. Both criteria had to be fulfilled for a subject to pass the practical test. Two ECMO-trained physicians (S.S., M.F.) judged the performance of the individual trainees during the simulation test on predetermined specific trigger events and expected interventions. Both examiners were blinded to each other’s results Fig 1.

Fig. 1
figure 1

Flow diagram of the study design

Full size image

The training course was conducted over a 6 day period, with 9 h of training per day embedded in 2 days each week, for a total of 3 weeks. The didactic lectures took place during the first 4 days, followed by a full day of simulation training and the exam day (written and practical) on the last day.

At registration, the participants received a training manual including a pretest. The training manual was written by the ECMO physicians and coordinators at this institution in accordance with existing ELSO guidelines.

Didactics

The didactic lectures, conducted over the first 4 days of the training, were structured according to the ELSO guidelines. The lecturers included ECMO-trained nurses and physicians. Didactic instruction included information on the pathophysiology of common disease processes requiring ECMO, basic hemodynamics, ECMO physiology, circuit anatomy, cannula placement, hemostasis on ECMO, and the basics of simulation. Special indications such as ECMO for congenital diaphragmatic hernia and cardiac support as well as initiation of ECMO during cardiopulmonary resuscitation (ECPR) were specifically discussed. Take-home case presentations were given at the conclusion of each day and discussed the next day. Water drills were performed at the end of the day when the ECMO coordinator described and demonstrated potential problems that could arise during ECMO.

The trainees were encouraged to participate verbally and physically in problem solving. They were given the opportunity to familiarize themselves with the circuit and to perform emergency procedures that included changing out tubing and pigtails.

Simulation Scenarios and Debriefing

The fifth day was the “simulation day” designed to allow a hands-on management of ECMO emergencies in an environment that posed no risk to patients. All the participants were familiarized with the high-fidelity simulation mannequin specially designed for ECMO training. This mannequin was configured by our team (R.D., A.P., S.S.) for ECMO simulation and included ECMO cannulas, which were attached to an ECMO circuit.

The mannequin was a basic, full-body cardiopulmonary resuscitation (CPR) dummy. We removed all previous components from the chest cavity and replaced them with a tubing loop and a bladder reservoir. A bellows mechanism was placed to simulate lung function during ventilation. An arterial line was placed in the femoral area that allowed the learner to draw “blood”. The latter was developed using water and chocolate coloring. A central venous line also was placed in the femoral area for the administration of medications and fluids. A manifold within the chest allowed the simulation operator either to infuse or to withdraw fluid from the circuit to simulate fluid overload or hypovolemia.

All course scenarios were conducted in situ within an actual intensive care unit (ICU) room using the same equipment (ventilator, defibrillator) and supplies (code cart, emergency medications) that were being used for real patients to maintain authenticity and reduce anxiety associated with unfamiliar equipment during testing. Basic vital signs and physiologic data including central venous pressure, arterial blood pressure, end-tidal carbon dioxide tracing, and an electrocardiogram were displayed in real time on a bedside monitor in accordance with the given ECMO emergency scenario.

To optimize authenticity and also to ensure clinical relevance for participants, we derived scenarios from common ECMO emergency situations, as shown in Table 2. The team or the individual had to act on physiologic data from the monitor, examination of the mannequin, and the ECMO circuit. Progression of the simulation scenario was controlled by course facilitators in real time in response to interventions applied by the care team. Scenarios were designed to include specific training goals and objectives including introduction to new equipment as well as specific intensive care unit guidelines and procedures.

Table 2 Simulated extracorporeal membrane oxygenation (ECMO) emergency scenarios
Full size table

The participants were expected to provide care within the context of a scenario the same way that they would during a clinical encounter within the limitations of the specific mannequin. This included drawing of blood for laboratory studies, administration of medications, airway management, and performance of CPR. Certain scenarios required participants to call for certain procedures such as recannulation or chest tube placement, although these were not performed at the mannequin.

As an example, Fig. 2 outlines the clinical progression of a scenario used, in which there was air in the venous site of the ECMO circuit. The learning objectives for this scenario included recognition that the air was prebladder (venous) and thus required no cross-clamping, assessment for leaks, knowledge of how to remove air, assessment of the bubble detector, and confirmation of the correct cannula position with a chest X-ray. These learning objectives were our critical performance criteria for passing the practical test.

Fig. 2
figure 2

Flow chart of the critical performance criteria in a venous air emergency scenario

Full size image

The typical scenario lasted approximately 15–30 min. During the training session, debriefings were facilitated by the coordinators and occurred immediately after each scenario. Debriefings typically lasted two to three times as long as the accompanying scenario. All the scenarios mentioned in Table 2 were conducted. No debriefings were performed during the test. Figures 3, 4, 5, 6, and 7 outline the clinical progression of the other scenarios.

Fig. 3
figure 3

Flow chart of the critical performance criteria in a sick circuit emergency scenario

Full size image
Fig. 4
figure 4

Flow chart of the critical performance criteria in a power failure emergency scenario

Full size image
Fig. 5
figure 5

Flow chart of the critical performance criteria in an arterial air emergency scenario

Full size image
Fig. 6
figure 6

Flow chart of the critical performance criteria in a hole-in-tubing emergency scenario

Full size image
Fig. 7
figure 7

Flow chart of the critical performance criteria in a hole-in-raceway/raceway change-out emergency scenario

Full size image

Examination

The last day of the curriculum was the examination day, which was divided into two sessions. During the first session, conducted in the morning, the participants had to answer a total of 100 questions on the written test within 2 h. The questions were compiled by the facilitators according to the topics given at the didactic lectures, case presentations, and water drills. A 90 % correct response rate was required for a participant to pass the test.

At the next session, in the afternoon, the participants had to perform the simulation-based practical test consisting of two ECMO emergency management scenarios. The two scenarios were chosen randomly from a list of previously simulated ECMO emergency scenarios (Table 2). The participants were tested in accordance with their clinical capacity, either as physician or as ECMO specialist.

Two ECMO-trained physicians evaluated the simulation test according to predetermined critical performance criteria specific for each scenario, as shown in Figs. 3, 4, 5, 6, 7, and 8. The participant had to fulfill all the criteria to pass the scenario. In addition, the participant had to pass both scenarios separately to pass the practical test.

Fig. 8
figure 8

Box plot of knowledge scores

Full size image

The participants were required to pass both the written and the practical test to be certified as ECMO practitioners. Nurses, respiratory therapists, and nurse practitioners who wanted to be certified as ECMO specialists were required to undergo further bedside training as per ELSO guidelines.

Reevaluation

The subjects were required to participate in a repeated exam within 6 months after the previous exam for reevaluation to assess maintenance of competency skills. The reevaluation consisted only of a simulation-based practical exam, proctored in similar fashion to the previous exam except that the ECMO coordinators evaluated it and not the previous two ECMO-trained physicians. Pre- and postparticipation questionnaires were obtained. The Institutional Review Board approved the study.

Statistical Analysis

The results were analyzed with IBM SPSS 19 statistical software (IBM Corp®, Armonk, NY). The variables comprising ability, confidence, and knowledge were presented as group medians and interquartile ranges (IQR 25–75 %) because they were not normally distributed. The experimental design consisted of repeated measurements.

Each category was tested initially with Friedman’s analysis of variance (ANOVA). Subsequently, the preplanned comparisons were tested with the paired Wilcoxon test. Bonferroni correction was applied, and all effects were reported at a 0.05/n (n = number of comparisons) level of significance. The results were depicted as box plots. The written test results were presented as means and standard deviations.

Results

The participants in the initial curriculum were 26 health providers. Their demographics are shown in Table 3. Of the 26 participants, 24 passed the written and practical tests. One participant failed the written test, whereas another failed the practical test. A total of 22 participants scored the didactic and scenarios education as useful, at 4 or higher (5 = very useful), in improving their perception of their overall knowledge, ability to fulfill the required critical performance criteria in simulated ECMO emergencies, and overall confidence. Those able to act as code leaders, chosen by their clinical role, felt more comfortable after the training. The written test score results were 94 %.

Table 3 Demographic data of the 26 participants
Full size table

Of the previous 26 providers, 22 participated at the reevaluation practical exam, performed within a 6- to 8-month period to assess maintenance of competency skills. The missing providers included the two who failed the initial exam and two who decided not to participate in the reevaluation due to personal reasons.

All 20 participants who appeared for the reevaluation demonstrated success in the simulated ECMO emergencies. Of the 20 participants, 18 answered the questionnaires completely.

Data Analysis

The knowledge median score at pretest 1 was 2 (IQR, 1–2). It increased to 3 (IQR, 2–4) after the test and dropped to 2 (IQR, 2–3) before the second test (6 months later). It increased to 3 (IQR, 2–3.25) after the second test. The knowledge test scores changed significantly over the four times the test was taken (Friedman ANOVA test χ2 [15] = 31; p < 0.001). Wilcoxon tests were used to follow up this finding. Bonferroni correction was applied to report all effects at a 0.0167 level of significance. The participants’ knowledge increased significantly (<0.0167) after the first time the test was taken (T = 153; r = −0.771. It also was significantly increased after the test was taken a second time (T = 55; r = −0.701). The knowledge dropped significantly after the first test and before the second test (T = 91; r = −0.791). The results are depicted in Fig. 8.

The ability median score at pretest 1 was 2 (IQR, 2–3). It increased to 3.5 (IQR, 3–4) after the test and dropped to 3 (IQR, 2–3) before the second test (6 months later). It increased to 3 (IQR, 3–4) after the second test. The ability test scores changed significantly over the four times the test was taken (Friedman ANOVA test χ2 [15] = 31.97; p < 0.001). Wilcoxon tests were used to confirm this finding. The significance was set at 0.0167 after Bonferroni correction. The ability increased significantly after the test was taken the first time (T = 136; r = −0.747). It also was significantly different after the test was taken the second time (T = 36; r = −0.640). The ability dropped significantly after the first test and before the second test (T = 78; r = −0.772). The results are depicted in Fig. 9.

Fig. 9
figure 9

Box plot of ability scores

Full size image

The confidence median score at pretest 1 was 2 (IQR, 1–2). It increased to 3 (IQR, 2–4) after the test and dropped to 2 (IQR, 2–3) before the second test (6 months later). It increased to 2.5 (IQR, 2–4) after the second test. The confidence test scores changed significantly over the four times the test was taken (Friedman test χ2 [15] = 26.55; p < 0.001). Wilcoxon tests were used to follow up this finding. The significance was set at 0.0167 after Bonferroni correction. The confidence increased significantly after the first time the test was taken (T = 171; r = −0.790). It also was significantly different after the test was taken the second time (T = 50; r = −0.588). The confidence dropped significantly after the first test and before the second test (T = 91; r = −0.791). The results are depicted in Fig. 10.

Fig. 10
figure 10

Box plot of confidence scores

Full size image

Discussion

Simulation-based training has long been thought to improve team function in a high-acuity clinical situation [17]. Simulations appeal to trainees because they provide significantly more active learning experiences than traditional training experiences. This characteristic aligns simulation-based training with the tenets of adult learning as outlined by Bloom [4]. Adult learners are independent, self-directed, and internally motivated to learn. They seek immediate application of their knowledge and use their accumulated experience to craft their own ongoing intellectual development. Adult learners perform better when they can apply new knowledge and use past experiences to solve problems. Simulation-based training provides them with these opportunities.

Cognitive learning theory argues that learning does not automatically occur with the transfer of information from an instructor to a learner. A learner must process and apply the information to learn [12]. Simulation-based training emphasizes the learning environment as opposed to the teaching environment and by its design encourages trainees to participate actively in their own education. The use of simulation in medicine has been suggested as a useful teaching method for clinical situations that although infrequent are critical in nature and require that a high level of skill and preparedness be maintained.

The recommendations for resuscitation training from the International Liaison Committee on Resuscitation from 2003 include the specific recommendations that training should move toward “scenario-based, facilitated, interactive training” and that “high-fidelity simulation-directed training should increasingly supplement instructor-directed training” in advanced life support courses [5]. Thus, current advanced life support courses such as advanced cardiac life support (ACLS), pediatric advanced life support (PALS), and neonatal adjuvant life support (NALS) often include simulations, and it is common practice to have routine mock codes in pediatric residency programs.

Simulation-based medical training programs were first developed in adult anesthesia and more recently in critical care, neonatology, and obstetrics [8, 9, 11, 13]. The use of a simulation-based training program in the management of ECMO emergencies in either pediatric or adult patient populations was documented by Anderson et al. [2, 3].

We report the positive impact of a multidisciplinary in situ simulation-based ECMO training implemented for novice trainees. We show that participation in the course led to improved self-perception of knowledge, ability, and confidence among multidisciplinary team members with regard to managing future real crisis events.

To our knowledge, this is the first report of simulation-based ECMO training implemented within the highly specialized environment of the PCICU. Emergencies in the PCICU often involve complex physiology, and resuscitation frequently falls outside the scope of PALS guidelines. Knowledge of the latter is necessary but not sufficient for resuscitating patients in this environment, particularly ECMO patients. Simulation training reinforces the optimum application of context-specific skills. Moreover, in situ training emphasizes the realistic approach. Practice within the actual PCICU using unit-specific equipment has uncovered important latent safety threats within the system.

Postcourse and posttest evaluations demonstrate a significant increase in the participations’ perception of their knowledge, ability, and confidence levels. This would lead to improvement in their sense of preparedness and confidence regarding participation in future ECMO emergencies. It likely is due in part to the multidisciplinary nature of the course design, particularly during the simulation training.

Close collaboration between ECMO specialists, physicians, and registered nurses during the curriculum design serves several important purposes. First, it ensures that the scenarios and materials are true and relevant to clinical practice. Second, multidisciplinary involvement pays attention to the specific educational needs of all novice trainees (physicians, registered nurses, nurse practitioners, respiratory therapists) and models teamwork and collaborative practice to be applied to the real-world patient care setting.

This is a short-term outcome study, and assessment of the rate of skills degradation is beyond the scope of this study. Studies have shown that skills learned in traditional resuscitation courses degrade over 3–6 months, and our data demonstrate a similar trend [5]. Not surprisingly, the increase in knowledge, ability, and confidence levels from the first test does not last 6 months. Most of the ECMO specialists and physician trainees scored their knowledge, ability, and confidence levels lower at the 6 month follow-up preassessment than at the postassessment survey after their initial training. These findings show that a regular 6 month follow-up simulation test may benefit the ECMO specialists and physician trainees and may potentially improve their comfort and confidence levels.

Subgroup analysis separating the different professional roles was not obtained due to the small number of the participants. It would have been interesting to know whether there were any differences between physicians and other ECMO providers. The other providers usually have further intensified their training to become ECMO specialists with obligatory shifts at the bedside of an ECMO patient and thus have more real-life experiences. It is expected that they would have a better maintenance of their competencies. This also is expected for PCICU fellows versus PICU fellows and specialists with more shifts versus those with fewer shifts.

Since the need for ECMO for Cardiac indications has been low in our PCICU over the 8 month duration of this study, no trainee based quality performance results can currently be drawn.

Our study had a few limitations. First, our PCICU had just started an ECMO program, so the number of curriculum participants was small. Second, the fact that course facilitators worked closely with course participants may have introduced bias into the evaluation process because participants may have felt compelled to give the course favorable reviews. Third, we had no rational data to prove the perceptions of the participants to be true. Data were gathered using questionnaires and represent subjective evaluations of the participants’ own perceptions. Fourth, this study was not randomized. Each participant had the same simulation-based curriculum, and no comparisons were made with previous participants trained by the traditional curriculum. Finally, the two test scenarios were selected randomly at both time points (initial vs. 6 month follow-up scenario) among the seven previously simulated scenarios. Thus, the participants may have been tested with the same scenario on both occasions.

Attempts to improve the aforementioned limitations and further research should include a randomized trial comparing simulation-based training and traditional (without simulation) ECMO training, as well as ongoing data collection with new fellows and other providers for a larger number to improve the power of the study. Future studies may use trained facilitators or evaluators from other clinical specialties to eliminate the second limitation. Evaluation of the simulation scenarios may be done by videotape. Real-time and more objective demonstration of skills can be evaluated during ECMO emergencies, for example, by a detailed description of critical task performance during the emergency (similar to the description during a cardiopulmonary resuscitation). An attempt could be made to test all seven scenarios, although this is difficult to realize due to individual time constraints.

Conclusion

Simulation-based training is an effective method for improving the perception of knowledge, ability, and confidence levels among ECMO novice trainees. Simulations are conducted in controlled settings. Trainees must actively demonstrate technical and behavioral skills. Moreover, simulation-based training poses no risk to real patients, allowing trainees to make mistakes safely and learn from them.

Routine reevaluation (e.g., a simulation test every 6 months and a written test every 12 months) is recommended because of our findings to ensure maintenance of competencies.

Further research is needed to assess real-time demonstration of skills retention during ECMO emergencies. The ultimate goal is to have documented improved operational performance with real patients and improved patient outcomes. This goal may be difficult due to the rarity of the clinical events but may be possible with longitudinal multicenter studies.