Background

Cardiovascular disease (CVD) accounts for 30% of all-cause mortality. On average, over 17.5 million people die of CVD annually worldwide, with approximately one death every 10 s [1]. Out-of-hospital cardiac arrest (OHCA) claims nearly 1.5 million lives annually [1]. High-quality cardiopulmonary resuscitation (CPR) is essential to successful OHCA. The American Heart Association (AHA) CPR Guidelines encourage closing the knowledge–practice gap and saving more lives [1]. However, a lack of practice remains a common complaint in medical education [2]. Simulation, as a novel method, is increasingly being used for the education of medical professionals. Simulation-based education has demonstrated many research benefits in improving skill performance, knowledge, and patient outcomes [3, 4]. High-fidelity (HF) manikins are widely used as part of the experiential learning component of advanced life-support (ALS) courses [5]. HF simulation provides real-time feedback on chest compression rate, depth, and recoil during advanced cardiac life support (ACLS) training and pediatric advanced life support (PALS) [5, 6].

Learners can assess physical findings and make clinical decisions regarding the simulated patients [5]. This may be ideal for providing opportunities for paramedics and medical students to practice their theoretical knowledge in simulated environments [7]. Studies have demonstrated the benefit of simulation training in various aspects of medical training [8, 9]. For ACLS training, the use of HF simulation has also been indicated to improve the knowledge translation of ACLS training [10,11,12,13]. Recent systematic reviews have revealed moderate benefits for improving skill performance in ALS training. However, research findings examining the benefit of HF simulation compared with low-fidelity (LF) simulation have yielded mixed findings. Several studies have demonstrated no distinct advantage of HF over LF simulations. Hence, teachers are perplexed by the conflicting nature of the evidence. HF is costly and time-consuming, as it requires specialized personnel, equipment, and space [14]. Therefore, a systematic review and meta-analysis to synthesize the current evidence is needed to assess the effectiveness of HF simulation in the learning process of ALS. This study aimed to identify the educational efficacy of HF simulation compared with no simulation or LF simulation in ALS training.

Methods

This systematic review and meta-analysis was performed according to the Cochrane Reviewers’ Handbook and presented according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines. The protocol for this systematic review and meta-analysis is available at PROSPERO (CRD42022333898).

Trial search

Randomized controlled trials (RCTs) were identified from PubMed, ISI (Web of Science: Science Citation Index Expanded), Cochrane Library (2022, Issue 8), China Knowledge Resource Integrated Database, World Health Organization Global Index Medicus, and Chinese Biomedicine Database from their inception dates to April 31, 2022. The search keywords and MESH terms were (“simulation” OR “patient simulation” OR “mannequin” OR “manikin”) AND (“life support care” OR “advanced life support” OR “neonatal resuscitation” OR “infant resuscitation”) AND (“education” OR “training” OR “teaching”). References from the RCTs were browsed, and the corresponding authors were consulted for any further information that they have not reported publicly. Ongoing RCTs were reviewed using clinical trial registers. The complete terms and strategies for identifying these articles are listed in the supplementary document (supplement 1).

Inclusion and exclusion criteria

Only RCTs assessing the efficacy of HF simulation or manikins in ALS training in any language were included. Trials that did not address any of the primary or secondary outcomes were excluded. Intervention groups that received LF simulations and traditional training at any stage were accepted. All the participants were medical students and practitioners.

Outcome measures

The primary outcome was the improvement in knowledge and skill performance. The secondary outcomes included the confidence of participants at the course conclusion, satisfaction of participants at the course conclusion, skill performance at one year; skill performance in resuscitation (compression rate, compression depth and compression fraction), teamwork behaviors and patient outcomes.

Study selection and risk of bias assessments

The studies identified from the electronic searches were evaluated independently by 2 researchers (W.K. and Q.Z.) using a study eligibility form based on the inclusion criteria. Relevant studies were initially screened using titles and abstracts. Potential articles for inclusion were independently assessed by the two reviewers (W.K. and R.S.Q.), and any dissentions were resolved by a third reviewer (W.M.X.). The methodological quality of the included studies was assessed based on the Cochrane Reviewers’ Handbook [15]. Each study was evaluated for bias using the following items: randomization sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, and other bias. If both allocation concealment and randomization had a low risk of bias and all other items had a low or unclear risk of bias, the trials were graded as high quality [16]. If either randomization or allocation concealment had a high risk of bias, the trials were considered of low quality, regardless of other items. If the trials did not meet the criteria of high or low risk of bias, they were graded as moderate quality.

Data extraction and analysis

Data were independently extracted by two researchers (W.K. and Q.Z.) from the full text of the studies and compiled into shared sheets. The following data were collected from the included studies: study identifier (lead author and year of publication), country of origin, duration of ALS training, study design, inclusion and exclusion criteria, HF simulator type, number of participants, and primary and secondary outcomes. Only the data of interest were extracted when the trials had more than two group designs and permitted multiple comparisons. The data were validated by a third reviewer (W.M.X.) using a standardized method. In the case of inconclusive or missing data, the original authors were contacted to obtain missing details. RCTs reporting the same level of outcome were included in the quantitative synthesis. Risk ratio was used to report discrete numerical variables along with 95% confidence intervals (CIs). The standardized mean difference (SMD) was reported to estimate continuous outcomes. A fixed effect model was used when heterogeneity across studies was not detected. Otherwise, a random effect model was used, then the source of the heterogeneity would be analyzed by subgroup analysis. The I2 statistic was used to quantify heterogeneity, and forest plots were generated and counterchecked by two reviewers (S.Q.R. and W.M.X.). If I2 < 25%, the pooled outcomes were considered to have low statistical heterogeneity; if I2 > 75%, the pooled outcomes were considered to have high statistical heterogeneity. We also performed sensitivity analysis by the sequential removal of trials for each outcome. Publication bias was assessed by the funnel plots when no less than 10 trials were included in the meta-analysis. If data which were reported instead of mean and standard deviation (e.g., in case of median and range), we would transform it by the methods created by Hozo et al. and Higgins and Green. Data were synthesized using the RevMan 5.4 software. The research protocol, outcomes, and relevant items in this systematic review are reported following the PRISMA Statement [17].

Results

Included studies

A total of 4,046 studies were identified across databases in the initial literature review. Of these studies, 3,745 were excluded after reviewing the titles. The initial screening resulted in 57 candidate studies from a review of the abstracts. The reasons for exclusion are detailed in the PRISMA diagram (Fig. 1). In total, 25 RCTs [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42] were identified for full analysis based on the inclusion and exclusion criteria. The design characteristics of the included studies are presented in Table 1. Of the included studies, eighteen studies compared HF simulation with LF simulation [18, 19, 21, 23, 25, 26, 28,29,30,31,32,33,34,35,36,37, 41, 42], and the other seven studies [20, 22, 24, 27, 38, 39, 42] compared traditional training. In total, 1,987 participants were included in the meta-analysis, of whom 998 were randomized to use HF manikins, whereas 989 received LF simulation-based or traditional training.

Fig. 1
figure 1

PRISMA diagram detailing the literature search and the study selection/exclusion process

Full size image
Table 1 Overview of the RCTs Included
Full size table

Participants and intervention

Of the 25 included studies, the participants included medical students (1,129), nursing students (90), residents (715), medical officers and paramedics (53). SimMan 3G by Laerdal is the most widely used simulator. All studies reported details of ALS training, which included the AHA course and group training using HF simulation cases. Twelve RCTs [18,19,20, 25, 27, 31,32,33,34,35, 37, 40] reported training duration and test time.

Improvement of knowledge

Fifteen studies [18,19,20,21,22,23,24,25, 27, 31, 34, 36, 37, 40, 42] reported data on knowledge measurement at the course conclusion. Pooled data from the RCTs demonstrated a benefit in improvement of knowledge for HF simulation when compared with LF simulation and traditional training (SMD = 0.38; 95% CI: 0.18–0.59, P = 0.0003, I2 = 70%]. (Fig. 2). I2 test discover significantly heterogeneity. Then we performed subgroup and sensitivity analyses, which found variation in course design, participants types, and outcomes measures in these studies [19, 20, 24] might be the source of heterogeneity. Subgroup analysis revealed that a benefit in knowledge with HF simulation compared with LF simulation (SMD = 0.22; 95% CI: 0.09–0.35, P = 0.001, I2 = 0%) and traditional training (SMD = 0.71; 95% CI: 0.51–0.97, P = 0.00001, I2 = 29%). (Supplement 2). Furthermore, we also performed a subgroup analysis depending on years of participation of the participant. The result show that residents (SMD = 0.28; 95% CI: 0.13–0.43, P = 0.0003, I2 = 0%) and three- and four-year medical/nurse students (SMD = 0.37; 95% CI: 0.04–0.69, P = 0.03, I2 = 77%) were benefit from HF simulation. However, fist-year medical/nurse students had no benefit from HF simulation (SMD = 0.66; 95% CI: − 0.31–1.64, P = 0.18, I2 = 85%). (Supplement 3). Four RCTs [19, 32, 36, 40] measured knowledge 3 months after training and have reported that both groups suffered a loss of knowledge (P < 0.001). However, no significant difference was observed between the HF and control groups (P = 0.28).

Fig. 2
figure 2

Forest plot of pooled weighted standardized mean difference from RCTs that evaluated the effects of improving knowledge with high-fidelity simulation at course conclusion. TT, traditional training

Full size image

Improvement of skill performance

A total of twenty-one RCTs [18,19,20, 22,23,24,25,26,27,28,29,30,31, 33, 36,37,38,39,40,41,42] measured skill performance at the course conclusion. No significant difference was observed between the HF and control groups (SMD = 0.63; 95% CI: 0.21–1.04, P = 0.003, I2 = 92%). (Fig. 3) For high heterogeneity, we performed subgroup and sensitivity analyses, which found these studies [18,19,20, 22, 28, 29, 33, 39,40,41,42] with different training setting and outcomes measures might be the source of heterogeneity. In the subgroup analysis, a moderate benefit was observed for HF simulation compared with LF simulation at the course conclusion (SMD = 0.36; 95% CI: 0.16–0.57, P = 0.0004, I2 = 0%).(Supplement 4) However, no improvement in skill performance was observed in the HF group compared with those who received traditional training (SMD = -0.08; 95% CI: −0.37–0.22, P = 0.62, I2 = 0%). (Supplement 4). Furthermore, a subgroup analysis depending on years of participation of the participant also been conducted and found that only residents benefit from HF simulation in skill (SMD = 0.55; 95% CI: 0.05–1.05, P = 0.03, I2 = 86%). (Supplement 5). Only two RCTs [19, 40] measured skill 3 months after training and demonstrated that HF simulation was associated with moderate benefits for retaining ALS skills at 3 months after training (P ≤ 0.001).

Fig. 3
figure 3

Forest plot of randomized controlled trials that evaluated the efficacy of high-fidelity simulation for improving skill performance at course conclusion. TT, traditional training

Full size image

Confidence and satisfaction of participants

Regarding the secondary outcomes, five studies [20, 28, 31, 33, 37] surveyed participants’ confidence, and three studies [31, 33, 40] reported participants’ satisfaction. Quantitative pooled data revealed that there was no difference in participants’ satisfaction between the HF and control groups (SMD = 0.47; 95% CI: -0.06–0.99, P = 0.08, I2 = 77%) (Fig. 4). Similarly, the confidence of the participants and teamwork behaviors in the HF group was not different from that of the control group (SMD = 0.03; 95% CI: −0.21–0.28, P = 0.78, I2 = 0%) (SMD = 0.43; 95% CI: −0.05–0.91, P = 0.08, I2 = 79%) (Figs. 5 and 6).

Fig. 4
figure 4

Random-effects meta-analysis of studies comparing HF simulation versus LF simulation or traditional training and reporting participants’ satisfaction

Full size image
Fig. 5
figure 5

Fixed-effects meta-analysis of studies comparing HF simulation versus LF simulation or traditional training and reporting participants’ confidence

Full size image
Fig. 6
figure 6

Random-effects meta-analysis of studies comparing HF simulation versus LF simulation or traditional training and reporting teamwork behaviors

Full size image

Risk of bias within included studies

The risk of bias assessment of the included RCTs is summarized in Fig. 7. Performance bias existed in all RCTs because participant blinding to the level of fidelity is difficult to achieve. The concealment of 12 studies [18, 19, 25, 27, 31,32,33,34,35, 37, 40, 42] was unclear and incomplete. Meanwhile, two studies have reported moderate dropout rates [19, 33]. Finally, the risk of bias graph of the RCTs is presented in Fig. 8.

Fig. 7
figure 7

Risk of bias summary of the included randomized controlled trials. Green ‘+’ means low risk of bias, and yellow ‘?’ means unclear risk of bias

Full size image
Fig. 8
figure 8

Risk of bias graph of the included randomized controlled trials

Full size image

Discussion

The results of this systematic review and meta-analysis indicated that participants benefited from improving knowledge and skill performance at the course conclusion with HF simulation. The subgroup analysis showed a greater benefit in knowledge with HF simulation compared with traditional training at the course conclusion. However, the use of HF simulation in comparison to LF simulation and traditional training showed no benefit for knowledge at 3 months, teamwork, and participant’s confidence. Higher means of satisfaction were observed in the HF group.

To the best of our knowledge, this is the first systematic review and meta-analysis of RCTs to identify the efficacy of HF simulation in ALS training. The benefits of HF simulation for improving skill performance and satisfaction in this study are consistent with the results of previous studies [1, 6, 43]. Cheng et al. compared HF manikins with LF manikins in ALS training [1]. They identified that HF manikins are moderately beneficial for improving skill performance at the conclusion of ALS training. In our study, the subgroup analysis also demonstrated a small to moderate benefit in skill performance for HF simulation when compare with LF simulation, but no benefit for HF simulation when compare with traditional training. A meta-analysis of 15 studies assessed the educational efficacy of simulations of a neonatal resuscitation program (NRP). The results revealed moderate effects favoring HF simulation for improving resuscitation knowledge and skill performance at the course conclusion [6]. Mundell et al. have reported that computer-controlled manikins are slightly beneficial for improving learner satisfaction and skills in ALS training [43]. A systematic review of emergency medicine training has demonstrated that technology-enhanced simulation is associated with greater benefits than traditional training [44]. These low-degree benefits may reveal the limitations of simulation, which is just a device that provides realistic feedback. A high efficacy ALS training also needs optimized course design, the case scenarios, experienced instructors and debriefing sessions. Different types of participants may not get the same benefit from HF simulation. The subgroup analysis of our study demonstrated that residents and three- and four-years medical/nurse students benefit from HF simulation, but first- and second-years medical/nurse students not. So, the high efficacy of HF simulation may be achieved in proper participants with well-design course, experienced instructors and debriefing in ALS training.

The retention of ALS knowledge and skills is widely recognized as a significant factor in actual resuscitation. Only two RCTs in this study [19, 40] have demonstrated that HF simulation was associated with moderate benefits for retaining ALS skills at 3 months after training. However, HF simulation did not significantly improve long-term retention of resuscitation knowledge [19, 32, 40]. Lack of retention might result from the quality of the content and the limitation of the training duration. Course duration and spaced practice are two key points for retaining knowledge and skill performance in the long term [43,44,45,46,47]. Wayne et al. have revealed that the use of eight hour of additional simulation training is associated with a greater benefit for retaining knowledge compared with none [9]. Evidence demonstrates that knowledge and skills deteriorate at 3 months after training course without ongoing practice [45]. The actual resuscitation performance of participants could be suboptimal during this interval. Efficacy of ALS training may be improved by increasing the frequency of HF simulation training, which may protect against knowledge and skill deterioration [46]. Additionally, spaced practice may improve the efficacy of training through elaborate learning and process information into a deeper memory. Studies found that after the initial training, repetition after a period of rest (weeks to months) better learning than practice massed within a very short period [47]. Therefore, further studies need to focus on improving the long-term retention of resuscitation knowledge and skills.

Teamwork is the key to resuscitation in a real clinical setting, and it could be improved through briefing and debriefing [48, 49]. During a debriefing, the leader should identify strengths and weaknesses in a positive forum, where everyone understands that the only goal is to enhance future performance. This step provides necessary feedback to team members and is crucial in enhancing team performance. Some studies have demonstrated that debriefing is associated with greater benefits for improving patient outcomes after cardiac arrest [50, 51]. Traditional training settings may not reflect the environment of actual team-based resuscitation. Since HF simulation-based training is often team-based, improving teamwork and debriefing may be more appropriate [33]. Previous studies have identified a small or moderate benefit for HF simulation-based training [42, 52]. One study has demonstrated that HF simulation improves teamwork performance in the NRP and PALS training [42]. Only five studies [27, 28, 33, 38, 39], including RCTs, assessed teamwork performance in ALS training. Pooled data from these RCTs demonstrated a trend benefit for improving teamwork behaviors for HF simulation when compared with LF simulation or traditional training. However, there were no significant difference in teamwork behaviors between HF simulation and control arm. Therefore, more high-quality RCTs are needed to assess the efficacy of HF simulation in improving teamwork performance. The best means of training debriefing sessions should be determined in order to enhance teamwork performance in HF simulation-based training.

While the goal of HF simulation-based training is to enhance knowledge and skills, cost-effectiveness is also a key point for educational intervention. If the cost of HF simulation-based training is prohibitively high, it could not be a viable option. Previous studies have analyzed the cost-effectiveness of simulation-based training programs on a learner basis [53]. Isaranuwatchai et al. conducted a cost-effectiveness analysis of HF and LF simulation training. The results of the study revealed that the HF program had the highest implementation cost [54]. At a willingness to pay of $100, HF programs had only a 6% probability of being cost-effective when compared with the LF program [54]. It is means that HF program was less cost-effective when compared to LF program. There was only a 6% chance that decision-makers were willing to pay $100 to buy HF program. None of the included RCTs assessed the cost-effectiveness of ALS training in HF simulations. Therefore, future studies should analyze the cost-effectiveness of HF simulation-based training.

Limitations

This study had some limitations that need to be considered. First, of the included RCTs, no studies measured skill performance during actual resuscitation or patient outcomes. Furthermore, high heterogeneity was observed in a meta-analysis of knowledge and skill performance. Hence, the studies might have had different training settings, outcome measurements, and types of participants. Third, no uniform standard assessment of confidence and satisfaction was implemented. The included RCTs used various questionnaires, which may have led to unobjective and incomparable results. Finally, only few of the included studies met the high-quality standards of evidence-based medicine because the participants were not blinded to the level of simulation. We would suggest future work commit more resources to optimize instructional design, instructor and debriefing training.

Conclusions

Learners using HF simulation benefited from knowledge and skill performance in ALS at the course conclusion than those using LF simulation or traditional training. A high-quality multicenter RCT is needed to enhance retention of knowledge and skills in actual resuscitation and patient outcomes in the future.