Randomized controlled trials of simulation-based interventions in Emergency Medicine: a methodological review
The number of trials assessing Simulation-Based Medical Education (SBME) interventions has rapidly expanded. Many studies show that potential flaws in design, conduct and reporting of randomized controlled trials (RCTs) can bias their results. We conducted a methodological review of RCTs assessing a SBME in Emergency Medicine (EM) and examined their methodological characteristics. We searched MEDLINE via PubMed for RCT that assessed a simulation intervention in EM, published in 6 general and internal medicine and in the top 10 EM journals. The Cochrane Collaboration risk of Bias tool was used to assess risk of bias, intervention reporting was evaluated based on the “template for intervention description and replication” checklist, and methodological quality was evaluated by the Medical Education Research Study Quality Instrument. Reports selection and data extraction was done by 2 independents researchers. From 1394 RCTs screened, 68 trials assessed a SBME intervention. They represent one quarter of our sample. Cardiopulmonary resuscitation (CPR) is the most frequent topic (81%). Random sequence generation and allocation concealment were performed correctly in 66 and 49% of trials. Blinding of participants and assessors was performed correctly in 19 and 68%. Risk of attrition bias was low in three-quarters of the studies (n = 51). Risk of selective reporting bias was unclear in nearly all studies. The mean MERQSI score was of 13.4/18.4% of the reports provided a description allowing the intervention replication. Trials assessing simulation represent one quarter of RCTs in EM. Their quality remains unclear, and reproducing the interventions appears challenging due to reporting issues.
KeywordsSimulation Randomized controlled trials Quality Emergency Medicine Cardiopulmonary resuscitation Reproducibility
We thank Laura Smales (BioMedEditing, Toronto, ON) for copyediting the manuscript.
Conception and design: AC, JT, PP, DP and YY; acquisition of data, AC and JT; analysis: AC; interpretation of data: AC, JT, PP, DP and YY; drafting the article: AC, AB, JT and YY; critical revision for important intellectual content: DP, AB and PP; final approval of the version to be published: AC, JT, PP, AB, DP and YY.
Compliance with ethical standards
Conflict of interest
The authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare no support from any organization other than the funding agency listed above for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.
Statement of human and animal rights
This article does not contain any studies with human participants or animals performed by any of the authors.
For this type of study formal consent is not required.
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
- 4.Society for simulation in Healthcare (2016) About simulation. http://ssih.org/About-Simulation. Accessed 26 Dec 2016
- 7.Fincher RME, Lewis LA (2002) Simulations used to teach clinical skills. In: Norman GR, van der Vleuten CPM, Newble DI (eds) International handbook of research in Medical Education, part one. Kluwer Academic, DordrechtGoogle Scholar
- 8.Haute Autorité de santé. Rapport de mission-Etat de l’art (national et international) en matière de pratiques de simulation dans le domaine de santé (2012). http://www.has-sante.fr/portail/jcms/c_930641/fr/simulation-en-sante. Accessed 22 Mar 2017
- 15.Douglass AM, Elder J, Watson R et al (2017) A Randomized controlled trial on the effect of a double check on the detection of medication errors. Ann Emerg Med S0196–0644(17):30318–30319Google Scholar
- 21.Bristol Medical Simulation Centre. Worldwide sim database. http://www.bmsc.co.uk/. Accessed 26 Dec 2016
- 22.Okuda Y, Bond W, Bonfante G et al (2011) National growth in simulation training within Emergency Medicine Residency Program, 2003–2008. Acad Emerg Med 12:455–460Google Scholar
- 38.Higgins JPT, Green S (2011) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]: Wiley-Blackwell, Chapter 9.5. http://handbook.cochrane.org/. Accessed 21 Mar 2017
- 40.Birkenes TS, Myklebust H, Kramer-Johansen J (2013) New pre-arrival instructions can avoid abdominal hand placement for chest compressions. Scand J Trauma Resusc Emerg Med 22(6):21–47Google Scholar
- 42.Godfred R, Huszti E, Fly D et al (2013) A randomized trial of video self-instruction in cardiopulmonary resuscitation for lay persons. Scand J Trauma Resusc Emerg Med 10(5):21–36Google Scholar
- 61.Park SO, Hong CK, Shin DH et al (2013) Efficacy of metronome sound guidance via a phone speaker during dispatcher-assisted compression-only cardiopulmonary resuscitation by an untrained layperson: a randomised controlled simulation study using a manikin. Emerg Med J 30(8):657–661CrossRefPubMedGoogle Scholar
- 64.Shin J, Hwang SY, Lee HJ et al (2014) Comparison of CPR quality and rescuer fatigue between standard 30: 2 CPR and chest compression-only CPR: a randomized crossover manikin trial. Scand J Trauma Resusc Emerg Med 28(10):22–59Google Scholar
- 96.Park SO, Shin DH, Lee KR et al (2013) Efficacy of the Disposcope endoscope, a new video laryngoscope, for endotracheal intubation in patients with cervical spine immobilisation by semirigid neck collar: comparison with the Macintosh laryngoscope using a simulation study on a manikin. Emerg Med J 30(4):270–274CrossRefPubMedGoogle Scholar
- 107.The Commonwealth fund task force on academic health centers (2002) Training tomorrow’s doctors: the Medical Education Mission of Academic Health Centers. The Commonwealth Fund, New York. http://www.commonwealthfund.org/publications/fund-reports/2002/apr/training-tomorrows-doctors–the-medical-education-mission-of-academic-health-centers. Accessed 02 June 2017