Patient Outcomes in Simulation-Based Medical Education: A Systematic Review
- 2.5k Downloads
Evaluating the patient impact of health professions education is a societal priority with many challenges. Researchers would benefit from a summary of topics studied and potential methodological problems. We sought to summarize key information on patient outcomes identified in a comprehensive systematic review of simulation-based instruction.
Systematic search of MEDLINE, EMBASE, CINAHL, PsychINFO, Scopus, key journals, and bibliographies of previous reviews through May 2011.
Original research in any language measuring the direct effects on patients of simulation-based instruction for health professionals, in comparison with no intervention or other instruction.
APPRAISAL and SYNTHESIS
Two reviewers independently abstracted information on learners, topics, study quality including unit of analysis, and validity evidence. We pooled outcomes using random effects.
From 10,903 articles screened, we identified 50 studies reporting patient outcomes for at least 3,221 trainees and 16,742 patients. Clinical topics included airway management (14 studies), gastrointestinal endoscopy (12), and central venous catheter insertion (8). There were 31 studies involving postgraduate physicians and seven studies each involving practicing physicians, nurses, and emergency medicine technicians. Fourteen studies (28 %) used an appropriate unit of analysis. Measurement validity was supported in seven studies reporting content evidence, three reporting internal structure, and three reporting relations with other variables. The pooled Hedges’ g effect size for 33 comparisons with no intervention was 0.47 (95 % confidence interval [CI], 0.31–0.63); and for nine comparisons with non-simulation instruction, it was 0.36 (95 % CI, −0.06 to 0.78).
Focused field in education; high inconsistency (I2 > 50 % in most analyses).
Simulation-based education was associated with small-moderate patient benefits in comparison with no intervention and non-simulation instruction, although the latter did not reach statistical significance. Unit of analysis errors were common, and validity evidence was infrequently reported.
KEY WORDSmedical education outcomes research simulation educational technology program evaluation quantitative research methods
The authors thank Rose Hatala, MD, MSc, Stanley J. Hamstra, Phd, Jason H. Szostek, MD, and Patricia J. Erwin, MLS, for their assistance in the literature search and initial data acquisition
This work was supported by intramural funds, including an award from the Division of General Internal Medicine, Mayo Clinic. The funding sources for this study played no role in the design and conduct of the study; in the collection, management, analysis, and interpretation of the data; or in the preparation of the manuscript. The funding sources did not review the manuscript.
Conflicts of Interest
No relevant conflicts of interest.
- 14.Cook DA, Brydges R, Hamstra S, Zendejas B, Szostek JH, Wang AT, et al. Comparative Effectiveness of Technology-Enhanced Simulation vs. Other Instructional Methods: A Systematic Review and Meta-Analysis. Simul Healthc. 2012;7:308–20.Google Scholar
- 15.Cook DA, Hamstra S, Brydges R, Zendejas B, Szostek JH, Wang AT, et al. Comparative Effectiveness of Instructional Design Features in Simulation-based Education: Systematic Review and Meta-analysis. Med Teach. 2012; Online early (doi: 10.3109/0142159X.2012.714886).
- 36.Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum; 1988.Google Scholar
- 42.Hochberger J, Matthes K, Maiss J, Koebnick C, Hahn EG, Cohen J. Training with the compactEASIE biologic endoscopy simulator significantly improves hemostatic technical skill of gastroenterology fellows: a randomized controlled comparison with clinical endoscopy training alone. Gastrointest Endosc. 2005;61:204–15.PubMedCrossRefGoogle Scholar
- 60.Hosking EJ. Does practising intubation on a manikin improve both understanding and clinical performance of the task by medical students? Anaesthesia Points West. 1998;31:25–8.Google Scholar
- 70.Zamora Z, Clark MJ, Winslow B, Schatzschneider M, Burkard J. Orthotopic neobladder irrigation: competency assessment through simulation. Urol Nurs. 2011;31:113–20.Google Scholar
- 73.Limpaphayom K, Ajello C, Reinprayoon D, Lumbiganon P, Graffikin L. The effectiveness of model-based training in accelerating IUD skill acquisition. A study of midwives in Thailand. Br J Fam Plann. 1997;23:58–61.Google Scholar
- 82.Trooskin SZ, Rabinowitz S, Eldridge C, McGowan DE, Flancbaum L. Teaching endotracheal intubation using animals and cadavers. Prehospital Disaster Med. 1992;7:179–82.Google Scholar
- 90.Cook DA, West CP. Reconsidering the focus on “outcomes research” in medical education: a cautionary note. Acad Med. 2013; Accepted (in press).Google Scholar