Advances in Health Sciences Education

, Volume 14, Issue 2, pp 159–171 | Cite as

Simulation-based medical education is no better than problem-based discussions and induces misjudgment in self-assessment

  • Manuel Wenk
  • René Waurick
  • David Schotes
  • Melanie Wenk
  • Christina Gerdes
  • Hugo K. Van Aken
  • Daniel M. Pöpping
Article

Abstract

Simulation-based teaching (SBT) is increasingly used in medical education. As an alternative to other teaching methods there is a lack of evidence concerning its efficacy. The aim of this study was to evaluate the potency of SBT in anesthesia in comparison to problem-based discussion (PBD) with students in a randomized controlled setting. Thirty-three fourth-year medical students attending a curricular anesthesiology course were randomly allocated to either a session of SBT or a session of PBD on an emergency induction method. Ten days later all students underwent examination in a simulator. The performance of each student was evaluated by weighted tasks, established according to a modified Delphi process. Confidence and a multiple-choice questionnaire were additionally performed pre- and post-intervention. A total of 32 students completed the study. Participants in the SBT group presented with significantly higher self-assessment scores after the intervention than students in the PBD group. However, students in the SBT group achieved only slightly and statistically insignificantly higher scores in the theoretical and simulator examination (p > 0.05) with only a moderate effect size of d = 0.52. The current study demonstrates that both PBD and SBT lead to comparable short-term outcomes in theoretical knowledge and clinical skills. However, undesirably, SBT students overrated their anticipated clinical abilities and knowledge improvement.

Keywords

Human patient simulation Problem-based discussion Randomized Delphi process Education Undergraduate medical students 

Notes

Acknowledgements

We thank Dr. Trevor G. Cooper, Ph.D., for carefully revising the manuscript.

References

  1. Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Education Research, 4, 4–16.Google Scholar
  2. Byrick, R., Cleave-Hogg, D., & McKnight, D. (1998). A crisis management program for residents in anesthesia. Academic Medicine: Journal of the Association of American Medical Colleges, 73(5), 592.Google Scholar
  3. Chopra, V., Gesink, B. J., & DeJong, J. (1994). Does training on an anaesthesia simulator lead to improvement in performance? British Journal of Anaesthesia, 73(3), 293–297.CrossRefGoogle Scholar
  4. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  5. Colliver, J. A. (2000). Effectiveness of problem-based learning curricula: Research and theory. Academic Medicine: Journal of the Association of American Medical Colleges, 75(3), 259–266.Google Scholar
  6. Cooper, J. B., & Taqueti, V. R. (2004). A brief history of the development of mannequin simulators for clinical education and training. Quality & Safety in Health Care, 13(Suppl 1), i11–i18.CrossRefGoogle Scholar
  7. Cumin, D., & Merry, A. F. (2007). Simulators for use in anaesthesia. Anaesthesia, 62(2), 151–162.CrossRefGoogle Scholar
  8. DeVita, M. A., Schaefer, J., Lutz, J., Dongilli, T., & Wang, H. (2004). Improving medical crisis team performance. Critical Care Medicine, 32(Suppl 2), S61–S65.CrossRefGoogle Scholar
  9. Eva, K. W., Cunnington, J. P., Reiter, H. I., Keane, D. R., & Norman, G. R. (2004). How can I know what I don’t know? Poor self assessment in a well-defined domain. Advances in Health Sciences Education: Theory and Practice, 9(3), 211–224.CrossRefGoogle Scholar
  10. Eva, K. W., & Regehr, G. (2005). Self-assessment in the health professions: A reformulation and research agenda. Academic Medicine: Journal of the Association of American Medical Colleges, 80(Suppl 10), S46–S54.Google Scholar
  11. Eva, K. W., & Regehr, G. (2007). Knowing when to look it up: A new conception of self-assessment ability. Academic Medicine: Journal of the Association of American Medical Colleges, 82(Suppl 10), S81–S84.Google Scholar
  12. Flanagan, B., Nestel, D., & Joseph, M. (2004). Making patient safety the focus: Crisis resource management in the undergraduate curriculum. Medical Education, 38(1), 56–66.CrossRefGoogle Scholar
  13. Gaba, D. M., & DeAnda, A. (1988). A comprehensive anesthesia simulation environment: Re-creating the operating room for research and training. Anesthesiology, 69(3), 387–394.CrossRefGoogle Scholar
  14. Gardiner, J. (2006). Simulators in obstetric anaesthesia are a waste of time. International Journal of Obstetric Anesthesia, 15(1), 44–46.CrossRefGoogle Scholar
  15. Goldmann, K., & Ferson, D. Z. (2005). Education and training in airway management. Best Practice & Research. Clinical Anaesthesiology, 19(4), 717–732.CrossRefGoogle Scholar
  16. Hebert, R., & Bravo, G. (1996). Development and validation of an evaluation instrument for medical students in tutorials. Academic Medicine: Journal of the Association of American Medical Colleges, 71(5), 488–494.Google Scholar
  17. Issenberg, S. B., McGaghie, W. C., Petrusa, E. R., Lee Gordon, D., & Scalese, R. J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Medical Teacher, 27(1), 10–28.CrossRefGoogle Scholar
  18. Jude, D. C., Gilbert, G. G., & Magrane, D. (2006). Simulation training in the obstetrics and gynecology clerkship. American Journal of Obstetrics Gynecology, 195(5), 1489–1492.Google Scholar
  19. Knudson, M. M., & Sisley, A. C. (2000). Training residents using simulation technology: Experience with ultrasound for trauma. The Journal of Trauma, 48(4), 659–665.CrossRefGoogle Scholar
  20. Lighthall, G. K. (2006). The value of simulation training during anesthesia residency. Anesthesiology, 105(2), 433.CrossRefGoogle Scholar
  21. Lippert, A., Nielsen, M. S., & Ostergaard, D. (2004). Medical simulation in anesthesiology. The Danish Society of Anesthesiology and Intensive Medicine. Ugeskr Laeger, 166(12), 1102.Google Scholar
  22. Marshall, R. L., Smith, J. S., Gorman, P. J., Krummel, T. M., Haluck, R. S., & Cooney, R. N. (2001). Use of a human patient simulator in the development of resident trauma management skills. The Journal of Trauma, 51(1), 17–21.CrossRefGoogle Scholar
  23. Matsumoto, E. D., Hamstra, S. J., Radomski, S. B., & Cusimano, M. D. (2002). The effect of bench model fidelity on endourological skills: A randomized controlled study. The Journal of Urology, 167(3), 1243–1247.CrossRefGoogle Scholar
  24. McFetrich, J. (2006). A structured literature review on the use of high fidelity patient simulators for teaching in emergency medicine. Emergency Medicine Journal, 23(7), 509–511.CrossRefGoogle Scholar
  25. Morgan, P. J., & Cleave-Hogg, D. (2002a). Comparison between medical students’ experience, confidence and competence. Medical Education, 36(6), 534–539.CrossRefGoogle Scholar
  26. Morgan, P. J., & Cleave-Hogg, D. (2002b). A worldwide survey of the use of simulation in anesthesia. Canadian Journal of Anaesthesia, 49(7), 659–662.Google Scholar
  27. Morgan, P. J., & Cleave-Hogg, D. M. (2001). Cost and resource implications of undergraduate simulator-based education. Canadian Journal of Anaesthesia, 48(8), 827–828.Google Scholar
  28. Morgan, P. J., Cleave-Hogg, D., McIlroy, J., & Devitt, J. H. (2002). Simulation technology: A comparison of experiential and visual learning for undergraduate medical students. Anesthesiology, 96(1), 10–16.CrossRefGoogle Scholar
  29. Murray, D. (2005). Clinical simulation: Measuring the efficacy of training. Current Opinion in Anaesthesiology, 18(6), 645–648.CrossRefGoogle Scholar
  30. Ostergaard, D. (2004). National medical simulation training program in Denmark. Critical Care Medicine, 32(Suppl 2), S58–S60.CrossRefGoogle Scholar
  31. Scavone, B. M., Sproviero, M. T., McCarthy, R. J., Wong, C. A., Sullivan, J. T., Siddall, V. J., & Wade, L. D. (2006). Development of an objective scoring system for measurement of resident performance on the human patient simulator. Anesthesiology, 105(2), 260–266.CrossRefGoogle Scholar
  32. Steadman, R. H., Coates, W. C., Huang, Y. M., Matevosian, R., Larmon, B. R., McCullough, L., & Ariel, D. (2006). Simulation-based training is superior to problem-based learning for the acquisition of critical assessment and management skills. Critical Care Medicine, 34(1), 151–157.CrossRefGoogle Scholar
  33. Weller, J. M. (2004). Simulation in undergraduate medical education: Bridging the gap between theory and practice. Medical Education, 38(1), 32–38.CrossRefGoogle Scholar
  34. Weller, J., Wilson, L., & Robinson, B. (2003). Survey of change in practice following simulation-based training in crisis management. Anaesthesia, 58(5), 471–473.CrossRefGoogle Scholar
  35. Yee, B., Naik, V. N., Joo, H. S., Savoldelli, G. L., Chung, D. Y., Houston, P. L., Karatzoglou, B. J., & Hamstra, S. J. (2005). Nontechnical skills in anesthesia crisis management with repeated exposure to simulation-based education. Anesthesiology, 103(2), 241–248.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  • Manuel Wenk
    • 1
  • René Waurick
    • 1
  • David Schotes
    • 1
  • Melanie Wenk
    • 2
  • Christina Gerdes
    • 1
  • Hugo K. Van Aken
    • 1
  • Daniel M. Pöpping
    • 1
  1. 1.Department of Anesthesiology and Intensive CareUniversity Hospital MuensterMuensterGermany
  2. 2.Department of PsychiatryUniversity Hospital MuensterMuensterGermany

Personalised recommendations