Skip to main content

Advertisement

SpringerLink
Log in

Feedback for simulation-based procedural skills training: a meta-analysis and critical narrative synthesis

  • Review
  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

Abstract

Although feedback has been identified as a key instructional feature in simulation based medical education (SBME), we remain uncertain as to the magnitude of its effectiveness and the mechanisms by which it may be effective. We employed a meta-analysis and critical narrative synthesis to examine the effectiveness of feedback for SBME procedural skills training and to examine how it works in this context. Our results demonstrate that feedback is moderately effective during procedural skills training in SBME, with a pooled effect size favoring feedback for skill outcomes of 0.74 (95 % CI 0.38–1.09; p < .001). Terminal feedback appears more effective than concurrent feedback for novice learners’ skill retention. Multiple sources of feedback, including instructor feedback, lead to short-term performance gains although data on long-term effects is lacking. The mechanism by which feedback may be operating is consistent with the guidance hypothesis, with more research needed to examine other mechanisms such as cognitive load theory and social development theory.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Archer, J. C. (2010). State of the science in health professional education: Effective feedback. Medical Education, 44(1), 101–108.

    Article  Google Scholar 

  • Backstein, D., Agnidis, Z., Sadhu, R., & MacRae, H. (2005). Effectiveness of repeated video feedback in the acquisition of a surgical technical skill. Canadian Journal of Surgery, 48(3), 195–200.

    Google Scholar 

  • Borenstein, M. (2009). Effect sizes for continuous data. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed.). New York, NY, US: Russell Sage Foundation.

    Google Scholar 

  • Byrne, A. J., Sellen, A. J., Jones, J. G., Aitkenhead, A. R., Hussain, S., Gilder, F., et al. (2002). Effect of videotape feedback on anaesthetists’ performance while managing simulated anaesthetic crises: A multicentre study. Anaesthesia, 57(2), 176–179.

    Article  Google Scholar 

  • Chang, J.-Y., Chang, G.-L., Chien, C.-J. C., Chung, K.-C., & Hsu, A.-T. (2007). Effectiveness of two forms of feedback on training of a joint mobilization skill by using a joint translation simulator. Physical Therapy, 87(4), 418–430.

    Article  Google Scholar 

  • Chang, S., Waid, E., Martinec, D. V., Zheng, B., & Swanstrom, L. L. (2008). Verbal communication improves laparoscopic team performance. Surgical Innovation, 15(2), 143–147.

    Article  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

  • Cook, D. A., Bordage, G., & Schmidt, H. G. (2008). Description, justification and clarification: A framework for classifying the purposes of research in medical education. Medical Education, 42(2), 128–133.

    Article  Google Scholar 

  • Cook, D. A., Brydges, R., Hamstra, S. J., Zendejas, B., Szostek, J. H., Wang, A. T., et al. (2012). Comparative effectiveness of technology-enhanced simulation versus other instructional methods: A systematic review and meta-analysis. Simulation in Healthcare: Journal of the Society for Simulation in Healthcare, 7(5), 308–320.

    Article  Google Scholar 

  • Cook, D. A., Hamstra, S. J., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., et al. (2013). Comparative effectiveness of instructional design features in simulation-based education: Systematic review and meta-analysis. Medical Teacher, 35(1), e867-98.

    Google Scholar 

  • Cook, D. A., Hatala, R., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., et al. (2011). Technology-enhanced simulation for health professions education: A systematic review and meta-analysis. The Journal of the American Medical Association, 306(9), 978–988.

    Article  Google Scholar 

  • Day, T., Iles, N., & Griffiths, P. (2009). Effect of performance feedback on tracheal suctioning knowledge and skills: Randomized controlled trial. Journal of Advanced Nursing, 65(7), 1423–1431.

    Article  Google Scholar 

  • Dine, C. J., Gersh, R. E., Leary, M., Riegel, B. J., Bellini, L. M., & Abella, B. S. (2008). Improving cardiopulmonary resuscitation quality and resuscitation training by combining audiovisual feedback and debriefing*. Critical Care Medicine, 36(10), 2817–2822.

    Article  Google Scholar 

  • Domuracki, K. J., Moule, C. J., Owen, H., Kostandoff, G., & Plummer, J. L. (2009). Learning on a simulator does transfer to clinical practice. Resuscitation, 80(3), 346–349.

    Article  Google Scholar 

  • Dunphy, B., & Dunphy, S. (2003). Assisted performance and the Zone of Proximal Development (ZPD); a potential framework for providing surgical education. Australian Journal of Educational & Developmental Psychology, 3, 48–58.

    Google Scholar 

  • Ende, J. (1983). Feedback in clinical medical education. The Journal of the American Medical Association, 250, 777–781.

    Google Scholar 

  • Enebo, B., & Sherwood, D. (2005). Experience and practice organization in learning a simulated high-velocity low-amplitude task. Journal of Manipulative and Physiological Therapeutics, 28(1), 33–43.

    Article  Google Scholar 

  • Gerling, G. J., & Thomas, G. W. (2003). Effect of augmented visual performance feedback on the effectiveness of clinical breast examination training with a dynamically configurable breast model. In Systems, man and cybernetics, 2003. IEEE international conference on, 3, 2095–2100 vol. 3.

  • Gurusamy, K., Aggarwal, R., Palanivelu, L., & Davidson, B. R. (2008). Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. British Journal of Surgery, 95(9), 1088–1097.

    Article  Google Scholar 

  • Higgins, E. T. (1997). Beyond pleasure and pain. The American Psychologist, 52(12), 1280–1300.

    Article  Google Scholar 

  • Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327(7414), 557–560. (Clinical research ed.).

    Article  Google Scholar 

  • Hunter, J., & Schmidt, F. (2004). Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Issenberg, S. B., McGaghie, W. C., Petrusa, E. R., Lee Gordon, D., & Scalese, R. J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Medical Teacher, 27(1), 10–28.

    Article  Google Scholar 

  • Kahol, K., French, J., McDaniel, T., Panchanathan, S., & Smith, M. (2007). Augmented virtual reality for laparoscopic surgical tool training. In Proceedings of the 12th international conference on human–computer interaction: Applications and services, pp. 459–467.

  • Kirkpatrick, D. L., & Kirkpatrick, J. D. (2009). Evaluating training programs (3rd ed.). San Francisco: Berrett-Koehler.

    Google Scholar 

  • Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254–284.

    Article  Google Scholar 

  • Kovacs, G., Bullock, G., Ackroyd-Stolarz, S., Cain, E., & Petrie, D. (2000). A randomized controlled trial on the effect of educational interventions in promoting airway management skill maintenance. Annals of Emergency Medicine, 36(4), 301–309.

    Article  Google Scholar 

  • Kruglikova, I., Grantcharov, T. P., Drewes, A. M., & Funch-Jensen, P. (2010). The impact of constructive feedback on training in gastrointestinal endoscopy using high-fidelity virtual-reality simulation: A randomised controlled trial. Gut, 59(2), 181–185.

    Article  Google Scholar 

  • Lazarski, M. P., Susarla, S. M., Bennett, N. L., & Seldin, E. B. (2007). How do feedback and instructions affect the performance of a simulated surgical task? Journal of Oral and Maxillofacial Surgery, 65(6), 1155–1161.

    Article  Google Scholar 

  • Ma, I. W. Y., Brindle, M. E., Ronksley, P. E., Lorenzetti, D. L., Sauve, R. S., & Ghali, W. A. (2011). Use of simulation-based education to improve outcomes of central venous catheterization: A systematic review and meta-analysis. Academic Medicine, 86(9), 1137–1147.

    Article  Google Scholar 

  • McGaghie, W. C., Issenberg, S. B., Petrusa, E. R., & Scalese, R. J. (2010). A critical review of simulation-based medical education research: 2003–2009. Medical Education, 44(1), 50–63.

    Article  Google Scholar 

  • Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., PRISMA Group. (2009, October). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Journal of Clinical Epidemiology, 62, 1006–1012.

    Google Scholar 

  • Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7(1), 105–125.

    Article  Google Scholar 

  • Moulton, C., Tabak, D., Kneebone, R., Nestel, D., MacRae, H., & LeBlanc, V. R. (2009). Teaching communication skills using the integrated procedural performance instrument (IPPI): A randomized controlled trial. American Journal of Surgery, 197(1), 113–118.

    Article  Google Scholar 

  • O’Connor, A., Schwaitzberg, S. D., & Cao, C. G. L. (2007). How much feedback is necessary for learning to suture? Surgical Endoscopy, 22(7), 1614–1619.

    Article  Google Scholar 

  • Passiment, M., Sacks, H., & Huang, G. (2011). Medical simulation in medical education: Results of an AAMC Survey (pp. 1–48). Washington, DC: Association of American Medical Colleges.

    Google Scholar 

  • Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005). Realist review—A new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy, 10(Suppl 1), 21–34.

    Article  Google Scholar 

  • Porte, M. C., Xeroulis, G., Reznick, R. K., & Dubrowski, A. (2007). Verbal feedback from an expert is more effective than self-accessed feedback about motion efficiency in learning new surgical skills. American Journal of Surgery, 193(1), 105–110.

    Article  Google Scholar 

  • Pugh, C. M., Srivastava, S., Shavelson, R., Walker, D., Cotner, T., Scarloss, B., et al. (2001). The effect of simulator use on learning and self-assessment: The case of Stanford University’s E-Pelvis simulator. Studies in Health Technology and Informatics, 81, 396–400.

    Google Scholar 

  • Quinn, F., Keogh, P., McDonald, A., & Hussey, D. (2003). A study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students. European Journal of Dental Education : Official Journal of the Association for Dental Education in Europe, 7(4), 164–169.

    Article  Google Scholar 

  • Rafiq, A., Tamariz, F., Boanca, C., Lavrentyev, V., & Merrell, R. C. (2008). Objective assessment of training surgical skills using simulated tissue interface with real-time feedback. Journal of Surgical Education, 65(4), 270–274.

    Article  Google Scholar 

  • Reed, D. A., Cook, D. A., Beckman, T. J., Levine, R. B., Kern, D. E., & Wright, S. M. (2007). Association between funding and quality of published medical education research. The Journal of the American Medical Association, 298(9), 1002–1009.

    Article  Google Scholar 

  • Rissanen, M. J., Kume, N., Kuroda, Y., Kuroda, T., Yoshimura, K., & Yoshihara, H. (2008). Asynchronous teaching of psychomotor skills through VR annotations: Evaluation in digital rectal examination. Studies in Health Technology and Informatics, 132, 411–416.

    Google Scholar 

  • Roediger, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20–27.

    Article  Google Scholar 

  • Rosser, J. C, Jr, Colsant, B. J., Lynch, P. J., Herman, B., Klonsky, J., & Young, S. M. (2006). The use of a “hybrid” trainer in an established laparoscopic skills program. Journal of the Society of Laparoendoscopic Surgeons, 10(1), 4.

    Google Scholar 

  • Scaringe, J. (2002). The effects of augmented sensory feedback precision on the acquisition and retention of a simulated chiropractic task. Journal of Manipulative and Physiological Therapeutics, 25(1), 34–41.

    Article  Google Scholar 

  • Schmidt, R., & Lee, T. (2011). Motor control and learning: A behavioral emphasis (5th ed.). Champaign: Human Kinetics.

    Google Scholar 

  • Shute, V. (2008). Focus on formative feedback. Review of Educational Research, 78, 153–189.

    Google Scholar 

  • Spooner, B. B., Fallaha, J. F., Kocierz, L., Smith, C. M., Smith, S. C. L., & Perkins, G. D. (2007). An evaluation of objective feedback in basic life support (BLS) training. Resuscitation, 73(3), 417–424.

    Article  Google Scholar 

  • Stefanidis, D., Korndorffer, J. R, Jr, Heniford, B. T., & Scott, D. J. (2007). Limited feedback and video tutorials optimize learning and resource utilization during laparoscopic simulator training. Surgery, 142(2), 202–206.

    Article  Google Scholar 

  • Sutherland, L. M., Middleton, P. F., Anthony, A., Hamdorf, J., Cregan, P., Scott, D., et al. (2006). Surgical simulation: A systematic review. Annals of Surgery, 243(3), 291–300.

    Article  Google Scholar 

  • van Merrienboer, J. J. G., & Sweller, J. (2010). Cognitive load theory in health professional education: Design principles and strategies. Medical Education, 44(1), 85–93.

    Article  Google Scholar 

  • Van Merriënboer, J. J. G., & Sweller, J. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17(2), 147–177.

    Article  Google Scholar 

  • Van Sickle, K. R., Gallagher, A. G., & Smith, C. D. (2007). The effect of escalating feedback on the acquisition of psychomotor skills for laparoscopy. Surgical Endoscopy, 21(2), 220–224.

    Article  Google Scholar 

  • Veloski, J., Boex, J. R., Grasberger, M. J., Evans, A., & Wolfson, D. B. (2006). Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Medical Teacher, 28(2), 117–128.

    Article  Google Scholar 

  • Vygotsky, L. (1978). Mind in society: The development of higher mental processes. In M. Cole, V. John-Steiner, S. Scribner, & E. Souberman (Eds.), (14th ed.). Harvard University Press.

  • Walsh, C. M., Ling, S. C., Wang, C. S., & Carnahan, H. (2009). Concurrent versus terminal feedback: It may be better to wait. Academic Medicine : Journal of the Association of American Medical Colleges, 84(10 Suppl), S54–S57.

    Article  Google Scholar 

  • Watling, C., Driessen, E., van der Vleuten, C. P. M., Vanstone, M., & Lingard, L. (2012). Understanding responses to feedback: The potential and limitations of regulatory focus theory. Medical Education, 46(6), 593–603.

    Google Scholar 

  • Wierinck, E., Puttemans, V., Swinnen, S., & van Steenberghe, D. (2005). Effect of augmented visual feedback from a virtual reality simulation system on manual dexterity training. European Journal of Dental Education: Official Journal of the Association for Dental Education in Europe, 9(1), 10–16.

    Article  Google Scholar 

  • Wierinck, E., Puttemans, V., & van Steenberghe, D. (2006a). Effect of reducing frequency of augmented feedback on manual dexterity training and its retention. Journal of Dentistry, 34(9), 641–647.

    Article  Google Scholar 

  • Wierinck, E., Puttemans, V., & van Steenberghe, D. (2006b). Effect of tutorial input in addition to augmented feedback on manual dexterity training and its retention. European Journal of Dental Education: Official Journal of the Association for Dental Education in Europe, 10(1), 24–31.

    Article  Google Scholar 

  • Wulf, G., & Shea, C. (2004). Understanding the role of augmented feedback: The good, the bad, and the ugly. In A. Williams & N. Hodges (Eds.), Skill acquisition in sport: Research, theory and practice (1st ed., pp. 121–144). London: Routledge.

    Google Scholar 

  • Wulf, G., Shea, C., & Lewthwaite, R. (2010). Motor skill learning and performance: A review of influential factors. Medical Education, 44(1), 75–84.

    Article  Google Scholar 

  • Xeroulis, G. J., Park, J., Moulton, C. A., Reznick, R. K., LeBlanc, V., & Dubrowski, A. (2007). Teaching suturing and knot-tying skills to medical students: A randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback. Surgery, 141(4), 442–449.

    Article  Google Scholar 

  • Yasukawa, Y. (2009). The effectiveness of cavity preparation training using a virtual reality simulation system with or without feedback. Kōkūbyō Gakkai zasshi. The Journal of the Stomatological Society, Japan, 76(2), 73–80.

  • Zendejas, B., Brydges, R., Wang, A. T., & Cook, D. A. (2013). Patient outcomes in simulation-based medical education: A systematic review. Journal of General Internal Medicine. doi:10.1007/s11606-012-2264-5

Download references

Acknowledgments

The authors thank Patricia Erwin, Jason Szostek, and Amy Wang for their assistance with literature searching, abstract reviewing and data extraction. This work was supported by intramural funds, including an award from the Division of General Internal Medicine, Mayo Clinic.

Conflicts of interest

No relevant conflicts of interest for any of the authors. There was no industry relationship with this work.

Author information

Authors and Affiliations

  1. Department of Medicine, University of British Columbia, Vancouver, BC, Canada

    Rose Hatala

  2. Office of Education Research, Mayo Medical School, Rochester, MN, USA

    David A. Cook

  3. Division of General Internal Medicine, Mayo Clinic College of Medicine, Rochester, MN, USA

    David A. Cook

  4. Department of Surgery, Mayo Clinic College of Medicine, Rochester, MN, USA

    Benjamin Zendejas

  5. University of Ottawa Skills and Simulation Centre, Academy for Innovation in Medical Education, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada

    Stanley J. Hamstra

  6. Department of Medicine and The Wilson Centre, University of Toronto, Toronto, ON, Canada

    Ryan Brydges

  7. St. Paul’s Hospital, Suite 5907, Burrard Bldg, 1081 Burrard St, Vancouver, BC, V6Z 1Y6, Canada

    Rose Hatala

Authors
  1. Rose Hatala
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David A. Cook
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benjamin Zendejas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stanley J. Hamstra
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ryan Brydges
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rose Hatala.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hatala, R., Cook, D.A., Zendejas, B. et al. Feedback for simulation-based procedural skills training: a meta-analysis and critical narrative synthesis. Adv in Health Sci Educ 19, 251–272 (2014). https://doi.org/10.1007/s10459-013-9462-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10459-013-9462-8

Keywords

Search

Navigation