Advertisement

Advances in Health Sciences Education

, Volume 19, Issue 2, pp 251–272 | Cite as

Feedback for simulation-based procedural skills training: a meta-analysis and critical narrative synthesis

  • Rose HatalaEmail author
  • David A. Cook
  • Benjamin Zendejas
  • Stanley J. Hamstra
  • Ryan Brydges
Review

Abstract

Although feedback has been identified as a key instructional feature in simulation based medical education (SBME), we remain uncertain as to the magnitude of its effectiveness and the mechanisms by which it may be effective. We employed a meta-analysis and critical narrative synthesis to examine the effectiveness of feedback for SBME procedural skills training and to examine how it works in this context. Our results demonstrate that feedback is moderately effective during procedural skills training in SBME, with a pooled effect size favoring feedback for skill outcomes of 0.74 (95 % CI 0.38–1.09; p < .001). Terminal feedback appears more effective than concurrent feedback for novice learners’ skill retention. Multiple sources of feedback, including instructor feedback, lead to short-term performance gains although data on long-term effects is lacking. The mechanism by which feedback may be operating is consistent with the guidance hypothesis, with more research needed to examine other mechanisms such as cognitive load theory and social development theory.

Keywords

Simulation-based medical education Procedural skills training Feedback Motor learning Technical skills 

Notes

Acknowledgments

The authors thank Patricia Erwin, Jason Szostek, and Amy Wang for their assistance with literature searching, abstract reviewing and data extraction. This work was supported by intramural funds, including an award from the Division of General Internal Medicine, Mayo Clinic.

Conflicts of interest

No relevant conflicts of interest for any of the authors. There was no industry relationship with this work.

References

  1. Archer, J. C. (2010). State of the science in health professional education: Effective feedback. Medical Education, 44(1), 101–108.CrossRefGoogle Scholar
  2. Backstein, D., Agnidis, Z., Sadhu, R., & MacRae, H. (2005). Effectiveness of repeated video feedback in the acquisition of a surgical technical skill. Canadian Journal of Surgery, 48(3), 195–200.Google Scholar
  3. Borenstein, M. (2009). Effect sizes for continuous data. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed.). New York, NY, US: Russell Sage Foundation.Google Scholar
  4. Byrne, A. J., Sellen, A. J., Jones, J. G., Aitkenhead, A. R., Hussain, S., Gilder, F., et al. (2002). Effect of videotape feedback on anaesthetists’ performance while managing simulated anaesthetic crises: A multicentre study. Anaesthesia, 57(2), 176–179.CrossRefGoogle Scholar
  5. Chang, J.-Y., Chang, G.-L., Chien, C.-J. C., Chung, K.-C., & Hsu, A.-T. (2007). Effectiveness of two forms of feedback on training of a joint mobilization skill by using a joint translation simulator. Physical Therapy, 87(4), 418–430.CrossRefGoogle Scholar
  6. Chang, S., Waid, E., Martinec, D. V., Zheng, B., & Swanstrom, L. L. (2008). Verbal communication improves laparoscopic team performance. Surgical Innovation, 15(2), 143–147.CrossRefGoogle Scholar
  7. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  8. Cook, D. A., Bordage, G., & Schmidt, H. G. (2008). Description, justification and clarification: A framework for classifying the purposes of research in medical education. Medical Education, 42(2), 128–133.CrossRefGoogle Scholar
  9. Cook, D. A., Brydges, R., Hamstra, S. J., Zendejas, B., Szostek, J. H., Wang, A. T., et al. (2012). Comparative effectiveness of technology-enhanced simulation versus other instructional methods: A systematic review and meta-analysis. Simulation in Healthcare: Journal of the Society for Simulation in Healthcare, 7(5), 308–320.CrossRefGoogle Scholar
  10. Cook, D. A., Hamstra, S. J., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., et al. (2013). Comparative effectiveness of instructional design features in simulation-based education: Systematic review and meta-analysis. Medical Teacher, 35(1), e867-98.Google Scholar
  11. Cook, D. A., Hatala, R., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., et al. (2011). Technology-enhanced simulation for health professions education: A systematic review and meta-analysis. The Journal of the American Medical Association, 306(9), 978–988.CrossRefGoogle Scholar
  12. Day, T., Iles, N., & Griffiths, P. (2009). Effect of performance feedback on tracheal suctioning knowledge and skills: Randomized controlled trial. Journal of Advanced Nursing, 65(7), 1423–1431.CrossRefGoogle Scholar
  13. Dine, C. J., Gersh, R. E., Leary, M., Riegel, B. J., Bellini, L. M., & Abella, B. S. (2008). Improving cardiopulmonary resuscitation quality and resuscitation training by combining audiovisual feedback and debriefing*. Critical Care Medicine, 36(10), 2817–2822.CrossRefGoogle Scholar
  14. Domuracki, K. J., Moule, C. J., Owen, H., Kostandoff, G., & Plummer, J. L. (2009). Learning on a simulator does transfer to clinical practice. Resuscitation, 80(3), 346–349.CrossRefGoogle Scholar
  15. Dunphy, B., & Dunphy, S. (2003). Assisted performance and the Zone of Proximal Development (ZPD); a potential framework for providing surgical education. Australian Journal of Educational & Developmental Psychology, 3, 48–58.Google Scholar
  16. Ende, J. (1983). Feedback in clinical medical education. The Journal of the American Medical Association, 250, 777–781.Google Scholar
  17. Enebo, B., & Sherwood, D. (2005). Experience and practice organization in learning a simulated high-velocity low-amplitude task. Journal of Manipulative and Physiological Therapeutics, 28(1), 33–43.CrossRefGoogle Scholar
  18. Gerling, G. J., & Thomas, G. W. (2003). Effect of augmented visual performance feedback on the effectiveness of clinical breast examination training with a dynamically configurable breast model. In Systems, man and cybernetics, 2003. IEEE international conference on, 3, 2095–2100 vol. 3.Google Scholar
  19. Gurusamy, K., Aggarwal, R., Palanivelu, L., & Davidson, B. R. (2008). Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. British Journal of Surgery, 95(9), 1088–1097.CrossRefGoogle Scholar
  20. Higgins, E. T. (1997). Beyond pleasure and pain. The American Psychologist, 52(12), 1280–1300.CrossRefGoogle Scholar
  21. Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327(7414), 557–560. (Clinical research ed.).CrossRefGoogle Scholar
  22. Hunter, J., & Schmidt, F. (2004). Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks, CA: Sage.Google Scholar
  23. Issenberg, S. B., McGaghie, W. C., Petrusa, E. R., Lee Gordon, D., & Scalese, R. J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Medical Teacher, 27(1), 10–28.CrossRefGoogle Scholar
  24. Kahol, K., French, J., McDaniel, T., Panchanathan, S., & Smith, M. (2007). Augmented virtual reality for laparoscopic surgical tool training. In Proceedings of the 12th international conference on humancomputer interaction: Applications and services, pp. 459–467.Google Scholar
  25. Kirkpatrick, D. L., & Kirkpatrick, J. D. (2009). Evaluating training programs (3rd ed.). San Francisco: Berrett-Koehler.Google Scholar
  26. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254–284.CrossRefGoogle Scholar
  27. Kovacs, G., Bullock, G., Ackroyd-Stolarz, S., Cain, E., & Petrie, D. (2000). A randomized controlled trial on the effect of educational interventions in promoting airway management skill maintenance. Annals of Emergency Medicine, 36(4), 301–309.CrossRefGoogle Scholar
  28. Kruglikova, I., Grantcharov, T. P., Drewes, A. M., & Funch-Jensen, P. (2010). The impact of constructive feedback on training in gastrointestinal endoscopy using high-fidelity virtual-reality simulation: A randomised controlled trial. Gut, 59(2), 181–185.CrossRefGoogle Scholar
  29. Lazarski, M. P., Susarla, S. M., Bennett, N. L., & Seldin, E. B. (2007). How do feedback and instructions affect the performance of a simulated surgical task? Journal of Oral and Maxillofacial Surgery, 65(6), 1155–1161.CrossRefGoogle Scholar
  30. Ma, I. W. Y., Brindle, M. E., Ronksley, P. E., Lorenzetti, D. L., Sauve, R. S., & Ghali, W. A. (2011). Use of simulation-based education to improve outcomes of central venous catheterization: A systematic review and meta-analysis. Academic Medicine, 86(9), 1137–1147.CrossRefGoogle Scholar
  31. McGaghie, W. C., Issenberg, S. B., Petrusa, E. R., & Scalese, R. J. (2010). A critical review of simulation-based medical education research: 2003–2009. Medical Education, 44(1), 50–63.CrossRefGoogle Scholar
  32. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., PRISMA Group. (2009, October). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Journal of Clinical Epidemiology, 62, 1006–1012.Google Scholar
  33. Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7(1), 105–125.CrossRefGoogle Scholar
  34. Moulton, C., Tabak, D., Kneebone, R., Nestel, D., MacRae, H., & LeBlanc, V. R. (2009). Teaching communication skills using the integrated procedural performance instrument (IPPI): A randomized controlled trial. American Journal of Surgery, 197(1), 113–118.CrossRefGoogle Scholar
  35. O’Connor, A., Schwaitzberg, S. D., & Cao, C. G. L. (2007). How much feedback is necessary for learning to suture? Surgical Endoscopy, 22(7), 1614–1619.CrossRefGoogle Scholar
  36. Passiment, M., Sacks, H., & Huang, G. (2011). Medical simulation in medical education: Results of an AAMC Survey (pp. 1–48). Washington, DC: Association of American Medical Colleges.Google Scholar
  37. Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005). Realist review—A new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy, 10(Suppl 1), 21–34.CrossRefGoogle Scholar
  38. Porte, M. C., Xeroulis, G., Reznick, R. K., & Dubrowski, A. (2007). Verbal feedback from an expert is more effective than self-accessed feedback about motion efficiency in learning new surgical skills. American Journal of Surgery, 193(1), 105–110.CrossRefGoogle Scholar
  39. Pugh, C. M., Srivastava, S., Shavelson, R., Walker, D., Cotner, T., Scarloss, B., et al. (2001). The effect of simulator use on learning and self-assessment: The case of Stanford University’s E-Pelvis simulator. Studies in Health Technology and Informatics, 81, 396–400.Google Scholar
  40. Quinn, F., Keogh, P., McDonald, A., & Hussey, D. (2003). A study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students. European Journal of Dental Education : Official Journal of the Association for Dental Education in Europe, 7(4), 164–169.CrossRefGoogle Scholar
  41. Rafiq, A., Tamariz, F., Boanca, C., Lavrentyev, V., & Merrell, R. C. (2008). Objective assessment of training surgical skills using simulated tissue interface with real-time feedback. Journal of Surgical Education, 65(4), 270–274.CrossRefGoogle Scholar
  42. Reed, D. A., Cook, D. A., Beckman, T. J., Levine, R. B., Kern, D. E., & Wright, S. M. (2007). Association between funding and quality of published medical education research. The Journal of the American Medical Association, 298(9), 1002–1009.CrossRefGoogle Scholar
  43. Rissanen, M. J., Kume, N., Kuroda, Y., Kuroda, T., Yoshimura, K., & Yoshihara, H. (2008). Asynchronous teaching of psychomotor skills through VR annotations: Evaluation in digital rectal examination. Studies in Health Technology and Informatics, 132, 411–416.Google Scholar
  44. Roediger, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20–27.CrossRefGoogle Scholar
  45. Rosser, J. C, Jr, Colsant, B. J., Lynch, P. J., Herman, B., Klonsky, J., & Young, S. M. (2006). The use of a “hybrid” trainer in an established laparoscopic skills program. Journal of the Society of Laparoendoscopic Surgeons, 10(1), 4.Google Scholar
  46. Scaringe, J. (2002). The effects of augmented sensory feedback precision on the acquisition and retention of a simulated chiropractic task. Journal of Manipulative and Physiological Therapeutics, 25(1), 34–41.CrossRefGoogle Scholar
  47. Schmidt, R., & Lee, T. (2011). Motor control and learning: A behavioral emphasis (5th ed.). Champaign: Human Kinetics.Google Scholar
  48. Shute, V. (2008). Focus on formative feedback. Review of Educational Research, 78, 153–189.Google Scholar
  49. Spooner, B. B., Fallaha, J. F., Kocierz, L., Smith, C. M., Smith, S. C. L., & Perkins, G. D. (2007). An evaluation of objective feedback in basic life support (BLS) training. Resuscitation, 73(3), 417–424.CrossRefGoogle Scholar
  50. Stefanidis, D., Korndorffer, J. R, Jr, Heniford, B. T., & Scott, D. J. (2007). Limited feedback and video tutorials optimize learning and resource utilization during laparoscopic simulator training. Surgery, 142(2), 202–206.CrossRefGoogle Scholar
  51. Sutherland, L. M., Middleton, P. F., Anthony, A., Hamdorf, J., Cregan, P., Scott, D., et al. (2006). Surgical simulation: A systematic review. Annals of Surgery, 243(3), 291–300.CrossRefGoogle Scholar
  52. van Merrienboer, J. J. G., & Sweller, J. (2010). Cognitive load theory in health professional education: Design principles and strategies. Medical Education, 44(1), 85–93.CrossRefGoogle Scholar
  53. Van Merriënboer, J. J. G., & Sweller, J. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17(2), 147–177.CrossRefGoogle Scholar
  54. Van Sickle, K. R., Gallagher, A. G., & Smith, C. D. (2007). The effect of escalating feedback on the acquisition of psychomotor skills for laparoscopy. Surgical Endoscopy, 21(2), 220–224.CrossRefGoogle Scholar
  55. Veloski, J., Boex, J. R., Grasberger, M. J., Evans, A., & Wolfson, D. B. (2006). Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Medical Teacher, 28(2), 117–128.CrossRefGoogle Scholar
  56. Vygotsky, L. (1978). Mind in society: The development of higher mental processes. In M. Cole, V. John-Steiner, S. Scribner, & E. Souberman (Eds.), (14th ed.). Harvard University Press.Google Scholar
  57. Walsh, C. M., Ling, S. C., Wang, C. S., & Carnahan, H. (2009). Concurrent versus terminal feedback: It may be better to wait. Academic Medicine : Journal of the Association of American Medical Colleges, 84(10 Suppl), S54–S57.CrossRefGoogle Scholar
  58. Watling, C., Driessen, E., van der Vleuten, C. P. M., Vanstone, M., & Lingard, L. (2012). Understanding responses to feedback: The potential and limitations of regulatory focus theory. Medical Education, 46(6), 593–603.Google Scholar
  59. Wierinck, E., Puttemans, V., Swinnen, S., & van Steenberghe, D. (2005). Effect of augmented visual feedback from a virtual reality simulation system on manual dexterity training. European Journal of Dental Education: Official Journal of the Association for Dental Education in Europe, 9(1), 10–16.CrossRefGoogle Scholar
  60. Wierinck, E., Puttemans, V., & van Steenberghe, D. (2006a). Effect of reducing frequency of augmented feedback on manual dexterity training and its retention. Journal of Dentistry, 34(9), 641–647.CrossRefGoogle Scholar
  61. Wierinck, E., Puttemans, V., & van Steenberghe, D. (2006b). Effect of tutorial input in addition to augmented feedback on manual dexterity training and its retention. European Journal of Dental Education: Official Journal of the Association for Dental Education in Europe, 10(1), 24–31.CrossRefGoogle Scholar
  62. Wulf, G., & Shea, C. (2004). Understanding the role of augmented feedback: The good, the bad, and the ugly. In A. Williams & N. Hodges (Eds.), Skill acquisition in sport: Research, theory and practice (1st ed., pp. 121–144). London: Routledge.Google Scholar
  63. Wulf, G., Shea, C., & Lewthwaite, R. (2010). Motor skill learning and performance: A review of influential factors. Medical Education, 44(1), 75–84.CrossRefGoogle Scholar
  64. Xeroulis, G. J., Park, J., Moulton, C. A., Reznick, R. K., LeBlanc, V., & Dubrowski, A. (2007). Teaching suturing and knot-tying skills to medical students: A randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback. Surgery, 141(4), 442–449.CrossRefGoogle Scholar
  65. Yasukawa, Y. (2009). The effectiveness of cavity preparation training using a virtual reality simulation system with or without feedback. Kōkūbyō Gakkai zasshi. The Journal of the Stomatological Society, Japan, 76(2), 73–80.Google Scholar
  66. Zendejas, B., Brydges, R., Wang, A. T., & Cook, D. A. (2013). Patient outcomes in simulation-based medical education: A systematic review. Journal of General Internal Medicine. doi: 10.1007/s11606-012-2264-5

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Rose Hatala
    • 1
    • 7
    Email author
  • David A. Cook
    • 2
    • 3
  • Benjamin Zendejas
    • 4
  • Stanley J. Hamstra
    • 5
  • Ryan Brydges
    • 6
  1. 1.Department of MedicineUniversity of British ColumbiaVancouverCanada
  2. 2.Office of Education ResearchMayo Medical SchoolRochesterUSA
  3. 3.Division of General Internal MedicineMayo Clinic College of MedicineRochesterUSA
  4. 4.Department of SurgeryMayo Clinic College of MedicineRochesterUSA
  5. 5.University of Ottawa Skills and Simulation Centre, Academy for Innovation in Medical Education, Faculty of MedicineUniversity of OttawaOttawaCanada
  6. 6.Department of Medicine and The Wilson CentreUniversity of TorontoTorontoCanada
  7. 7.St. Paul’s HospitalVancouverCanada

Personalised recommendations