Abstract
Introduction and Hypothesis
We developed a summative assessment tool to evaluate competent performance on three procedure-specific low fidelity simulation models for vaginal surgery. Our purpose was to determine a pass–fail score for each model.
Methods
We enrolled participants (2011–2023, three Canadian academic centers) and grouped them according to operative competency in vaginal procedures. Novice operators were medical students recruited through targeted advertisement to clerkship level medical students. Proficient operators consisted of gynecology residents from the intervention arm of a randomized controlled trial, trained to competence in the use of the models; urogynecology fellows and attending gynecologic surgeons recruited through departmental rounds. All participants were asked to perform the three procedures on the models, were videotaped, and their performance assessed by evaluators familiar with the procedure and the scoring system, blinded to operator identity. A total performance score (range 0–400) assessed timing and errors. Basic skill deductions were set a priori. We calculated sensitivity and specificity scores and obtained an optimal cutoff based on Youden’s J statistic.
Results
For anterior repair, we rated 46 novice and 16 proficient videos. The pass–fail score was 170/400. For posterior repair, we rated 54 novice and 14 proficient videos. The pass–fail score was 140/400. For vaginal hysterectomy, we rated 47 novice and 12 proficient videos. The pass–fail score was 180/400. Scores of proficient operators were significantly better than those of novice participants (p < 0.001 for all).
Conclusions
A pass–fail score can distinguish between novice and proficient operators and can be used for summative assessment of surgical skill.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Abbreviations
- AR:
-
Anterior repair
- PR:
-
Posterior repair
- VH:
-
Vaginal hysterectomy
- MISTELS:
-
McGill Inanimate System for Training and Evaluation of Laparoscopic Skills
- FLS:
-
Fundamentals of Laparoscopic Surgery
References
Walters MD, Ridgeway BM. Increasing Utilization of Minimally Invasive Hysterectomy. Clin Obstet Gynecol. 2017;60(2):273–85. https://doi.org/10.1097/GRF.0000000000000283.
Washburn EE, Cohen SL, Manoucheri E, Zurawin RK, Einarsson JI. Trends in reported resident surgical experience in hysterectomy. J Minimally Invasive Gynecol. 2014;21(6):1067–70. https://doi.org/10.1016/j.jmig.2014.05.005.
Geoffrion R. Standing on the shoulders of giants: contemplating a standard national curriculum for surgical training in gynaecology. J Obstet Gynaecol Canada: JOGC = Journal d’obstetrique et gynecologie du Canada: JOGC. 2008;30(8):684–95. https://doi.org/10.1016/S1701-2163(16)32917-6.
Sroka G, Feldman LS, Vassiliou MC, Kaneva PA, Fayez R, Fried GM. Fundamentals of laparoscopic surgery simulator training to proficiency improves laparoscopic performance in the operating room-a randomized controlled trial. Am J Surg. 2010;199(1):115–20. https://doi.org/10.1016/j.amjsurg.2009.07.035.
Fraser SA, Klassen DR, Feldman LS, Ghitulescu GA, Stanbridge D, Fried GM. Evaluating laparoscopic skills: setting the pass/fail score for the MISTELS system. Surg Endoscopy. 2003;17(6):964–7. https://doi.org/10.1007/s00464-002-8828-4.
Geoffrion R, Suen MW, Koenig NA, Yong P, Brennand E, Mehra N, Larouche M, Lee T, Todd NJ. Teaching vaginal surgery to junior residents: initial validation of 3 novel procedure-specific low-fidelity models. J Surg Educ. 2016;73(1):157–61. https://doi.org/10.1016/j.jsurg.2015.09.004.
Geoffrion R, Koenig NA, Sanaee MS, Lee T, Todd NJ. Optimizing resident operative self-confidence through competency-based surgical education modules: are we there yet? Int Urogynecol J. 2019;30(3):423–8. https://doi.org/10.1007/s00192-018-3654-y.
Guntupalli SR, Doo DW, Guy M, Sheeder J, Omurtag K, Kondapalli L, Valea F, Harper L, Muffly TM. Preparedness of obstetrics and gynecology residents for fellowship training. Obstet Gynecol. 2015;126(3):559–68. https://doi.org/10.1097/AOG.0000000000000999.
Caccia N, Nakajima A, Scheele F, Kent N. Competency-based medical education: developing a framework for obstetrics and gynaecology. J Obstet Gynaecol Canada: Journal d’obstetrique et gynecologie du Canada. 2015;37(12):1104–12. https://doi.org/10.1016/s1701-2163(16)30076-7.
https://www.acgme.org/globalassets/pfassets/programresources/obgyncaseloginfo.pdf. Accessed 26 July 2023.
Dreyfus S. The five-stage model of adult skill acquisition. Bull Sci Technol Soc. 2004;24:177–81.
Wohlrab K, Jelovsek JE, Myers D. Incorporating simulation into gynecologic surgical training. Am J Obstet Gynecol. 2017;217(5):522–6. https://doi.org/10.1016/j.ajog.2017.05.017.
Blades ML, Glaze S, McQuillan SK. Resident Perspectives on Competency-By-Design Curriculum. J Obstet Gynaecol Canada: Journal d’obstetrique et gynecologie du Canada. 2020;42(3):242–7. https://doi.org/10.1016/j.jogc.2019.07.005.
Funding
This work was supported by a Medical Education Research Grant from the Royal College of Physicians and Surgeons of Canada and a Summer Student Research Project Grant from the University of British Columbia. Funding sources had no role in the design, conduct, analysis, interpretation or dissemination of study data. These grant providers do not assign individual grant numbers.
Author information
Authors and Affiliations
Contributions
R. Geoffrion: Project development, design of surgical tools and videos, manuscript writing.
N. Koenig: Project development, data collection and analysis, manuscript review.
Z. Sunderji: Data collection, manuscript writing.
T. Lee: Project development, statistical support, manuscript writing.
Corresponding author
Ethics declarations
Conflicts of Interest
None.
Additional information
Handling Editor: Rufus Cartwright
Editor in Chief: Maria A. Bortolini
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Geoffrion, R., Koenig, N.A., Sunderji, Z. et al. Summative Evaluation of Vaginal Surgery Skills: Setting A Pass–Fail Score. Int Urogynecol J 35, 451–456 (2024). https://doi.org/10.1007/s00192-023-05717-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00192-023-05717-9