Abstract
Background
Surgical assessment instruments are used for formative and summative trainee evaluations. To characterize the features of existing instruments and a novel 12-item objective, procedure-specific assessment tool for Roux-en-Y Gastric Bypass (RYGB-OPSA), we evaluated the progress of a single surgical fellow over 17 consecutive surgeries.
Methods
Seventeen consecutive RYGB videos completed between 8/2021 and 1/2022 by an academic hospital surgical fellow were de-identified and assessed by four board-certified bariatric surgeons using Global Operative Assessment of Laparoscopic Skills (GOALS), General Assessment of Surgical Skill (GASS), and RYGB-OPSA which includes the reflection of transverse colon, identification of ligament of Treitz, biliopancreatic and Roux limbs orientation, jejunal division point selection, stapler use, mesentery division, bleeding control, jejunojejunostomy (JJ) anastomotic site selection, apposition of JJ anastomotic site, JJ creation, common enterotomy closure of JJ, and integrity of anastomosis. The GASS measured economy of motion, tissue handling, appreciating operative anatomy, bimanual dexterity, and achievement of hemostasis. RYGB-OPSA and GASS items were scored “poor—unsafe,” “acceptable—safe,” or “good—safe.” Change in performance was measured by linear trendline slope.
Results
Over the course of 17 procedures, significant improvement was demonstrated by three GOALS items, GOALS overall score, GASS bimanual dexterity, and three RYGB-OPSA tasks: JJ creation, jejunal division point selection, and stapler use. Achievement of hemostasis declined but never rated “poor—unsafe.” Overall RYGB-OPSA and GOALS trendlines documented significant increase across the 17 procedures.
Conclusion
This examination of a bariatric surgery fellow’s operative training experience as measured by three surgical assessment instruments demonstrated anticipated improvements in general skills and safe completion of procedure-specific tasks. Effective surgical assessment instruments have enough sensitivity to show improvement to enable meaningful trainee feedback (low-stakes assessments) as well as the ability to determine safe surgical practice to enable promotion to greater autonomous practice.
Graphical abstract
Similar content being viewed by others
References
Ibrahim AM, Ghaferi AA, Thumma JR, Dimick JB (2017) Variation in outcomes at bariatric surgery centers of excellence. JAMA Surg. https://doi.org/10.1001/jamasurg.2017.0542
Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJO, Collaborative MBS (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med. https://doi.org/10.1056/nejmsa1300625
Varban OA, Thumma JR, Finks JF, Carlin AM, Ghaferi AA, Dimick JB (2021) Evaluating the effect of surgical skill on outcomes for laparoscopic sleeve gastrectomy: a video-based study. Ann Surg. https://doi.org/10.1097/sla.0000000000003385
Stulberg JJ, Huang R, Kreutzer L, Ban K, Champagne BJ, Steele SR, Johnson JK, Holl JL, Greenberg CC, Bilimoria KY (2020) Association between surgeon technical skills and patient outcomes. JAMA Surg. https://doi.org/10.1001/jamasurg.2020.3007
Woods MS, Liberman JN, Rui P, Wiggins E, White J, Ramshaw B, Stulberg JJ (2023) Association between surgical technical skills and clinical outcomes; a systematic literature review and meta—analysis. JSLS. https://doi.org/10.4293/JSLS.2022.00076
Ahmet A, Gamze K, Rustem M, Sezen KA (2018) Is video-based education an effective method in surgical education? A systematic review. J Surg Educ. https://doi.org/10.1016/j.jsurg.2018.01.014
McQueen S, McKinnon V, VanderBeek L, McCarthy C, Sonnadara R (2019) Video-based assessment in surgical education: a scoping review. J Surg Educ. https://doi.org/10.1016/j.jsurg.2019.05.013
Huang RJ, Limsui D, Triadafilopoulos G (2018) Video-based performance assessment in endoscopy: moving beyond “see one, do one, teach one”? Gastrointest Endosc. https://doi.org/10.1016/j.gie.2017.09.014
Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. https://doi.org/10.1046/j.1365-2168.1997.02502.x
Balvardi S, Semsar-Kazerooni K, Kaneva P, Mueller C, Vassiliou M, Al Mahroos M, Fiore JF Jr, Schwartzman K, Feldman LS (2023) Validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy. Surg Endosc. https://doi.org/10.1007/s00464-022-09466-6
Kramp KH, van Det MJ, Hoff C, Lamme B, Veeger NJGM, Pierie JPEN (2015) Validity and reliability of global operative assessment of laparoscopic skills (GOALS) in novice trainees performing a laparoscopic cholecystectomy. J Surg Educ. https://doi.org/10.1016/j.jsurg.2014.08.006
Anderson DD, Long S, Thomas GW, Putnam MD, Bechtold JE, Karam MD (2016) Objective structured assessments of technical skills (OSATS) does not assess the quality of the surgical result effectively. Clin Orthop Relat Res. https://doi.org/10.1007/s11999-015-4603-4
Vaidya A, Aydin A, Ridgely J, Raison N, Dasgupta P, Ahmed K (2020) Current stats of technical skills assessment in surgery: a systematic review. J Surg Res. https://doi.org/10.1016/j.jss.2019.09.006
Zevin B, Bonrath EM, Aggarwal R, Dedy NJ, Ahmed N, Grantcharov TP, ATLAS Group (2013) Development, feasibility, validity, and reliability of a scale for objective assessment of operative performance in laparoscopic gastric bypass surgery. J Am Coll Surg. https://doi.org/10.1016/j.jamcollsurg.2013.01.003
Greenberg JA, Minter RM (2019) Entrustable professional activities: the future of competency-based education in surgery may already be here. Ann Surg. https://doi.org/10.1097/sla.0000000000003153
Wagner JP, Lewis CE, Tillou A, Agopian VG, Quach C, Donahue TR, Hines OJ (2018) Use of entrustable professional activities in the assessment of surgical resident competency. JAMA Surg. https://doi.org/10.1001/jamasurg.2017.4547
American Board of Surgery (2022) Entrustable professional activities. https://www.absurgery.org/default.jsp?epahome. Accessed 1 Mar 2023
White A, Moran HRM, Ryan J, Mador B, Campbell S, Turner SR (2022) Validity evidence for procedure-specific competency assessment tools in cardiovascular and thoracic surgery: a scoping review. J Surg Educ 79:1016–1023
Hogle NJ, Liu Y, Ogden RT, Fowler DL (2014) Evaluation of surgical fellows’ laparoscopic performance using global operative assessment of laparoscopic skills (GOALS). Surg Endosc. https://doi.org/10.1007/s00464-013-3324-6
Bello RJ, Sarmiento S, Meyer ML, Rosson GD, Cooney DS, Cooney DS, Lifchez SD, Cooney CM (2018) Understanding surgical resident and fellow perspectives on their operative performance feedback needs: a qualitative study. J Surg Educ. https://doi.org/10.1016/j.jsurg.2018.04.002
Ducournau F, Meyer N, Xavier F, Facca S, Liverneaux P (2021) Learning a MIPO technique for distal radius fractures: mentoring versus simple experience versus deliberate practice. Orthop Traumatol Surg Res. https://doi.org/10.1016/j.otsr.2021.102939
Bonrath EM, Dedy NJ, Gordon LE, Grantcharov TP (2015) Comprehensive surgical coaching enhances surgical skill in the operating room: a randomized controlled trial. Ann Surg. https://doi.org/10.1097/sla.0000000000001214
Wehrtmann FS, de la Garza JR, Kowalewski KF, Schmidt MW, Müller K, Tapking C, Probst P, Diener MK, Fischer L, Müeller-Stich BP, Nickel F (2020) Learning curves of laparascopic Roux-en-Y gastric bypass and sleeve gastrectomy in bariatric surgery: a systematic review and introduction of a standardization. Obes Surg. https://doi.org/10.1007/s11695-019-04230-7
Champagne BJ (2013) Effective teaching and feedback strategies in the OR and beyond. Clin Colon Rectal Surg. https://doi.org/10.1055/s-0033-1356725
Groenier M, Brummer L, Bunting BP, Gallagher AG (2020) Reliability of observational assessment methods for outcome-based assessment of surgical skill: systematic review and meta-analysis. J Surg Educ. https://doi.org/10.1016/j.jsurg.2019.07.007
Acknowledgements
The authors wish to thank Justin Smith for analytical support and Health Analytics, LLC for administrative and medical writing support in preparing the manuscript.
Funding
Caresyntax sponsored this project.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Disclosures
Nau reported receipt of support for this manuscript and payment or honoraria from Caresyntax. Worden reported receipt of support for this manuscript from Caresyntax. Lehmann reported receipt of support for this manuscript, consulting fees and payment or honoraria from Caresyntax. Kleppe reported receipt of support for this manuscript from Caresyntax. G. Mancini reported receipt of support for this manuscript, consulting fees and payment or honoraria from Caresyntax. M. Mancini reported receipt of support for this manuscript, consulting fees and payment or honoraria from Caresyntax. Dr. Ramshaw reported receipt of support for this manuscript and consulting fees from Caresyntax. Dr. Woods is an employee of Caresyntax.
Ethical approval
This study was approved by the University of Iowa Hospitals & Clinics Institutional Review Board (Approval No. 202109638).
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
464_2023_10425_MOESM1_ESM.jpg
Supplementary file1 (JPG 166 kb) Supplemental Figure 1. Objective Procedure-Specific Assessment (OPSA) for the 12 Consecutive Jejunojejunostomy Tasks in Laparoscopic Roux-en-Y Gastric Bypass (RYGB) Surgery
464_2023_10425_MOESM3_ESM.jpg
Supplementary file3 (JPG 256 kb) Supplemental Figure 3. Global Operative Assessment of Laparoscopic Skills (GOALS) is a Validated Assessment Tool for Grading Overall Technical Proficiency for Laparoscopic Surgery. While it Measures Domains, such as Fluidity of Motion, it does not Define Surgical or Clinical Judgement, e.g., Autonomy.
464_2023_10425_MOESM5_ESM.jpg
Supplementary file5 (JPG 131 kb) Supplemental Figure 5. Average Rating for the Individual Components of the GASS Rating over Time.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Nau, P., Worden, E., Lehmann, R. et al. Using video-based assessment (VBA) to document fellow improvement in safely completing the jejunojejunostomy portion of laparoscopic Roux-en-Y gastric bypass (RYGB) surgery. Surg Endosc 37, 8853–8860 (2023). https://doi.org/10.1007/s00464-023-10425-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00464-023-10425-y