Skip to main content

Advertisement

SpringerLink
Log in

Using video-based assessment (VBA) to document fellow improvement in safely completing the jejunojejunostomy portion of laparoscopic Roux-en-Y gastric bypass (RYGB) surgery

  • 2023 SAGES Poster
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Surgical assessment instruments are used for formative and summative trainee evaluations. To characterize the features of existing instruments and a novel 12-item objective, procedure-specific assessment tool for Roux-en-Y Gastric Bypass (RYGB-OPSA), we evaluated the progress of a single surgical fellow over 17 consecutive surgeries.

Methods

Seventeen consecutive RYGB videos completed between 8/2021 and 1/2022 by an academic hospital surgical fellow were de-identified and assessed by four board-certified bariatric surgeons using Global Operative Assessment of Laparoscopic Skills (GOALS), General Assessment of Surgical Skill (GASS), and RYGB-OPSA which includes the reflection of transverse colon, identification of ligament of Treitz, biliopancreatic and Roux limbs orientation, jejunal division point selection, stapler use, mesentery division, bleeding control, jejunojejunostomy (JJ) anastomotic site selection, apposition of JJ anastomotic site, JJ creation, common enterotomy closure of JJ, and integrity of anastomosis. The GASS measured economy of motion, tissue handling, appreciating operative anatomy, bimanual dexterity, and achievement of hemostasis. RYGB-OPSA and GASS items were scored “poor—unsafe,” “acceptable—safe,” or “good—safe.” Change in performance was measured by linear trendline slope.

Results

Over the course of 17 procedures, significant improvement was demonstrated by three GOALS items, GOALS overall score, GASS bimanual dexterity, and three RYGB-OPSA tasks: JJ creation, jejunal division point selection, and stapler use. Achievement of hemostasis declined but never rated “poor—unsafe.” Overall RYGB-OPSA and GOALS trendlines documented significant increase across the 17 procedures.

Conclusion

This examination of a bariatric surgery fellow’s operative training experience as measured by three surgical assessment instruments demonstrated anticipated improvements in general skills and safe completion of procedure-specific tasks. Effective surgical assessment instruments have enough sensitivity to show improvement to enable meaningful trainee feedback (low-stakes assessments) as well as the ability to determine safe surgical practice to enable promotion to greater autonomous practice.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Ibrahim AM, Ghaferi AA, Thumma JR, Dimick JB (2017) Variation in outcomes at bariatric surgery centers of excellence. JAMA Surg. https://doi.org/10.1001/jamasurg.2017.0542

    Article  PubMed  PubMed Central  Google Scholar 

  2. Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJO, Collaborative MBS (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med. https://doi.org/10.1056/nejmsa1300625

    Article  PubMed  Google Scholar 

  3. Varban OA, Thumma JR, Finks JF, Carlin AM, Ghaferi AA, Dimick JB (2021) Evaluating the effect of surgical skill on outcomes for laparoscopic sleeve gastrectomy: a video-based study. Ann Surg. https://doi.org/10.1097/sla.0000000000003385

    Article  PubMed  Google Scholar 

  4. Stulberg JJ, Huang R, Kreutzer L, Ban K, Champagne BJ, Steele SR, Johnson JK, Holl JL, Greenberg CC, Bilimoria KY (2020) Association between surgeon technical skills and patient outcomes. JAMA Surg. https://doi.org/10.1001/jamasurg.2020.3007

    Article  PubMed  PubMed Central  Google Scholar 

  5. Woods MS, Liberman JN, Rui P, Wiggins E, White J, Ramshaw B, Stulberg JJ (2023) Association between surgical technical skills and clinical outcomes; a systematic literature review and meta—analysis. JSLS. https://doi.org/10.4293/JSLS.2022.00076

    Article  PubMed  PubMed Central  Google Scholar 

  6. Ahmet A, Gamze K, Rustem M, Sezen KA (2018) Is video-based education an effective method in surgical education? A systematic review. J Surg Educ. https://doi.org/10.1016/j.jsurg.2018.01.014

    Article  PubMed  Google Scholar 

  7. McQueen S, McKinnon V, VanderBeek L, McCarthy C, Sonnadara R (2019) Video-based assessment in surgical education: a scoping review. J Surg Educ. https://doi.org/10.1016/j.jsurg.2019.05.013

    Article  PubMed  Google Scholar 

  8. Huang RJ, Limsui D, Triadafilopoulos G (2018) Video-based performance assessment in endoscopy: moving beyond “see one, do one, teach one”? Gastrointest Endosc. https://doi.org/10.1016/j.gie.2017.09.014

    Article  PubMed Central  Google Scholar 

  9. Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. https://doi.org/10.1046/j.1365-2168.1997.02502.x

    Article  PubMed  Google Scholar 

  10. Balvardi S, Semsar-Kazerooni K, Kaneva P, Mueller C, Vassiliou M, Al Mahroos M, Fiore JF Jr, Schwartzman K, Feldman LS (2023) Validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy. Surg Endosc. https://doi.org/10.1007/s00464-022-09466-6

    Article  PubMed  Google Scholar 

  11. Kramp KH, van Det MJ, Hoff C, Lamme B, Veeger NJGM, Pierie JPEN (2015) Validity and reliability of global operative assessment of laparoscopic skills (GOALS) in novice trainees performing a laparoscopic cholecystectomy. J Surg Educ. https://doi.org/10.1016/j.jsurg.2014.08.006

    Article  PubMed  Google Scholar 

  12. Anderson DD, Long S, Thomas GW, Putnam MD, Bechtold JE, Karam MD (2016) Objective structured assessments of technical skills (OSATS) does not assess the quality of the surgical result effectively. Clin Orthop Relat Res. https://doi.org/10.1007/s11999-015-4603-4

    Article  PubMed  PubMed Central  Google Scholar 

  13. Vaidya A, Aydin A, Ridgely J, Raison N, Dasgupta P, Ahmed K (2020) Current stats of technical skills assessment in surgery: a systematic review. J Surg Res. https://doi.org/10.1016/j.jss.2019.09.006

    Article  PubMed  Google Scholar 

  14. Zevin B, Bonrath EM, Aggarwal R, Dedy NJ, Ahmed N, Grantcharov TP, ATLAS Group (2013) Development, feasibility, validity, and reliability of a scale for objective assessment of operative performance in laparoscopic gastric bypass surgery. J Am Coll Surg. https://doi.org/10.1016/j.jamcollsurg.2013.01.003

    Article  Google Scholar 

  15. Greenberg JA, Minter RM (2019) Entrustable professional activities: the future of competency-based education in surgery may already be here. Ann Surg. https://doi.org/10.1097/sla.0000000000003153

    Article  Google Scholar 

  16. Wagner JP, Lewis CE, Tillou A, Agopian VG, Quach C, Donahue TR, Hines OJ (2018) Use of entrustable professional activities in the assessment of surgical resident competency. JAMA Surg. https://doi.org/10.1001/jamasurg.2017.4547

    Article  PubMed  PubMed Central  Google Scholar 

  17. American Board of Surgery (2022) Entrustable professional activities. https://www.absurgery.org/default.jsp?epahome. Accessed 1 Mar 2023

  18. White A, Moran HRM, Ryan J, Mador B, Campbell S, Turner SR (2022) Validity evidence for procedure-specific competency assessment tools in cardiovascular and thoracic surgery: a scoping review. J Surg Educ 79:1016–1023

    Article  PubMed  Google Scholar 

  19. Hogle NJ, Liu Y, Ogden RT, Fowler DL (2014) Evaluation of surgical fellows’ laparoscopic performance using global operative assessment of laparoscopic skills (GOALS). Surg Endosc. https://doi.org/10.1007/s00464-013-3324-6

    Article  PubMed  Google Scholar 

  20. Bello RJ, Sarmiento S, Meyer ML, Rosson GD, Cooney DS, Cooney DS, Lifchez SD, Cooney CM (2018) Understanding surgical resident and fellow perspectives on their operative performance feedback needs: a qualitative study. J Surg Educ. https://doi.org/10.1016/j.jsurg.2018.04.002

    Article  Google Scholar 

  21. Ducournau F, Meyer N, Xavier F, Facca S, Liverneaux P (2021) Learning a MIPO technique for distal radius fractures: mentoring versus simple experience versus deliberate practice. Orthop Traumatol Surg Res. https://doi.org/10.1016/j.otsr.2021.102939

    Article  Google Scholar 

  22. Bonrath EM, Dedy NJ, Gordon LE, Grantcharov TP (2015) Comprehensive surgical coaching enhances surgical skill in the operating room: a randomized controlled trial. Ann Surg. https://doi.org/10.1097/sla.0000000000001214

    Article  PubMed  Google Scholar 

  23. Wehrtmann FS, de la Garza JR, Kowalewski KF, Schmidt MW, Müller K, Tapking C, Probst P, Diener MK, Fischer L, Müeller-Stich BP, Nickel F (2020) Learning curves of laparascopic Roux-en-Y gastric bypass and sleeve gastrectomy in bariatric surgery: a systematic review and introduction of a standardization. Obes Surg. https://doi.org/10.1007/s11695-019-04230-7

    Article  PubMed  Google Scholar 

  24. Champagne BJ (2013) Effective teaching and feedback strategies in the OR and beyond. Clin Colon Rectal Surg. https://doi.org/10.1055/s-0033-1356725

    Article  PubMed  PubMed Central  Google Scholar 

  25. Groenier M, Brummer L, Bunting BP, Gallagher AG (2020) Reliability of observational assessment methods for outcome-based assessment of surgical skill: systematic review and meta-analysis. J Surg Educ. https://doi.org/10.1016/j.jsurg.2019.07.007

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors wish to thank Justin Smith for analytical support and Health Analytics, LLC for administrative and medical writing support in preparing the manuscript.

Funding

Caresyntax sponsored this project.

Author information

Authors and Affiliations

  1. Section of Bariatric Surgery, Department of Surgery, University of Iowa Hospitals & Clinics, 200 Hawkins Drive, Iowa City, IA, 52242, USA

    Peter Nau, Erin Worden & Ryan Lehmann

  2. Section of Foregut Surgery, Department of Surgery, University of Tennessee – Knoxville, 1924 Alcoa Highway, Knoxville, TN, 37920, USA

    Kyle Kleppe, Gregory J. Mancini & Matt L. Mancini

  3. CQInsights PBC, Knoxville, TN, USA

    Bruce Ramshaw

  4. Caresyntax Corporation, Boston, MA, USA

    Bruce Ramshaw & Michael S. Woods

Authors
  1. Peter Nau
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Erin Worden
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ryan Lehmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kyle Kleppe
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gregory J. Mancini
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Matt L. Mancini
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Bruce Ramshaw
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Michael S. Woods
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Nau.

Ethics declarations

Disclosures

Nau reported receipt of support for this manuscript and payment or honoraria from Caresyntax. Worden reported receipt of support for this manuscript from Caresyntax. Lehmann reported receipt of support for this manuscript, consulting fees and payment or honoraria from Caresyntax. Kleppe reported receipt of support for this manuscript from Caresyntax. G. Mancini reported receipt of support for this manuscript, consulting fees and payment or honoraria from Caresyntax. M. Mancini reported receipt of support for this manuscript, consulting fees and payment or honoraria from Caresyntax. Dr. Ramshaw reported receipt of support for this manuscript and consulting fees from Caresyntax. Dr. Woods is an employee of Caresyntax.

Ethical approval

This study was approved by the University of Iowa Hospitals & Clinics Institutional Review Board (Approval No. 202109638).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

464_2023_10425_MOESM1_ESM.jpg

Supplementary file1 (JPG 166 kb) Supplemental Figure 1. Objective Procedure-Specific Assessment (OPSA) for the 12 Consecutive Jejunojejunostomy Tasks in Laparoscopic Roux-en-Y Gastric Bypass (RYGB) Surgery

Supplementary file2 (JPG 104 kb) Supplemental Figure 2. General Assessment of Surgical Skill instrument.

464_2023_10425_MOESM3_ESM.jpg

Supplementary file3 (JPG 256 kb) Supplemental Figure 3. Global Operative Assessment of Laparoscopic Skills (GOALS) is a Validated Assessment Tool for Grading Overall Technical Proficiency for Laparoscopic Surgery. While it Measures Domains, such as Fluidity of Motion, it does not Define Surgical or Clinical Judgement, e.g., Autonomy.

Supplementary file4 (JPG 146 kb) Supplemental Figure 4. Average Scores for Each Task in the RYGB OPSA over Time.

464_2023_10425_MOESM5_ESM.jpg

Supplementary file5 (JPG 131 kb) Supplemental Figure 5. Average Rating for the Individual Components of the GASS Rating over Time.

Supplementary file6 (JPG 85 kb) Supplemental Figure 6. GOALS Performance Over Time.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nau, P., Worden, E., Lehmann, R. et al. Using video-based assessment (VBA) to document fellow improvement in safely completing the jejunojejunostomy portion of laparoscopic Roux-en-Y gastric bypass (RYGB) surgery. Surg Endosc 37, 8853–8860 (2023). https://doi.org/10.1007/s00464-023-10425-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-023-10425-y

Keywords

Search

Navigation