Skip to main content
SpringerLink
Log in

A comparative analysis of video-based surgical assessment: is evaluation of the entire critical portion of the operation necessary?

  • 2021 SAGES Oral
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Previous studies of video-based operative assessments using crowd sourcing have established the efficacy of non-expert evaluations. Our group sought to establish the equivalence of abbreviating video content for operative assessment.

Methods

A single institution video repository of six core general surgery operations was submitted for evaluation. Each core surgery included three unique surgical performances, totaling 18 unique operative videos. Each video was edited using four different protocols based on the critical portion of the operation: (1) custom edited critical portion (2) condensed critical portion (3) first 20 s of every minute of the critical portion, and (4) first 10 s of every minute of the critical portion. In total, 72 individually edited operative videos were submitted to the C-SATS (Crowd-Sourced Assessment of Technical Skills) platform (C-SATS) for evaluation. Aggregate score for study protocol was compared using the Kruskal–Wallis test. A multivariable, multilevel mixed-effects model was constructed to predict total skill assessment scores.

Results

Median video lengths for each protocol were: custom, 6:20 (IQR 5:27–7:28); condensed, 10:35 (8:50–12:06); 10 s, 4:35 (2:11–6:09); and 20 s, 9:09 (4:20–12:14). There was no difference in aggregate median score among the four study protocols: custom, 15.7 (14.4–16.2); condensed, 15.8 (15.2–16.4); 10 s, 15.8 (15.3–16.1); 20 s, 16.0 (15.1–16.3); χ2 = 1.661, p = 0.65. Regression modeling demonstrated a significant, but minimal effect of the 10 s and 20 s editing protocols compared to the custom method on individual video score: condensed, + 0.33 (− 0.05–0.70), p = 0.09; 10 s, + 0.29 (0.04–0.55), p = 0.03; 20 s, + 0.40 (0.15–0.66), p = 0.002.

Conclusion

A standardized protocol for video editing abbreviated surgical performances yields reproducible assessment of surgical aptitude when assessed by non-experts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Representation of varying edited formats of video submission of the four separate study protocols (VM1 Lap Chole 19:20)

Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Garrett K, Kaups K (2014) The aging surgeon: when is it time to leave active practice? Bull Am Coll Surg 99:32–35

    PubMed  Google Scholar 

  2. Mellinger J, Williams R, Sanfey H, Fryer J, DaRosa D, George B, Bohnen J, Schuller M, Sandhu G, Minter R, Gardner A, Scott D (2017) Teaching and assessing operative skills: from theory to practice. Curr Probl Surg 54:44–81

    Article  Google Scholar 

  3. Bieliauskaus L, Langenecker S, Graver C, Lee H, O’Neill J, Greenfield L (2008) Cognitive changes and retirement among senior surgeons (CCRASS): Results from the CCRASS study. J Am Coll Surg 207:69–78

    Article  Google Scholar 

  4. Deal S, Lendvay T, Haque M, Brand T, Comstock B, Warren J, Alseidi A (2016) Crowd-sourced assessment of technical skills: an opportunity for improvement in the assessment of laparoscopic surgical skills. Am J Surg 211:398–404

    Article  Google Scholar 

  5. Scully R, Deal S, Clark M, Yang K, Wnuk G, Smink D, Fryer J, Bohnen J, Teitelbaum E, Meyerson S, Meier A, Gauger P, Reddy R, Kendrick D, Stern M, Hughes D, Chipman J, Patel J, Alseidi A, George B (2020) Concordance between expert and nonexpert ratings of condensed video-based trainee operative performance assessment. J Surg Ed 77:627–634

    Article  Google Scholar 

  6. McQueen S, McKinnon V, Vanderbeek L, McCarthy C, Sonnadara R (2019) Video-based assessment in surgical education: a scoping review. J Surg Ed 76:1645–1654

    Article  Google Scholar 

  7. Greenberg C, Dombrowski J, Dimick J (2016) Video-Based Surgical Coaching. JAMA Surg 151:282–283

    Article  Google Scholar 

Download references

Acknowledgements

Funding for this project was made possible through the generosity of the Virginia Mason Digestive Disease Institute Research Grant (#0920801) and the Honoraria Fund for Surgical Education.

Funding

Virginia Mason Digestive Disease Institute Grant #0920801.

Author information

Authors and Affiliations

  1. Virginia Mason Medical Center, 1100 9th Ave., C6-GS, Seattle, WA, 98101, USA

    Caleb W. Barnhill, Stephen J. Kaplan, Adnan A. Alseidi & Shanley B. Deal

Authors
  1. Caleb W. Barnhill
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephen J. Kaplan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adnan A. Alseidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shanley B. Deal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Caleb W. Barnhill.

Ethics declarations

Disclosures

Doctors Barnhill, Kaplan, Alseidi and Deal have no conflicts of interest or financial ties to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barnhill, C.W., Kaplan, S.J., Alseidi, A.A. et al. A comparative analysis of video-based surgical assessment: is evaluation of the entire critical portion of the operation necessary?. Surg Endosc 36, 6719–6723 (2022). https://doi.org/10.1007/s00464-021-08945-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-021-08945-6

Keywords

Search

Navigation