Skip to main content
SpringerLink
Log in

The association between video-based assessment of intraoperative technical performance and patient outcomes: a systematic review

  • Review Article
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Efforts to improve surgical safety and outcomes have traditionally placed little emphasis on intraoperative performance, partly due to difficulties in measurement. Video-based assessment (VBA) provides an opportunity for blinded and unbiased appraisal of surgeon performance. Therefore, we aimed to systematically review the existing literature on the association between intraoperative technical performance, measured using VBA, and patient outcomes.

Methods

Major databases (Medline, Embase, Cochrane Database, and Web of Science) were systematically searched for studies assessing the association of intraoperative technical performance measured by tools supported by validity evidence with short-term (≤ 30 days) and/or long-term postoperative outcomes. Study quality was assessed using the Newcastle–Ottawa Scale. Results were appraised descriptively as study heterogeneity precluded meta-analysis.

Results

A total of 11 observational studies were identified involving 8 different procedures in foregut/bariatric (n = 4), colorectal (n = 4), urologic (n = 2), and hepatobiliary surgery (n = 1). The number of surgeons assessed ranged from 1 to 34; patient sample size ranged from 47 to 10,242. High risk of bias was present in 5 of 8 studies assessing short-term outcomes and 2 of 6 studies assessing long-term outcomes. Short-term outcomes were reported in 8 studies (i.e., morbidity, mortality, and readmission), while 6 reported long-term outcomes (i.e., cancer outcomes, weight loss, and urinary continence). Better intraoperative performance was associated with fewer postoperative complications (6 of 7 studies), reoperations (3 of 4 studies), and readmissions (1 of 4 studies). Long-term outcomes were less commonly investigated, with mixed results.

Conclusion

Current evidence supports an association between superior intraoperative technical performance measured using surgical videos and improved short-term postoperative outcomes. Intraoperative performance analysis using video-based assessment represents a promising approach to surgical quality-improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Zegers M, de Bruijne MC, Wagner C et al (2009) Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Saf Health Care 18(4):297–302. https://doi.org/10.1136/qshc.2007.025924

    Article  CAS  PubMed  Google Scholar 

  2. Kable AK, Gibberd RW, Spigelman AD (2002) Adverse events in surgical patients in Australia. Int J Qual Health Care 14(4):269–276. https://doi.org/10.1093/intqhc/14.4.269

    Article  CAS  PubMed  Google Scholar 

  3. Fabri PJ, Zayas-Castro JL (2008) Human error, not communication and systems, underlies surgical complications. Surgery 144(4):557–63; discussion 563–5. https://doi.org/10.1016/j.surg.2008.06.011

  4. Dimick JB, Varban OA (2015) Surgical video analysis: an emerging tool for improving surgeon performance. BMJ Qual Saf 24(8):490–491. https://doi.org/10.1136/bmjqs-2015-004439

    Article  PubMed  Google Scholar 

  5. Feldman LS, Pryor AD, Gardner AK et al (2020) SAGES video-based assessment (VBA) program: a vision for life-long learning for surgeons. Surg Endosc 34(8):3285–3288. https://doi.org/10.1007/s00464-020-07628-y

    Article  PubMed  Google Scholar 

  6. Yanes AF, McElroy LM, Abecassis ZA, Holl J, Woods D, Ladner DP (2016) Observation for assessment of clinician performance: a narrative review. BMJ Qual Saf 25(1):46–55. https://doi.org/10.1136/bmjqs-2015-004171

    Article  PubMed  Google Scholar 

  7. Bilgic E, Valanci-Aroesty S, Fried GM (2020) Video assessment of surgeons and surgery. Adv Surg 54:205–214. https://doi.org/10.1016/j.yasu.2020.03.002

    Article  PubMed  Google Scholar 

  8. Greenberg CC, Dombrowski J, Dimick JB (2016) Video-based surgical coaching: an emerging approach to performance improvement. JAMA Surg 151(3):282–283. https://doi.org/10.1001/jamasurg.2015.4442

    Article  PubMed  PubMed Central  Google Scholar 

  9. Grenda TR, Pradarelli JC, Dimick JB (2016) Using surgical video to improve technique and skill. Ann Surg 264(1):32–33. https://doi.org/10.1097/SLA.0000000000001592

    Article  PubMed  Google Scholar 

  10. Mackenzie H, Cuming T, Miskovic D et al (2015) Design, delivery, and validation of a trainer curriculum for the national laparoscopic colorectal training program in England. Ann Surg 261(1):149–156. https://doi.org/10.1097/SLA.0000000000000437

    Article  PubMed  Google Scholar 

  11. Mori T, Kimura T, Kitajima M (2010) Skill accreditation system for laparoscopic gastroenterologic surgeons in Japan. Minim Invasive Ther Allied Technol 19(1):18–23. https://doi.org/10.3109/13645700903492969

    Article  PubMed  Google Scholar 

  12. Birkmeyer JD, Finks JF, O’Reilly A et al (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369(15):1434–1442. https://doi.org/10.1056/NEJMsa1300625

    Article  CAS  PubMed  Google Scholar 

  13. Fecso AB, Szasz P, Kerezov G, Grantcharov TP (2017) The effect of technical performance on patient outcomes in surgery: a systematic review. Ann Surg 265(3):492–501. https://doi.org/10.1097/SLA.0000000000001959

    Article  PubMed  Google Scholar 

  14. Page MJ, McKenzie JE, Bossuyt PM et al (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372:71. https://doi.org/10.1136/bmj.n71

    Article  Google Scholar 

  15. Kelley WE Jr (2008) The evolution of laparoscopy and the revolution in surgery in the decade of the 1990s. JSLS 12(4):351–357

    PubMed  PubMed Central  Google Scholar 

  16. McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C (2016) PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol 75:40–46. https://doi.org/10.1016/j.jclinepi.2016.01.021

    Article  PubMed  Google Scholar 

  17. Horsley T, Dingwall O (2011) Sampson M (2011) Checking reference lists to find additional studies for systematic reviews. Cochrane Database Syst Rev 2011(8):MR000026. https://doi.org/10.1002/14651858.MR000026.pub2

    Article  PubMed Central  Google Scholar 

  18. Peterson J, Welch V, Losos M, Tugwell P (2011) The Newcastle-Ottawa scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa Hospital Research Institute, Ottawa

    Google Scholar 

  19. Wells G, Shea B, O’connell D et al (2011) The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa Hospital Research Institute. Oxford. Asp, Ottawa

    Google Scholar 

  20. Viale L, Allotey J, Cheong-See F et al (2015) Epilepsy in pregnancy and reproductive outcomes: a systematic review and meta-analysis. Lancet 386(10006):1845–1852. https://doi.org/10.1016/S0140-6736(15)00045-8

    Article  PubMed  Google Scholar 

  21. Sobhy S, Zamora J, Dharmarajah K et al (2016) Anaesthesia-related maternal mortality in low-income and middle-income countries: a systematic review and meta-analysis. Lancet Glob Health 4(5):e320–e327. https://doi.org/10.1016/S2214-109X(16)30003-1

    Article  PubMed  Google Scholar 

  22. Papola D, Ostuzzi G, Thabane L, Guyatt G, Barbui C (2018) Antipsychotic drug exposure and risk of fracture: a systematic review and meta-analysis of observational studies. Int Clin Psychopharmacol 33(4):181–196. https://doi.org/10.1097/YIC.0000000000000221

    Article  PubMed  Google Scholar 

  23. Wang B, An X, Shi X, Zhang JA (2017) Management of endocrine disease: suicide risk in patients with diabetes: a systematic review and meta-analysis. Eur J Endocrinol 177(4):R169–R181. https://doi.org/10.1530/EJE-16-0952

    Article  CAS  PubMed  Google Scholar 

  24. Visser A, Geboers B, Gouma DJ, Goslings JC, Ubbink DT (2015) Predictors of surgical complications: a systematic review. Surgery 158(1):58–65. https://doi.org/10.1016/j.surg.2015.01.012

    Article  PubMed  Google Scholar 

  25. Dettori JR (2011) Loss to follow-up. Evid Based Spine Care J 2(1):7–10. https://doi.org/10.1055/s-0030-1267080

    Article  PubMed  PubMed Central  Google Scholar 

  26. Popay J, Roberts H, Sowden A, et al (2006) Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme Version. 1:b92.

  27. Arvidsson D, Berndsen FH, Larsson LG et al (2005) Randomized clinical trial comparing 5-year recurrence rate after laparoscopic versus Shouldice repair of primary inguinal hernia. Br J Surg 92(9):1085–1091. https://doi.org/10.1002/bjs.5137

    Article  CAS  PubMed  Google Scholar 

  28. Mills JT, Hougen HY, Bitner D, Krupski TL, Schenkman NS (2017) Does robotic surgical simulator performance correlate with surgical skill? J Surg Educ 74(6):1052–1056. https://doi.org/10.1016/j.jsurg.2017.05.011

    Article  PubMed  Google Scholar 

  29. Brajcich BC, Stulberg JJ, Palis BE et al (2021) Association between surgical technical skill and long-term survival for colon cancer. JAMA Oncol 7(1):127–129. https://doi.org/10.1001/jamaoncol.2020.5462

    Article  PubMed  Google Scholar 

  30. Varban OA, Thumma JR, Finks JF, Carlin AM, Ghaferi AA, Dimick JB (2021) Evaluating the effect of surgical skill on outcomes for laparoscopic sleeve gastrectomy: a video-based study. Ann Surg 273(4):766–771. https://doi.org/10.1097/SLA.0000000000003385

    Article  PubMed  Google Scholar 

  31. Stulberg JJ, Huang R, Kreutzer L et al (2020) Association between surgeon technical skills and patient outcomes. JAMA Surg 19:19

    Google Scholar 

  32. Curtis NJ, Foster JD, Miskovic D et al (2020) Association of surgical skill assessment with clinical outcomes in cancer surgery. JAMA Surg 155(7):590–598. https://doi.org/10.1001/jamasurg.2020.1004

    Article  PubMed  Google Scholar 

  33. Fecso AB, Bhatti JA, Stotland PK, Quereshy FA, Grantcharov TP (2019) Technical performance as a predictor of clinical outcomes in laparoscopic gastric cancer surgery. Ann Surg 270(1):115–120. https://doi.org/10.1097/SLA.0000000000002741

    Article  PubMed  Google Scholar 

  34. Goldenberg MG, Goldenberg L, Grantcharov TP (2017) Surgeon performance predicts early continence after robot-assisted radical prostatectomy. J Endourol 31(9):858–863. https://doi.org/10.1089/end.2017.0284

    Article  PubMed  Google Scholar 

  35. Scally CP, Varban OA, Carlin AM, Birkmeyer JD, Dimick JB, Michigan Bariatric Surgery C. (2016) Video ratings of surgical skill and late outcomes of bariatric surgery. JAMA Surg 151(6):1160428. https://doi.org/10.1001/jamasurg.2016.0428

    Article  Google Scholar 

  36. Paterson C, McLuckie S, Yew-Fung C, Tang B, Lang S, Nabi G (2016) Videotaping of surgical procedures and outcomes following extraperitoneal laparoscopic radical prostatectomy for clinically localized prostate cancer. J Surg Oncol 114(8):1016–1023. https://doi.org/10.1002/jso.24484

    Article  PubMed  Google Scholar 

  37. Hogg ME, Zenati M, Novak S et al (2016) Grading of surgeon technical performance predicts postoperative pancreatic fistula for pancreaticoduodenectomy independent of patient-related variables. Ann Surg 264(3):482–491. https://doi.org/10.1097/SLA.0000000000001862

    Article  PubMed  Google Scholar 

  38. Mackenzie H, Ni M, Miskovic D et al (2015) Clinical validity of consultant technical skills assessment in the English National Training Programme for Laparoscopic Colorectal Surgery. Br J Surg 102(8):991–997. https://doi.org/10.1002/bjs.9828

    Article  CAS  PubMed  Google Scholar 

  39. Madani A, Vassiliou MC, Watanabe Y et al (2017) What are the principles that guide behaviors in the operating room?: Creating a framework to define and measure performance. Ann Surg 265(2):255–267. https://doi.org/10.1097/SLA.0000000000001962

    Article  PubMed  Google Scholar 

  40. Varban OA, Sheetz KH, Cassidy RB et al (2017) Evaluating the effect of operative technique on leaks after laparoscopic sleeve gastrectomy: a case-control study. Surg Obes Relat Dis 13(4):560–567. https://doi.org/10.1016/j.soard.2016.11.027

    Article  PubMed  Google Scholar 

  41. Ponce J (2018) Impact of different surgical techniques on outcomes in laparoscopic sleeve gastrectomies: first report from the metabolic and bariatric surgery accreditation and quality improvement program (MBSAQIP). Ann Surg 267(3):e52. https://doi.org/10.1097/SLA.0000000000002076

    Article  PubMed  Google Scholar 

  42. Le AT, Huang B, Hnoosh D et al (2017) Effect of complications on oncologic outcomes after pancreaticoduodenectomy for pancreatic cancer. J Surg Res 214:1–8. https://doi.org/10.1016/j.jss.2017.02.036

    Article  PubMed  Google Scholar 

  43. Park EJ, Baik SH, Kang J et al (2016) The impact of postoperative complications on long-term oncologic outcomes after laparoscopic low anterior resection for rectal cancer. Medicine (Baltimore) 95(14):e3271. https://doi.org/10.1097/MD.0000000000003271

    Article  Google Scholar 

  44. Beecher SM, O’Leary DP, McLaughlin R, Kerin MJ (2018) The impact of surgical complications on cancer recurrence rates: a literature review. Oncol Res Treat 41(7–8):478–482. https://doi.org/10.1159/000487510

    Article  PubMed  Google Scholar 

  45. Greenberg CC, Ghousseini HN, Pavuluri Quamme SR, Beasley HL, Wiegmann DA (2015) Surgical coaching for individual performance improvement. Ann Surg 261(1):32–34. https://doi.org/10.1097/SLA.0000000000000776

    Article  PubMed  Google Scholar 

  46. ABS to Explore Video-Based Assessment in Pilot Program Launching June 2021. The American Board of Surgery; 2021. Accessed 2021/4/22. https://www.absurgery.org/default.jsp?news_vba04.21

  47. Trehan A, Barnett-Vanes A, Carty MJ, McCulloch P, Maruthappu M (2015) The impact of feedback of intraoperative technical performance in surgery: a systematic review. BMJ Open 5(6):e006759. https://doi.org/10.1136/bmjopen-2014-006759

    Article  PubMed  PubMed Central  Google Scholar 

  48. Ritter EM, Gardner AK, Dunkin BJ, Schultz L, Pryor AD, Feldman L (2020) Video-based assessment for laparoscopic fundoplication: initial development of a robust tool for operative performance assessment. Surg Endosc 34(7):3176–3183. https://doi.org/10.1007/s00464-019-07089-y

    Article  PubMed  Google Scholar 

  49. Watanabe Y, Bilgic E, Lebedeva E et al (2016) A systematic review of performance assessment tools for laparoscopic cholecystectomy. Surg Endosc 30(3):832–844. https://doi.org/10.1007/s00464-015-4285-8

    Article  PubMed  Google Scholar 

  50. Bilgic E, Al Mahroos M, Landry T, Fried GM, Vassiliou MC, Feldman LS (2019) Assessment of surgical performance of laparoscopic benign hiatal surgery: a systematic review. Surg Endosc 33(11):3798–3805. https://doi.org/10.1007/s00464-019-06662-9

    Article  PubMed  Google Scholar 

  51. Tam V, Zeh HJ 3rd, Hogg ME (2017) Incorporating metrics of surgical proficiency into credentialing and privileging pathways. JAMA Surg 152(5):494–495. https://doi.org/10.1001/jamasurg.2017.0025

    Article  PubMed  Google Scholar 

  52. Deijen CL, Velthuis S, Tsai A et al (2016) COLOR III: a multicentre randomised clinical trial comparing transanal TME versus laparoscopic TME for mid and low rectal cancer. Surg Endosc 30(8):3210–3215. https://doi.org/10.1007/s00464-015-4615-x

    Article  PubMed  Google Scholar 

  53. Scott DJ, Rege RV, Bergen PC et al (2000) Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation. J Laparoendosc Adv Surg Tech A 10(4):183–190. https://doi.org/10.1089/109264200421559

    Article  CAS  PubMed  Google Scholar 

  54. Joosten M, Bokkerink GMJ, Verhoeven BH, Sutcliffe J, de Blaauw I, Botden S (2021) Are self-assessment and peer assessment of added value in training complex pediatric surgical skills? Eur J Pediatr Surg 31(1):25–33. https://doi.org/10.1055/s-0040-1715438

    Article  PubMed  Google Scholar 

  55. Scully RE, Deal SB, Clark MJ et al (2020) Concordance between expert and nonexpert ratings of condensed video-based trainee operative performance assessment. J Surg Educ 77(3):627–634. https://doi.org/10.1016/j.jsurg.2019.12.016

    Article  PubMed  Google Scholar 

  56. Deal SB, Lendvay TS, Haque MI et al (2016) Crowd-sourced assessment of technical skills: an opportunity for improvement in the assessment of laparoscopic surgical skills. Am J Surg 211(2):398–404. https://doi.org/10.1016/j.amjsurg.2015.09.005

    Article  PubMed  Google Scholar 

  57. Feldman M, Lazzara EH, Vanderbilt AA, DiazGranados D (2012) Rater training to support high-stakes simulation-based assessments. J Contin Educ Health Prof Fall 32(4):279–286. https://doi.org/10.1002/chp.21156

    Article  Google Scholar 

  58. Bilgic E, Watanabe Y, McKendy K et al (2016) Reliable assessment of operative performance. Am J Surg 211(2):426–430. https://doi.org/10.1016/j.amjsurg.2015.10.008

    Article  PubMed  Google Scholar 

Download references

Funding

Fonds de la recherche en Sante du Quebec (FRSQ)- Grant Number 288097.

Author information

Authors and Affiliations

  1. Department of Surgery, McGill University, 1650 Cedar Ave, D6-136, Montreal, QC, H3G 1A4, Canada

    Saba Balvardi, Anitha Kammili, Melissa Hanson, Carmen Mueller, Melina Vassiliou, Lawrence Lee, Julio F. Fiore Jr. & Liane S. Feldman

  2. Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, Montreal, QC, Canada

    Saba Balvardi, Anitha Kammili, Melissa Hanson, Carmen Mueller, Melina Vassiliou, Lawrence Lee, Julio F. Fiore Jr. & Liane S. Feldman

  3. Respiratory Division, Department of Medicine, McGill University, Montreal, QC, Canada

    Kevin Schwartzman

  4. McGill International Tuberculosis Centre, Research Institute of the McGill University Health Centre, Montreal, QC, Canada

    Kevin Schwartzman

Authors
  1. Saba Balvardi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anitha Kammili
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Melissa Hanson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Carmen Mueller
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Melina Vassiliou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lawrence Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kevin Schwartzman
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Julio F. Fiore Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Liane S. Feldman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liane S. Feldman.

Ethics declarations

Disclosures

Saba Balvardi, Anitha Kammili, Melissa Hanson, Carmen Mueller, Melina Vassiliou, Lawrence Lee, Kevin Schwartzman, Julio F Fiore Jr., Liane S Feldman declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 47 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Balvardi, S., Kammili, A., Hanson, M. et al. The association between video-based assessment of intraoperative technical performance and patient outcomes: a systematic review. Surg Endosc 36, 7938–7948 (2022). https://doi.org/10.1007/s00464-022-09296-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-022-09296-6

Keywords

Search

Navigation