Skip to main content
SpringerLink
Log in

A multifactorial evaluation of objective performance indicators and video analysis in the context of case complexity and clinical outcomes in robotic-assisted cholecystectomy

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

The increased digitization in robotic surgical procedures today enables surgeons to quantify their movements through data captured directly from the robotic system. These calculations, called objective performance indicators (OPIs), offer unprecedented detail into surgical performance. In this study, we link case- and surgical step-specific OPIs to case complexity, surgical experience and console utilization, and post-operative clinical complications across 87 robotic cholecystectomy (RC) cases.

Methods

Videos of RCs performed by a principal surgeon with and without fellows were segmented into eight surgical steps and linked to patients’ clinical data. Data for OPI calculations were extracted from an Intuitive Data Recorder and the da Vinci ® robotic system. RC cases were each assigned a Nassar and Parkland Grading score and categorized as standard or complex. OPIs were compared across complexity groups, console attributions, and post-surgical complication severities to determine objective relationships across variables.

Results

Across cases, differences in camera control and head positioning metrics of the principal surgeon were observed when comparing standard and complex cases. Further, OPI differences across the principal surgeon and the fellow(s) were observed in standard cases and include differences in arm swapping, camera control, and clutching behaviors. Monopolar coagulation energy usage differences were also observed. Select surgical step duration differences were observed across complexities and console attributions, and additional surgical task analyses determine the adhesion removal and liver bed hemostasis steps to be the most impactful steps for case complexity and post-surgical complications, respectively.

Conclusion

This is the first study to establish the association between OPIs, case complexities, and clinical complications in RC. We identified OPI differences in intra-operative behaviors and post-surgical complications dependent on surgeon expertise and case complexity, opening the door for more standardized assessments of teaching cases, surgical behaviors and case complexities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Vivek MA, Augustine AJ, Rao R (2014) A comprehensive predictive scoring method for difficult laparoscopic cholecystectomy. J Minim Access Surg 10(2):62–67. https://doi.org/10.4103/0972-9941.129947

    Article  PubMed  PubMed Central  Google Scholar 

  2. Griffiths EA, Hodson J, Vohra RS et al (2019) Utilisation of an operative difficulty grading scale for laparoscopic cholecystectomy. Surg Endosc 33(1):110–121. https://doi.org/10.1007/s00464-018-6281-2

    Article  PubMed  Google Scholar 

  3. Madni TD, Leshikar DE, Minshall CT et al (2018) The Parkland grading scale for cholecystitis. Am J Surg 215(4):625–630. https://doi.org/10.1016/j.amjsurg.2017.05.017

    Article  PubMed  Google Scholar 

  4. Nassar AHM, Hodson J, Ng HJ et al (2020) Predicting the difficult laparoscopic cholecystectomy: development and validation of a pre-operative risk score using an objective operative difficulty grading system. Surg Endosc 34(10):4549–4561. https://doi.org/10.1007/s00464-019-07244-5

    Article  PubMed  Google Scholar 

  5. Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ (2012) Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol 187(1):247–252. https://doi.org/10.1016/j.juro.2011.09.032

    Article  PubMed  Google Scholar 

  6. Aghazadeh MA, Jayaratna IS, Hung AJ et al (2015) External validation of Global Evaluative Assessment of Robotic Skills (GEARS). Surg Endosc 29(11):3261–3266. https://doi.org/10.1007/s00464-015-4070-8

    Article  PubMed  Google Scholar 

  7. Martin JA, Regehr G, Reznick R et al (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84(2):273–278. https://doi.org/10.1046/j.1365-2168.1997.02502.x

    Article  CAS  PubMed  Google Scholar 

  8. Fard MJ, Ameri S, Darin Ellis R, Chinnam RB, Pandya AK, Klein MD (2018) Automated robot-assisted surgical skill evaluation: predictive analytics approach. Int J Med Robot. https://doi.org/10.1002/rcs.1850

    Article  PubMed  Google Scholar 

  9. Jarc AM, Curet MJ (2017) Viewpoint matters: objective performance metrics for surgeon endoscope control during robot-assisted surgery. Surg Endosc 31(3):1192–1202. https://doi.org/10.1007/s00464-016-5090-8

    Article  PubMed  Google Scholar 

  10. Lyman WB, Passeri M, Murphy K et al (2018) Novel objective approach to evaluate novice robotic surgeons using a combination of kinematics and stepwise cumulative sum analyses. J Am Coll Surg 227(4):S223–S224

    Article  Google Scholar 

  11. Hung AJ, Oh PJ, Chen J et al (2019) Experts vs super-experts: differences in automated performance metrics and clinical outcomes for robot-assisted radical prostatectomy. BJU Int 123(5):861–868. https://doi.org/10.1111/bju.14599

    Article  PubMed  Google Scholar 

  12. Lyman WB, Passeri MJ, Murphy K et al (2021) An objective approach to evaluate novice robotic surgeons using a combination of kinematics and stepwise cumulative sum (CUSUM) analyses. Surg Endosc 35(6):2765–2772. https://doi.org/10.1007/s00464-020-07708-z

    Article  PubMed  Google Scholar 

  13. Ma R, Lee RS, Nguyen JH et al (2022) Tailored feedback based on clinically relevant performance metrics expedites the acquisition of robotic suturing skills—an unblinded pilot randomized controlled trial. J Urol 208(2):414–424. https://doi.org/10.1097/JU.0000000000002691

    Article  PubMed  Google Scholar 

  14. Hung AJ, Chen J, Jarc A, Hatcher D, Djaladat H, Gill IS (2018) Development and validation of objective performance metrics for robot-assisted radical prostatectomy: a pilot study. J Urol 199(1):296–304. https://doi.org/10.1016/j.juro.2017.07.081

    Article  PubMed  Google Scholar 

  15. Hung AJ, Chen J, Che Z et al (2018) Utilizing machine learning and automated performance metrics to evaluate robot-assisted radical prostatectomy performance and predict outcomes. J Endourol 32(5):438–444. https://doi.org/10.1089/end.2018.0035

    Article  PubMed  Google Scholar 

  16. Brown KC, Bhattacharyya KD, Kulason S, Zia A, Jarc A (2020) How to bring surgery to the next level: interpretable skills assessment in robotic-assisted surgery. Visc Med 36(6):463–470. https://doi.org/10.1159/000512437

    Article  PubMed  PubMed Central  Google Scholar 

  17. Dindo D, Demartines N, Clavien PA (2004) Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg 240(2):205–213. https://doi.org/10.1097/01.sla.0000133083.54934.ae

    Article  PubMed  PubMed Central  Google Scholar 

  18. Slankamenac K, Graf R, Barkun J, Puhan MA, Clavien PA (2013) The comprehensive complication index: a novel continuous scale to measure surgical morbidity. Ann Surg 258(1):1–7. https://doi.org/10.1097/SLA.0b013e318296c732

    Article  PubMed  Google Scholar 

  19. Mehta HB, Dimou F, Adhikari D et al (2016) Comparison of comorbidity scores in predicting surgical outcomes. Med Care 54(2):180–187. https://doi.org/10.1097/mlr.0000000000000465

    Article  PubMed  PubMed Central  Google Scholar 

  20. Salky BA, Edye MB (1998) The difficult cholecystectomy: problems related to concomitant diseases. Semin Laparosc Surg 5(2):107–114. https://doi.org/10.1177/155335069800500205

    Article  CAS  PubMed  Google Scholar 

  21. Lauder CI, Garcea G, Strickland A, Maddern GJ (2010) Abdominal adhesion prevention: still a sticky subject? Dig Surg 27(5):347–358. https://doi.org/10.1159/000314805

    Article  PubMed  Google Scholar 

  22. Weibel MA, Majno G (1973) Peritoneal adhesions and their relation to abdominal surgery. A postmortem study. Am J Surg 126(3):345–353. https://doi.org/10.1016/s0002-9610(73)80123-0

    Article  CAS  PubMed  Google Scholar 

  23. Menzies D, Ellis H (1990) Intestinal obstruction from adhesions—how big is the problem? Ann R Coll Surg Engl 72(1):60–63

    CAS  PubMed  PubMed Central  Google Scholar 

  24. Ten Broek RP, Issa Y, van Santbrink EJ et al (2013) Burden of adhesions in abdominal and pelvic surgery: systematic review and met-analysis. BMJ 347:f5588. https://doi.org/10.1136/bmj.f5588

    Article  PubMed  PubMed Central  Google Scholar 

  25. Nkere UU (2000) Postoperative adhesion formation and the use of adhesion preventing techniques in cardiac and general surgery. ASAIO J 46(6):654–656. https://doi.org/10.1097/00002480-200011000-00003

    Article  CAS  PubMed  Google Scholar 

  26. Bhandari TR, Khan SA, Jha JL (2021) Prediction of difficult laparoscopic cholecystectomy: an observational study. Ann Med Surg (Lond) 72:103060. https://doi.org/10.1016/j.amsu.2021.103060

    Article  PubMed  Google Scholar 

  27. Kaushik R (2010) Bleeding complications in laparoscopic cholecystectomy: incidence, mechanisms, prevention and management. J Minim Access Surg 6(3):59–65. https://doi.org/10.4103/0972-9941.68579

    Article  PubMed  PubMed Central  Google Scholar 

  28. Melnyk R, Chen Y, Holler T et al (2022) Utilizing head-mounted eye trackers to analyze patterns and decision-making strategies of 3D virtual modelling platform (IRIS(™)) during preoperative planning for renal cancer surgeries. World J Urol 40(3):651–658. https://doi.org/10.1007/s00345-021-03906-z

    Article  PubMed  Google Scholar 

  29. Lindeman B, Brasel K, Minter RM, Buyske J, Grambau M, Sarosi G (2021) A phased approach: the general surgery experience adopting entrustable professional activities in the United States. Acad Med 96(7s):S9–S13. https://doi.org/10.1097/acm.0000000000004107

    Article  PubMed  Google Scholar 

  30. American Board of Surgery (2022) ABS announces transition to entrustable professional activities for general surgery resident evaluation. https://www.absurgery.org/default.jsp?news_epas0222. Accessed 7 March 2023

  31. Lam K, Chen J, Wang Z et al (2022) Machine learning for technical skill assessment in surgery: a systematic review. NPJ Digit Med 5(1):24. https://doi.org/10.1038/s41746-022-00566-0

    Article  PubMed  PubMed Central  Google Scholar 

  32. Vedula SS, Ghazi A, Collins JW et al (2022) Artificial intelligence methods and artificial intelligence-enabled metrics for surgical education: a multidisciplinary consensus. J Am Coll Surg 234(6):1181–1192. https://doi.org/10.1097/xcs.0000000000000190

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors acknowledge Michelle Liu and Busisiwe Mlambo, MD for their help in the surgical step annotations and video breakdown.

Funding

No funding was received for this study.

Author information

Authors and Affiliations

  1. Good Samaritan Medical Center, Brockton, MA, USA

    Georges Kaoukabani & Fahri Gokcal

  2. Applied Research, Intuitive Surgical Inc., Peachtree City, GA, USA

    Abeselom Fanta, Xi Liu, Mallory Shields & Catherine Stricklin

  3. Tufts Medical Center, Boston, MA, USA

    Alexander Friedman

  4. Tufts University School of Medicine, Boston, MA, USA

    Omar Yusef Kudsi

Authors
  1. Georges Kaoukabani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fahri Gokcal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abeselom Fanta
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mallory Shields
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Catherine Stricklin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Alexander Friedman
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Omar Yusef Kudsi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Omar Yusef Kudsi.

Ethics declarations

Disclosures

Drs. Georges Kaoukabani, Alexander Friedman, and Fahri Gokcal have no conflicts of interest or financial ties to disclose. Abeselom Fanta, Xi Liu, and Mallory Shields are employees at Intuitive Surgical and do not report any other conflict of interest of financial ties. Dr. Omar Yusef Kudsi has received a teaching course and/or consultancy fees from Intuitive Surgical, Bard-Davol, and W.L. Gore outside the submitted work.

Ethical approval

The database used for this study approved by the Institutional Review Board.

Human and animal rights

All procedures performed in studies involving human participants were under the ethical standards of the Institutional and/or National Research Committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kaoukabani, G., Gokcal, F., Fanta, A. et al. A multifactorial evaluation of objective performance indicators and video analysis in the context of case complexity and clinical outcomes in robotic-assisted cholecystectomy. Surg Endosc 37, 8540–8551 (2023). https://doi.org/10.1007/s00464-023-10432-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-023-10432-z

Keywords

Search

Navigation