Skip to main content
SpringerLink
Log in

Active control time: an objective performance metric for trainee participation in robotic surgery

  • Research
  • Published:
Journal of Robotic Surgery Aims and scope Submit manuscript

Abstract

Trainee participation and progression in robotic general surgery remain poorly defined. Computer-assisted technology offers the potential to provide and track objective performance metrics. In this study, we aimed to validate the use of a novel metric—active control time (ACT)—for assessing trainee participation in robotic-assisted cases. Performance data from da Vinci Surgical Systems was retrospectively analyzed for all robotic cases involving trainees with a single minimally invasive surgeon over 10 months. The primary outcome metric was percent ACT—the amount of trainee console time spent in active system manipulations over total active time from both consoles. Kruskal–Wallis and Mann–Whitney U statistical tests were applied in analyses. A total of 123 robotic cases with 18 general surgery residents and 1 fellow were included. Of these, 56 were categorized as complex. Median %ACT was statistically different between trainee levels for all case types taken in aggregate (PGY1s 3.0% [IQR 2–14%], PGY3s 32% [IQR 27–66%], PGY4s 42% [IQR 26–52%], PGY5s 50% [IQR 28–70%], and fellow 61% [IQR 41–85%], p =  < 0.0001). When stratified by complexity, median %ACT was higher in standard versus complex cases for PGY5 (60% vs. 36%, p = 0.0002) and fellow groups (74% vs. 47%, p = 0.0045). In this study, we demonstrated an increase in %ACT with trainee level and with standard versus complex robotic cases. These findings are consistent with hypotheses, providing validity evidence for ACT as an objective measurement of trainee participation in robotic-assisted cases. Future studies will aim to define task-specific ACT to guide further robotic training and performance assessments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data Availability

The dataset generated during and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.

References

  1. Sheetz KH, Claflin J, Dimick JB (2020) Trends in the adoption of robotic surgery for common surgical procedures. JAMA Netw Open. https://doi.org/10.1001/jamanetworkopen.2019.18911

    Article  PubMed  PubMed Central  Google Scholar 

  2. Juo YY, Mantha A, Abiri A, Lin A, Dutson E (2018) Diffusion of robotic-assisted laparoscopic technology across specialties: a national study from 2008 to 2013. Surg Endosc 32:1405–1413

    Article  PubMed  Google Scholar 

  3. Mustafa S, Handren E, Farmer D, Ontiveros E, Ogola GO, Leeds SG (2019) Robotic curriculum enhances minimally invasive general surgery residents’ education. J Surg Educ 76:548–553

    Article  PubMed  Google Scholar 

  4. Winder JS, Juza RM, Sasaki J, Rogers AM, Pauli EM, Haluck RS, Estes SJ, Lyn-Sue JR (2016) Implementing a robotics curriculum at an academic general surgery training program: our initial experience. J Robot Surg 10:209–2013

    Article  PubMed  Google Scholar 

  5. Gerull W, Zihni A, Awad M (2020) Operative performance outcomes of a simulator-based robotic skills curriculum. Surg Endosc 34:4543–4548

    Article  PubMed  Google Scholar 

  6. Chen R, Rodrigues Armijo P, Krause C, Siu KC, Oleynikov D, SAGES Robotic Task Force (2020) A comprehensive review of robotic surgery curriculum and training for residents, fellows, and postgraduate surgical education. Surg Endosc 34:361–367

    Article  PubMed  Google Scholar 

  7. Fernandes E, Elli E, Giulianotti P (2014) The role of the dual console in robotic surgical training. Surgery. https://doi.org/10.1016/j.surg.2013.06.023

    Article  PubMed  Google Scholar 

  8. Zhao B, Lam J, Hollandsworth HM, Lee AM, Lopez NE, Abbadessa B, Eisenstein S, Cosman BC, Ramamoorthy SL, Parry LA (2020) General surgery training in the era of robotic surgery: a qualitative analysis of perceptions from resident and attending surgeons. Surg Endosc 34:1712–1721

    Article  PubMed  Google Scholar 

  9. Mehaffey JH, Michaels AD, Mullen MG, Yount KW, Meneveau MO, Smith PW, Friel CM, Schirmer BD (2017) Adoption of robotics in a general surgery residency program: at what cost? J Surg Res. https://doi.org/10.1016/j.jss.2017.02.052

    Article  PubMed  PubMed Central  Google Scholar 

  10. Khalafallah YM, Bernaiche T, Ranson S, Liu C, Collins DT, Dort J, Hafner G (2021) Residents’ views on the impact of robotic surgery on general surgery education. J Surg Educ 78:1007–1012

    Article  PubMed  Google Scholar 

  11. Shaw RD, Eid MA, Bleicher J, Broecker J, Caesar B, Chin R, Meyer C, Mitsakos A, Stolarksi AE, Theiss L, Smith BK, Ivatury SJ (2021) Current barriers in robotic surgery training for general surgery residents. J Surg Educ. https://doi.org/10.1016/j.jsurg.2021.11.005

    Article  PubMed  Google Scholar 

  12. Meyerson SL, Teitelbaum EN, George BC, Schuller MC, DaRosa DA, Fryer JP (2014) Defining the autonomy gap: when expectations do not meet reality in the operating room. J Surg Educ. https://doi.org/10.1016/j.jsurg.2014.05.002

    Article  PubMed  Google Scholar 

  13. Meyerson SL, Sternbach JM, Zwischenberger JB, Bender EM (2017) Resident autonomy in the operating room: expectations versus reality. Ann Thorac Surg. https://doi.org/10.1016/j.athoracsur.2017.05.034

    Article  PubMed  Google Scholar 

  14. Pernar LIM, Robertson FC, Tavakkoli A, Sheu EG, Brooks DC, Smink DS (2017) An appraisal of the learning curve in robotic general surgery. Surg Endosc. https://doi.org/10.1007/s00464-017-5520-2

    Article  PubMed  Google Scholar 

  15. Soomro NA, Hashimoto DA, Porteous AJ, Ridley CJA, Marsh WJ, Ditto R, Roy S (2020) Systematic review of learning curves in robot-assisted surgery. BJS Open. https://doi.org/10.1002/bjs5.50235

    Article  PubMed  PubMed Central  Google Scholar 

  16. Hung AJ, Chen J, Gill IS (2018) Automated performance metrics and machine learning algorithms to measure surgeon performance and anticipate clinical outcomes in robotic surgery. JAMA Surg. https://doi.org/10.1001/jamasurg.2018.1512

    Article  PubMed  PubMed Central  Google Scholar 

  17. Kumar R, Jog A, Vagvolgyi B, Nguyen H, Hager G, Chen CC, Yuh D (2012) Objective measures for longitudinal assessment of robotic surgery training. J Thorac Cardiovasc Surg. https://doi.org/10.1016/j.jtcvs.2011.11.002

    Article  PubMed  Google Scholar 

  18. Chen J, Oh PJ, Cheng N, Shah A, Montez J, Jarc A, Guo L, Gill IS, Hung AJ (2018) Use of automated performance metrics to measure surgeon performance during robotic vesicourethral anastomosis and methodical development of a training tutorial. J Urol. https://doi.org/10.1016/j.juro.2018.05.080

    Article  PubMed  PubMed Central  Google Scholar 

  19. Hung AJ, Chen J, Che Z, Nilanon T, Jarc A, Titus M, Oh PJ, Gill IS, Liu Y (2018) Utilizing machine learning and automated performance metrics to evaluate robot-assisted radical prostatectomy performance and predict outcomes. J Endourol. https://doi.org/10.1089/end.2018.0035

    Article  PubMed  Google Scholar 

  20. Kadakia N, Malek K, Lee SK, Lee EJ, Burruss S, Srikureja D, Mukherjee K, Lum SS (2020) Impact of robotic surgery on residency training for herniorrhaphy and cholecystectomy. Am Surg. https://doi.org/10.1177/0003134820964430

    Article  PubMed  Google Scholar 

  21. George BC, Bohnen JD, Williams RG, Meyerson SL, Schuller MC, Clark MJ, Meier AH, Torbeck L, Mandell SP, Mullen JT, Smink DS, Scully RE, Chipman JG, Auyang ED, Terhune KP, Wise PE, Choi JN, Foley EF, Dimick JB, Choti MA, Soper NJ, Lillemoe KD, Zwischenberger JB, Dunnington GL, DaRosa DA, Fryer JP, Procedural Learning and Safety Collaborative (PLSC) (2017) Readiness of US general surgery residents for independent practice. Ann Surg. https://doi.org/10.1097/SLA.0000000000002414

    Article  PubMed  Google Scholar 

  22. Booth KK, Stewart KE, Lewis SL, Garwe T, Kempenich JW, Lees JS (2021) Correlation of supervised independence and performance with procedure difficulty amongst surgical residents stratified by post graduate year. J Surg Educ. https://doi.org/10.1016/j.jsurg.2021.08.002

    Article  PubMed  Google Scholar 

  23. Gofton WT, Dudek NL, Wood TJ, Balaa F, Hamstra SJ (2012) The Ottawa surgical competency operating room evaluation (O-SCORE): a tool to assess surgical competence. Acad Med 87(10):1401–1407

    Article  PubMed  Google Scholar 

  24. Bohnen JD, George BC, Williams RG, Schuller MC, DaRosa DA, Torbeck L, Mullen JT, Meyerson SL, Auyang ED, Chipman JG, Choi JN, Choti MA, Endean ED, Foley EF, Mandell SP, Meier AH, Smink DS, Terhune KP, Wise PE, Soper NJ, Zwischenberger JB, Lillemoe KD, Dunnington GL, Fryer JP, Procedural Learning and Safety Collaborative (PLSC) (2016) The feasibility of real-time intraoperative performance assessment with SIMPL (system for improving and measuring procedural learning): early experience from a multi-institutional trial. J Surg Educ 73(6):118–130

    Article  Google Scholar 

  25. Eaton M, Scully R, Schuller M, Yang A, Smink D, Williams RG, Bohnen JD, George BC, Fryer JP, Meyerson SL (2019) Value and barriers to use of the SIMPL tool for resident feedback. J Surg Educ. https://doi.org/10.1016/j.jsurg.2019.01.012

    Article  PubMed  Google Scholar 

  26. Sánchez R, Rodríguez O, Rosciano J, Vegas L, Bond V, Rojas A, Sanchez-Ismayel A (2016) Robotic surgery training: construct validity of global evaluative assessment of robotic skills (GEARS). J Robot Surg. https://doi.org/10.1007/s11701-016-0572-1

    Article  PubMed  Google Scholar 

  27. Liu M, Purohit S, Mazanetz J, Allen W, Kreaden US, Curet M (2018) Assessment of Robotic Console Skills (ARCS): construct validity of a novel global rating scale for technical skills in robotically assisted surgery. Surg Endosc. https://doi.org/10.1007/s00464-017-5694-7

    Article  PubMed  PubMed Central  Google Scholar 

  28. Intuitive Surgical, Inc. 2021 Annual Report. https://www.annualreports.com/HostedData/AnnualReports/PDF/NASDAQ_ISRG_2021.pdf. Accessed 25 Mar 2022

  29. Tousignant MR, Liu X, Ershad Langroodi M, Jarc AM (2022) Identification of main influencers of surgical efficiency and variability using task-level objective metrics: a five-year robotic sleeve gastrectomy case series. Front Surg. https://doi.org/10.3389/fsurg.2022.756522

    Article  PubMed  PubMed Central  Google Scholar 

  30. Zia A, Zhang C, Xiong X, Jarc AM (2017) Temporal clustering of surgical activities in robot-assisted surgery. Int J Comput Assist Radiol Surg. https://doi.org/10.1007/s11548-017-1600-y

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors thank Sylvia Padilla for her assistance with organization of performance metric data obtained from the da Vinci Surgical System.

Funding

The authors declare that no funds, grants, or other support were received during the study or preparation of this manuscript.

Author information

Authors and Affiliations

  1. Department of Surgery, Section of Minimally Invasive Surgery, Washington University School of Medicine, 660 South Euclid Avenue, Mailstop 8109-22-9905, Campus Box 8109, St. Louis, MO, 63110-1093, USA

    Julie M. Clanahan & Michael M. Awad

  2. Data and Analytics, Intuitive Surgical, Inc., Peachtree Corners, GA, 30092, USA

    Andrew Yee

Authors
  1. Julie M. Clanahan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew Yee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael M. Awad
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Author MMA conceived and designed the study. Authors JMC and AY performed data collection, assembly, analysis, and interpretation. All authors contributed to manuscript writing and final approvals.

Corresponding author

Correspondence to Julie M. Clanahan.

Ethics declarations

Conflict of interest

Dr. Awad has educational grants from Applied Medical, Bard/BD Medical, Baxter, Ethicon, Medtronic, and Stryker for simulation training. He is also a consultant for Ethicon, Intuitive Surgical, and Medtronic. Dr. Yee had no conflicts during this study, however, has become an employee of Intuitive Surgical after its completion. Dr. Clanahan has no conflicts of interest or financial ties to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Clanahan, J.M., Yee, A. & Awad, M.M. Active control time: an objective performance metric for trainee participation in robotic surgery. J Robotic Surg 17, 2117–2123 (2023). https://doi.org/10.1007/s11701-023-01628-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11701-023-01628-5

Keywords

Search

Navigation