Skip to main content

Robotic Skills Assessment: Crowd-Sourced Evaluation in Surgery and Future Directions in Plastic Surgery

  • Chapter
  • First Online:
Robotics in Plastic and Reconstructive Surgery
  • 415 Accesses

Abstract

Surgical technique is paramount in achieving desirable surgical outcomes. This statement holds true across all surgical disciplines, including plastic, urologic, and gynecologic surgery. Currently, surgical skills acquisition during training in several subspecialties (including plastic surgery and urology) is graded in accordance with the ACGME’s (Accredited Council of Graduate Medical Education) milestone projects. This method of assessment is predicated upon the subjective evaluation of individual observers. Thus, significant potential for observer bias exists in these systems. Furthermore, at most hospitals in the United States, the credentialing process is based upon peer recommendations or case currency. These methods result in a system of assessment that lacks in efficiency, cost-effectiveness, and standardization. The creation of structed tools for the assessment of surgical skill, such as Objective Structured Assessment of Technical Skills (OSATS), provided a framework to produce validated assessments of technical skill. OSATS provided a set of parameters by which many types of surgery could be analyzed on their technical merits. However, while a validated system of assessment, grading a surgeon using systems such as OSATS is often a time-consuming and resource-intensive process – still performed by individual expert surgeons with all the associated costs. As such, these methods have been adopted only within certain subfields where they can be deployed in an efficient manner. Currently, within plastic and reconstructive surgery, few of these methods have been adopted. This chapter presents the current use of a crowd-sourced platform for surgical skills assessment – the Crowd-Sourced Assessment of Technical Skill (CSATS). The platform has successfully been applied to several procedures in both robotic and endoscopic urologic and general surgery. This novel platform of surgical assessment provides a cost-effective, timely, and validated method of surgical skills assessment. While the platform has not been widely adopted within plastic surgery, the authors point to possible opportunities for an analogous system used by plastic surgeons.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 16.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Birkmeyer JD, et al. Surgical skill and complication rates after bariatric surgery. N Engl J Med. 2013;369(15):1434–42.

    Article  CAS  Google Scholar 

  2. Donaldson MS. An overview of to err is human: re-emphasizing the message of patient safety. In: Hughes RG, editor. Patient safety and quality: an evidence-based handbook for nurses. Rockville: Agency for Healthcare Research and Quality; 2008.

    Google Scholar 

  3. Weiser TG, et al. An estimation of the global volume of surgery: a modelling strategy based on available data. Lancet. 2008;372(9633):139–44.

    Article  Google Scholar 

  4. American College of Surgeons. Health Policy Research Institute; Association of American Medical Colleges. The surgery workforce in the United States: profile and recent trends. Washington, DC: Association of American Medical Colleges; 2010.

    Google Scholar 

  5. Nasca TJ, et al. The next GME accreditation system--rationale and benefits. N Engl J Med. 2012;366(11):1051–6.

    Article  CAS  Google Scholar 

  6. Sosa JA, et al. The importance of surgeon experience for clinical and economic outcomes from thyroidectomy. Ann Surg. 1998;228(3):320–30.

    Article  CAS  Google Scholar 

  7. Schmidt CM, et al. Effect of hospital volume, surgeon experience, and surgeon volume on patient outcomes after pancreaticoduodenectomy: a single-institution experience. Arch Surg. 2010;145(7):634–40.

    Article  Google Scholar 

  8. Mendivil A, Holloway RW, Boggess JF. Emergence of robotic assisted surgery in gynecologic oncology: American perspective. Gynecol Oncol. 2009;114(2 Suppl):S24–31.

    Article  Google Scholar 

  9. Zorn KC, et al. Training, credentialing, proctoring and medicolegal risks of robotic urological surgery: recommendations of the society of urologic robotic surgeons. J Urol. 2009;182(3):1126–32.

    Article  Google Scholar 

  10. Matsuda T, et al. The endoscopic surgical skill qualification system in urological laparoscopy: a novel system in Japan. J Urol. 2006;176(5):2168–72; discussion 2172.

    Article  Google Scholar 

  11. Matsuda T, et al. Reliability of laparoscopic skills assessment on video: 8-year results of the endoscopic surgical skill qualification system in Japan. J Endourol. 2014;28(11):1374–8.

    Article  Google Scholar 

  12. Martin JA, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84(2):273–8.

    CAS  PubMed  Google Scholar 

  13. Hogg ME, et al. Grading of surgeon technical performance predicts postoperative pancreatic fistula for pancreaticoduodenectomy independent of patient-related variables. Ann Surg. 2016;264(3):482–91.

    Article  Google Scholar 

  14. Goh AC, et al. Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol. 2012;187(1):247–52.

    Article  Google Scholar 

  15. Vassiliou MC, et al. A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg. 2005;190(1):107–13.

    Article  Google Scholar 

  16. Ezra DG, et al. Skills acquisition and assessment after a microsurgical skills course for ophthalmology residents. Ophthalmology. 2009;116(2):257–62.

    Article  Google Scholar 

  17. Kurashima Y, et al. A tool for training and evaluation of laparoscopic inguinal hernia repair: the Global Operative Assessment of Laparoscopic Skills-Groin Hernia (GOALS-GH). Am J Surg. 2011;201(1):54–61.

    Article  Google Scholar 

  18. Korndorffer JR Jr, et al. Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg. 2005;201(1):23–9.

    Article  Google Scholar 

  19. Ghani KR, et al. Measuring to improve: peer and crowd-sourced assessments of technical skill with robot-assisted radical prostatectomy. Eur Urol. 2016;69(4):547–50.

    Article  Google Scholar 

  20. Chen C, et al. Crowd-sourced assessment of technical skills: a novel method to evaluate surgical performance. J Surg Res. 2014;187(1):65–71.

    Article  Google Scholar 

  21. White LW, et al. Crowd-sourced assessment of technical skill: a valid method for discriminating basic robotic surgery skills. J Endourol. 2015;29(11):1295–301.

    Article  Google Scholar 

  22. Holst D, et al. Crowd-sourced assessment of technical skills: differentiating animate surgical skill through the wisdom of crowds. J Endourol. 2015;29(10):1183–8.

    Article  Google Scholar 

  23. Lendvay TS, Ghani KR, Peabody JO, Linsell S, Miller DC, Comstock B. Is crowdsourcing surgical skill assessment reliable? An analysis of robotic prostatectomies. J Urol. 2017;197(4):E890–1.

    Google Scholar 

  24. White LW, Lendvay TS, Holst D, Borbely Y, Bekele A, Wright A. Using crowd-assessment to support surgical training in the developing world. J Am Coll Surg. 2014;219(4):e40.

    Article  Google Scholar 

  25. Sweet RM, et al. Introduction and validation of the American Urological Association Basic Laparoscopic Urologic Surgery skills curriculum. J Endourol. 2012;26(2):190–6.

    Article  Google Scholar 

  26. Kowalewski TM, et al. Validation of the AUA BLUS tasks. J Urol. 2016;195(4 Pt 1):998–1005.

    Article  Google Scholar 

  27. Kowalewski TM, et al. Crowd-sourced assessment of technical skills for validation of basic laparoscopic urologic skills tasks. J Urol. 2016;195(6):1859–65.

    Article  Google Scholar 

  28. Ghani KR, Comstock B, Miller DC, Dunn RL, Kim T, Linsell S, Lane BR, Sarle R, Lendvay T, Montie J, Peabody JO. Technical skill assessment of surgeons performing robot-assisted radical prostatectomy: relationship between crowdsourced review and patient outcomes. J Urol. 2017;197(4):e609.

    Google Scholar 

  29. Powers MK, et al. Crowdsourcing assessment of surgeon dissection of renal artery and vein during robotic partial nephrectomy: a novel approach for quantitative assessment of surgical performance. J Endourol. 2016;30(4):447–52.

    Article  Google Scholar 

  30. Deal SB, et al. Evaluation of crowd-sourced assessment of the critical view of safety in laparoscopic cholecystectomy. Surg Endosc. 2017;31(12):5094–100.

    Article  Google Scholar 

  31. Alrasheed T, et al. Robotic microsurgery: validating an assessment tool and plotting the learning curve. Plast Reconstr Surg. 2014;134(4):794–803.

    Article  CAS  Google Scholar 

Download references

Disclosure

Authors Lendvay and Smartt were prior shareholders in CSATS Inc.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to James Smartt .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Lendvay, T., Smartt, J. (2021). Robotic Skills Assessment: Crowd-Sourced Evaluation in Surgery and Future Directions in Plastic Surgery. In: Selber, J.C. (eds) Robotics in Plastic and Reconstructive Surgery. Springer, Cham. https://doi.org/10.1007/978-3-030-74244-7_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-74244-7_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-74243-0

  • Online ISBN: 978-3-030-74244-7

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics