Skip to main content
SpringerLink
Log in

External validation of Global Evaluative Assessment of Robotic Skills (GEARS)

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

We demonstrate the construct validity, reliability, and utility of Global Evaluative Assessment of Robotic Skills (GEARS), a clinical assessment tool designed to measure robotic technical skills, in an independent cohort using an in vivo animal training model.

Methods

Using a cross-sectional observational study design, 47 voluntary participants were categorized as experts (>30 robotic cases completed as primary surgeon) or trainees. The trainee group was further divided into intermediates (≥5 but ≤30 cases) or novices (<5 cases). All participants completed a standardized in vivo robotic task in a porcine model. Task performance was evaluated by two expert robotic surgeons and self-assessed by the participants using the GEARS assessment tool. Kruskal–Wallis test was used to compare the GEARS performance scores to determine construct validity; Spearman’s rank correlation measured interobserver reliability; and Cronbach’s alpha was used to assess internal consistency.

Results

Performance evaluations were completed on nine experts and 38 trainees (14 intermediate, 24 novice). Experts demonstrated superior performance compared to intermediates and novices overall and in all individual domains (p < 0.0001). In comparing intermediates and novices, the overall performance difference trended toward significance (p = 0.0505), while the individual domains of efficiency and autonomy were significantly different between groups (p = 0.0280 and 0.0425, respectively). Interobserver reliability between expert ratings was confirmed with a strong correlation observed (r = 0.857, 95 % CI [0.691, 0.941]). Experts and participant scoring showed less agreement (r = 0.435, 95 % CI [0.121, 0.689] and r = 0.422, 95 % CI [0.081, 0.0672]). Internal consistency was excellent for experts and participants (α = 0.96, 0.98, 0.93).

Conclusions

In an independent cohort, GEARS was able to differentiate between different robotic skill levels, demonstrating excellent construct validity. As a standardized assessment tool, GEARS maintained consistency and reliability for an in vivo robotic surgical task and may be applied for skills evaluation in a broad range of robotic procedures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Lowrance WT, Eastham JA, Savage C, Maschino AC, Laudon VP, Dechet CB, Stephenson RA, Scardino PT, Sandhu JS (2012) Contemporary open and robotic radical prostatectomy practice patterns among urologists in the United States. J Urol 187(6):2087–2093

    Article  PubMed Central  PubMed  Google Scholar 

  2. Scott DJ (2006) Patient safety, competency, and the future of surgical simulation. Simul Healthc 1(3):164–170

    Article  PubMed  Google Scholar 

  3. Sweet RM, Beach R, Sainfort F, Gupta P, Reihsen T, Poniatowski LH, McDougall EM (2012) Introduction and validation of the American Urological Association basic laparoscopic urologic surgery skills curriculum. J Endourol 26(2):190–196. doi:10.1089/end.2011.0414

    Article  PubMed  Google Scholar 

  4. Seixas-Mikelus SA, Stegemann AP, Kesavadas T, Srimathveeravalli G, Sathyaseelan G, Chandrasekhar R, Wilding GE, Peabody JO, Guru KA (2011) Content validation of a novel robotic surgical simulator. BJU Int 107(7):1130–1135. doi:10.1111/j.1464-410X.2010.09694.x

    Article  PubMed  Google Scholar 

  5. van der Meijden OA, Broeders IA, Schijven MP (2010) The SEP “robot”: a valid virtual reality robotic simulator for the Da Vinci surgical system? Surg Technol Int 19:51–58

    PubMed  Google Scholar 

  6. Jonsson MN, Mahmood M, Askerud T, Hellborg H, Ramel S, Wiklund NP, Kjellman M, Ahlberg G (2011) ProMIS can serve as a da Vinci(R) simulator—a construct validity study. J Endourol 25(2):345–350. doi:10.1089/end.2010.0220

    Article  PubMed  Google Scholar 

  7. Kenney PA, Wszolek MF, Gould JJ, Libertino JA, Moinzadeh A (2009) Face, content, and construct validity of dV-trainer, a novel virtual reality simulator for robotic surgery. Urology 73(6):1288–1292. doi:10.1016/j.urology.2008.12.044

    Article  PubMed  Google Scholar 

  8. Hung AJ, Patil MB, Zehnder P, Cai J, Ng CK, Aron M, Gill IS, Desai MM (2012) Concurrent and predictive validation of a novel robotic surgery simulator: a prospective, randomized study. J Urol 187(2):630–637. doi:10.1016/j.juro.2011.09.154

    Article  PubMed  Google Scholar 

  9. Ramos P, Montez J, Tripp A, Ng CK, Gill IS, Hung AJ (2014) Face, content, construct and concurrent validity of dry laboratory exercises for robotic training using a global assessment tool. BJU Int 113(5):836–842. doi:10.1111/bju.12559

    Article  PubMed  Google Scholar 

  10. Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ (2012) Global Evaluative Assessment of Robotic Skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol 187(1):247–252. doi:10.1016/j.juro.2011.09.032

    Article  PubMed  Google Scholar 

  11. Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondre K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190(1):107–113. doi:10.1016/j.amjsurg.2005.04.004

    Article  PubMed  Google Scholar 

  12. Goh A, Goldfarb DW, Sander J, Miles B, Dunkin B (2012) Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol 187:247–252

    Article  PubMed  Google Scholar 

  13. Hung AJ, Jayaratna IS, Teruya K, Desai MM, Gill IS, Goh AC (2013) Comparative assessment of three standardized robotic surgery training methods. BJU Int 112(6):864–871. doi:10.1111/bju.12045

    Article  PubMed  Google Scholar 

  14. Patel VR, Tully AS, Holmes R (2005) Robotic radical prostatectomy in the community setting—the learning curve and beyond: initial 200 cases. J Urol 174(1):269–272

    Article  PubMed  Google Scholar 

  15. Nordin P, van der Linden W (2008) Volume of procedures and risk of recurrence after repair of groin hernia: national register study. BMJ 336(7650):934–937. doi:10.1136/bmj.39525.514572.25

    Article  PubMed Central  PubMed  Google Scholar 

Download references

Acknowledgment

Authors received Fund Source from Institutional, Ethicon, and Intuitive.

Conflict of interest

Drs. Aghazadeh, Jayaratna, Hung, Desai, Gill, Goh, and Mr. Pan have no conflicts of interest or financial ties to disclose.

Author information

Authors and Affiliations

  1. Department of Urology, Methodist Institute for Technology, Innovation, and Education, Houston Methodist Hospital, 6560 Fannin Street, Suite 2100, Houston, TX, 77030, USA

    Monty A. Aghazadeh & Alvin C. Goh

  2. Scott Department of Urology, Baylor College of Medicine, Houston, TX, USA

    Monty A. Aghazadeh & Michael M. Pan

  3. USC Institute of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA

    Isuru S. Jayaratna, Andrew J. Hung, Mihir M. Desai & Inderbir S. Gill

Authors
  1. Monty A. Aghazadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Isuru S. Jayaratna
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrew J. Hung
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael M. Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mihir M. Desai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Inderbir S. Gill
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Alvin C. Goh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alvin C. Goh.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aghazadeh, M.A., Jayaratna, I.S., Hung, A.J. et al. External validation of Global Evaluative Assessment of Robotic Skills (GEARS). Surg Endosc 29, 3261–3266 (2015). https://doi.org/10.1007/s00464-015-4070-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-015-4070-8

Keywords

Search

Navigation