Development and validation of surgical training tool: cystectomy assessment and surgical evaluation (CASE) for robot-assisted radical cystectomy for men

  • Ahmed A. Hussein
  • Kevin J. Sexton
  • Paul R. May
  • Maxwell V. Meng
  • Abolfazl Hosseini
  • Daniel D. Eun
  • Siamak Daneshmand
  • Bernard H. Bochner
  • James O. Peabody
  • Ronney Abaza
  • Eila C. Skinner
  • Richard E. Hautmann
  • Khurshid A. Guru



We aimed to develop a structured scoring tool: cystectomy assessment and surgical evaluation (CASE) that objectively measures and quantifies performance during robot-assisted radical cystectomy (RARC) for men.


A multinational 10-surgeon expert panel collaborated towards development and validation of CASE. The critical steps of RARC in men were deconstructed into nine key domains, each assessed by five anchors. Content validation was done utilizing the Delphi methodology. Each anchor was assessed in terms of context, score concordance, and clarity. The content validity index (CVI) was calculated for each aspect. A CVI ≥ 0.75 represented consensus, and this statement was removed from the next round. This process was repeated until consensus was achieved for all statements. CASE was used to assess de-identified videos of RARC to determine reliability and construct validity. Linearly weighted percent agreement was used to assess inter-rater reliability (IRR). A logit model for odds ratio (OR) was used to assess construct validation.


The expert panel reached consensus on CASE after four rounds. The final eight domains of the CASE included: pelvic lymph node dissection, development of the peri-ureteral space, lateral pelvic space, anterior rectal space, control of the vascular pedicle, anterior vesical space, control of the dorsal venous complex, and apical dissection. IRR > 0.6 was achieved for all eight domains. Experts outperformed trainees across all domains.


We developed and validated a reliable structured, procedure-specific tool for objective evaluation of surgical performance during RARC. CASE may help differentiate novice from expert performances.


Robot-assisted Radical cystectomy Training Skill acquisition Quality Assessment 



This research was supported in part by funding from the National Cancer Institute of the National Institutes of Health under Award Number: R25CA181003, and Roswell Park Alliance Foundation.

Compliance with ethical standards


Ronney Abaza: Research grant from Intuitive Surgical. Ahmed A. Hussein, Kevin J. Sexton, Paul R. May, Maxwell V. Meng, Abolfazl Hosseini, Daniel D. Eun, Siamak Daneshmand, Bernard H. Bochner, James O. Peabody, Eila C. Skinner, Richard E. Hautmann, Khurshid A. Guru have no conflicts of interest or financial ties to disclose.


  1. 1.
    Raza SJ, Wilson T, Peabody JO et al (2015) Long-term oncologic outcomes following robot-assisted radical cystectomy: results from the International Robotic Cystectomy Consortium. Eur Urol 68(4):721–728CrossRefPubMedGoogle Scholar
  2. 2.
    Leow JJ, Reese SW, Jiang W et al (2014) Propensity-matched comparison of morbidity and costs of open and robot-assisted radical cystectomies: a contemporary population-based analysis in the United States. Eur Urol 66(3):569–576CrossRefPubMedGoogle Scholar
  3. 3.
    Wilson TG, Guru K, Rosen RC et al (2015) Best practices in robot-assisted radical cystectomy and urinary reconstruction: recommendations of the Pasadena Consensus Panel. Eur Urol 67(3):363–375CrossRefPubMedGoogle Scholar
  4. 4.
    Lee JY, Mucksavage P, Sundaram CP, McDougall EM (2011) Best practices for robotic surgery training and credentialing. J Urol 185(4):1191–1197CrossRefPubMedGoogle Scholar
  5. 5.
    Polit DF, Beck CT, Owen SV (2007) Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health 30(4):459–467CrossRefPubMedGoogle Scholar
  6. 6.
    Hussein AA, Hinata N, Dibaj S et al (2017) Development, validation and clinical application of Pelvic Lymphadenectomy Assessment and Completion Evaluation: intraoperative assessment of lymph node dissection after robot-assisted radical cystectomy for bladder cancer. BJU Int 119(6):879–884CrossRefPubMedGoogle Scholar
  7. 7.
    Schiffmann J, Gandaglia G, Larcher A et al (2014) Contemporary 90-day mortality rates after radical cystectomy in the elderly. Eur J Surg Oncol J Eur Soc Surg Oncol Br Assoc Surg Oncol 40(12):1738–1745Google Scholar
  8. 8.
    Eisenberg MS, Boorjian SA, Cheville JC et al (2013) The SPARC score: a multifactorial outcome prediction model for patients undergoing radical cystectomy for bladder cancer. J Urol 190(6):2005–2010CrossRefPubMedGoogle Scholar
  9. 9.
    Hussein AA, Dibaj S, Hinata N et al. Development and validation of a quality assurance score for robot-assisted radical cystectomy: a 10-year analysis. Urology 2016Google Scholar
  10. 10.
    Herr HW, Faulkner JR, Grossman HB et al (2004) Surgical factors influence bladder cancer outcomes: a cooperative group report. J Clin Oncol 22(14):2781–2789CrossRefPubMedGoogle Scholar
  11. 11.
    Cooperberg MR, Odisho AY, Carroll PR (2012) Outcomes for radical prostatectomy: is it the singer, the song, or both? J Clin Oncol 30(5):476–478CrossRefPubMedGoogle Scholar
  12. 12.
    Rashid HH, Leung YY, Rashid MJ, Oleyourryk G, Valvo JR, Eichel L (2006) Robotic surgical education: a systematic approach to training urology residents to perform robotic-assisted laparoscopic radical prostatectomy. Urology 68(1):75–79CrossRefPubMedGoogle Scholar
  13. 13.
    Lerner MA, Ayalew M, Peine WJ, Sundaram CP (2010) Does training on a virtual reality robotic simulator improve performance on the da Vinci surgical system? J Endourol 24(3):467–472CrossRefPubMedGoogle Scholar
  14. 14.
    Abboudi H, Khan MS, Guru KA et al (2014) Learning curves for urological procedures: a systematic review. BJU Int 114(4):617–629CrossRefPubMedGoogle Scholar
  15. 15.
    Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ (2012) Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol 187(1):247–252CrossRefPubMedGoogle Scholar
  16. 16.
    Hussein AA, Ghani KR, Peabody J et al (2017) Development and validation of an objective scoring tool for robot-assisted radical prostatectomy: prostatectomy assessment and competency evaluation. J Urol 197(5):1237–1244CrossRefPubMedGoogle Scholar
  17. 17.
    Apramian T, Cristancho S, Sener A, Lingard L (2017) How do thresholds of principle and preference influence surgeon assessments of learner performance? Ann SurgGoogle Scholar
  18. 18.
    Frederick PJ, Szender JB, Hussein AA et al (2017) Surgical competency for robot-assisted hysterectomy: development and validation of a robotic hysterectomy assessment score (RHAS). J Minim Invasive Gynecol 24(1):55–61CrossRefPubMedGoogle Scholar
  19. 19.
    Diamond IR, Grant RC, Feldman BM et al (2014) Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 67(4):401–409CrossRefPubMedGoogle Scholar
  20. 20.
    Birkmeyer JD, Finks JF, O’Reilly A et al (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369(15):1434–1442CrossRefPubMedGoogle Scholar
  21. 21.
    Stolzenburg JU, Rabenalt R, Do M, Horn LC, Liatsikos EN (2006) Modular training for residents with no prior experience with open pelvic surgery in endoscopic extraperitoneal radical prostatectomy. Eur Urol 49(3):491–498; discussion 499–500Google Scholar
  22. 22.
    Volpe A, Ahmed K, Dasgupta P et al (2015) Pilot validation study of the European Association of Urology Robotic Training Curriculum. Eur Urol 68(2):292–299CrossRefPubMedGoogle Scholar
  23. 23.
    Vassiliou MC, Feldman LS, Andrew CG et al (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190(1):107–113CrossRefPubMedGoogle Scholar
  24. 24.
    Ahmed N, Devitt KS, Keshet I et al (2014) A systematic review of the effects of resident duty hour restrictions in surgery: impact on resident wellness, training, and patient outcomes. Ann Surg 259(6):1041–1053CrossRefPubMedPubMedCentralGoogle Scholar
  25. 25.
    McAlister C (2015) Breaking the silence of the switch-increasing transparency about trainee participation in surgery. N Engl J Med 372(26):2477–2479CrossRefPubMedGoogle Scholar
  26. 26.
    Tiferes J, Hussein AA, Bisantz A et al (2016) The loud surgeon behind the console: understanding team activities during robot-assisted surgery. J Surg Educ 73(3):504–512CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Ahmed A. Hussein
    • 1
    • 2
  • Kevin J. Sexton
    • 1
  • Paul R. May
    • 1
  • Maxwell V. Meng
    • 3
  • Abolfazl Hosseini
    • 4
  • Daniel D. Eun
    • 5
  • Siamak Daneshmand
    • 6
  • Bernard H. Bochner
    • 7
  • James O. Peabody
    • 8
  • Ronney Abaza
    • 9
  • Eila C. Skinner
    • 10
  • Richard E. Hautmann
    • 11
  • Khurshid A. Guru
    • 1
  1. 1.Department of UrologyRoswell Park Cancer InstituteBuffaloUSA
  2. 2.Cairo UniversityCairoEgypt
  3. 3.University of CaliforniaSan FranciscoUSA
  4. 4.Karolinska University HospitalStockholmSweden
  5. 5.Temple University HospitalPhiladelphiaUSA
  6. 6.University of Southern CaliforniaLos AngelesUSA
  7. 7.Memorial Sloan Kettering Cancer CenterNew YorkUSA
  8. 8.Henry Ford Health SystemsDetroitUSA
  9. 9.OhioHealthDublinUSA
  10. 10.Stanford UniversityPalo AltoUSA
  11. 11.University of UlmUlmGermany

Personalised recommendations