Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure
- 18 Downloads
The use of robotic surgery for minimally invasive procedures has increased considerably over the last decade. Robotic surgery has potential advantages compared to laparoscopic surgery but also requires new skills. Using virtual reality (VR) simulation to facilitate the acquisition of these new skills could potentially benefit training of robotic surgical skills and also be a crucial step in developing a robotic surgical training curriculum. The study's objective was to establish validity evidence for a simulation-based test for procedural competency for the vaginal cuff closure procedure that can be used in a future simulation-based, mastery learning training curriculum.
Eleven novice gynaecological surgeons without prior robotic experience and 11 experienced gynaecological robotic surgeons (> 30 robotic procedures) were recruited. After familiarization with the VR simulator, participants completed the module ‘Guided Vaginal Cuff Closure’ six times. Validity evidence was investigated for 18 preselected simulator metrics. The internal consistency was assessed using Cronbach’s alpha and a composite score was calculated based on metrics with significant discriminative ability between the two groups. Finally, a pass/fail standard was established using the contrasting groups’ method.
The experienced surgeons significantly outperformed the novice surgeons on 6 of the 18 metrics. The internal consistency was 0.58 (Cronbach’s alpha). The experienced surgeons’ mean composite score for all six repetitions were significantly better than the novice surgeons’ (76.1 vs. 63.0, respectively, p < 0.001). A pass/fail standard of 75/100 was established. Four novice surgeons passed this standard (false positives) and three experienced surgeons failed (false negatives).
Our study has gathered validity evidence for a simulation-based test for procedural robotic surgical competency in the vaginal cuff closure procedure and established a credible pass/fail standard for future proficiency-based training.
KeywordRobotic surgery Virtual reality simulation Gynaecology Assessment Proficiency-based training
The research group sincerely thank all doctors from the Centre for Robotic Surgery at Copenhagen University Hospital, Herlev-Gentofte and from Robotic Surgery Section at Department of Gynaecology, Copenhagen University Hospital, Rigshospitalet-Glostrup for participation.
Compliance with ethical standards
Lisette H. Hovgaard and Drs. Steven A. W. Andersen, Lars Konge, Torur Dalsgaard and Christian R. Larsen have no conflicts of interest or financial ties to disclose.
- 11.Sandadi S, Gadzinski JA, Lee S, Chi DS, Sonoda Y, Jewell EL, Brown CL, Gardner GJ, Barakat RR, Leitao MM (2014) Fellowship learning curve associated with completing a robotic assisted total laparoscopic hysterectomy. Gynecol Oncol 132(1):102–106. https://doi.org/10.1016/J.Ygyno.2013.11.017 CrossRefPubMedGoogle Scholar
- 12.Seamon LG, Fowler JM, Richardson DL, Carlson MJ, Valmadre S, Phillips GS, Cohn DE (2009) A detailed analysis of the learning curve: robotic hysterectomy and pelvic-aortic lymphadenectomy for endometrial cancer. Gynecol Oncol 114(2):162–167. https://doi.org/10.1016/J.Ygyno.2009.04.017 CrossRefPubMedGoogle Scholar
- 16.Lendvay TS, Brand TC, White L, Kowalewski T, Jonnadula S, Mercer LD, Khorsand D, Andros J, Hannaford B, Satava R (2013) Virtual reality robotic surgery warm-up improves task performance in a dry laboratory environment: a prospective randomized controlled study. J Am Coll Surg 216(6):1181–1192. https://doi.org/10.1016/J.Jamcollsurg.2013.02.012 CrossRefPubMedPubMedCentralGoogle Scholar
- 20.Kiely DJ, Gotlieb WH, Lau S, Zeng X, Samouelian V, Ramanakumar AV, Zakrzewski H, Brin S, Fraser SA, Korsieporn P, Drudi L (2015) Virtual reality robotic surgery simulation curriculum to teach robotic suturing: a randomized controlled trial. J Robot Surg 9(3):179–186. https://doi.org/10.1007/S11701-015-0513-4 CrossRefPubMedGoogle Scholar
- 23.Raza SJ, Froghi S, Chowriappa A, Ahmed K, Field E, Stegemann AP, Rehman S, Sharif M, Shi Y, Wilding GE, Kesavadas T (2014) Construct validation of the key components of fundamental skills of robotic surgery (FSRS) curriculum—a multi-institution prospective study. J Surg Educ 71(3):316–324. https://doi.org/10.1016/J.Jsurg.2013.10.006 CrossRefPubMedGoogle Scholar
- 25.Simbionix (2017) http://simbionix.com/simulators/robotix-mentor/robotix-library-of-modules/. Accessed 21 Feb 2017
- 33.American Educational Research Association APA, National Council on Measurement in Education, Joint Committee on Standards for Educational and Psychological Testing (2014) Standards for educational and psychological testing. American Educational Research Association, Washington, DCGoogle Scholar
- 35.Downing SM, Yudkowsky R (2009) Assessment in Health Professions Education. Routledge, New YorkGoogle Scholar
- 37.Madsen ME, Konge L, Norgaard LN, Tabor A, Ringsted C, Klemmensen AK, Ottesen B, Tolsgaard M (2014) Assessment of performance measures and learning curves for use of a virtual-reality ultrasound simulator in transvaginal ultrasound examination. Ultrasound Obstet Gynecol 44(6):693–699. https://doi.org/10.1002/Uog.13400 CrossRefPubMedGoogle Scholar