Abstract
Increasing focus on patient safety makes it important to ensure surgical competency among surgeons before operating on patients. The objective was to gather validity evidence for a virtual-reality simulator test for robotic surgical skills and evaluate its potential as a training tool. Surgeons with varying experience in robotic surgery were recruited: novices (zero procedures), intermediates (1–50), experienced (> 50). Five experienced surgeons rated five exercises on the da Vinci Skills Simulator. Participants were tested using the five exercises. Participants were invited back 3 times and completed a total of 10 attempts per exercise. The outcome was the average simulator performance score for the 5 exercises. 32 participants from 5 surgical specialties were included. 38 participants completed all 4 sessions. A moderate correlation between the average total score and robotic experience was identified for the first attempt (Spearman r = 0.58; p = 0.0004). A difference in average total score was observed between novices and intermediates [median score 61% (IQR 52–66) vs. 83% (IQR 75–91), adjusted p < 0.0001], as well as novices and experienced [median score 61% (IQR 52–66) vs. 80 (IQR 69–85), adjusted p = 0.002]. All three groups improved their performance between the 1st and 10th attempts (p < 0.00). This study describes validity evidence for a virtual-reality simulator for basic robotic surgical skills, which can be used for assessment of basic competency and as a training tool. However, more validity evidence is needed before it can be used for certification or high-stakes assessment.
Similar content being viewed by others
References
Aggarwal R, Mytton OT, Derbrew M et al (2010) Training and simulation for patient safety. Qual Saf Health Care 19 Suppl 2:i34–i43. https://doi.org/10.1136/qshc.2009.038562
Carreyrou J (2010) Botched operation using Da Vinci robot spurs lawsuit. Wall Street J. https://www.wsj.com/articles/SB10001424052748703341904575266952674277806. Accessed 30 Apr 2018
Schreuder HWR, Wolswijk R, Zweemer RP et al (2012) Training and learning robotic surgery, time for a more structured approach: a systematic review. BJOG 119:137–149. https://doi.org/10.1111/j.1471-0528.2011.03139.x
Larsen CR, Oestergaard J, Ottesen BS, Sorensen JL (2012) The efficacy of virtual reality simulation training in laparoscopy: a systematic review of randomized trials. Acta Obstet Gynecol Scand 91:1015–1028. https://doi.org/10.1111/j.1600-0412.2012.01482.x
Ahlberg G, Heikkinen T, Iselius L et al (2002) Does training in a virtual reality simulator improve surgical performance? Surg Endosc 16:126–129. https://doi.org/10.1007/s00464-001-9025-6
Zendejas B, Cook DA, Hernández-Irizarry R et al (2012) Mastery learning simulation-based curriculum for laparoscopic TEP inguinal hernia repair. J Surg Educ 69:208–214. https://doi.org/10.1016/j.jsurg.2011.08.008
Bric JD, Lumbard DC, Frelich MJ, Gould JC (2016) Current state of virtual reality simulation in robotic surgery training: a review. Surg Endosc 30:2169–2178. https://doi.org/10.1007/s00464-015-4517-y
Aydin A, Shafi AMA, Shamim Khan M et al (2016) Current status of simulation and training models in urological surgery: a systematic review. J Urol 196:312–320. https://doi.org/10.1016/j.juro.2016.01.131
Moglia A, Ferrari V, Morelli L et al (2016) A systematic review of virtual reality simulators for robot-assisted surgery. Eur Urol 69:1065–1080. https://doi.org/10.1016/j.eururo.2015.09.021
Downing SM, Yudkowsky R (2009) Assessment in health professions education. Routledge, New York
Thinggaard E, Bjerrum F, Strandbygaard J et al (2016) Ensuring competency of novice laparoscopic surgeons-exploring standard setting methods and their consequences. J Surg Educ 73:986–991. https://doi.org/10.1016/j.jsurg.2016.05.008
Smith R, Patel V, Satava R (2014) Fundamentals of robotic surgery: a course of basic robotic surgery skills based upon a 14-society consensus template of outcomes measures and curriculum development. Int J Med Robot 10:379–384. https://doi.org/10.1002/rcs.1559
Foote JR, Valea FA (2016) Robotic surgical training: where are we? Gynecol Oncol 143:179–183. https://doi.org/10.1016/j.ygyno.2016.05.031
Downing SM (2003) Validity: on meaningful interpretation of assessment data. Med Educ 37:830–837
Zijlstra F (1993) Efficiency in work behaviour: a design approach for modern tools. Delft University Press
Bharathan R, Vali S, Setchell T et al (2013) Psychomotor skills and cognitive load training on a virtual reality laparoscopic simulator for tubal surgery is effective. Eur J Obstet Gynecol Reprod Biol 169:347–352. https://doi.org/10.1016/j.ejogrb.2013.03.017
Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate—a practical and powerful approach to multiple testing. J R Stat Soc Ser B Methodol 57:289–300
Yang K, Zhen H, Hubert N et al (2017) From dV-trainer to real robotic console: the limitations of robotic skill training. J Surg Educ 74:1074–1080. https://doi.org/10.1016/j.jsurg.2017.03.006
Cook DA (2015) Much ado about differences: why expert-novice comparisons add little to the validity argument. Adv Health Sci Educ Theory Pract 20:829–834. https://doi.org/10.1007/s10459-014-9551-3
Julian D, Tanaka A, Mattingly P et al (2018) A comparative analysis and guide to virtual reality robotic surgical simulators. Int J Med Robot 14:e1874. https://doi.org/10.1002/rcs.1874
Ahmed K, Khan R, Mottrie A et al (2015) Development of a standardised training curriculum for robotic surgery: a consensus statement from an international multidisciplinary group of experts. BJU Int 116:93–101. https://doi.org/10.1111/bju.12974
Acknowledgements
The authors would like to thank all the surgeons from the eastern part of Denmark who volunteered and participated in this study.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Drs. Havemann, Dalsgaard, Sorensen, Røssaak, Brisling, Moesgaard, Høgdall and Bjerrum declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Havemann, M.C., Dalsgaard, T., Sørensen, J.L. et al. Examining validity evidence for a simulation-based assessment tool for basic robotic surgical skills. J Robotic Surg 13, 99–106 (2019). https://doi.org/10.1007/s11701-018-0811-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11701-018-0811-8