Procedural virtual reality simulation training for robotic surgery: a randomised controlled trial

Background Virtual reality (VR) training is widely used for surgical training, supported by comprehensive, high-quality validation. Technological advances have enabled the development of procedural-based VR training. This study assesses the effectiveness of procedural VR compared to basic skills VR in minimally invasive surgery. Methods 26 novice participants were randomised to either procedural VR (n = 13) or basic VR simulation (n = 13). Both cohorts completed a structured training programme. Simulator metric data were used to plot learning curves. All participants then performed parts of a robotic radical prostatectomy (RARP) on a fresh frozen cadaver. Performances were compared against a cohort of 9 control participants without any training experience. Performances were video recorded and assessed blindly using GEARS post hoc. Results Learning curve analysis demonstrated improvements in technical skill for both training modalities although procedural training was associated with greater training effects. Any VR training resulted in significantly higher GEARS scores than no training (GEARS score 11.3 ± 0.58 vs. 8.8 ± 2.9, p = 0.002). Procedural VR training was found to be more effective than both basic VR training and no training (GEARS 11.9 ± 2.9 vs. 10.7 ± 2.8 vs. 8.8 ± 1.4, respectively, p = 0.03). Conclusions This trial has shown that a structured programme of procedural VR simulation is effective for robotic training with technical skills successfully transferred to a clinical task in cadavers. Further work to evaluate the role of procedural-based VR for more advanced surgical skills training is required. Electronic supplementary material The online version of this article (10.1007/s00464-020-08197-w) contains supplementary material, which is available to authorized users.

Keywords Robotic surgery · Surgical education · Simulation · Virtual reality Training simulators trace their origins to the mechanical Link trainer developed in 1929 for pilots to practice flying by instruments. It was not until 1993 that Richard Satava et al. [1] developed the first medical VR simulator. Since then, advances in computer hardware and software have continued to drive the development of ever more realistic and complex VR surgical simulators. Currently, the majority of VR trainers, especially for laparoscopic and robotic surgery, offer basic surgical skills training for core motor skills. In robotic surgery they include endowrist manipulation, clutching, three-dimensional vision, dexterity, tissue handling, instrument control and camera control. VR modules mostly use abstract exercises such as placing hoops on pegs or manipulating objects for specific skills.
Whilst VR simulation is acknowledge as being effective in teaching basic surgical skills [2], bridging the gap between such isolated skills training and undertaking full surgical procedures in the operating room (OR) requires further extensive training.
Advances in software and hardware have led to the development of increasingly realistic VR environments. Procedural-based simulation aims to extend training beyond abstract tasks and recreate complete or part of a surgical procedure. This allows training of both basic and advanced 1 3 skills such as managing bleeding. Even for basic surgical VR, modelling elements such as shadows, the effects of collision, and topological changes due to tearing, grasping or cutting is challenging. With procedural VR, the complexity is greatly increased with the need to accommodate surgical factors such as the effects of instruments or sutures on tissues, physiological responses such as bleeding and accurate anatomical modelling [3]. The simulator should also be able to provide useful and objective assessments of performance for training and assessment. Procedural VR training has been shown to be effective in training laparoscopic cholecystectomy and salpingectomy [4,5]. A number of robotic VR simulators offer procedural or part procedural training, however, to date assessment of the effectiveness of this training has been limited. The RobotiX Mentor (3D Systems, Airport City, Israel) offers seven different procedural training modules covering gynaecological, thoracic, urological and general surgery.
This study aims to compare the effectiveness of structured procedural VR training against basic VR training and no training for robotic surgery.

Methods
A multi-institutional, randomised controlled trial was conducted in the Vattikuti Institute for Robotic Surgery, King's College London. Data were collected between the March and November 2016.
The RobotiX Mentor robot-assisted radical prostatectomy (RARP) module was selected for this study. Alongside video-based didactic training, hands-on VR training can be undertaken either with or without step-by-step procedural guidance. Guided training modules were used throughout the study. The module consists of four training tasks; bladder neck dissection (BND), neurovascular bundle dissection, apical dissection and urethrovesical anastomosis (UVA) (Fig. 1). Given that the neurovascular bundle and apical dissection tasks require specialist anatomical knowledge for successful completion, it was decided by the authors to exclude these modules. The primary outcome measure was the operative technical performance, assessed on fresh frozen cadaveric models within a simulated operating room. The secondary outcome measure was the training effect evaluated through learning curve analysis.

Subjects
Novice participants (without any experience in robotic sur-

Training programme
Initially, all novice participants underwent generic robotic skills training (Fig. 2). This involved completing three Fundamentals of Robotic Surgery (FRS) tasks during a 1-h training session (Ring Tower Transfer, Railroad Track, Vessel Energy Dissection). These tasks were selected to provide the exposure to core robotic skills including endowrist manipulation, camera navigation, dissection and diathermy use. No data were collected during this familiarisation training.
Following familiarisation, all novice participants were randomised using a block randomisation protocol (http:// www.rando mizat ion.com). Participants were randomised into two groups; procedural VR training or basic VR training (for CONSORT flow diagram see Supplementary Figure 1). For either cohort a programme of structured training was developed based on the simulator training modules. Training followed a competency-based approach whereby participants were encouraged to complete tasks in sequential manner.
The procedural VR cohort underwent training over the course of 5 weeks consisting of 1-h sessions using the guided BND and UVA tasks. During these sessions, ad hoc training and guidance was provided by TA, a study author and expert robotic surgeon. Similarly, proctoring was also provided to basic VR cohort.
The basic VR training group underwent a parallel training programme using the FRS curriculum and a continuous A further group of novice participants were recruited by open invitation as described above and acted as controls. They did not undergo any robotic skills training prior to the cadaveric performance assessment task.

Learning curve analysis
Training effects were compared directly through learning curve analysis. Two tasks were selected from each training curriculum. For basic VR training, the Rail Road suturing and Ring Tower Transfer tasks were selected which were compared to BND and UVA (procedural VR). Tasks were carefully selected by authors on the basis of skill that each assessed to ensure they were comparable. Analysis was limited to common metrics to all four modules. Learning curve assessment was limited to the first five attempts. To allow comparison between the different exercises, Z scores were calculated.

Cadaveric performance assessment
Following training, both training groups and the control group underwent an assessment of skills transfer using human fresh frozen cadavers. Cadavers were set up within an "Igloo" disseminated operating room to provide a realistic surgical environment [6]. To complement the procedural training, RARP was used as the assessment task and the cadaver was placed in the Trendelenburg position. The aim was to evaluate the transfer of generic robotic skills, developed through the skills training programme, to the OR. All participants in either the intervention or control cohorts were allocated a 15-min assessment slot. A Da Vinci Si robotic system (Intuitive Surgical, CA) was used. Participants were given a short introduction on using the robot. For each assessment, the participants were guided through steps of a RARP by two study investigators, NR and TA. Complex steps such as the urethrovesical anastomosis were avoided to enable fair analysis of all participants. All participant performances were recorded using the robotic laparoscopic camera. Video recordings for one participant in the basic training cohort were corrupted and their results were therefore excluded from further analysis. Technical performance was evaluated post hoc using the GEARS assessment tool with each performance scored out of a maximum of 30. The assessor was blinded to the participants' identity or allocation status.

Statistical analysis
Performance metric data were retrieved from the simulator and Z scores calculated. Learning curves were plotted and analysed visually using GraphPad (Prism version 8.4.1, GraphPad Software, La Jolla California USA). Parametric analyses of the cadaveric assessment performance data were undertaken. Statistical analyses were performed using SPSS (IBM SPSS Statistics for Macintosh, Version 25.0, Armonk, NY: IBM Corp).

Results
26 novice participants were recruited to the study. None had experience in robotic surgery (either live or simulated). Following informed consent all participants underwent baseline training and were then randomised (Fig. 2).
Over the course of five weeks participants underwent training according to their randomisation status. In total, participants completed a 3.0 ± 0.9 h of simulation training (procedural VR group: 2.7 ± 1.2 h; basic VR group: 3.2 ± 0.4 h; p = 0.1).
Metric data from the simulation training were analysed. Learning curves demonstrated differences between basic and procedural training groups. Overall, there was a noticeable improvement in scores with both simulation techniques Fig. 2 Evaluation of procedural virtual reality simulation training for robotic surgery trial protocol particularly during the first three attempts. Training effects were seen most markedly in clutch usage, instrument collisions and the number of movements of left and right instruments. Basic and procedural training both demonstrated improvements particularly in these metrics. Greatest improvements were seen in the procedural (BND and UVA) tasks especially for total time, number of movements for right instrument and path length for right instrument (Fig. 3, see Supplementary Figure 2 for full results).

Skills transfer assessment on fresh frozen cadavers
Performance data from 25 study participants who completed the skill transfer assessment were compared to 9 control participants. Any VR training (procedural or basic training) resulted in a significantly higher GEARS score than no training (mean GEARS score 11.3 ± 0.58 vs. 8.8 ± 2.9 p = 0.002) (Fig. 4). Procedural training was found to be more effective than either basic or no training; mean GEARS 11.9 ± 2.9 vs. 10.7 ± 2.8 vs. 8.8 ± 1.4, respectively, p = 0.03 (Fig. 4).

Discussion
This randomised controlled trial firstly provides objective evidence that structured VR training (either basic or procedural) is effective in improving robotic surgical skills compared to no training. Importantly, procedural VR training resulted in better skill acquisition and training outcomes than basic VR simulation or no training.
The effectiveness of simulation training is increasingly well established across numerous surgical disciplines. VR simulation for laparoscopic surgery is supported by landmark studies showing that training can translate into improved operative performances [7,8]. Particularly for a training tool, it may be argued that the consequences or extrapolation inferences of training are one of the most important facets of validity evidence [9]. Direct effects on performance within the OR, either live or simulated, are key for supporting the implementation of simulation tools. Yet, these results do assume further downstream consequences on healthcare outcomes such as patient-reported outcomes. Few studies offer evidence for these outcomes [10].
For robotic surgery, basic VR simulators have undergone extensive validation but important evidence for the consequences of training remains limited [11]. Culligan et al. [12] demonstrated that completing a training programme using the dVSS simulator led to successful completion of a supravesical hysterectomy equivalent to experienced robotic surgeons. Hung et al. [13] also assessed skills transfer effect of basic skills VR training using an ex vivo animal model but found no significance difference compared to a nontraining group. The evidence from our study adds to the body of evidence supporting the role of basic VR simulation.
In comparison, evidence for procedural VR simulators in robotic surgery is very limited. Of the six VR simulators commercially available, three offer procedural-based training (dV-Trainer, Mimic Technologies, USA; RobotiX Mentor, 3D Systems, USA, USA; RoSS, Simulated Surgical Systems, USA; SEP Robot, SimSurgery, Norway). Initial validation of the RobotiX Mentor has been reported supporting its content [14]. This current study builds on this initial experience. Using a randomised design, our data show beneficial effects of training both on performance on the VR simulator as assessed through learning curves and the transfer of learning effects have been shown. The cadaveric task was chosen to both provide an accurate assessment of operative robotic skill in line with the training from the RobotiX Mentor. Importantly, greater benefit was seen with proceduralbased training. Whilst this is the first direct comparison of procedural and basic VR training, data from prior studies do suggest greater benefit with procedural-based training [15]. The reasons for this are not clear but the results of this study have important implications for surgical training. Anecdotal feedback from participants was that the procedural training was more enjoyable than the abstract basic skills tasks. Such more clinically relevant training may help to motivate participants and focus their training.
Limitations to this study need to be considered. The results need to be reviewed in the context of the trial design and the limited numbers of participants. Closer correlations between educational interventions and outcomes are found with more experienced participants as well as with specific clinical encounters rather than general impressions [10]. This study may therefore underestimate the beneficial effects of procedural VR. Further analysis of the outcomes of training in a larger cohort of trainees is also required to be able to assess the value of procedural over basic VR training. It is also important to recognise that validity evidence is specific to the context in which assessments were undertaken [16]. Outcomes from this study result from a period of structured training, however, the assessment should not be considered summative. Assessment time was set to 15 min to balance adequate assessment with resource limitations. It has been shown that 5-min recordings are adequate for GEARS assessment by both expert and amateur raters [17]. Further research is required to evaluate the role of procedural VR for high stakes assessments. Likewise, this study used RARP performed on human cadavers in a simulated operating room environment as a surrogate for live surgery. Disseminated surgical environments provide a realistic the reliability of procedural training needs to be established through additional studies. Development of objective competency criteria will also be required to support the integration of procedural VR into robotic curricula.
This study aimed to evaluate the role of procedural VR training for robotic surgery. Procedural VR training has been found to be more effective than both no training and basic VR simulation. These results offer important considerations for the development of robotic surgical training programmes. In addition, such training offers the possibility of developing more advanced surgical competencies than the basic motor skills that are the focus of current VR programmes. Including non-technical skill such as decision making and judgement will become an increasingly important aspect of surgical curriculum development.
Funding This funding was supported by a grant from the Vattikuti Foundation.

Compliance with ethical standards
Disclosures Drs. N. Raison, P. Harrison, T. Abe, A. Aydin, K. Ahmed, P. Dasgupta have no conflicts of interest or financial ties to disclose. The study was supported by 3D Systems, Inc. with the loan of the RobotiX Mentor Simulator only. 3D Systems, Inc. provided no other support and had no input in the study or manuscript. The manuscript was not shown to 3D Systems, Inc. staff prior to submission. Dr. Nicholas Raison acknowledges research support by the Vattikuti Foundation.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/.