1 Introduction

The implementation of robot-assisted surgery (RAS) has surged over the past two decades (Bric et al. 2014; Diana and Marescaux 2015; Zelhart and Kaiser 2018; Park et al. 2015). Articulating instruments cause less pivot and fulcrum effects. Besides, RAS offers improved tremor filtering, instrument coordination, visuals and surgeon ergonomics (Willuth et al. 2021; Schmidt et al. 2021; Kowalewski et al. 2021). Therefore, compared to laparoscopic surgery, it can improve surgical performance during complex procedures (Baik et al. 2009; Mirnezami et al. 2010; Stefanidis et al. 2010; Widmar et al. 2017). However, tele-robotic surgery (often referred to as RAS) implies that there is no direct physical contact between the surgeon and the patient. The lack of haptic feedback results in a less realistic perception of tissue properties (Diana and Marescaux 2015; Sung and Gill 2001; Rizun et al. 2006; Alleblas et al. 2017). Consequently, robotic surgeons in the early phase of the learning curve have to rely on visual and spatial cues, such as depth perception, the degree of tissue deformation and tension of anatomical structures, instead (Sung and Gill 2001; Meijden and Schijven 2009; Simorov et al. 2012; Tan et al. 2009; Bahler and Sundaram 2014; Hagen et al. 2008). The conversion rate (16–20%) during the robotic learning curve while having limited experience with conventional minimally invasive surgery, such as laparoscopy, is high (Parisi et al. 2017; Reitz et al. 2018). This stresses the need for simulation training initiatives to improve.

Currently, RAS training is often carried out using the Da Vinci’s SimNow virtual reality (VR) simulator. This system is relatively expensive, bulky and not easily transported. This implies that surgeons have to train at the hospital during Operation Room (OR) downtime, in the evening or at night (Abboudi et al. 2013). Moreover, compared to hands-on box training, VR systems for laparoscopic surgery are known for their unrealistic simulation. Although some VR simulators now provide haptic feedback, the lower quality of the unidirectional force feedback compared to hands-on box trainers using real instruments potentially feels unnatural and may potentially hampers the learning curve (Strom et al. 2006; Tanaka et al. 2016 Sep; Postema et al. 2021 Aug; Horeman et al. 2014; Hiemstra et al. 2011; Overtoom et al. 2019; Amirabdollahian et al. 2018; Miller et al. 2021). As surgeons mainly rely on their spatial visualization abilities associated with a relatively long learning curve (Jourdes et al. 2022), technical innovators have been working on the integration of haptic and force feedback mechanisms into VR trainers with accumulating evidence (Overtoom et al. 2019; Kaul et al. 2006 Mar; Othman et al. 2022). These sensors could improve simulation for novice surgeons and enhance the learning curves and showed promising results (Meijden and Schijven 2009 Jun; Hiemstra et al. 2011; Othman et al. 2022; Rangarajan et al. 2020; Prasad et al. 2016). Moreover, with regards to VR training, the use of this prompt feedback has even proven to be the most effective way to train (Hagelsteen et al. 2019; Alaker et al. 2016). However, the haptic feedback sensors should not cause additional friction forces that often result in a less realistic training setup (Våpenstad et al. 2013a, 2013b). Prasad et al., concluded that visual force and motion feedback holds potential to increase the learning curve for residents when utilizing a VR-based haptic simulator (Prasad et al. 2016). In another study, by Hagelsteen et al., trainees were able to acquire skills faster, performed tasks more efficient and reached the proficiency level in fewer attempts (Hagelsteen et al. 2017). Although benefits of force feedback and haptic feedback enhanced VR training have been showed, no firm consensus exist on the implementation in training of surgical skills for Minimally Invasive Surgery (Meijden and Schijven 2009).

Recently, a new Portable Laparoscopy Robot Simulator (PoLaRS-VR) with integrated haptic and visual feedback was validated at the Delft University of Technology (Hardon et al. 2021). This portable system was designed to simulate different types of instruments and thus to be compatible with different robotic surgical platforms for laparoscopic surgery. Due to its integrated haptic and visual feedback, this system allows for the investigation of instrument–instrument and instrument-environment collisions and its influence on surgical performance and the RAS learning curve.

The aim of this study was to determine the influence of visual and haptic collision feedback on the acquisition and retention of fundamental robotic skills on a novel VR robotic simulator for advanced laparoscopic surgery. It was hypothesized that the integration of haptic and visual feedback during training results in fewer instrument collisions and increased instruments movements due to avoidance of undesired collisions.

2 Methods

2.1 Participants

Robotically- and laparoscopically-naïve trainees participated in this study. Important exclusion criteria were: visual or haptic disability and prior experience in either robotic simulator training or actual experience in robotic surgery. Participation was voluntary, and all subjects signed an informed consent form before the start of the experiment. The study was conducted in accordance with the World Medical Association (WMA) Declaration of Helsinki.

2.2 Study design

The study was designed as a randomized controlled trial, in which participants were randomized in a 1:1 ratio into a group either receiving haptic and visual Feedback (HVF) when collision occurs between instruments and environment or a group receiving no Feedback (NF) when collision occurs. Each group consisted of 11 participants. Randomization was performed by using the randomization function in Microsoft Excel software. After randomization, a demographics questionnaire was distributed to collect baseline characteristics such as gender, gaming experience, and previous experience with simulation training.

2.3 Systems hardware and software

This research was conducted using a custom build PoLaRS VR (Fig. 1). The hardware of the system consists of a Portable Robot Master with 14 degrees of freedom (DOF), seven in each arm. In each arm, three DOF's, measured by three potentiometers, describe the global motion of the handle. Another three DOF's, Roll, Pitch and Yaw, describe the local orientation of the tip at the distal side of the instrument. The final DOF describes closing and opening of the tip measured by a small hall sensor in the grip and a magnet in the leaver that is activated by the index finger. The output of each potentiometer is connected to an analog input of a single Labjack U3 HV. This device has 16 available analog input channels, of which we use 14, that use an ADC with 12 bit accuracy. The source voltage for the potentiometers is provided through a DAC output of the Labjack U3 HV. We created a low pass filter in the circuit to stabilize the dac voltage even further with a single cap and a resistor. Our measurements show that 1, 7 s is needed to charge the capacitor and the simulation can start. The general latency depends on the laptop capacity, frame size and resolution but remains acceptable on a 1280 × 1024 resolution with a delay of less than 0.05 s.

Fig. 1
figure 1

Portable PoLaRS VR simulator master system

Full size image

2.4 Training tasks

The simulator offers a variety of 3D tasks, all of which represent important skills for basic robotic surgery skills training (Hardon et al. 2021). Before the execution of the pre-test, participants received general instructions on the procedures and hardware, what was to be done during the pre-test, training phase, and post-tests, and what outcome measures were measured. They were instructed to handle the PoLaRS system with caution. To familiarize themselves with the equipment, all participants first executed an ”instrument handling test” without feedback before proceeding to the pre-test; (Task 1). Subjects had to retrieve the marbles from the grey bowl, and collect them into the corresponding bowls without making contact with one of the bowls (Fig. 2). This task (Fig. 3A) was the same for the pre-test and post-test (Fig. 4). During the tasks, no additional instructions were given, however the participants were allowed to ask questions. To keep the effect of learning at a minimum and comparable for all subjects, a measurement was ended after two minutes, or sooner when the task was completed.

Fig. 2
figure 2

Visual collision feedback; the local red discoloration shows when the grasper and bowl make contact

Full size image
Fig. 3
figure 3

A Task 1: Subjects had to retrieve the marbles from the grey bowl, and collect them into the corresponding bowls. They had to avoid making contact with one of the bowls. B Task 2: Subjects had to collect the marbles into the corresponding bowls. To ensure that both graspers were being used, they were instructed to pick up the marbles from the left bowls with the left grasper, and those in the right bowls with the right grasper. C Task 3: Subjects had to pick up the marble from the left bowl with the left grasper and put it through the opening of the screen into the blue tray. When the marble was dropped into the blue tray, it rolled down to the right side. Here, the subjects had to retrieve the marble with the right grasper through the opening and collect it into the green bowl. Besides the bowls, the subjects also had to avoid making contact with the object in the middle

Full size image
Fig. 4
figure 4

Study set up flowchart. One group received haptic and visual feedback about instrument and environment interaction during training (HVF). The other group received no feedback during training (NF). Finally both groups conducted post-test 1 and post-test 2

Full size image

After the familiarization trial, the pre-test (Fig. 3A) was conducted to measure the baseline performances in both groups. No haptic or visual feedback was provided. This baseline assessment test was followed by the execution of Task 2 (Figs. 3B and 4), where the HVF group received haptic and visual feedback from instrument and environment interaction. Before execution, the HVF group received additional instructions about the vibration in the grasper and the red signal that would appear on the screen (Fig. 2) in case of a collision. During this task, subjects had to collect the marbles into the corresponding bowls. To ensure that both graspers were being used, they were instructed to pick up the marbles from the left bowls with the left grasper, and those in the right bowls with the right grasper. The NF group did not receive any form of feedback during the performance of this task. Based on our previous reported results, the training phase was limited to 6 trials by each participant (Postema et al. 2021; Hardon et al. 2021). After completing the training phase, both groups performed the post-training assessment on two tasks: Task 1 and Task 3 (Fig. 3B and C. During Task 3, subjects had to pick up the marble from the left bowl with the left grasper and put it through the opening of the screen into the blue tray. When the marble was dropped into the blue tray, it rolled down to the right side. Here, the subjects had to retrieve the marble with the right grasper through the opening and collect it into the green bowl. Besides the bowls, the subjects also had to avoid making contact with the object in the middle. Task 1 was considered relatively easy and was similar to the pre-test to determine the effect of the training. Task 3 considered to be more difficult, and was performed to evaluate the influence of HVF during training in a more complex and demanding procedure. No form of feedback was provided during the post-training assessments. The time limit was set to 5 min.

2.5 Data mapping

The Labjack is connected to a laptop running the compiled version of the simulator developed with Unity 3D. The LJUDDotNet library for.NET provided by the Labjack manufacturer is used in the c# backend of the Unity 3D simulator. A separate thread loads the data from the labjack in a single roundtrip, by using a package that requests 14 analog channels and sets 2 digital channels for the feedback motors. The standard documentation provided with the labjack gives typical execution times for such a package of 12.2 ms. The maximum and minimum values and positions of the potentiometers are known and calibrated before starting the tests. During the physics cycle of the simulation (once every 20 ms), the data are retrieved from the labjack thread and converted to angles using a linear scale. The position and orientation of the handle are then calculated as a moving average of the last 4 data retrievals. This moving average is used to correct for fluctuations in the signal. A window of 4 frames was a good middle point between visible lag and small movement noise. The calculated position variables are sampled every 50 ms and saved in a file after completion of the exercise. The orientation and position of the master handle is calculated relative to the base using the dimensions of the arms and the angles provided by the potentiometers. The position of the handle relative to the base is translated as a position of the instrument in the simulator relative to a fixed point. The orientation of the handles is mirrored in the orientation of the instrument tips.

2.6 Virtual environment

Using the Unity game engine a square training box with a dimension of 16 cm is simulated (Figs. 2 and 3). The instrument positions and orientations are solely determined by input from the Portable Robot Master. Most other objects in the environment are rigid objects that are also not influenced by the physics engine. The only object that use the physics engine for collision solving are the spheres with a diameter of 0.7 cm. The spheres collide with each other, the rigid objects in the training area and the instruments. The area is visualized with perspective view and a viewing angle of 60 degrees. The perspective in combination with 2 point light sources that cast shadows allow participants to see depth on the monitor. Flat bowls of approximately 6 cm are used as containers for the spheres. Spheres reset to a starting position upon falling out of the training area or when making contact with the ground. Upon all the colored spheres being in contact with their respective colored bowl, the training is finished. The Left and Right instruments bars are constrained by the two origins that represent the insertion Points that allow the instrument to enter the virtual training box and the two global coordinates of the handle. When the instrument takes hold of a sphere, it becomes part of the instrument and physics for the sphere are turned off. Intersections of the instruments with rigid objects or the training box are counted as instrument collisions but not solved using the physics engine. These recorded collisions are programmed with an area of effect, time constraint and cannot overlap; thus, greater movement or remaining for a longer time in a collided position can accumulate collisions. The full orientation and position of both instruments, sphere resets and collisions are saved 20 times per second. These data are used to determine instrument path length and participant performance.

2.7 Collision feedback and data acquisition

In the HVF group, haptic and visual feedback was provided when one of the instruments collided with the environment that should have been avoided, for example the bowl or one of the obstacles in between. The haptic feedback was provided in the form of vibrations in the graspers’ handles. This vibration stopped when the contact between the grasper and the entity that it should have avoided was lost. The visual feedback presented itself as a red light appearing at the site of the grasper and environment contact (Fig. 2). Again, this collision indicator switched off when contact between the grasper and the environment was lost.

The number of collisions were measured for both graspers. Other measurements were the total time taken to complete the task, both graspers’ path length and the tip to tip grasper distance. The tip to tip grasper distance metric is the mean distance between the two instrument tips and represents the efficiency of instrument trajectories based on 3D coordinates. These motion analysis parameters have been shown to discriminate between levels of expertise (Uemura et al. 2014; Empel et al. 2013). During a trial, multiple marbles had to be collected into the corresponding bowl. The time it took to collect all marbles was set as the total time needed for one trial. The distance covered by the instruments’ tips, during this time, was set as the instruments’ path lengths. The distance between these tips was set as the tip to tip grasper distance. The participants in the NF group did not receive any feedback about instrument and environment interactions. When participants dropped the marble too late, they could accidentally push the marble through the bowl, which resulted in a reset of the marble to its starting position. Directly after a trial was finished, the duration, path length’s, and the number of collisions of both graspers were displayed on the screen.

2.8 Statistics

To evaluate improvement in skills, pre- and post-test measurement outcomes were analyzed and compared. To identify the effect of training with or without feedback, mixed ANOVA tests for the training trials, and between the pre-test and post-test 1 data, were executed. Differences in performance between the feedback group (HVF) and no feedback group (NF) during the first post-test (Task 1) and second post-test (Task 3) were analyzed using the nonparametric Mann–Whitney U test for independent samples, as the Shapiro–Wilk test (and visual inspection) showed that this data was not normally distributed. Analysis was done using SPSS Statistics for Apple Mac version 27 (IBM SPSS Inc., Chicago, Illinois, USA). The level of statistical significance was set at p ≤ 0.05.

3 Results

Twenty-two participants (18 males, all right-handed, mean age: 25.5 (± 1.9 SD)), who were equally divided over the two groups, performed a total of 198 trials. None of the participants had previous experience in laparoscopic or robotic surgery. Seven participants had substantial gaming experience, but there was no significant difference between groups. In total, 22 trials were not completed within the 5 min timeframe and were excluded from analysis (Supplemental files).

3.1 Collisions outcomes

Significant differences in the number of collisions of the right grasper (dominant hand) were observed between the HVF group and the NF group in trial 1 (HVF mean rank = 3.75 versus NF mean rank = 9.00, U = 5.00, p = 0.034), trial 2 (HVF Mean Rank = 5.43 versus NF Mean Rank = 10.89, U = 10.00, p = 0.023), trial 3 (HVF Mean Rank = 6.78 versus NF Mean Rank = 13.55, U = 16.00, p = 0.011), and trial 5 (HVF Mean Rank = 7.50 versus NF Mean Rank = 13.50, U = 20.00, p = 0.023) (Figs. 5 and 6). Moreover, significantly less collisions occurred with the right grasper in the HVF group (Mean Rank = 8.73) compared to the NF group (Mean Rank = 14.27) in the second post-test assessment (Task 3) (U = 30.00, p = 0.045).

Fig. 5
figure 5

A Left grasper collision outcomes. B Right grasper collision outcomes (numbers on graph points represent data outliers)

Full size image
Fig. 6
figure 6

Tip-to-tip grasper distances outcomes. Boxplot of the Tip-to-tip distance, collisions left/right and path length left/right for the pre-test, training phase and post-tests. A significant difference is indicated with the asterisk

Full size image

Moreover, although not observed in the first trial of the training phase, significant differences in the left grasper (non-dominant hand) were observed in trial 2 (HVF Mean Rank = 5.43 versus NF Mean Rank = 10.89, U = 10.00, p = 0.023), trial 3 (HVF M = 133.22, SD = 120.84 versus NF M = 401.73, SD = 308.83, t(18) =  − 2.450, p = 0.025), trial 4 (HVF Mean Rank = 7.30 versus NF Mean Rank = 13.70, U = 18.00, p = 0.016), trial 5 (HVF M = 68.40, SD = 67.22 versus NF M = 348.20, SD = 245.93, t(18) = -3.471, p = 0.003), and trial 6 (HVF M = 72.27, SD = 52.98 versus NF M = 255.50, SD = 244.33, t(19) =  − 2.431, p = 0.025). However, there were no differences in left grasper collisions during the post-test assessments.

3.2 Instrument handling parameters

During the training phase, a significant difference in total time to complete the trial was observed in trial 1 (HVF group Mean Rank = 14.32 versus NF group Mean Rank = 8.68, U = 29.50, p = 0.037), and in trial 3 (HVF group M = 231.58, SD = 48.34 s versus NF group M = 188.09, SD = 35.81 s, t(20) = 2.397, p = 0.026). No differences were found in the other trials during training phase, nor during the pre- and post-test assessment phases (Fig. 7).

Fig. 7
figure 7

Total time to complete tasks outcomes. Boxplots of total time explained, for pre-test, training phase and post-tests. The actual tasks performed during the different phases are indicated by the colored squares (dashed lines are used in the figures to separate the pre-test, training phase, and post-tests)

Full size image

The ANOVA test showed a significant difference in the distances traveled by the instrument tips (Path length parameter) between the pre-test and the post-test (Task 1) (p = 0.012) (Fig. 8). There was no difference in the distances traveled by instrument tips (Path Length parameter) between both groups during the training phase.

Fig. 8
figure 8

A Path length left grasper outcomes. B Path length right grasper outcomes

Full size image

4 Discussion

This study shows the impact of haptic and visual feedback on the occurrence of undesired collisions during robotic-assisted VR simulation training. The hypothesis was that training with haptic and visual feedback would result in increased quality performance (less instrument–instrument and instrument-environment collisions), but with decreased efficiency of motion (more time to complete the task and a larger instrument path length). The results of this study confirm only part of the hypothesis. During the training phase, a clear learning effect was apparent when collision feedback (HVF) was present. There was no significant difference between the groups for post-test assessment 1 (Task 1). However, a difference was found in collisions made with the dominant hand in favor of HVF in post-test assessment 2 (Task 3).

4.1 Effects of haptic and visual collision feedback on training outcomes

Figure 7 shows that the total time needed for the participants to finish the task in the feedback group during the training phase is higher compared to the group that received no collision feedback. This can be considered as evidence that feedback during training influences technical performance and the strategy of execution, because participants slow down as they become aware of their errors during the performance. The control group participants were not aware of doing something wrong, causing them to continue with the exercise. This is in line with other studies showing that robotic surgery has a distinct learning curve that needs to be addressed and is de-pendent on the learning of visual haptics (Bahler and Sundaram 2014; Overtoom et al. 2019). Figure 5 shows that the longer trial time during the training phase was accompanied by a much lower number of collisions. This implies that provided with feedback, participants execute with less speed, but increased awareness of events: They learn how to prevent undesired collisions, with all the associated risks.

4.2 Skills transfer to known task

Apart from a reduction in one of the collision parameters, no evidence of a residual effect of feedback on the time and motion parameters was found during and after training in either group. This indicates that participants are able to deal with additional forms of feedback that help to avoid collisions with the environment during training, without having it influencing performance. Although previous research showed similar results for training with visual feedback in a box trainer or suture trainer with conventional instruments (Horeman et al. 2014; Smit et al. 2017), this study indicates that adding more complexity due to more degrees of feedback does not necessarily reduce overall task performance.

Only one main effect was found for the path length parameter for the dominant right hand between the pre-test and post-test 1 for both groups indicating that increased efficiency of instrument handling measured after training was not related to the presence of feedback during the training. From the literature it is known that a decrease in the dominant hand activity leads to an increase in the use of the non-dominant hand (Rodrigues et al. 2014; Horeman et al. 2012). Although indications of a similar effect can be found in the data of the left path length, no significant difference was present in this sample. As the time to complete the task for the pre-test did not significantly differ from post-test 1 it can be concluded that the learned skills did not reduce the task time. In contrast to previous research reporting outcomes of a standard box trainer with conventional instruments, this study did not report a natural reduction of task time between the pre- and post-test after training. This could be explained by the possibility that the participants mainly focused on avoiding collisions without speeding up performance, which was in line with the received instruction (Smit et al. 2017).

During this research it was noted that some participants were skillful at executing the different tasks, and others were less. This is in line with previous research, showing very variable innate technical skill sets at the baseline, and heterogeneity in duration of training to reach proficiency goals between individuals (Schmidt et al. 2020; Romero et al. 2018; Nickel et al. 2016). Inspection of data indicated that this was not related to gender, age or gaming experience. This is in line with other studies that found conflicting results regarding influence of gaming experience and gender with regard to minimally invasive surgical skill acquisition (Kowalewski et al. 2018; Nickel et al. 2015; Javier et al. 2019). It could be that using a more homogeneous population, for example medical students who may have more affinity with medical equipment, could result in less difference in this respect.

4.3 Limitations

Although a normal distribution was expected based on the nature of the movements and tasks, the presence of outliers and high dispersion in each group resulted in a skewed distribution. This may have negatively influenced the comparison of collision parameters between HVF and NF groups, resulting in less significant outcomes. Moreover, due to the 5 min time limit, one trial was excluded from the pre-test analyses and 21 trials were excluded from the training phase analyses. No trials were excluded from the post-test analyses. Potentially, the exclusion of these data had a negative effect on the dispersion too. Nevertheless, still in 9 out of 12 trials a significant difference was present. The training with haptic and visual interaction feedback resulted in a 68% and 85% lower collision average for the HVF group compared with the NF group in post-test 1 and 2 for the dominant hand. This result was much lower compared to the 32% and 11% decrease measured for the non-dominant hand. A post-hoc power analysis (sampsizepwr.m, MathWorks, Natick, MA) conducted on the means and standard deviations indicated that for the Left and Right hand in post-test 1; respectively, 179 and 11 participants are needed for significant differences(Amirabdollahian et al. 2018). For post-test 2 this is 1127 and 3 participants. This indicates that significant differences can be expected for the dominant right hand path length when a slightly larger group size is used. Arguably, a more homogenous sample would also lead to lower dispersion. Considering generalizability and transferability of results to clinical practice, it is advised to only perform the study with such a cohort if these participants are all experienced surgeons with high levels of skill.

4.4 Future perspectives

During the study, it was expected that the non-dominant hand becomes more active during training resulting in a higher path length; while, the dominant hand becomes less active, resulting in a decrease in path length. Although the latter was not observed, it remains interesting to investigate how residents in surgery can be motivated to use their passive hand more efficient in minimally invasive surgery to increase bimanual dexterity. Therefore, it could be interesting to conduct a randomized controlled crossover study that allows students to first use their non-dominant hand during training followed by a bimanual post-test, before switching to a second training session conducted only with the dominant hand followed by a bimanual post-test.

5 Conclusions

This study shows the impact of haptic and visual feedback during VR training for robotic-assisted surgery. This feedback resulted in less undesired collisions, compared to training without feedback. These results underline the importance of haptic feedback in VR simulation training for robotic surgery. The PoLaRS system can be utilized to remotely optimize technical skills before commencing robotic surgery in the operating room.