Extensive training is needed to acquire proficiency in image-based surgery, such as for laparoscopic surgery. Trainees can pass through a major part of the first steep segment of the learning curve for basic laparoscopic skills preclinically by training on simulators [1–3]. Virtual-reality (VR) simulators have the additional advantage that they can be utilised as a tool for objective proficiency assessment. Various VR simulators are available that provide validated tasks to train in laparoscopic tissue manipulation, laparoscope navigation, or both [3, 4]. The character of the skills required to perform laparoscopic tissue manipulation or laparoscope navigation tasks differs, while the most important difference between the tasks is the on-screen visual feedback [5]. In laparoscopic tissue manipulation the visual feedback consists of an instrument tip depicted on-screen as a moving object within a static environment, while navigational manipulation of the laparoscope changes the on-screen representation of the observed environment as a whole. This difference and its influence on hand–eye coordination are not always taken into account when laparoscopic skills are considered. However, fundamental differences in user-interface interaction for different basic laparoscopy skills are important enough to consider before actual implementation of simulators for training or proficiency assessment purposes.

In one of our previous studies, experienced laparoscopic surgeons expressed the feeling that the virtual environment surrounding the laparoscope navigation task affected their performance negatively, in particular due to the high level of abstraction and their unfamiliarity with the landscape [5]. They said to be so used to have anatomical landmarks as reference points when manipulating the laparoscope that they found it very difficult to orientate themselves without these landmarks. Likewise, Stefanidis et al. and Maithel et al. [6, 7] pointed out that the performance of expert participants in their studies appeared to be subject to a simulator-associated learning curve when performing a laparoscope navigation task in an abstract environment, needing several repetitions to become acquainted with the task on the simulator.

The aim of this study is to investigate the influence of anatomical landmarks in the visual environment on the performance of angled laparoscope navigation on a VR simulator. Is the presence of familiar anatomic landmarks beneficial for the performance of experienced laparoscopic surgeons? To answer this question, a group of experienced laparoscopic surgeons performed the Camera Navigation (CN) task on the SimSurgery SEP simulator (SimSurgery AS, Oslo, Norway) in two different virtual environments: the standard SEP abstract environment (CN-box) and a VR representation of the lower abdomen (CN-abdomen). A group of medical trainees with no laparoscopy experience also performed these tasks to assess whether the results could be related to level of experience.

Methods and materials

Fifty-four participants took part in this study. During three advanced laparoscopic skills courses and five basic surgical skills courses, trainees, faculty and staff of the institutes where the courses took place were invited to participate. In the information they received it was clearly stated that the researchers were not affiliated with the manufacturer of the simulator and that all data would be analysed anonymously. The participants filled out an informed consent form and a questionnaire about their demographics and general medical and laparoscopy experience.

Participants were allotted to one of two groups based on their experience with laparoscopic surgery as indicated in the questionnaire. Participants in the experienced group (Table 1) indicated to have performed more than 100 basic laparoscopic procedures clinically (such as cholecystectomies or appendectomies) and at least five advanced laparoscopic procedures (such as Nissen fundoplication or bariatric surgery), plus to be experienced with using a 30° angled laparoscope. In addition, performance on the Place Arrow (PA) task was used to verify the expertise level of the experienced participants. The performances on the PA task of three participants allotted to the experienced group were labelled as extreme outliers (z-score > 3.29), so these participants were excluded from this group. A novice group was formed by medical trainees with the minimum knowledge level of a general surgical intern, but with no clinical experience in performing laparoscopic procedures (Table 1).

Table 1 Demographic data of the participants
Full size table

Protocol

After filling out the questionnaire, participants received an introduction to the simulator and explanation of the tasks following a standardised procedure. Next, they performed three tasks twice on the SimSurgery SEP VR simulator (SimSurgery AS, Oslo, Norway) (Figs. 1 and 2): the Place Arrow (PA) task, the Camera Navigation task with a 30° angled laparoscope in an abstract virtual environment (CN-box) and the Camera Navigation task with a 30° angled laparoscope in a virtual representation of the lower abdomen (CN-abdomen). The order of the CN tasks was randomised. All participants started with performing the PA task twice to become acquainted with the simulator. The PA task performance also functioned as an indicator to verify the expertise level of the experienced participants. Participants were encouraged to use the first repetition of each task as a tryout run to become acquainted with the exercise. Only the second repetition of each task was used to analyse performances. The assignment given to the participants was to perform the second repetition of each task as best they could, but also as quickly as possible. After performing all tasks on the simulator, participants filled out the remaining part of the questionnaire, in which they were asked to rate the tasks on seven-point Likert scales.

Fig. 1
figure 1

The study protocol. (PA place arrow task, CN-box camera navigation task with a 30° angled laparoscope in an abstract virtual environment; CN-abdomen camera navigation task with a 30° angled laparoscope in a virtual representation of the lower abdomen)

Full size image
Fig. 2
figure 2

Screenshots of the PA task (top), CN-box task (middle) and CN-abdomen task (below). The screenshot of the CN-box task shows one of the (blue) targets correctly visualised (zoomed-in sufficiently, centred on screen, with horizon level, and the bull’s-eye visible); it turns green and should be held steady for five consecutive seconds. (Color figure online)

Full size image

The SEP software was used in combination with the SimPack surgical interface (SimSurgery AS, Oslo, Norway). The SEP software includes a variety of tasks in a VR environment to train different laparoscopy skills and provides learning objectives, instructions and a demonstration video before each task. After each task, the software provides numerical scores and a graph of performances. In addition to the scores provided by the simulator (including time to accomplish the task, total tip trajectory and various error scores), the overall average speed per instrument tip was calculated (by dividing the total tip trajectory per tip by the time to accomplish the task).

The PA task represents a basic bimanual tissue manipulation task in which an arrow-shaped object needs to be grasped at both ends and placed over another arrow-shaped target elsewhere in the abstract box-like environment. The simulator calculates the quality of the manipulations by assessing the position, spatial orientation and size of the grabbed object with respect to the position, spatial orientation and size of the target object. When the object remains still in the approved position for 5 s, the target is regarded as successfully placed, after which a new target appears elsewhere in the environment. In total, five targets are presented one after another. During the exercise, the viewpoint is fixed. The on-screen visual feedback consists of the graspers moving while being manipulated within the static abstract surroundings. In the CN task the participant has to locate a pyramid-shaped target placed somewhere in the virtual three-dimensional environment and accurately visualise the target and the bull’s-eye inside the target, which can only be seen through an opening in the pyramid’s top (Fig. 2). Again the task involves meeting several quality parameters; the target is visualised correctly when it is displayed and held steady on screen for 5 s from a proper distance, centred on the screen, horizontally oriented and with its bull’s-eye visible. The CN task requires extensive manipulation of the angled laparoscope controls to visualise the targets properly. The position of the target and the surroundings are tightly fixed to each other and static. Each manipulation of the laparoscope controls alters the on-screen representation of the target within its surrounding, resulting in a very dynamic on-screen image. The CN tasks included five targets, presented one after another. For each of the mentioned quality parameters the acceptable range can be set to alter the level of difficulty of the exercise. In this study we used the standard settings of the simulator, which represent a medium level of difficulty according to the manufacturer. To ensure that all participants experienced a similar test situation, the exercise sets were carefully selected on the location of the targets within each task repetition (specified as the ‘random value’ setting) (Fig. 1).

Data analysis

SPSS 16.0 software (SPSS Inc., Chicago, USA) was used for statistical analysis of the data. The Wilcoxon signed-rank test (two-tailed) was used to compare performances on the tasks within each group. To compare performances between the experienced and novices groups, the Mann–Whitney U test was used (two-tailed). In addition, we investigated whether the simulator performances within the experienced group were affected by the age or the years of laparoscopic experience of the surgeons. Using the Whitney U test (two-tailed), the performances in the experienced group of the youngest five surgeons (≤35 years) were compared with the performances of the five oldest surgeons (≥48 years). Also, the performances of five senior residents with 6 years or less laparoscopy experience were compared with the performances of the six senior surgeons who stated to have at least 15 years of experience in laparoscopic surgery. A p < 0.05 was considered statistically significant.

Results

Within both the experienced (N = 28) and novice (N = 23) groups, no significant differences were found between the performances of subgroups A (CN-box followed by CN-abdomen) and B (CN-abdomen followed by CN-box) on the angled laparoscope navigation task in the abstract environment (CN-box task) or in the virtual representation of the lower abdomen (CN-abdomen task). This implies that the influence of the order in which the CN tasks were performed is negligible. The performances of the subgroups were therefore combined for further analysis. None of the participants encountered tool–tool collision in the PA task, while in the CN tasks only two participants (one novice and one experienced participant) recorded one collision each with the camera to the target. Therefore, the scores on tool–tool collision, tool–tool collision time and camera target collision were excluded from the analysis.

In the experienced group, the CN-box task was accomplished in significantly less time (Fig. 3) and with shorter total tip trajectory (Fig. 4) than the CN-abdomen task. The same held for the novice group. No significant differences were found between the CN task in the two different surroundings for the number of targets lost out of view or the average speed per instrument tip (Figs. 5 and 6).

Fig. 3
figure 3

Time to accomplish the tasks. Presented p-values represent significant difference within the groups between tasks (Wilcoxon signed-rank test; only significant differences are presented)

Full size image
Fig. 4
figure 4

Total tip trajectory during the tasks. Presented p-values represent significant difference within the groups between tasks (Wilcoxon signed-rank test; only significant differences are presented)

Full size image
Fig. 5
figure 5

Number of main task errors made during the tasks

Full size image
Fig. 6
figure 6

Average speed per instrument tip during the tasks

Full size image

The scores of the experienced group did not differ significantly from the scores of the novice group on both the CN-box task and the CN-abdomen task. However, on the PA task the experienced group significantly outperformed the novice group. The experienced group accomplished the PA task in significantly less time (p < 0.001) and with shorter total tip trajectory (p < 0.001). In the experienced group the speed per instrument tip was also significantly faster (p = 0.032).

Mann–Whitney U tests did not present any significant differences in performances in the experienced group between the performances of the youngest five surgeons (≤35 years) and the five oldest surgeons (≥48 years), or between the performances of five senior residents and the six senior surgeons with 15 or more years of experience in laparoscopic surgery.

After performing all tasks on the simulator, the participants were asked to rate the educational value of the tasks and whether they experienced a difference between the perceived and anticipated level of difficulty of the tasks (Table 2). The opinion of the experienced and novice groups only differed significantly for the difference between the perceived and anticipated level of difficulty of the PA task (p = 0.011).

Table 2 Opinion of the participants about the SEP tasks, rated on a seven-point Likert scale (1 = strongly disagree, 4 = neutral, 7 = strongly agree)
Full size table

Discussion

Due to the new European working-time directives (EWTD) and economical and ethical considerations, clinical training of surgical skills is losing acceptance and preclinical training and proficiency assessment on VR simulators are becoming increasingly more common [1–3]. Handling an angled laparoscope and navigating with it within the abdominal cavity are basic laparoscopy skills for which the general value of simulator training and transfer of skills to the clinical setting has been proven [8–10]. However, more in-depth knowledge is still needed on the physical and cognitive aspects of the interaction with the instrumentarium and interpretation of visual information during image-based procedures such as laparoscopic surgery. The visual environment in which simulator tasks are to be performed could be of influence, as was confirmed in our study.

The results of our study indicate that performances on the angled laparoscope navigation task on SimSurgery SEP differed significantly between the abstract environment and the virtual abdomen environment. The performances in the abstract virtual environment surpassed the performances in the virtual environment with anatomic landmarks in both the experienced group and the novice group. These results could (partly) be clarified by the fact that the abdominal environment used in this study was still a virtual-reality representation of the abdominal cavity. The anatomy of the abdominal cavity is a very familiar environment to laparoscopic surgeons. Hence, minor deviations in the representation of the anatomy or flawed computer graphical representation could have drawn their attention immediately and distracted them from performance of the CN task. Also, even though the novice group was inexperienced in laparoscopic surgery, with the minimum knowledge level of a surgical intern they were all supposed to be familiar with the anatomy of the abdomen. Consequently, just like the experts, they may also have been distracted from their task. It could also be that, due to the limitations of computer graphics, the level of detail of the anatomy was not sufficient to create a realistic experience of perceiving the abdominal cavity as in the operating room setting to allow skill-based behaviour by the experienced surgeons. Some surgeons commented that the CN task required too precise and sometimes too close visualisation of the targets, in this way not representing the flexibility of visualisation of anatomic structures in the operating room. This relates to personal preferences of some surgeons on visualisation of the operating field or variations that are dependent on the therapeutic task to be visualised. Finally, the performance assessment on the basis of the scores provided by the simulator could have had an influence. Each task on SimSurgery SEP incorporates several quality parameters which play a key role in accomplishing the tasks successfully. Still, the output generated by the simulator at the end of each task only presents time needed to accomplish the task, total tip trajectory and specific error scores. It could be that a potential difference in navigation with an angled laparoscope is not reflected by these individual rudimentary scores, but should be identified by other, more sophisticated performance parameters (e.g. parameters related to specific aberrations from the optimal tip trajectory, or parameters that take multiple factors into account).

VR simulators could play an important role in fulfilling the desire for objective proficiency assessment and in accomplishing a shift towards criterion-based training. The development of criterion-based training programmes with also a stronger focus on the needs of the individual trainee will most likely improve the efficiency and efficacy of laparoscopy curricula and decrease the incidence of under- or over-training [2]. However, this approach necessitates better understanding of the influence and function of different types of intrinsic and extrinsic feedback when performing different types of tasks on a simulator. The focus of future research should therefore be more on the quality of the performance and the development of more sophisticated and comprehensive performance metrics.

While the CN-box task was accomplished in significantly less time and with shorter total tip trajectory, the average speed of the instrument tips did not significantly differ. It appears that the participants followed a longer route to find and properly visualise the targets in the CN-abdomen task. The simulator data does not provide details on the actual path followed with the instrument, only on the total length. Chmarra et al. [11] analysed the shape of the path of instrument tips for a simple eye–hand coordination task with a laparoscopic grasper and identified a retracting phase and a seeking phase. Instead of following the actual shortest route between two points, being a straight line, a pull-back movement is performed first before the target is approached [11]. This pull-back movement is essential to minimise the chance of touching tissue unnecessarily and ensure patient safety. Possibly, the presence of anatomic landmarks instigated a more cautious behaviour. However, this could also be related to the irregular shape of the environment and the presence of obstacles in the CN-abdomen task versus the open and clear space in the CN-box task. Then again, similar results were found in both the novices and experienced groups, which suggests that the latter is less likely to be the case.

The questionnaire data (Table 1) show that the assisting surgeon usually handles the laparoscope. In the early days of laparoscopic surgery, many surgeons started immediately with performing the therapeutic actions with an assistant or resident holding the laparoscope, while in many teaching hospitals the senior surgeon would fulfil the role of assisting surgeon in less complicated laparoscopic procedures performed by surgical residents. This implies that senior surgeons would handle the laparoscope on a regular basis, but only during procedures involving uncomplicated utilisation of the angled laparoscope. In contrast, the younger generation of laparoscopic surgeons generally started their laparoscopic career by handling the laparoscope to assist the primary operating surgeon, during both straightforward procedures and procedures requiring more sophisticated manipulation of the laparoscope. Consequently, one might expect that the younger generation has more experience with manoeuvring laparoscopes. This could be a plausible explanation for the lack of difference in performance of the CN tasks between novices and experienced laparoscopic surgeons. Therefore, we also investigated whether the simulator performances within the experienced group were affected by the age or the years of laparoscopic experience of the participants in the experienced group. The results though did not reveal any significant differences.

In the questionnaire, the experienced participants were asked to rate whether the CN-abdomen task was a more effective tool than the CN-box task to train novices or assess experienced laparoscopists in angled laparoscope navigation (Table 2). Although the opinions upon these questions were divided, the CN-abdomen task was rated to be more effective to train novices than the CN-box task. This suggests that the requirements for a simulator for training are perceived to be different from the requirements for a simulator for assessment of laparoscopic proficiency. Just as in one of our previous studies, the CN task was rated as being more challenging than anticipated beforehand, and also more so than the PA task [5]. The study was performed during laparoscopic courses that made extensive use of box trainers or living animals, often using 30° angled laparoscopes, and sometimes also VR simulators. Frequently, the participants worked in pairs, with one person handling the laparoscope while the other performed the training task. However, somehow, the participants did not recognise this as simulation training, because many of the experienced surgeons stated to have none or only brief experience with simulations for either laparoscopic tissue manipulation or laparoscope navigation (Table 1).

Conclusions

The performances of experienced laparoscopic surgeons on an angled laparoscope navigation task on the SimSurgery SEP VR simulator differed significantly between the abstract environment and the virtual environment of the abdomen with anatomic landmarks. The task was performed with better simulator scores in the abstract environment, opposite to the propositions of several experienced laparoscopic surgeons in a previous study. In the group with inexperienced medical trainees, a similar difference in performances was found. This means that the influence of the virtual environment on the task performance was not related to level of experience. Further research should focus on extending knowledge on the influence and function of different types of intrinsic and extrinsic feedback provided by the simulator on the effectiveness of preclinical training (e.g. the realism of the task and its surroundings, and the added value of haptic feedback). The criteria and parameters used to assess task performance for different types of laparoscopy skills need to be further investigated as well.