Introduction

Over the last 15 years, laparoscopy has been inexorably integrated into urologic practice. However, during this transition period, urologic training programs are only now beginning to provide systematic and fundamental minimally invasive surgery (MIS) training to residents. Additionally, there remains a significant proportion of practicing urologists who have limited laparoscopic experience, but who wish to add this to their therapeutic armamentarium. The road to laparoscopic expertise involves a steep and prolonged learning curve, which may discourage physicians from undertaking this challenge [1, 2]. With the advent of modern technology, surgical robotics have come of age and now represent a viable solution to technically complex MIS [3–5]. The benefits of robotic assistance over conventional laparoscopy including increased degrees of freedom, elimination of tremor, 3D visualization, and motion scaling have proved so advantageous as to introduce the possibility of MIS without laparoscopic training [6, 7].

In a similar vein, the impact of advanced surgical technologies on surgeons of varying MIS skills has not been evaluated previously. Just as laparoscopy has been integrated into training programs, as surgical robotics becomes more common, the question of how these advanced platforms influence surgical education for future trainees needs to be addressed [9]. With the proliferation of robotic technology, should trainees expect to learn robotic skills rather than conventional laparoscopy?

In the personal computer world, we have come to expect technological leaps at every quarter, as advances in miniaturization and engineering efficiency continue apace. Computational speed does not equal power, however, as architecture and design have been found to be significant and limiting factors, and more evidence-based metrics were developed to suit these purposes. Analogous to this phenomenon, surgical robotics is a burgeoning field with limitless potential for the future. Standard, validated, objective evaluations have not yet been widely adopted to evaluate their performance, and this remains a limiting factor in their comparisons [9]. As well, task-specific robotics further complicates comparisons, as one robot may not be suited to all tasks.

In an effort to evaluate the surgical performance of different advanced surgical platforms in standardized tasks we compared standard laparoscopy, Zeus (ComputerMotion, Sunnyvale, CA) surgical robot and daVinci (Intuitive Surgical, Sunnyvale, CA) surgical performance across varying operator skill levels.

Methods

Between April 2004 and June 2005, 12 participants were recruited to the study: four staff surgeons with laparoscopic expertise (laparoscopic aneurysm repair, pyeloplasty, cholecystectomy), four senior level postgraduate trainees (PGY) with moderate levels of laparoscopic training, and four laparoscopic naïve medical student interns (MSI). Thirty-minute acclimation sessions on each platform were allowed prior each study trial for all study groups. Each participant was required to complete four trials each of two standardized drylab modules: suturing and knot tying with 5-0 prolene sutures (Fig. 1) on each of the three test platforms (Lap, Zeus, daVinci). The suture task consisted of driving the needle through dots spaced 3 mm apart on a latex sheet, then running the suture six passes through consecutive dot series. The knot-tying task consisted of an identical setup; however, instead of running the suture, the participants were required to throw three half hitches to knot the prolene suture. Introduction to the tasks were allowed, as were informal practice task runups to the trial. Tasks were monitored by a preceptor who recorded task times and task errors. Maximum time permitted for individual tasks was 15 min. Following completion of the task on each platform, participants filled out surveys designed to subjectively score system performance in eight domains: visualization, fluidity, efficacy, precision, dexterity, tremor, tactile feedback and coordination. This survey was developed in-house through review of the current literature as well as interviews with experts in the field. CSTAR is currently working towards validating this tool for future minimally invasive clinical trials.

Fig. 1
figure 1

Drylab module setup for suturing skills. Participants were required to pass a 5-0 prolene suture immediately into and out of the adjacent marked dots spaced 3 mm apart on a latex sheet, and run six passes through consecutive dot series. Setup is identical for knot-tying tasks; however, participants are required to drive one pass of a 5-0 prolene suture through adjacent marked dots, followed by three throws of a half-hitch knot

Full size image

A standard “black box” laparoscopic simulator was used for these experiments consisting of a plexiglass enclosure with three ports on the angled upper surface for one laparoscope and two working ports. For conventional laparoscopy, two Ethicon self-locking needle drivers were used for all tasks. Zeus and daVinci trials were conducted in a similar fashion, placing the robotic laparoscope and working elements through their respective ports on the simulator. A hollow aluminum block with one face cut off was used to tent the latex sheet for the suturing surface. An IC chip was recruited for the template to generate the target rows of ink dots on the latex sheet. Twelve-centimeter lengths of 5-0 prolene suture were used for all tasks in this study. Endowristed Zeus and daVinci robotic working elements were used throughout.

All data were reported as mean ± SD (range). Continuous variables were analyzed using single-sided unpaired Student's t-tests, and categorical variables from Likert scales were analyzed using Kruskal–Wallis nonparametric tests with P < 0.05 considered statistically significant.

Results

All participants completed all tasks for the study. Overall platform performance times and errors are illustrated in Fig. 2a, b. Zeus robot-assisted tasks demonstrated significantly inferior performance compared to conventional laparoscopy and daVinci platforms in both task times (suturing: 485.7 s vs 302.0 vs 156.7 s; knot tying: 456 s vs 183.7 s vs 93.5 s, respectively) and task errors (suturing: 12 vs 5 vs 2; knot tying: 12 vs 4 vs 1, respectively; P < 0.05).

Fig. 2
figure 2

a Overall platform performance times for suturing and knot-tying modules. Zeus assisted task times were significantly longer compared to conventional laparoscopy and DaVinci platforms for both suturing and knot-tying tasks. A trend towards significance was demonstrated in task performance times between DaVinci and laparoscopy (*P < 0.05, **P < 0.05). b Overall platform performance errors for suturing and knot-tying modules. Zeus-assisted task errors were significantly more numerous compared to conventional laparoscopy and DaVinci platforms for both suturing and knot-tying tasks. A trend towards significance was demonstrated in task performance errors between DaVinci and laparoscopy (*P < 0.05, **P < 0.05)

Full size image

When stratified by skill level, suturing task performance paralleled overall platform performance in that Zeus robotic-assisted procedures were lengthy and introduced more task errors than either the laparoscopic or daVinci platforms. These findings were most significant at the MSI skill level, and incremental loss of significance between platforms was observed with increasing skill level (Fig. 3a, b).

Fig. 3
figure 3

a Suturing task performance times stratified by skill level of participant. The Zeus robotic platform demonstrated significantly inferior times compared with laparoscopic and DaVinci platforms across skill levels. However, with increasing surgeon skill, there is an incremental loss of significant performance time benefit seen with laparoscopic and DaVinci platforms (*P < 0.05, **P < 0.05). b Suturing task performance errors stratified by skill level of participant. The Zeus robotic platform demonstrated significantly higher error rates compared with laparoscopic and DaVinci platforms across skill levels. In a similar fashion as to performance times, surgical experience appears to blunt the significance of platform dependence (*P < 0.05, **P < 0.05)

Full size image

In a similar analysis, knot-tying task performance stratified by skill level again demonstrated the inferiority of the Zeus platform compared to conventional laparoscopy and daVinci. With knot-tying task times, significant differences were observed across skill levels; however, knot-tying errors demonstrated significance with MSI and PGY trainees only (Fig. 4a, b).

Fig. 4
figure 4

a Knot-tying task times stratified by skill level of participant. DaVinci robotic-assisted knot-tying times were significantly shorter compared to laparoscopic and Zeus-assisted platforms (*P < 0.05, **P < 0.05). b Knot-tying errors stratified by skill level of participant. A significantly reduced error rate was found between Zeus and the remaining two platforms. This trend was maintained across MSI and PGY skill levels, but lost when staff surgeons performed this task

Full size image

Overall subjective assessment by platform demonstrated a significant benefit to the daVinci platform versus both Zeus and conventional laparoscopy. Analysis of subjective data by individual domains grouped by skill level is shown in Fig. 5a–c. As surgical skill increases, the degree of conformity of the data across platforms increased except for the domain tactile feedback, which increases in disparity.

Fig. 5
figure 5

a–c Subjective assessment of surgical platform across eight domains stratified by platform type for MSI, PGY, and staff study participants. Note that Likert scores ranged from 0 to 10 where 0 was the worst possible score, and 10 was the best (*P < 0.05)

Full size image

Discussion

As technology continues to be refined in the setting of advanced MIS platforms, the previously daunting task of introducing laparoscopic approaches into established practices is becoming increasingly facile. The question of whether increasing surgical experience can facilitate this process while foregoing formal laparoscopic training has so far been unanswered. Additionally, although a single surgical robotic platform dominates the market at this point, head-to-head comparisons of existing technologies across surgical skill levels has not been examined [10–12].

We show in this study that not all robotic platforms are created equal in their impact on surgical performance and surgeon preference. The daVinci robotic assistant outperformed both Zeus and conventional laparoscopy in all task times and error rates as well as proving subjective superiority. The increased degrees of freedom, enhanced visualization provided by binocular laparoscopic vision, smooth motion scaling and translation, as well as ergonomically superior controls combined to provide a stable, consistently efficacious platform for task performance regardless of skill level. The Zeus robotic platform scored subjectively and objectively worse in all areas than the daVinci platform or laparoscopy. With increasing surgical experience, however, the negative effects associated with Zeus assistance were reduced. Conventional laparoscopy fell somewhere in between these results with most participants yielding better performance than with the Zeus platform and inferior performance than with the daVinci platform in all tasks. The finding that MSI participants performed worse in Zeus trials versus conventional laparoscopy is interpreted as a result of removing two degrees of freedom, a lack of tactile feedback, and 2D visualization with inexperienced surgeons. Additionally, the Zeus operating platform provides restrictions on arm movement in terms of radii that are traversable at each joint, a generally unrefined human-machine interface, and poor visualization. The combination of these factors seems to impact the operator more if the subject lacks surgical experience, as it represents additional complexities to be overcome in completing the task at hand. Operator compensation can occur for deficits of platform functionality as seen in correlating PGY and staff trials, but this appears to be a learned behavior. Perhaps the lesson here is that the human–machine interface is a crucial component of any new surgical technology, as the inability to realize the full potential of the technology at hand will result in frustration and poor adoption. All subjective scoring in the MSI group demonstrated daVinci superiority across domains except in tremor reduction and coordination, although a trend towards significance was found. The lack of surgical experience may have led to the finding of non-significance in the domain of tactile feedback in the MSI group who may not yet appreciate the subtleties of surgery. The PGY group demonstrated significant differences across all subjective domains in favor of the daVinci platform, except in tactile feedback in which laparoscopy was superior. This was the predicted result of senior PGY trainees who were operating on a daily, continuous basis with some sensitivity to the tools of the trade. Finally, experienced staff surgeons were indifferent as to platform preference except in demonstrating a preference for laparoscopy in the domain of tactile feedback. Again, this is the expected result as experienced surgeons likely had the technical knowledge to surpass deficiencies of platform dependence noting only the most obvious of inequalities.

Historically, surgeons have relied on the sense of touch to provide invaluable direction during surgery; however, current robotic systems do not yet have integrated haptics to provide feedback to the surgeon at the console. While this deficiency is observed in the results of the subjective scoring in this study, it is not borne out in the objective results of task times or errors. This is likely because the in vitro model is too simplistic in nature to mimic the tissue planes, tensile fragilities, and their complex interactions found in vivo. Even without the ability to provide force-feedback, the daVinci robot excelled at suturing and knot tying without increased error rates secondary to tearing of the latex membrane.

The effect of surgical experience was also examined in this study and demonstrated globally improved performance with increasing experience. If the gold standard is considered the staff experience with each platform, then we can see that the graduation of surgical experience from MSI to PGY to staff decreases the learning curve of the skill, whereas the platform type dictates the overall difficulty level of the task. Analysis of performance trends across repeated trials again demonstrated quicker skills acquisition and earlier plateau with increasing surgical experience. The most inexperienced participants produced more random results with respect to skills adaptation, an indicator of the steep learning curve, confirmed by the correspondingly prolonged task times. Our study suggests that increasing surgical experience can negate the significant effects of platform advantage, which explains why highly proficient surgeons whether in open, laparoscopic or robotic arenas may report equivalent results of similar surgeries with different approaches. Our study also implies that experienced laparoscopic surgeons can recruit the daVinci platform with a short learning curve and utilize it to facilitate more complex procedures not otherwise feasible through pure laparoscopy. Even with procedures currently performed through pure laparoscopic means, we support the possibility that in experienced hands, daVinci robotic assistance can facilitate or improve on these operations as well.

In terms of surgical education, this study demonstrates the significant benefits of daVinci robotic assistance in task performance across trainee skill levels. It also highlights the fact that while performance is enhanced, there is a narrow range in which the most technologically advanced platform is superior to conventional laparoscopy. While some might argue that with increasing task complexity, one would see a concomitant increase in the difference between these two systems, in all likelihood there remains a baseline performance function for each platform, for which surgical experience alone may make up the difference.

A search of the current literature regarding surgical robotics elicits numerous studies comparing laparoscopy to one or another robotic platform, but few which compare all three directly in terms of task performance [13–15]. As well, results from these studies are at odds with each other with similar platform comparisons. This is likely a function of the fact that the Zeus surgical system has undergone several revisions since its introduction, and as a result, its performance has likely been improved.

This study used a drylab simulator to evaluate surgeon and platform performance. However, as mentioned previously, this model may be overly simplistic to account for the multifactorial environment of operating in vivo. As well, subjective scoring was carried out using an in-house developed survey, which was constructed in a standard manner, however, not validated on a separate dataset. Our group in the Canadian Surgical Technology and Advanced Robotics (CSTAR) facility at our center is currently constructing a validated subjective tool to assess platform performance. Additionally, the results of this study only imply that robots may benefit the experienced surgeon in practice who is laparoscopically naïve. As senior residents were used as the comparator to trained MIS surgeons, this remains a limitation of this study.

In summary, we confirm the results of previous literature documenting the inferior performance of Zeus robotic surgical system versus laparoscopy as well as demonstrate that the daVinci surgical platform is superior to conventional laparoscopy for both task performance and usability. In fact, in finding the Zeus platform inferior to conventional laparoscopy, this reinforces the notion that as new technologies are introduced into the surgical armamentarium, each must be carefully and comprehensively evaluated in the provision of benefit over established means.