Journal of Robotic Surgery

, Volume 7, Issue 3, pp 235–240

Lessons learned using the insertable robotic effector platform (IREP) for single port access surgery

Authors

    • A.R.M.A. Laboratory, Department of Mechanical EngineeringVanderbilt University
    • Vanderbilt Initiative in Surgery and Engineering (VISE)
  • A. Bajo
    • A.R.M.A. Laboratory, Department of Mechanical EngineeringVanderbilt University
    • Vanderbilt Initiative in Surgery and Engineering (VISE)
  • A. Reiter
    • Department of Computer ScienceColumbia University
  • Long Wang
    • A.R.M.A. Laboratory, Department of Mechanical EngineeringVanderbilt University
    • Vanderbilt Initiative in Surgery and Engineering (VISE)
  • P. Allen
    • Department of Computer ScienceColumbia University
  • D. Fowler
    • Department of SurgeryColumbia University
Special Issue: Hamlyn Symposium 2012

DOI: 10.1007/s11701-013-0400-9

Cite this article as:
Simaan, N., Bajo, A., Reiter, A. et al. J Robotic Surg (2013) 7: 235. doi:10.1007/s11701-013-0400-9

Abstract

This paper presents the preliminary evaluation of a robotic system for single port access surgery. This system may be deployed through a 15-mm incision. It deploys two surgical arms and a third arm manipulating a stereo-vision module that tracks instrument location. The paper presents the design of the robot along with experiments demonstrating the capabilities of this robot. The evaluation includes use of tasks from fundamentals of laparoscopic surgery, evaluation of telemanipulation accuracy, knot tying, and vision tracking of tools.

Keywords

Single port access surgeryMinimally invasive surgeryNatural orifice surgery

Introduction

Single port access surgery (SPAS) is driven by the potential for added patient benefits due to reduction of the number of access incisions to only one (or to no incisions when using a natural orifice). Compared to open or minimally invasive surgery (MIS), these potential benefits include improved cosmesis and patient self-image, reduced risk of wound site infection, and reduced pain. However, in order to deliver the potential patient benefits of SPAS over MIS, new technologies that address the technical demands of SPAS are needed. As an effort towards achieving this goal, our team has developed the Insertable Robotic Effector Platform (IREP), Fig. 1.
https://static-content.springer.com/image/art%3A10.1007%2Fs11701-013-0400-9/MediaObjects/11701_2013_400_Fig1_HTML.gif
Fig. 1

The IREP mounted on a rotational and a translational stage: (1) central stem, (2) three-DoF camera control arm, (3) vision module, (4) passively flexible stem, (5) parallelogram linkage, (6) first continuum segment, (7) second continuum segment, (8) rotational wrist, (9) gripper

To enable SPAS, researchers investigated manual and robotic solutions. Examples of manual instruments include Realhand® from Novare, Cambridge Endo and EndoSAMURAI from Olympus Corp., or Spider from Transenterix. These instruments provide distal tool tip dexterity and can articulate to avoid collisions between the operator hands [9]. Animal studies of single port access laparoscopic cholecystectomy have been reported using these instruments [8]. However, the use of manual instruments presents ergonomic challenges, requires surgeons to operate using un-intuitive hand movements and relies on exceptional hand–eye coordination and substantial training. More importantly, manual instruments do not harness the full potential of computer-aided surgery. Manual instruments only augment the surgeon’s reach and dexterity. On the other hand, robotic systems are capable of augmenting sensory perception (e.g. by providing force sensing and feedback), interpretation of the surgical scene (e.g. by registering preoperative images to intraoperative secenes through the use of image overlay) and the accuracy and safety of surgical execution (e.g. by using active constraints or telemanipulation virtual fixtures).

For these reasons, researchers explored robotic assistance for SPAS and natural orifice transluminal endoscopic surgery (NOTES). Abbott [1] developed a wire-actuated dual-arm robotic system for NOTES which has 16 degrees of freedom (DoF) and a diameter (∅) larger than 20 mm. Kencana et al. [2] presented a 9 DoF ∅22 mm dual-arm robot. Lehman et al. [3] developed a NOTES robot that may be inserted into the abdomen via a ∅20 mm overtube and could be attached to the abdomen using external magnets. This design requires the surgeon to switch the device from a folded to a working configuration. Recently, Harada et al. [4] introduced a novel concept of a reconfigurable self-assembling robot for NOTES. This concept has yet to be experimentally proven. Lee et al. [5] presented a stackable four-bar mechanism for SPAS. Piccigallo et al. [6] presented a dual-arm robot for SPAS. This design embeds motors inside its surgical arms. Each arm is ∅23 mm and the requires access is ∅30 mm. Xu et al. [7] presented a concept for a trans-esophageal NOTES robot using design features of the IREP. Finally, Larkin et al. [8] presented a dual-arm SPAS system that uses wire-actuated snake-like articulated linkages. In contrast to this design, the IREP uses push–pull actuation and super-elastic NiTi backbones.

The IREP was designed by combining our past experience in designing systems for MIS of the throat [9] and for vision-guided automated surgical tool tracking [10]. The initial design considerations of this system have been described in [11, 12].

The IREP, we believe, is currently the smallest robotic system for SPAS, requiring an access port of only ∅15 mm while offering dual-arm dexterous operation with sub-millimeter accuracy, 3D visualization, and automated instrument tracking. It has two ∅6.4 mm dexterous surgical arms and a third arm manipulating a stereo-vision module. Each surgical arm includes a parallel mechanism, a passively flexible stem, an actively controlled continuum snake-like arm, a rotational wrist and a gripper. The parallel mechanisms control the distance between the bases of each snake arm. The total number of actuated DoF of the IREP is 21. These include seven actuated DoF for each surgical arm, three DoF for deploying and controlling the pan/tilt of a vision module, two DoF for each gripper and two DoF for axial insertion of each dexterous arm.

The detailed kinematic modeling and design specifications of the IREP were presented in [13]. These specifications included the coverage of a workspace of at least 50 mm in each Cartesian direction, roll of the gripper about its longitudinal axis to enable suturing in confined spaces, ability to triangulate both surgical arms, ability to apply a 2-N lateral force in correspondence with surgical tissue manipulation forces and suturing [14], and ±0.25 mm in positioning accuracy to allow micro-surgical interaction. The design avoids the use of serially connected motorized joints (such as in [15, 16]) in order to limit backlash and to enhance sterilizability. The IREP also provides a controllable distance between the bases of its dexterous surgical arms to increase kinematic dexterity and dual-arm triangulation. Ding et al. [12] showed these advantages compared to other designs using dexterous arms emanating from a single lumen (e.g. [17, 18]).

The IREP has also been designed as a modular platform for multi-modal use including energy and drug delivery and suction applications for SPAS and NOTES. This is achieved through the use of tubular access channels within each surgical arm. The flexible continuum arms can also be used as independent tools for NOTES by deploying them through an over-tube.

The following is a description of lessons learned during early evaluation experiments using the IREP.

Materials and methods

To evaluate the IREP’s functionality, we used several tasks including the Fundamentals of Laparoscopic Surgery (FLS) Manual Skills testing component (peg transfer and simple suture with intracorporeal knot). We also evaluated the accuracy of telemanipulation while following a circular path, tested the utility of our vision tracking algorithms and confirmed the workspace covered by a single arm against an adult life-size cholecystectomy trainer. Figure 2 shows the peg transfer experiment. Six rubber triangular parts with ∅6.3 mm holes were transferred from one side to the other side of a peg board with ∅3.2 mm pegs.
https://static-content.springer.com/image/art%3A10.1007%2Fs11701-013-0400-9/MediaObjects/11701_2013_400_Fig2_HTML.jpg
Fig. 2

Dexterity peg board used for FLS evaluation

The IREP was tested in extreme conditions assuming that the only available gross movement of the device is along the trocar’s longitudinal axis. We did not use all the four DoF available to standard MIS instruments since we wanted to determine the IREP’s dexterity and workspace without reliance on these additional DoF. The peg transfer task consists of grasping a plastic ring with one hand, removing the ring from the peg, transferring the ring to the other hand, and then placing the ring on another peg without dropping the ring. The simple suture and intracorporeal knot module consists of placing a suture through a dot on a rubber drain followed by tying intracorporeally a surgeon’s knot and two square knots. Using these two modules, we could assess functionality of the robotic device under direct vision. If completed, we recorded the time to complete each module.

To evaluate telemanipulation accuracy and the effect of vision feedback using the IREP’s cameras on user performance, we used a position-symmetric telemanipulation algorithm as described in [19]. A trained user was asked to move the robot’s gripper along the circumference of a ∅12 mm circle. A grid paper with 500 μm grid was used as a back-drop. A high-definition digital microscope was used to record a video of the robot movement and color-based image segmentation software was developed to locate the center of a printed red dot attached to the robot’s gripper. This setup was needed since the movement accuracy of the IREP was very close or beyond the accuracy of commercial vision trackers available at our laboratory. Twenty experiments were equally split using two visual feedback methods: (a) using a high-definition microscope, (b) using the IREP’s on-board cameras. We calculated the root mean square (RMS) and maximal tracking error and recorded competion times.

To demonstrate our tool tracking algorithm we overlaid the location of each gripper tracking box on the images of the vision module. We used vision tracking algorithms combining several different color and texture features in a probabilistic framework described in [20]. The features work together to assist each other when some cues are stronger than others, and the appearance of the tool is learned on-line. In this way, the tracker only requires an initial position and bounding box dimension on the first frame of the video sequence, and the appearance of the tool is learned on-the-fly as new views are presented to the camera and various conditions of the environment are changing. We have demonstrated this work in real surgical scenarios [21] as well as with the IREP gripper in this paper.

Finally, we used a Simulab LC-10 cholecystectomy model. A surgeon telemanipulated one arm of the IREP to verify the extent of its reachable workspace. A sequence of experiment images was recorded.

Results and discussion

Figure 2 shows the results of the FLS experiment. The experiment validated that the grippers of the IREP were able to firmly hold the triangular objects, that the movement of the dexterous arms was sufficient to allow object transfer, and that each dexterous arm could cover the entire FLS peg board (64 × 103 mm). The experiment, however, did reveal that the telemanipulation code suffered from several flaws, including: (a) there was an unnecessarily high scaling ratio (5:1) between the master and slave, (b) coupling of motion between the master and slave was not direct, (c) rotation of the grippers about their axis slightly affected the gripper tip position, (d) although movement in the mid-range of the trocar’s longitudinal insertion axis was intuitive for both hands, at the extremes of the range of movement in this axis the movement generated by the Cartesian stage carrying the IREP could only be controlled by the right master. Flaws (a–b) have later been fixed during debugging phases of the telemanipulation code. Flaw (c) is related to calibration of the continuum robotic arms and the fact that we used a Phantom Omni as the master interface, which made it difficult for the surgeon to rotate his hand in space without inducing translations. We have since then implemented an orientation telemanipulation mode. Exact calibration of the dexterous surgical arms remains a minor issue to address since our experience has shown that telemanipulation under surgeon vision allows for intuitive compensation for system imperfections.

Figure 3 shows the experiments in passing circular needles and knot tying. This experiment revealed that the limited roll of the gripper wrist made circular needle passing and hand exchange difficult. We initially designed the wrists to provide ±60° of roll. This highlighted the need for a redesign of the distal wrist to provide a larger rotation workspace. To validate the amount of wrist roll we carried a straight needle at the gripper tip and rolled the wrist throughout its range of motion. By segmenting these images we determined that the manufactured wrist provided a roll range of ±69°. Our new design goal is to provide at least 360° of roll. We have since then redesigned the wrists, but the new design has not been integrated yet.
https://static-content.springer.com/image/art%3A10.1007%2Fs11701-013-0400-9/MediaObjects/11701_2013_400_Fig3_HTML.jpg
Fig. 3

Visual tracking of the IREP’s dexterous arms

Figure 4 shows a sample result of the telemanipulation experiment along the circumference of a ∅12 mm circle. The RMS and maximal tracking error along with the average experiment completion time are presented in Table 1. The maximal error was correlated with mechanical backlash in the actuation unit of the parallelogram linkage. These results informed the redesign of the actuation unit and we are setting up for repeating this experiment with multiple users.
https://static-content.springer.com/image/art%3A10.1007%2Fs11701-013-0400-9/MediaObjects/11701_2013_400_Fig4_HTML.jpg
Fig. 4

Visual tracking of the IREP’s dexterous arms: (left) image segmentation of the gripper position, (right) the tracked trajectory against the desired movement trajectory

Table 1

Results of the circle-following experiments

Vision source

RMS error (mm)

Max. error (mm)

Average time (s)

Microscope

0.24

1.2

47.6

IREP cameras

0.33

1.9

40.4

Figure 5 shows successful tracking of the grippers using our probabilistic tracking framework. The red and blue boxes indicate the location of the gripper as detected by the tracking algorithm. The tracker was able to detect both grippers and to follow them successfully, but it was not able to deal with scenarios where the gripper is temporarily hidden behind anatomy and then reappears in the image.
https://static-content.springer.com/image/art%3A10.1007%2Fs11701-013-0400-9/MediaObjects/11701_2013_400_Fig5_HTML.jpg
Fig. 5

Visual tracking of the IREP’s dexterous arms

The cholecystectomy workspace verification experiment is shown in Fig. 6. We were successful in covering the extents of an adult life-size model of a human gallbladder using only a single arm of the IREP, which agrees with our design goals as first stated in [11].
https://static-content.springer.com/image/art%3A10.1007%2Fs11701-013-0400-9/MediaObjects/11701_2013_400_Fig6_HTML.jpg
Fig. 6

Workspace boundaries of a single IREP arm shown against a cholecystectomy trainer: a left, b top, c right, d bottom

Conclusion

We demonstrated that the IREP can complete object transfer, knot tying and automated vision tracking of its grippers. We also demonstrated that the workspace of a single arm of the IREP is suitable to cover the surgical field of cholecystectomy—even when the IREP is only mounted on an insertion slide and not all the traditional four DoF of MIS tools are used. This means that the IREP can be mounted on a simple support arm instead of a four-DoF robot. The evaluation also showed that the limited wrist roll complicated the passing of circular needles. We evaluated the actual roll range of the IREP and redesiged the wrist to provide at least one full turn. The telemanipulation experiments showed that the IREP is capable of sub-millimetric precision and that the 640 × 480-pixel cameras of the IREP degrade the user’s performance. We are working to incorporate higher resolution cameras. Evaluation of the IREP in the near future will include quantification of payload capabilities, telemanipulation latency, and evaluation on animals.

Acknowledgments

This work was supported by NIH Grant No. 7R21EB007779. A. Bajo and N. Simaan were also supported by NSF Career Grant No. IIS-1063750. The evaluation experiments were performed under a research agreement with Titan Medical Inc. Dennis Fowler serves as a consultant for Titan Medical Inc.

Conflict of interest

None.

Copyright information

© Springer-Verlag London 2013