1 Introduction

The need for healthcare robots is becoming more evident. Communities have experienced greater need since the start of the COVID-19 pandemic when there was a huge shortage of doctors, nurses, and other healthcare supports [1, 2]. The healthcare robot branches from surgical robots to robot assistants, service robots to assistive robots [3]. Moreover, this paper focuses on a robot that can support doctors and nurses in the outpatient department (OPD) and operation theater complex (OT), especially with the manipulation of surgical tools. We call it a robot assistant for surgical tool manipulation in healthcare. The role of these robots is to support doctors and nurses with surgical tool exchange during surgery, tool sorting and sterilization and cleaning surgical workspaces. Generally, the robot assistant consists of a multi-degree of freedom robotic arm with highly dexterous workspace movement and a gripper for object manipulation. In particular, grippers are required to manipulate various objects with unique design capabilities and autonomous control architectures to handle uncertainty and adapt to complex shapes. However, there is little research that focuses on gripper designs and manipulation of surgical instruments[4, 5].

Surgical instruments have many random shapes, sizes, and materials that pose key challenges to manipulate in a hospital environment. Several conventional and commercial robotic grippers, such as Shunk, Robotiq, and Kuka, have excellent precision and repetitive grasping performance. Still, such grippers cannot be a viable solution due to their limited capability to adapt to random objects, especially flat and thin. On the other hand, these grippers are mostly made of rigid materials with a low degree of dexterity.

Many research groups focus on developing either a new gripper design or control architecture to achieve high success rates in grasping small and thin objects. For example, Odhner et al. [6] grasp with a two-finger flat underactuated gripper by adapting a human-like manipulation strategy and performing flip and pinch tasks. They could particularly manipulate a flat key, a coin, a screwdriver, a USB drive etc. The proposed strategy is unique but it requires high-level control architecture and is limited to specific shapes and sizes. In 2013, Yamaguchi et al. proposed a three-finger gripper hand (iGRIPP-4) with a suction mechanism. The fingers can reconfigure their position and grasp multiple objects such as keys, USB drives, etc. [7], but they are unsuitable for uneven surfaces and random shapes. Kim et al. also used a suction gripper, a powerful tool for grasping thin objects [8]. However, they selected a specific gripper type based on the object’s size and shape. For example, they only used a suction gripper with a smaller suction hole than the size of the thin objects to grasp properly. Babin et al. proposed another solution, a scooping grasp mechanism with retractable nails and a grasp thin sheet using scooping-like closing and opening movements [9, 10]. However, the proposed system was rigid and limited to large flat surfaces. In 2017, Nishimura et al. proposed an underactuated soft gripper with a torsional spring and ratchet gear. The proposed gripper contains viscoelastic fluid at the fingertip and can operate in various modes such as parallel gripping, pinching and enveloping. They tested the gripper successfully on a PET sheet, a thin plate, a washer, a business card etc, which are limited to thin-flat 2D objects [11]. Recently, Hang et al. proposed pre-grasp sliding manipulation using a compliant, underactuated hand. The proposed system requires two manipulators, one for sliding the object and the other for grasping. However, this method is used only for picking a thin-flat flexible sheet [12]. Additional approaches include an electromagnetic gripper, which showed complex grasping of tiny, slender objects like a surgical instrument [13, 14], and levitation-based grasping for thin object manipulation using tilt control and haptic feedback [15]. The grippers are operated with passive compliance that can reorient and conform to the instrument surface. Still, they are difficult to control and limited to ferromagnetic objects. Another approach with a rigid commercial gripper, Barrett’s hand, is proposed for grasping an object on the table surface [16]. The force sensor installed on the gripper measures force while contacting the surface, and it is possible to grasp an object without damage. However, the gripper is difficult to grasp thin-flat objects and other types of surgical tools. The issue could be solved with a special gripper with softness and deformability that can grasp a small object with a large contact area [17]. A compliant grasp strategy, which is proposed to interact with an object and environment surface in a reliable and robot way [18]. However, they still lack a high degree of dexterous flat and thin object manipulation, especially the surgical instruments.

The previously described solutions are mostly based on either semi or fully-rigid materials. However, a fully soft gripper design could be a more viable solution [19, 20]. These designs are mostly either highly stretchable polymer-based pneumatic, tendon driven [21,22,23] or granular jamming actuated grippers [24]. These grippers have excellent manipulation capability of random objects in a highly unstructured environment, but they lack load carrying capacity and precise movement.

Fig. 1
figure 1

A complete 3D CAD model of SurgGrip with its part annotations: [1] force and torque sensors; [2] DC metal gear motors; [3] flexible coupling; [4] leadscrew; [5] lock nut; [6] gripper chassis; [7] base linkage; [8] compression springs; [9] gripper fingers; [10] fingertips; [11] soft silicone pad; [12] torsional springs

Full size image

Moreover, there are several solutions for flat and thin object manipulation. Still, they did not consider grasping in the surgical environment. For example, in [13, 25,26,27], the authors demonstrated that robotic grasping for handling ferromagnetic surgical instruments using the electromagnetic gripper. The grippers could only handle selected tools when the tools were cluttered and overlaid. In contrast, the gripper cannot be used for non-ferromagnetic instruments and the electromagnetic force can bend or deform thin and flat tools. For these reasons, we developed a finger-based gripper, which can handle 2D and 3D ferro and non-ferromagnetic instruments with dexterous grasping capability. We develop a solution for manipulating surgical tools using an underactuated system, which can perform paralleling, pinching, and enveloping movements. The SurgGrips is also capable of handling table shock and collisions, minimizing surface friction and adapting to the shape of 2D/3D objects.

The objectives of the SurgGrip are as follows: (1) The gripper is expected to hold the objects with greater stability using an overlap grasp mechanism (fingertips as mortise and tenon joint, zero gap) with soft pads. (2) The gripper is inherently compliant and implies passive adaptation to the working environment. (3) The gripper is expected to pick and place the various shapes, sizes and weights of surgical instruments. (4) The gripper is presumed to control both mechanisms simply with high precision movement. (5) The gripper is expected to automatically manipulate various surgical tools in a short time using computer vision without any human intervention.

We organize the paper as follows. In Sect. 2, we report the proposed design and working principle. Similarly, Sect. 3 shows the fabrication process, and Sect. 4 describes the kinematics and static analysis. Section 5 describes the experimental setup and results, and Sect. 6 shows the conclusions and future works.

2 Proposed design and working principle of SurgGrip

We developed a versatile gripper that can manipulate various shapes and sizes of surgical instruments with a high success rate. The Key features of gripper design consists of soft, compliant, resilient, high-precision movements and 3D printable. Herein, we report the first version of SurgGrip design (Fig. 1) with its kinematic scheme and working principle (Fig. 2a–b). The SurgGrip design includes a gripper chassis, base linkages, fingers and fingertips with soft pads. The gripper chassis is attached to a UR5 robotic arm in conjunction with a Force/Torque (F/T) sensor. Furthermore, it consists of a motor–leadscrew–nut drive connected to the base linkage. The fingers are designed with embedded compression springs, and each finger is connected to sharp wedge-shaped fingertips by a torsional spring. Both fingertips are designed like mortise and tenon joint with soft elastomer pads. (Fig. 1)

Fig. 2
figure 2

Gripper schematic and working principle. a kinematic diagram where \(l_{0}=\) fixed linkage, \(T_{0}=\) transnational joint for motor and leadscrew rotation to lead nut linear motion, \(l_{1}=\) base linkage, \(l_{2}=\) left finger linkage, \(l_{3}=\) right finger linkage, \(l_{4}=\) left prismatic linkage, \(l_{5}=\) right prismatic linkage, \(l_{6}=\) left fingertip linkage, \(l_{7}=\) right fingertip linkage, \(T_{1}=\) left transnational joint for helical spring linear motion, \(T_{2}=\) right transnational joint for helical spring linear motion, \(R_{0}=\) revolute joint to connect \(l_{1}\) and \(l_{2}\), \(R_{1}=\) revolute joint to connect \(l_{0}\) and \(l_{2}\), \(R_{2}=\) revolute joint to connect \(l_{4}\) and \(l_{6}\), \(R_{3}=\) revolute joint to connect \(l_{1}\) and \(l_{3}\), \(R_{4}=\) revolute joint to connect \(l_{0}\) and \(l_{3}\), \(R_{5}=\) revolute joint to connect \(l_{5}\) and \(l_{7}\); b Working principle of the gripper in four steps, [1] Preparing; [2] Paralleling; [3] Pinching; [4] Enveloping

Full size image

In Fig. 2, we have shown kinematic schematics and the basic working principle of SurgGrip. In the kinematic diagram, we presented that, the SurgGrip has three prismatic joints (\(T_{0}\)–\(T_{2}\)) and six revolute joints (\(R_{0}\)–\(R_{5}\)) (see Fig. 2a). Where, \(T_{0}\) represents the motor–leadscrew–nut drive which is connected to prismatic linkage \(l_{0}\) & \(l_{1}\). The four-bar linkage mechanism has four links \(l_{0}\)–\(l_{3}\) and connected by four revolute joints. In particular, \(l_{0}\) & \(l_{2}\) are connected by \(R_{1}\), \(l_{1}\) & \(l_{2}\) are connected through \(R_{0}\), \(l_{0}\) & \(l_{3}\) are connected through \(R_{4}\), and \(l_{1}\) & \(l_{3}\) connected through \(R_{3}\). The joints \(T_{1}\) & \(T_{2}\) are connected by linkage \(l_{2}\), \(l_{4}\) and \(l_{3}\), \(l_{5}\). Similarly, the fingertips joints \(R_{2}\) & \(R_{5}\) are connected by linkage \(l_{4}\), \(l_{6}\) and \(l_{5}\), \(l_{7}\).

The working principle of the gripper is explained in four steps; (1) Preparing: The gripper moves and touch the table/surface. (2) Paralleling: The fingertip flattens to the table using the UR5 arm. (3) Pinching: The gripper closes the workspace through DC motor–leadscrew–nut drive. The fingers move until it is exactly perpendicular to the table. (4) Enveloping: The DC motor rotates until both fingertips (mortise and tenon) overlap. In addition to overlapping fingertips, the soft pads conform and adapt to the tool shape for tight grasping, and then the UR5 arm lifts the object and places it in the desired location (see Fig. 2b).

To actuate the gripper, the motor–leadscrew–nut drive converts the rotational motion into linear motion. Then, the four-bar linkage mechanism controls the opening and closing of the gripper workspace. The high torque DC motor, and leadscrew-nut drive allow the gripper to produce precise motion and high load carrying capacity. In the gripper, we have used two types of springs (compression and torsional) and soft elastomeric pads to make them more compliant and adaptable. For example, the compression and torsional spring make the gripper more compliant, and resilient that could prevent failure due to table shocks and collisions, especially for instrument tables and nearby objects. Furthermore, soft elastomer pads make them more conformable and adaptable to increase graspability. To fabricate SurgGrip, we used 3D printing techniques to enable complex features, reduce manufacturing complexity and expedite the fabrication process with fast integration and assembly at a lower cost.

Table 1 Material properties of components used in gripper fabrication
Full size table

3 Fabrication

To build the SurgGrip, we used an additive manufacturing method. We first printed fingertips, fingers, base linkage and gripper chassis and the connectors for the UR5 robotic arm and F/T sensor. We used a 3D Systems ProJet HD 3000 printer to print the structures with acrylic resin (3D Systems VisiJet M3 Crystal), which is a biocompatible material [28] (Table 1).

Fig. 3
figure 3

A complete fabricated prototype of the SurgGrip with a four-bar linkage-based grasping mechanism

Full size image

For the DC motors, we used Pololu high-power metal gearmotors, a magnetic encoder and a DRV8835 dual motor driver to control the motor (Table 1). We used a high load and high accuracy trapezoidal-shaped leadscrew. To connect the motor shaft to the leadscrew, we used a flexible beam coupling (FBC) with constant velocity (for both high and low-speed operation), high torque capacity, and zero backlashes. Besides that, FBC can also handle misalignment with 3\(^{\circ }\) angular, 0.381 mm parallel, and 0.254 mm axial. For the springs, we used high spring steel alloy material for compression and torsional springs (Table 1). We fabricated the soft pad for the fingertips by using the replica molding technique and cast the pads on the 3D-printed mold. We poured Dragon Skin 20 medium resin [29] and cured it for 5 h at room temperature (Table 1). After fabricating all the components, we assembled them using conventional fastening techniques such as screw nuts and super glue. The complete assembled gripper prototype weights 298.73 g with opening and closing angles of 124.4\(^{\circ }\) and 90\(^{\circ }\). We show the final assembled prototype in Fig. 3.

4 Kinematic and static analysis

We synthesized the basic kinematic of the gripper mechanism to estimate the fingertip length and determine the workspace (see Fig. 2a). The static model estimates motor torque and the relation between friction force and stiffness.

Fig. 4
figure 4

Kinematics and statics models: a shows the kinematic synthesis to estimate the workspace and corresponding geometric parameters. b shows the gripper modeling for opening and closing workspace c describes the static analysis to determine the relationship between spring stiffness and motor torque. \(\emptyset _{1}=\) finger rotation angle, \(\emptyset _{2}=\) fingertip rotation angle, \(W_{1}=\) distance between fingertip to gripper center line at the state 1, \(W_{2}=\) object width, \(W_{3}=\) distance between fingertip to gripper center line at the state 2, \(L_{max}=\) finger link length at state 1, \(L_{min}=\) finger link length at state 2, \(\theta _{1}=\) finger hinge rotation angle, \(\theta _{2}=\) fingertip hinge rotation angle, \(\sim k_{1}=\) helical/compression spring stiffness, \(\sim k_{2}=\) torsional spring stiffness, \(F_{r}=\) ground reaction force, \(l_{tip}=\) length of the fingertips, \(\tau _{min}=\) minimum torque required, \(F_{r1}=\) ground reaction force at maximum finger length, \(F_{r2}=\) ground reaction force at minimum finger length, \(f_{1}=\) ground friction force at maximum finger length, \(f_{2}=\) ground reaction force at minimum finger length. \(\varDelta P=\) total leadscrew displacement, \(\lambda \) = helix angle of leadscrew, \(\phi _{0}=\) initial angle between linkage \(l_{3}\) (AB) and horizontal x axis, \(l_{l}=\) left and right finger link length, \(l_{f}=\) left and right prismatic link length

Full size image

4.1 Kinematics synthesis

The proposed design of SurgGrip contains nine degrees of freedom with three prismatic joints (T\(_{0}\)–T\(_{2}\)) and six revolute joints (R\(_{0}\)–R\(_{5}\)). We used a geometrical method to synthesize the gripper mechanism to estimate the fingertip length and workspace. Also, we assumed that SurgGrip movements are completely symmetric and considered only half of the movement. Therefore, we used a simplified model for the gripper kinematic synthesis (Fig. 4a). Here, we only have seven degrees of freedom with four revolute joints and three prismatic joints. We defined the grasping movement in three basic steps by assuming that the SurgGrip is already touching the ground and flattens its base parallel to the table surface (Fig. 4a). We defined steps and divided them into State 1, State 2, and State 3. Where the State 1: when the gripper touches the surface at its maximum finger length; State 2: when the gripper finger moves to its minimum length with maximum linear spring compression; and then State 3: when the finger starts restoring to its original length, the fingertip crosses the central axis to minimize the gap between two fingertips (mortise and tenon). We describe the gripper movement conditions in Table 2. Moreover, these parameters help to build a kinematic relationship between the required object width (half) \((max(W_{obj}), min(W_{obj}))\) and fingertip length. For example, if we assume the distance from fingertip to the centerline for State 1 and State 2 are \(W_{1}\), \(W_{3}\) respectively, and the distance from the finger to the centerline is \(W_{2}\). We create a parametric relationship to estimate the possible fingertip length and desired workspace. In this case \(l_{tip}\) is no more than two times of \(min(W_{obj})\) and the workspace is (\(max(W_{obj})\)-\(min(W_{obj})\)), see the Eq. 1. The proposed models were used for initial approximation. To have an accurate gripper workspace, we performed dedicated experiments (5.3)

Table 2 Kinematic synthesis conditions
Full size table
$$\begin{aligned}&W_{1} \ge max(W_{obj}), W_{2} \ge max(W_{obj}), \nonumber \\&W_{3} \le min(W_{obj}), \frac{l_{tip}}{2} \le min(W_{obj}) \end{aligned}$$
(1)

where \(L=\) arbitrary finger length, \(L_{max}=\) maximum finger length, \(L_{min}=\) minimum finger length, \(\emptyset _{1}=\) revolute joint angle 1, \(\emptyset _{2}=\) revolute joint angle 2, \(max(W_{obj})=\) half of instrument’s maximum width, \(min(W_{obj})=\) half of instrument’s minimum width, and \(l_{tip}=\) horizontal length of the fingertip.

4.2 Gripper workspace modeling

We modeled the closing and opening of the gripper workspace by correlating actuator space to gripper space. The basic schematic diagram is shown in Fig. 4b. The actuator space modeling was adapted from a design textbook [30]. In the proposed gripper design, the lead screw is connected to the motor shaft and the four-bar linkage mechanism, which changes the gripper workspace by rotating the leadscrew. We estimated the distance traveled by the leadscrew \(\varDelta P\) by multiplying the number of turns and leadscrew pitch (Eq. 2). In this case, it moves from \(O_{0}\) and \(O_{1}\) and forms a triangle (\(\varDelta A B C\)). However, when the gripper closes, it benefits from lateral deformation, which we have assumed to be negligible in our model. The correlation between gripper finger rotation and \(\varDelta P\) is formulated using the cosine rule (Eq. 3) The final correlation between angle and number turn total distance traveled by leadscrew are shown in Eqs. 4–5.

$$\begin{aligned} \varDelta P&= nP = \pi d_{m}tan(\lambda ) \end{aligned}$$
(2)
$$\begin{aligned} \varDelta P&= (l_{l})^2 + (l_{l})^2 - 2(l_{l})^2cos \phi _{1} \end{aligned}$$
(3)
$$\begin{aligned} (nP)^2&=2(l_{l})^2(1-cos \phi _{1})\nonumber \\&= 4(l_{l})^2 sin^2 \frac{\phi _{1}}{2} \end{aligned}$$
(4)
$$\begin{aligned} \phi _{1}&=2sin^{-1}\frac{nP}{l_{l}}\nonumber \\&=2sin^{-1}\frac{\pi d_{m}tan(\lambda )}{l_{l}} \end{aligned}$$
(5)

In Eqs. 2–5, we developed a relationship between finger rotational angle, leadscrew pitch and link length, which could be used to estimate the gripper opening and closing position. We first developed the initial position of the gripper \(x_{0}\) and then the finger position at \(\phi \) degree rotation. The gap between the gripper could be easily estimated by subtracting the tip length and finger position. The final formulation position and workspace are shown in Eqs. 7 and  10.

$$\begin{aligned} x_{0}&= \frac{l_{1}}{2} + \frac{l_{l}}{2}cos\phi _{0} + l_{f}sin\phi _{0} \end{aligned}$$
(6)
$$\begin{aligned} x_{\phi }&= \frac{l_{1}}{2} + \frac{l_{l}}{2}cos(\phi _{1}-\phi _{0}) + l_{f}sin(\phi _{1}-\phi _{0}) \end{aligned}$$
(7)
$$\begin{aligned} W_{1}&= x_{0} - l_{tip} \end{aligned}$$
(8)
$$\begin{aligned} W_{1\phi }&= x_{\phi } - l_{tip} \end{aligned}$$
(9)
$$\begin{aligned} W_{t\phi }&= 2(x_{\phi } - l_{tip}) \end{aligned}$$
(10)

\(\varDelta P=\) total leadscrew displacement from \(O_{0}\) to \(O_{1}\), \(\lambda =\) helix angle of leadscrew, \(\phi _{0}=\) initial angle between linkage \(l_{3}\) (AB) and horizontal axis (x axis), \(l_{l}=\) left and right finger link length, \(l_{f}=\) left and right prismatic link length, \(x_{0}=\) distance between \(O_{0}\) and \(D_{0}\), \(x_{\phi }=\)distance between \(O_{0}\) and \(D_{1}\), \(W_{1\phi }=\) half gap between both fingertips \(W_{t \phi }=\) total gap between both fingertips, \(n=\) number to pitch, \(P=\) pitch of leadscrew, \(\phi _{1}=\) angle between linkage \(l_{3}\) initial and final stage (\( \angle \)ABC), \(d_{m}=\) mean diameter of leadscrew.

4.3 Statics

This section focuses on developing a relationship between frictional forces, joint angles, and spring constants (Fig. 4c). Herein, from state 0, we assume that fingertips move down and change their angle from \(\theta _{2}\) to 0 from state 0 to state 1. In this case, we assumed minimal deformation in compression springs. Now, we can write the force equation from the free body diagram (FBD, Fig. 4c). Where we found reaction force is directly proportional to \(k_{2}\) and inversely proportional to \(l_{tip}\).

$$\begin{aligned} \tau _{ts}&= \theta _{2} \cdot k_{2} \nonumber \\ \tau _{ts}&= F_{r} \cdot l_{tip} \nonumber \\ F_{r}&= \frac{k_{2} \cdot \theta _{2}}{ l_{tip}} \end{aligned}$$
(11)

Considering the minimal displacement in compression spring, we can balance the reaction force with spring restoring force,(Eq. 11).

$$\begin{aligned}&F_{r} \cdot cos\theta _{1} = k_{} \cdot \varDelta l\nonumber \\&F_{r} = \frac{k_{1} \cdot \varDelta l}{cos\theta _{1}} \end{aligned}$$
(12)

After solving Eqs. 11 and 12, we develop a direct relation between \(\theta _{1}\), \(\theta _{2}\) and \(k_{1}\), \(k_{2}\) which suggest that \(\theta _{2} > \theta _{1}\) in as Eq. 13.

$$\begin{aligned} \frac{\theta _{2}}{\varDelta l} =\frac{k_{1}\cdot l_{tip}}{k_{2} \cdot cos\theta _{1}} \end{aligned}$$
(13)

where, \(k_{1}=\) stiffness of compression spring, \(k_{2}=\) stiffness of torsional spring, \(\theta _{1}=\) initial stage or state 0 joint angle \( R_{1}\), \(\theta _{2}=\) state 0 joint angle \( R_{2}\), \(F_{r}=\) ground reaction force, \(\tau _{ts}=\) torque of torsional spring, \(l_{tip}=\) length of fingertip, and \(\varDelta l=\) linear spring compression length or stretch.

We also developed a relationship between the required minimum torque between frictional, reaction, and spring restoring forces. As we mentioned in Sect. 2, the motor is activated after the fingertip touches the ground and moves the finger from state 1 to state 2. In this case, the FBD shows how the frictional force directly increases the minimum required torque for the motor (Eq. 14), and motor torque is directly proportional to the product of \(k_{2}\) and \(\theta _{2}\)

$$\begin{aligned} \tau _{min}&= (F_{r1} \cdot sin\theta _{1} + f_{1} \cdot cos\theta _{1}) \cdot L_{max} \nonumber \\ f_{1}&= F_{r1} \cdot \mu \nonumber \\ F_{r1}&= \frac{\theta _{2} \cdot k_{2}}{l_{tip}} \nonumber \\ \tau _{min}&= \frac{\theta _{2} \cdot k_{2}}{l_{tip}} \cdot (sin\theta _{1} + \mu \cdot cos\theta _{1} )\cdot L_{max} \end{aligned}$$
(14)

However, the motor torque required for state 1 and state 2 will be different. Here, we have defined the minimum required torque which is directly proportional to \(k_{1}\) (Eq. 15)

$$\begin{aligned} \tau _{min}&= f_{2} \cdot L_{min} \nonumber \\ f_{2}&= F_{r2} \cdot \mu \nonumber \\ F_{r2}&= k_{1} \cdot (L_{max} - L_{min})\nonumber \\ \tau _{min}&= k_{1} \cdot \mu \cdot (L_{max} - L_{min}) \cdot L_{min} \end{aligned}$$
(15)

where \(\mu =\) friction coefficient, \(f=\) friction force, and \(\tau _{min}=\) minimum torque required for the motor.

We chose the gripper dimensions and motor type based on the developed model by considering surgical instrument dimensions and weight. We found that the torque requirement changes over the angular positions of the finger and fingertip. However, in this case, we used simplified version of the model to synthesize the gripper.

5 Experimental setups

5.1 Surgical instruments

We selected surgical instruments frequently used in hospitals during minor and major surgeries, such as clamps, scissors, forceps and scalpels in various shapes, sizes and weights. The dimensions and the weight of these tools vary from 140 \(\times \) 3 \(\times \) 3 mm and 9.03 gs (minimum) to 300 \(\times \) 93 \(\times \) 55 mm and 116 gs (maximum). We have shown complete shape size with respective weight in detail in Fig. 5.

Fig. 5
figure 5

Target objects and dimensions: This information is used to design the SurgGrip. The objects are composed of scissors, knives, tweezers, etc. with different shapes and materials

Full size image
Fig. 6
figure 6

The experimental setups: for a the test of grasping and lifting surgical instruments using UR5 robotic arm, b the measurement of the workspace using position tracker with Aurora system, c the measurement of the pulling force with threads connected between a tool and the fixture using a F/T sensor (F: force, T: torque). d The friction force measurement between the fingertip and table surface with and without a load

Full size image

5.2 Manipulator and sensor system

The experimental environments include four different setups, which are grasping test, gripper workspace using the Aurora system, pulling, collision force and friction measurement setup, as shown in Fig. 6a–d. The environment comprises a UR5 robotic arm, a SurgGrip, a Robotiq F/T sensor, an Arduino Uno microcontroller board, surgical instruments and a table (see Fig. 6a). The test scenarios for grasping involve picking and lifting the instruments. We perform the experiments at least ten times for each tool (a total of 120 times) to estimate success rates and object grasping time in the manual and autonomous modes that validate grasping functionality.

5.3 Workspace measurement setup

NDI’s Aurora Technology, which consists of an electromagnetic (EM) field generator, sensor and sensor data amplifier, is used for the workspace analysis. The system can measure the data regarding positions (T\(_{x}\), T\(_{y}\), T\(_{z}\)) and orientation (R\(_{x}\), R\(_{y}\), R\(_{z}\)) at a 40 Hz rate with an accuracy of 0.8 mm and 0.70\(^{\circ } \)(see Fig. 6b). Only position data from the sensor installed at the fingertip is used. This test was repeated five times.

5.4 Pulling force measurement setup

We measured the pulling force (vertical force on the z-axis) of the gripper by adapting the experimental method [31,32,33]. After grasping an instrument, two threads connected to the fixtures are utilized to fix the instruments. Then F/T sensor (60 Hz rate) starts recording the pulling force while the arm lifts up the instrument in the z-axis direction until the gripper drops (Fig. 6c).

5.5 Gripper impact and frictional force measurement setup

In this experiment, we tested the gripper’s shock-absorbing capabilities and frictional force measurements. This experiment investigates the robustness and stability of our gripper model to apply the rigid surface and validation through captured images. ROS is introduced to communicate between Arduino Uno and the UR5 robotic arm. To measure the friction force between the gripper fingertip and the table surface, we used the same setup of the UR5 robotic arm with SurgGrip. We moved the SurgGrip towards the table until both fingertips touched the table and became parallel to the table surface (0\(^{\circ }\) between fingertip face and table) and moved the gripper from left to right. After that, we measure friction force and normal force in the x and z directions using the F/T sensor. To understand the frictional effect, we used multiple types of surfaces such as plastic sheets (pl), plastic coated wood sheets (wo), paper sheets (pa) and aluminum sheets (al). We also tested the gripper with (compressed) and without load (relaxed) conditions to understand the role of linear spring-based prismatic joint ( Fig. 6d).

5.6 Computer vision system

We applied a color segmentation method to detect and extract the tool’s grasp and orientation using the OpenCV library (Fig. 7). The input images from the Realsense depth camera (D435) are converted to hue, saturation and value (HSV). The upper and the lower bounds on the colors of the tool are provided during in range process. The function of structuring element and morphologyEx are produced for the initial segmented image from the HSV image. Before finding the contour, a black image with the same input size is defined and a white rectangular box is included in the image for preparing the workspace boundary. The output image from morphologyEx multiplies the black image so that only the tools on the workspace are considered for the segmentation. The next phase is finding a contour area for the tool segmentation, which enables the information for calculating grasp position (mean of segmentation area) and orientation of the tool using minAreaRect function.

Fig. 7
figure 7

Computer vision-based grasping block diagram

Full size image

5.7 Manual and autonomous mode

The manual mode, which a human operator controls, is used to grasp a surgical tool. Firstly, the environment of the mode is set on the fixed tool, gripper position, and orientation. Then the UR5 robotic arm is moved to a specific place with the predefined trajectory such that the gripper is close to the tool. Last, a motor of the gripper, controlled by two fingers, is operated to grasp the tool by human command using Arduino.

In contrast, in the autonomous mode, tools are placed on the table freely and segmented and detected by the vision system. The system operated on Kinectv2 using the OpenCV library with a color segmentation algorithm. As the segmentation and grasping location tasks are complete, the position and orientation of a tool are transferred to the arm by ROS communication. When the arm is close to the tool, two fingers are controlled by the motor to grasp the tool. We set the initial and end position of jaws empirically by counting motor rotation. We took motor counter feedback from the attached encoder based on the segmented surgical tool image. The motor also is communicated by ROS with Arduino. The entire process above is done automatically.

Additionally, in manual mode grasping, we must set the environment, such as the fixed position and orientation of the tool and gripper, motor counter, etc. Otherwise, the task would fail if the tool is not placed exactly. In contrast, the autonomous mode only considers that the tool should be placed in the workspace, so the robot could easily segment and find a grasping point to pick up the tools.

6 Results and discussion

In this section, we reported the results in two parts. First, a quantitative analysis is performed to examine the proposed gripper (SurgGrip) model. The second is the grasping of different objects to show the performance.

6.1 Quantitative analysis

Fig. 8
figure 8

The four experimental results: a the analysis of workspace from maximum and minimum displacement of left and right fingertips, b the success rate of grasping each surgical tool, c the evaluation of reaction force with and without linear and torsional springs to the rigid surface, d the estimation of pulling force from surgical tools, e the measurement of the friction and normal forces with and without load on various surfaces (Pa\(_{wo}\) and Pa\(_{w}\) are without and with a load on the paper surface. Wo\(_{wo}\), and Wo\(_{w}\): are without and with a load on the plastic-coated wood surface. Pl\(_{wo}\), and Pl\(_{w}\) are without and with a load on the plastic surface. Al\(_{wo}\), and Al\(_{w}\) are without and with a load on the aluminum surface), f the evaluation of friction force on the paper surface

Full size image

6.1.1 Workspace

The gripper is designed and developed for grasping various surgical instruments. Therefore, the instruments’ width is essential in defining the gripper’s workspace. Based on the information in Fig. 8a, the maximum gap is 92 mm, and the minimum gap is 3 mm. Using the Aurora system, the maximum and minimum displacement between two fingers are 72.1 mm and 5.1 mm (Fig. 8a). This result shows that the maximum workspace is less than the maximum gap between fingers. Also, the maximum workspace we set has a function to prevent the finger’s breakage since the gripper could be open to a wider one. Moreover, the soft pads can compensate for the minimum gap during grasping.

6.1.2 Success rate

As shown in Fig. 8b, the gripper succeeds in grasping 10 out of 12 surgical tools (83.33% in both manual and autonomous modes). The success rate of SurgGrip is higher than 67 % and 80% including and excluding Tool 1 and Tool 9. The grasping failure of Tool 1 and Tool 9 may be due to these reasons: (1) they are longer tools so it is difficult to find stable a grip position, which incurs grasping failure; (2) the encoder’s positional count error causes mismatches in reading fingertip gap position; and (3) the larger object width: restricts the overlap between the two fingertips, which reduces the holding force. We also think that recognizing the center of mass and object reflection from the complex tool shape incur incorrect grasp position, orientation and poor segmentation quality, which could be one of the reasons for failure.

6.1.3 Table contact reaction force with/without springs

The spring mechanism plays a crucial role in the proposed gripper because it should contact the surface of the table for the given tasks. When the gripper pushes to the surface with the springs, the F/T sensor quasi-statically goes to − 20 N and keeps the arm movement stable. However, as the gripper without spring reaches the surface, the sensor goes to − 120 N abruptly. The state of the arm then enters emergency mode due to the high reaction force from the table and possible arm failure (Fig. 8c). As a result, the spring mechanism keeps the gripper stable and creates a safe interaction with a rigid environment.

6.1.4 The pulling force measurement

The gripper’s grasping force has been measured in a vertical direction to evaluate its load carrying capability. We measured the average vertical pulling force (5.95 N) from the experiment, with only ten selected surgical tools after estimating the SurgGrip success rate (Fig. 8d). The result means that our gripper could lift up to 606.73 g mass. This is sufficient to grasp almost all surgical instruments. However, the gripper has some manufacturing and prototyping errors between the joints, which can be improved by stronger 3D printable materials.

6.1.5 Friction and normal force measurement

We tested the friction and normal force on various surfaces (see the setup section for further details). The experimental setup is shown in Fig. 8e. We validated that the most favorable (lowest friction force) surface for grasping is the plastic-coated wood surface which requires friction force without a load of 1.10 N and a load of 2.13 N. The paper-based table is the unfavorable surface for grasping, which requires the highest friction force with a load of 3.91 N and without a load of 2.93 N. Moreover, the transient zone in Fig. 8f shows that the linear-spring-based joint helps to absorb the contact reaction and adapt to the surface texture.

6.2 Qualitative analysis

6.2.1 Grasping demonstrations on flat and nonflat surfaces

Compliance and safety are fundamental properties for grasping surgical tools. Therefore, SurgGrip has been demonstrated by considering different sizes, shapes, and irregularities of the objects. As it is possible to observe from Fig. 9, SurgGrip can grasp most surgical tools except Tool 1 and 9. Moreover, a combination of rigid and soft material, passive actuation with linear and torsion springs, and the fingertip’s enveloping mechanism play a vital role in these experiments’ success. The compression spring-based prismatic joint demonstrated adaptability and agility, which we validated through testing it on the nonflat surface (slope: 8.5\(^{\circ }\)) in Fig. 9b. Based on the are on the surface, the length of the linear and torsion spring is automatically adapted to the table so that the gripper can grasp the tools with better stability.

Fig. 9
figure 9

The demonstration of SurgGrip’s performance with grasping success (from (1) to (12)) except (1) and (9) they failed) of the surgical tools in the flat and nonflat surface. a Flat surface: Grasping with a computer vision system by automatically selecting green circle as grasp position and blue box as orientation in the binary images b Nonflat surface: Grasping with surface adaptation

Full size image

6.2.2 Object grasping time

We measured object grasping time in manual (average 19.94 s) and autonomous mode (average 15.81 sec) to confirm the time-consumed during grasping. The autonomous mode could be completed 21% faster than the manual mode with the same success rate. During measuring the time, the manual mode fixed the position and orientation of the tool. In contrast, the autonomous mode is used to grasp the tools with random position and orientation using vision, which is more desirable while testing it in real-time.

7 Conclusions and future works

This paper presents the design, characterization, and testing of the SurgGrip, which grasps surgical tools using computer vision-based control. In our design, linear and torsional springs allow damping of the reaction force from the table to protect the gripper. Soft pads provide better grasping and holding of the tools and allow the fingertips (mortise and tenon) to grasp various surgical tools from thin to thick with a high degree of adaptability. In Fig 10, we have shown that there are few studies that have focused on surgical instrument manipulation. These authors [13, 14] have performed several works, but the mechanism is based on magnetic actuation, which limits its capability to only ferromagnetic tools.

We evaluate our proposed design experimentally (quantitatively and qualitatively) and demonstrate it through actual tasks (see Fig. 9 and supplementary video). The results show that a gripper can prevent a surface collision, grasp, and lift most of the surgical instruments perform stable grasping with thin objects. The proposed work aims to use the gripper for general purposes in hospitals with safer interaction as a robot assistant for handing over tools to doctors and nurses, cleaning, and sterilization. We also believe that our gripper could be used for non-surgical applications (reflected through the big red color in Fig. 10) such as cleaning workspace and picking non-surgical tools.

Fig. 10
figure 10

Comparative study of SurgGrip with the current state of the art. We compared object types and gripper materials. To confirm the object types, we divided them into two categories, which are Flat objects (FO) and Nonflat objects (NFO) with surgical instruments (SI). Then, the grippers are compared for grasping between soft, hybrid, and rigid materials

Full size image

In the future, we want to improve the vision algorithm and material properties of the gripper to manipulate the SurgGrip more efficiently. We also aim to develop improved design by adding reconfigurability, tendon-driven actuation and intricate softer design and developing sophisticated kinematic and dynamic models. The reconfigure soft robotic gripper is a promising design, and it can be integrated into SurgGrip design to obtain better performance [33, 34]. Furthermore, we also want to implement deep reinforcement learning for more agile and dexterous grasping of surgical instruments, which we will develop first in simulation and then test in real-time hospital environments [35, 36].