Introduction

The da Vinci (Intuitive Surgical Inc., CA, USA) is a notable surgical robot which is being used to facilitate complex minimally invasive surgery (MIS). Although the da Vinci is admittedly the most successful surgical robotic system to date, it suffers from a number of limitations. For instance, the lack of haptic feedback has not been satisfactorily resolved despite multiple attempts [1]. Additionally, the intrinsic accuracy of the system has been shown to be around 1 mm [2], which is insufficient for applications that require sub-millimetre accuracy, such as tissue scanning for optical biopsy. While higher accuracy has been achieved using the da Vinci, such approaches require additional markers and calibration steps [3].

Fig. 1
figure 1

One of the CYCLOPS endoscope robotic attachment designs [12]

Full size image

On the other hand, there is a demand from the surgeons to be able to utilize the imaging modality intraoperatively for accurate identification of malignant tissue to minimize unnecessary damage to healthy tissue. Optical biopsy using endomicroscopy (EM) provides the highest resolution images among current modalities and is potentially applicable to robotic surgeries as shown in [4]. EM relies on sub-millimetre diameter optical probes to obtain histological information. For accurate and reliable assessment of pre-cancerous lesions and their margins, large areas of tissue must be scanned and accurately reconstructed. The use of a large number of spatially distributed points is one way to cover wide tissue areas. At the same time, the force applied on the tissue by the scanning probe must remain within a very tight range for optimal signal acquisition and safety [5]. For the same reasons, the probe must be able to comply with tissue morphological variations. Research is being conducted and outcomes have been reported by measuring the contact force using a force sensor placed at the tip of an instrument [6], by using force adaptive device [7], or by using computer vision techniques such as edge detection or intensity recognition [3].

An alternative and simpler design, based on the use of a cable-driven parallel manipulator (CDPM), is proposed here for robotic EM. The mechanism is used to accurately control an EM probe and at the same time accurately measure tissue contact forces. The contact forces are derived simply by measuring the tension of the controlling cables/tendons. CDPMs exhibit several desirable qualities: high force transmission, large workspace, easy workspace reconfigurability, high dynamic capabilities, and low cost [8]. To demonstrate its potential and favourable characteristics, the proposed instrument is retro-fitted on a modified da Vinci robot instrument. Partial robotic hepatectomy is used as an exemplar target procedure, where EM can be very useful in identifying the malignant tissue margins intraoperatively [9, 10]. Therefore, the system is validated on bovine liver tissue. The following sections provide a description of the developed system and its principle of operation and conclude with experimental validation of its capabilities.

Instrument description

The concept of CDPM is introduced first followed by the design and specifications of the proposed instrument.

Cable-driven parallel robotic manipulator

The proposed instrument is based on a patented concept introduced in [11], named ‘CYCLOPS’, which describes a robotic attachment for endoscopes (Fig. 1). The CYCLOPS is based on a six-cable CDPM used to control an over-tube that can accommodate commercially available surgical instruments. The CDPM is supported within a deployable rigid or inflatable scaffold, which then becomes rigid or quasi-rigid upon deployment or inflation.

The CYCLOPS exhibits several desirable properties, which are inherent to most CDPM approaches. These include, dexterous manipulation, high accuracy, stability, simplicity and force sensitivity. The system is currently undergoing pre-clinical in vivo validation with a focus on colorectal Endoscopic Submucosal Dissection (ESD) [12]. Here, we use EM as an exemplar application to further highlight the advantages of the concept.

Fig. 2
figure 2

Conceptual design of the proposed instrument

Full size image
Fig. 3
figure 3

a Tension measurement per tendon and b contact force calculation

Full size image

Conceptual design of the proposed instrument

Unlike the six tendons per instrument used in the CYCLOPS, the proposed instrument uses four tendons to actuate a cylindrical over-tube with an outer diameter of 3.0 mm. The over-tube is used to accommodate an EM probe with an outer diameter of 2.8 mm. The four tendons can actuate the probe in three planar degrees of freedom (DoF). Figure 2 shows a schematic illustration of the concept.

Fig. 4
figure 4

The developed instrument retro-fitted on a standard da Vinci instrument base. On the right, the instrument is shown mounted on the da Vinci’s slave arm

Full size image
Fig. 5
figure 5

The test-setup to evaluate the contact force sensitivity of the system using a linear stage and ground truth load cell

Full size image

Force calculation and sensitivity

With reference to Fig. 2, the four-tendon configuration can be used to control the position of the probe and simultaneously sense the forces applied on it. The forces on the tip of the probe are estimated by measuring the tension \(T_i\) for each tendon i, as shown in Fig. 3a and (1). The calculated contact force \({ CF}_\mathrm{{cal}}\) is derived based on the instrument pose and the measured tensions, as shown in Fig. 3b and (2). The force sensitivity can be estimated by comparing the calculated forces \({ CF}_\mathrm{{cal}}\) with ‘ground truth’ forces CF, measured with a load cell placed at the interface with the probe tip. Discrepancies between \({ CF}_\mathrm{{cal}} \) and CF are expected due to static friction, elasticity, and hysteresis in the mechanism. \(T_0 \) is an initial preset tension to keep the over-tube stable before applying the external force.

$$\begin{aligned}&T_i =\frac{F_i }{2\sin \left( \frac{\pi }{4}\right) } \end{aligned}$$
(1)
$$\begin{aligned}&{ CF}_\mathrm{{cal}} =\sum \limits _{{\mathrm{i}}=1}^4 \left( {T_i -T_{i,0} } \right) \cos \theta _i \end{aligned}$$
(2)

Depending on the tissue type, the contact forces required for EM, can be below 1.0 N [13]. Therefore, the reduction of friction in the mechanical parts involved is essential to increase the forces sensitivity of the system. To reduce friction on the rounded corners where the tendons change direction on the scaffold, or where they interface with the load cells, we use Polytetrafluoroethylene (PTFE) tubes. Using pulleys is an alternative, which however takes more space and introduces more complexity. Moreover, through experimentation we have established that the difference between pulleys and PTFE tubes in terms of friction is negligible. For the proposed device, the tendons in use are 0.19 mm diameter UHMW-PE spectra wires (PowerPro, Shimano Inc., Japan), which can withstand up to 13 kgf.

Mechanical design of proposed instrument

All parts are mounted on a modified da Vinci instrument. Figure 4 shows the instrument. Four DC servo motors, 2232S024BX4 CCD 3830 (FAULHABER Inc., Daimlerstrabe, Germany) are used to control the tendons. With 3000 increments per revolution, a 1:25 gear ratio and a 10 mm tendon spool, the theoretical positional accuracy for each tendon is \(0.4~\upmu \hbox {m}\). Load cells (LCL-005, OMEGA Engineering, Inc, Manchester, UK) are used, which can measure up to 2.27 kgf with \(2~\hbox {mV/V} \pm 20\%\) rated output by a 5VDC excitation voltage. The system is controlled by a laptop PC in C++. Most structures have been manufactured in Poly-Lactic Acid (PLA) by a commercial 3D printer (Ultimaker2 Extended, Ultimaker Inc., Netherlands).

Experimental validation

The accuracy of force sensing is discussed by comparing the calculated contact force with the measured contact force in a static situation. The relation between contact forces and acquired EM images is evaluated in a simple 1 DoF axial movement. Subsequently, concurrent force sensing and image acquisition in a 2 DoF controlled movement is presented.

Fig. 6
figure 6

a Setup including the Cellvizio endomicroscope and b liver tissue sample

Full size image

Measurement of contact force

For experimental validation a testing rig has been created. The mechanism to test the contact force is shown in Fig. 5. A linear actuator, L2818S0604-T5X5 (Nanotec Electronic Inc., Feldkirchen, Germany) is placed at the bottom of the testing mechanism to simulate up–down contacting movement. Another load cell (LCL-005) is placed on the tip of the linear actuator and used to provide the ground truth. The position of the over-tube is set at the centre of the scaffold, where the tensions in all cables should be equal to satisfy the static equilibrium condition. The linear actuator is programmed to perform 10 consecutive 2 mm displacements, along the longitudinal axis of the probe. Both ground truth and calculated contact forces are acquired and evaluated.

Relation of contact force and EM image quality

To assess the capabilities of the system when applied to EM, the optimal applied force for adequate image quality should be assessed and compared to the force sensitivity of the proposed mechanism.

Experimental environment

The Cellvizio endomicroscope (Mauna Kea Technologies Inc., Paris, France) is used. An optical probe is inserted from the back-end of the instrument and placed into the over-tube to which the probe is fixed (Fig. 6a). A Coloflex probe with a confocal length of 55–65 \(\upmu \hbox {m}\) is used. The field-of-view diameter of the probe is approximately 0.5 mm.

Target tissue

A piece of healthy bovine liver was used as a sample tissue. As the normal liver does not have noticeable features on the surface, a piece of lens-cleaning paper (Thorlabs Inc., NJ, USA) was placed on top of the liver. The dimension of the tissue is approximately \(20\times 20\times 8~\hbox {mm}\), and both the tissue and the paper were dyed with 0.2% acriflavine hydrochloride solution to get a fluorescent effect (Fig. 6b).

Optimum contact force

As in EM there is a direct relation between the applied force and the image quality achieved, the optimum contact force for the tissue needs to be identified. This contact force can be determined by the ground truth load cell which is placed underneath the liver tissue (Fig. 7).

Fig. 7
figure 7

Test-setup used to determine the optimum contact force and determine the maximum force sensitivity during a 1 DoF motion

Full size image

Control method for image quality assessment

The over-tube is controlled autonomously by 0.01 mm incremental steps in the vertical direction. When the instrument detects contact with an object, the motors are stopped in holding mode. Contact is sensed through a rapid increase in the first derivative on the sensed cable tension values. After this point, the over-tube is moved backwards until the contact force becomes zero again. During this autonomous movement, ground truth force data and EM images are acquired and used for evaluation (Fig. 7). To assess whether the stiffness of the surface affects the relation between contact force and image quality, this experiment is conducted both on a rigid surface (PLA) as on liver tissue, each covered with lens testing paper.

Autonomous 2DoF control

Autonomous control is essential in a clinical case. Manual control would be difficult to achieve stable sub-mm accuracy required for vertical and horizontal control of the EM probe, based on a confocal length of 55–65 \(\upmu \hbox {m}\) and a field of view of approx. 0.5 mm, respectively. The liver tissue is used for testing the performance in 2 DoF movement using the inverse kinematics of the CDPM, based on a pre-defined surface scanning pathway. All pathways have been preliminarily programmed in \(\hbox {C++}\). The testing mechanism is the same as in “Relation of contact force and EM image quality” section. The difference is that we use the second derivative of tension data to detect contact force. Existing 2 DoF dynamic effects introduce changes in the first derivative even without any surface contact, and therefore additional threshold based on the second derivative is required to increase robustness of force detection. Threshold values P, \(Q_i\), and \(R_i\) (3, 4, 5) are used. P is a difference of the first derivative comparing the sum of tensions of tendon 1 and 2 with the sum of tensions of tendon 3 and 4. \(Q_i\) is the increase ratio of the first derivative of each tendon. \(R_i \) is the difference of the second derivative compared to a reference. For the tension T, the subscript i refers to the tendon number ranging from 1 to 4, and subscript t refers to the time at which the sample was collected: t being the current sample and \(t_0 \) and \(t_1 \) reference values at which the system was unloaded.

$$\begin{aligned}&P=\sum \limits _{i=1}^2 \frac{\mathrm{d}T_i }{\mathrm{d}t}-\sum \limits _{i=3}^4 \frac{\mathrm{d}T_i }{\mathrm{d}t} \end{aligned}$$
(3)
$$\begin{aligned}&Q_i =\frac{\mathrm{d}T_{i,t} }{\mathrm{d}t}\Bigg /\frac{\mathrm{d}T_{i,t_0 } }{\mathrm{d}t} \end{aligned}$$
(4)
$$\begin{aligned}&R_i =\frac{\mathrm{d}^{2}T_{n,t} }{\mathrm{d}t^{2}}-\frac{\mathrm{d}^{2}T_{i,t_1 } }{\mathrm{d}t^{2}} \end{aligned}$$
(5)

If the detected contact force is higher than the contact force required for obtaining an optimum quality image due to the diminished force sensitivity caused by the static friction, controlled step-back of the probe can be used to reach the required optimum force. The amount of stepping back needs to be determined empirically. During the 2 DoF autonomous movement, the ground truth force data and EM images are acquired for evaluation (Fig. 8). Six horizontal points are used to conduct effectively and repeatedly the same experiments. Consecutive mosaic images can be acquired between each horizontal point. At each point, the same control method as described in the previous section is applied, as shown in Fig. 8.

Fig. 8
figure 8

Pathway of the endomicroscope probe, when the contact force sensitivity is lower than the optimum contact force for high-quality images

Full size image
Fig. 9
figure 9

Contact force comparison. Area A shows the error between the calculated and ground truth contact force. Areas B1 and B2 indicate a discrepancy between measured and calculated contact forces as a result of hysteresis

Full size image

Results

Results of contact force measurements

Figure 9 shows the plots acquired by the procedure described in “Measurement of contact force”. The top graph illustrates how the four tensions change depending on the position of the end-effector. The bottom graph is the comparison between the ground truth contact force and the calculated contact force. The dotted area A shows that the calculated contact force is \(14.7\,\pm \, 2.0\%\) lower than the ground truth contact force. The dotted areas B1 and B2 show the hysteresis effects, in which the tensions did not return to the initial tension even after removing the external force applied by the actuator. It is considered that this discrepancy is due to static friction between the tendons and the PTFE and the elasticity of the structure. However, the overall performance is significantly stable over repeated motions, which demonstrates the high repeatability of the mechanism.

Results on the relation between contact force and EM image quality

Figure 10 shows EM images obtained using different contact forces. The contact force was measured by a load cell placed under the liver tissue. The image at 0.05 N shows the most noticeable contrast with clear edge features and was therefore selected as the optimal contact force value during subsequent experiments with the same liver tissue.

Fig. 10
figure 10

Contact force and respective endomicroscope image

Full size image
Fig. 11
figure 11

Contact force detection and endomicroscope image

Full size image

Figure 11 illustrates detection of the contact force and a relation between the contact force and images using the testing paper on a rigid surface. The moving average method was used to reduce the noise of the four tension values. Subsequently, by using the least square method, gradient values were derived as the first derivative. The red line shows the ground truth contact force which was measured on the rigid surface where the testing lens paper was placed. It is noticeable that the contact force was detected at approximately 0.2 N, at which point motor actuation was stopped. 0.2 N is the minimum contact force which the instrument could detect. This minimum force depends mainly on threshold conditions required to robustly detect forces. At the 0.2 N detecting point, the EM image shows certain features of the testing paper, but the image is blurred (Fig. 11(i)), as this value is higher than the earlier determined 0.05 N force required for acquiring the highest quality EM images. From this point, the EM probe was gradually stepped back from the paper at 0.01 mm steps, until the optimum force of 0.05 N was reached and the highest quality image was acquired (Fig. 11(ii)). The number of back-steps required to reach this optimum force was recorded. The same 0.2 N force sensitivity has been found while performing the 1 DoF movement on the liver tissue.

Fig. 12
figure 12

2DoF autonomous scanning control

Full size image

Results of autonomous 2 DoF control task

For the 2 DoF case, the average minimum detectable contact force on healthy liver tissue was approximately 0.6 N, which was larger than the 0.2 N found in the 1 DoF case due to the differences of the detecting algorithm. Similar to the 1 DoF controlled case, the minimum detectable contact force is higher than the required optimum contact force. Therefore, continuous images were taken while stepping back. The pathway in 2 DoF is shown in Fig. 8, and Fig. 12 illustrates the first three points of the 2 DoF movement extracting tension data and the first derivative data. CF (ground truth contact force measured on the object surface) in Fig. 12 shows that approximately 0.6 N contact force was detected during each stepping-in motion. Below the graph, representative images are shown. At the detecting point, EM image shows certain features of the testing paper, but the image was blurred (Fig. 12(i)). When the contacting force was approximately 0.05 N, the highest quality images were taken (Fig. 12(ii)). Additionally, clear images for mosaicking were also taken during the horizontal movement of the probe to the next point (Fig. 12(iii)). The 2 DoF point scanning continued for six points in total. Three additional cases showed minor differences in average detected contact force (0.62 N) when compared to the first case, and the optimum contact force (0.04 N) when compared to the determined force (0.05 N). However, there were no visible negative effects on the EM image quality.

Discussion

This paper shows a first proof-of-concept using a CDPM for safe and adequate contact force to acquire high-resolution EM images. While the detectable force sensitivity is high, the used healthy liver sample has shown to require an even higher force sensitivity for high-quality images. However, the proposed back-stepping technique effectively shows that the positional accuracy of the system is sufficiently high to acquire high-quality images. Additional bench-marking is required to fully validate the positional feed-forward accuracy of the end-effector. Hardware and software improvements can be implemented to further decrease friction and increase the detection sensitivity. It is noteworthy to mention that current results are found without any additional modeling in which the theoretical optimum tension distribution, dynamic and friction effects are taken into account. Implementing such models would further increase the force sensitivity of the system.

The large variability between different tissue pathologies, different tissue types, and different patients will make an adequate solution that is based on contact force alone as a metric for image quality unlikely. The combination of the current method and existing computer vision techniques would enable the system to autonomously and safely obtain high-quality EM images over large tissue sections. In such a case, no a priori information about the tissue is required to attain the optimum contact force for adequate images.

Further hardware developments will include a system with additional cables to allow for 5 Dof scanning of larger and complex surface area, while using the same algorithm. The force sensitivity is expected to reduce slightly, as the forces are distributed over more loadcells. Additional integration with the da Vinci and hybrid control methods can be used to increase the workspace of the instrument further. For clinical applicability, a deployable scaffold as used in the current CYCLOPS system needs to be incorporated.

While the research focuses primarily on EM imaging, it should be noted that the high force sensitivity can also be used for other clinical applications, including accurate control of a laser scalpel or a micro ultrasound probe. Haptic feedback and palpation are two other functionalities that the current system can be used for, giving the surgeon enhanced situational awareness and diagnostic tools.

Conclusion

An integration of a CDPM and a da Vinci instrument was introduced providing a viable solution for the safe and precise autonomous EM scanning. 0.2 and 0.6 N force sensitivity were found for 1 and 2 DoF image acquisition methods, respectively. In case of insufficient force sensitivity for specific tissues, the system is accurate enough to acquire high-quality images by gradually stepping back. As force sensing and accurate control have been technical challenges since the emergence of the surgical robots, this new instrument could offer an option not only for robotic EM but also for other applications which require force sensing and precise control.