Keywords

1 Introduction

Minimally invasive therapies such as biopsy, brachytherapy, radiofrequency ablation and cryoablation involve the insertion of multiple needles into the patient [1–3]. Accurate placement of the needle tip can result in reliable acquisition of diagnostic samples [4], effective drug delivery [5] or target ablation [2]. When the clinicians maneuver the needle to the target location, the needle is likely to bend due to the tissue-needle or hand-needle interaction, resulting in suboptimal placement of the needle. Mala et al. [3] have reported that in nearly 28 % of the cases, cryoablation of liver metastases was inadequate due to improper placement of the needles among other reasons. We propose to develop a real-time navigation system for better guidance while accounting for the needle bending caused by the needle-tissue interactions.

Many methods have been proposed to estimate the needle deflection. The most popular class of methods is the model-based estimation [6–8]. Roesthuis et al. proposed the virtual springs model considering the needle as a cantilever beam supported by a series of springs and utilized Rayleigh-Ritz method to solve for needle deflection [8]. The work of Dorileo et al. merged needle-tissue properties, tip asymmetry and needle tip position updates from images to estimate the needle deflection as a function of insertion depth [7]. However, since the model-based estimation is sensitive to model parameters and the needle-tissue interaction is stochastic in nature, needle deflection and insertion trajectory are not completely repeatable. The second type of estimation is achieved using an optical fiber based sensor. Park et al. designed an MRI-compatible biopsy needle instrumented with optical fiber Bragg gratings to track needle deviation [4]. However, the design and functionality of certain needles, such as cryoablation and radiofrequency ablation needles, do not allow for instrumentation of the optical fiber based sensor in the lumen of the needle. The third kind of estimation strategy was proposed in [9], where Kalman filter was employed to combine a needle bending model with the needle base and tip position measurements from two electromagnetic (EM) trackers to estimate the true tip position. This approach can effectively compensate for the quantification uncertainties of the needle model and therefore be more reliable. However, this method is not feasible in the MRI environment due to the use of MRI-unsafe sensors. In this work, we present a new fusion method using an optical tracker at the needle’s base and an MRI gradient field driven EM tracker attached to the shaft of the needle. By integrating the sensor data with the angular springs model presented in [10], the Kalman filter-based fusion model can significantly reduce the estimation error in presence of needle bending.

2 Methodology

2.1 Sensor Fusion

Needle Configuration.

In this study, we have used a cone-tip IceRod® 1.5 mm MRI Cryoablation Needle (Galil Medical, Inc.), as shown in Fig. 1. A frame with four passive spheres (Northern Digital Inc. and a tracking system from Symbow Medical Inc.) is mounted on the base of the needle, and an MRI-safe EndoScout® EM sensor (Robin Medical, Inc.) is attached to the needle’s shaft with 10 cm offset from the tip set by a depth stopper.

Fig. 1.
figure 1

Cryoablation needle mounted with Optical and EM sensor and a depth stopper.

Full size image

Through pivot calibration, the optical tracking system can provide the needle base position \( P_{Opt} \) and the orientation of the straight needle \( O_{Opt} \). The EM sensor obtains the sensor’s location \( P_{EM} \) and its orientation with respect to the magnetic field of the MR scanner \( O_{EM} \).

Kalman Filter Formulation.

The state vector is set as \( x_{k} = [P_{tip\left( k \right)} , \dot{P}_{tip\left( k \right)} ]^{T} \). The insertion speed during cryoablation procedure is slow enough to be considered as a constant. Therefore, the process model can be formulated in the form of \( x_{k} = Ax_{k - 1} + w_{k - 1} \) as follows:

$$ \left[ {\begin{array}{*{20}c} {P_{tip(k)} } \\ {\dot{P}_{tip(k)} } \\ \end{array} } \right] = \underbrace {{\left[ {\begin{array}{*{20}c} {I_{3} } & {T_{S} I_{3} } \\ {0_{3} } & {I_{3} } \\ \end{array} } \right]}}_{{\begin{array}{*{20}c} {transition } \\ {matrix A} \\ \end{array} }}\left[ {\begin{array}{*{20}c} {P_{tip(k - 1)} } \\ {\dot{P}_{tip(k - 1)} } \\ \end{array} } \right] + \left[ {\begin{array}{*{20}c} {\frac{{T_{s}^{2} }}{2}I_{3} } \\ {T_{s} I_{3} } \\ \end{array} } \right]{\ddot{\textit{P}}}_{tip(k)} $$
(1)

where \( T_{S} \), \( I_{3} \), \( 0_{3} \) stand for the time step, 3-order identity matrix and 3-order null matrix. \( P_{tip(k)} \), \( \dot{P}_{tip(k)} \), \( {\ddot{\textit{P}}}_{tip(k)} \) represent the tip position, velocity, acceleration, respectively. The acceleration element \( [\frac{{T_{s}^{2} }}{2}I_{3} , T_{s} I_{3} ]^{T} {\ddot{\textit{P}}}_{tip(k)} \) is taken as the process noise, denoted by \( w_{k - 1} \sim {\mathcal{N}}(0,{\mathcal{Q}}) \), where \( {\mathcal{Q}} \) is the process noise covariance matrix.

When considering the needle as straight, the tip position was estimated using the three sets of data as follows: \( TIP_{Opt} \) (using \( P_{Opt} \), \( O_{Opt} \) and needle length offset), \( TIP_{EM} \) (using \( P_{EM} \), \( O_{EM} \) and EM offset), and \( TIP_{OptEM} \) (drawing a straight line using \( P_{Opt} \) and \( P_{EM} \), and needle length offset). When taking the needle bending into account, we can estimate the needle tip position using the angular springs model with either the combination of \( P_{EM} \), \( P_{Opt} \), and \( O_{opt} \) (\( TIP_{EMOptOpt} ) \) or the combination of \( P_{Opt} \), \( P_{EM} \) and \( O_{EM} \) (\( TIP_{OptEMEM} ) \), which are formulated in (2) and (3).

$$ P_{EMOptOpt} = g_{1} \left( {P_{EM} ,P_{Opt} ,O_{Opt} } \right) $$
(2)
$$ P_{OptEMEM} = g_{2} \left( {P_{Opt} ,P_{EM} ,O_{EM} } \right) $$
(3)

In our measurement equation \( z_{k} = Hx_{k} + v_{k} \), as \( z_{k} \) is of crucial importance for the stability and accuracy, \( z_{k} = [g_{1} \left( {P_{opt} ,P_{EM} ,O_{opt} } \right), g_{2} \left( {P_{opt} ,P_{EM} ,O_{EM} } \right),TIP_{EM} ]^{T} \) is suggested by later experiments. Accordingly, H is defined as in (4):

$$ {\text{H}} = \left[ {\begin{array}{*{20}c} {\begin{array}{*{20}c} {I_{3} } & {O_{3} } \\ {I_{3} } & {O_{3} } \\ \end{array} } \\ {\begin{array}{*{20}c} {I_{3} } & {O_{3} } \\ \end{array} } \\ \end{array} } \right] $$
(4)

The measurement noise is denoted as \( v_{k} \sim {\mathcal{N}}(0,{\mathcal{R}}) \), where \( {\mathcal{R}} \) is the measurement noise covariance matrix. For finding the optimal noise estimation, we used the Nelder-Mead simplex method [11].

2.2 Bending Model

In order to estimate the flexible needle deflection from the sensor data, an efficient and robust bending model is needed. In [10], three different models are presented, including two models based on finite element method (FEM) and one angular springs model. Further in [8] and [12], a virtual springs model was proposed, which took the needle-tissue force interaction into consideration. In [5] and [9], a kinematic quadratic polynomial model is implemented to estimate the needle tip deflection. Since we assume that the deflection is planar and caused by the orthogonal force acting on the needle tip, we have investigated multiple models and here we present the angular springs formulation to model the needle.

Angular Springs Model.

In this method, the needle is modeled into n rigid rods connected by angular springs with the same spring constant k, which can be identified through experiment. Due to the orthogonal force acting on the needle tip, the needle deflects causing the springs to extend. The insertion process is slow enough to be considered as quasi-static, therefore the rods and springs are in equilibrium at each time step. Additionally, for the elastic range of deformations, the springs behave linearly, i.e., \( \tau_{i} = k \cdot q_{i} \), where \( \tau_{i} \) is the spring torque at each joint. The implementation of this method is demonstrated in Fig. 2, and the mechanical relations are expressed as in (5).

Fig. 2.
figure 2

Angular springs model, taking 5 rods as an example

Full size image
$$ \left\{ {\begin{array}{*{20}c} {kq_{5} = F_{tip} l} \\ {kq_{4} = F_{tip} l(1 + \cos q_{5} )} \\ {kq_{3} = F_{tip} l\left[ {1 + \cos q_{5} + \cos (q_{5} + q_{4} )} \right]} \\ {kq_{2} = F_{tip} l\left[ {1 + \cos q_{5} + \cos (q_{5} + q_{4} ) + \cos (q_{5} + q_{4} + q_{3} )} \right]} \\ {\begin{array}{*{20}c} {kq_{1} = F_{tip} l[1 + \cos q_{5} + \cos (q_{5} + q_{4} ) + \cos (q_{5} + q_{4} + q_{3} )} \\ { + \cos (q_{5} + q_{4} + q_{3} + q_{2} )]} \\ \end{array} } \\ \end{array} } \right. $$
(5)

The Eq. (5) can be written in the form of \( k \cdot\Phi = F_{tip} \cdot {\text{J}}(\Phi ) \), where \( \Phi = \left[ {q_{1} , q_{2} , \ldots , q_{n} } \right] \), and \( {\text{J}} \) is the parameter function calculating the force-deflection relationship vector. In order to implement this model into the tip estimation method in (2) and (3), one more equation is needed for relating sensor input data with (5). As the data of \( P_{EM} ,P_{Opt} ,O_{opt} \) and \( P_{opt} ,P_{EM} ,O_{EM} \) are received during insertion, the deflection of the needle can be estimated as:

$$ d_{EM} = l \cdot [\sin q_{1} + \sin \left( {q_{1} + q_{2} } \right)] $$
(6)
$$ d_{base} = l \cdot \left[ {\sin q_{3} + \sin \left( {q_{3} + q_{2} } \right)} \right] $$
(7)

where \( d_{EM} \) represents the deviation of the EM sensor from the optical-measured straight needle orientation and \( d_{base} \) stands for the relative deviation of the needle base from the EM measured direction.

To estimate the needle deflection from \( P_{EM} ,P_{Opt} ,O_{opt} \) or \( P_{opt} ,P_{EM} ,O_{EM} \), a set of nonlinear equations consisting of either (5) (6) or (5) (7) needs to be solved. However, as proposed in [10], the nonlinear system of (6) can be solved iteratively using Picard’s method, which is expressed in (8). Given the needle configuration \( \Phi ^{t} \), we can use the function J to estimate the needle posture at the next iteration.

$$ \Phi ^{t + 1} = k^{ - 1} J\left( {\Phi ^{t} } \right)F_{tip} $$
(8)

For minor deflections, it only takes less than 10 iterations to solve this nonlinear equations, which is efficient enough to achieve real-time estimation.

However, the implementation of Picard’s method requires the \( F_{tip} \) to be known. In order to find the \( F_{tip} \) using the sensor inputs, a series of simulation experiments are conducted and linearly-increasing simulated tip force \( F_{tip} \) with the corresponding \( d_{EM} \), \( d_{base} \) are collected. The simulation results are shown in Fig. 3. Left.

Fig. 3.
figure 3

Left: Tip force and deflection relation: tip force increases with 50 mN intervals. Right: Static tip bending experiment setup at MRI entrance.

Full size image

A least square method is implemented to fit the force-deviation data with a cubic polynomial. Thereafter, to solve the needle configuration using \( P_{EM} ,P_{Opt} ,O_{opt} \) and \( P_{Opt} ,P_{EM} ,O_{EM} \), the optimal cubic polynomial is used first to estimate the tip force from the measured \( d_{EM} \) and \( d_{base} \), and then (5) is solved iteratively using (8).

3 Experiments

In order to validate our proposed method, we designed the static tip bending experiment, which was performed at the isocenter and 650 mm offset along z-axis from the isocenter (entrance) of MRI shown in Fig. 3. Right. The experiment is conducted in two steps: first, the needle tip was placed at a particular point (such as inside a phantom marker) and kept static without bending the needle. The optical and EM sensor data were recorded for 10 s. Second, the needle’s tip remained at the same point and the needle was bent by maneuvering the needle base, with a mean magnitude of about 40 mm tip deviation for large bending validation and 20 mm for small bending validation. Similarly, the data were recorded from both sensors for an additional 20 s. Besides, needle was bent in three patterns: in the x-y plane of MRI, y-z plane and all directions, to evaluate the relevance between EM sensor orientation and its accuracy.

From the data collected in the first step, the estimated needle tip mean position without needle deflection compensation can be viewed as the gold standard reference point \( TIP_{gold} \). In the second step, the proposed fusion method, together with other tip estimation methods, was used to estimate the static tip position, which was compared with \( TIP_{gold} \). The results are shown in Fig. 4. For large bending, error of \( TIP_{Opt} \), \( TIP_{EM} \) and \( TIP_{fused} \) is 29.23 mm, 6.29 mm, 3.15 mm at isocenter, and 39.96 mm, 9.77 mm, 6.90 mm at MRI entrance, respectively. For small bending they become 21.00 mm, 3.70 mm, 2.20 mm at isocenter, and 16.54 mm, 5.41 mm, 4.20 mm at entrance, respectively.

Fig. 4.
figure 4

Top: Single experiment result. Each scattered point represent a single time step record. The left-side points represent the estimated tip positions using different methods. The light blue points in the middle and dark blue points to the right represent the raw data of EM sensor locations and needle base positions respectively. The black sphere is centered at the gold standard point, and encompasses 90 % of the fused estimation points (black). Lines connect the raw data and estimated tip positions of a single time step. Bottom: From left to right: large bending experiment at isocenter, large-entrance, small-isocenter, small-entrance. X axis, from 1 to 6, stand for \( TIP_{fused} \), \( TIP_{EM} \), \( TIP_{OptEMEM} \), \( TIP_{EMOptOpt} \), \( TIP_{OptEM} \), \( TIP_{Opt} \), respectively. Y axis indicates the mean estimation error (mm) and each dot represents a single experiment result.

Full size image

4 Discussion

By comparing the \( TIP_{fused} \) with \( TIP_{Opt} \) instead of \( TIP_{EM} \), it should be noted that the EM sensor is primarily used to augment the measurements of the optical sensor and compensate for its line-of-sight problem. Although EM sensor better estimates the needle tip position in presence of needle bending, it is sensitive to the MR gradient field nonlinearity and noise. Therefore, its performance is less reliable when performing the needle insertion procedure at the MRI entrance.

Although quantifying the range of bending during therapy is difficult, our initial insertion experiments in a homogeneous spine phantom using the same needle demonstrated a needle bending of over 10 mm. Therefore, we attempted to simulate a larger bending (40 mm tip deviation) that could be anticipated when needle is inserted through heterogeneous tissue composition. However, as small bending will be more commonly observed, validation experiments were conducted and demonstrated consistently better estimation using the data fusion method.

From Fig. 4 Bottom, we find that the green dots, which represent bending in the x-y plane, exhibit higher accuracy of the EM sensor, thus resulting in a better fusion result. For large bending experiment in the x-y plane at the entrance, the mean error of \( TIP_{Opt} \), \( TIP_{EM} \) and \( TIP_{fused} \) are 28.22 mm, 5.76 mm, 3.40 mm, respectively. The result suggests that by maneuvering the needle in the x-y plane, the estimation accuracy can be further improved.

It should be noted that the magnitude of estimation errors using fusion method still appears large due to the significant bending introduced in the needle. When the actual bending becomes less conspicuous, the estimation error can be much smaller. In addition, the estimation error is not equal to the overall targeting error. It only represents the real-time tracking error in presence of needle bending. By integrating the data fusion algorithm with the 3D Slicer-based navigation system [13], clinicians can be provided with better real-time guidance and maneuverability of the needle.

5 Conclusion

In this work, we proposed a Kalman filter based optical-EM sensor fusion method to estimate the flexible needle deflection. The data fusion method exhibits consistently smaller mean error than the methods without fusion. The EM sensor used in our method is MR-safe, and the method requires no other force or insertion-depth sensor, making it easy to integrate with the clinical workflow. In the future, we will improve the robustness of the needle bending model and integrate with our navigation system.