Registration-Free Simultaneous Catheter and Environment Modelling

Abstract

Endovascular procedures are challenging to perform due to the complexity and difficulty in catheter manipulation. The simultaneous recovery of the 3D structure of the vasculature and the catheter position and orientation intra-operatively is necessary in catheter control and navigation. State-of-art Simultaneous Catheter and Environment Modelling provides robust and real-time 3D vessel reconstruction based on real-time intravascular ultrasound (IVUS) imaging and electromagnetic (EM) sensing, but still relies on accurate registration between EM and pre-operative data. In this paper, a registration-free vessel reconstruction method is proposed for endovascular navigation. In the optimisation framework, the EM-CT registration is estimated and updated intra-operatively together with the 3D vessel reconstruction from IVUS, EM and pre-operative data, and thus does not require explicit registration. The proposed algorithm can also deal with global (patient) motion and periodic deformation caused by cardiac motion. Phantom and in-vivo experiments validate the accuracy of the algorithm and the results demonstrate the potential clinical value of the technique.

This work was supported by the FP7-ICT (601021) and the EPSRC (EP/L020688/1). Dr. Stamatia Giannarou is supported by the Royal Society (UF140290).

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Endovascular catheter procedures are among the most common surgical interventions used to treat Cardiovascular Diseases (CVD). Being minimally invasive, these procedures extend the range of patients able to receive interventional CVD treatment to age groups with high risks for open surgery [1]. However, the challenge associated with minimising access incisions lies in the increased complexity of catheter manipulations, which is mainly caused by the loss of direct access to the anatomy and the poor visualisation of the surgical site [2]. Thus, the 3D structure of the vasculature needs to be recovered intra-operatively in order to model the interaction between the catheter and its surroundings and assist catheter navigation.

The current clinical approaches to endovascular procedures mainly rely on 2D guidance based on X-ray fluoroscopy and the use of contrast agents [3]. An alternative imaging modality that does not depend on ionising radiation or nephrotoxic contrast agents is intravascular ultrasound (IVUS) [4]. In [5, 6], the 3D shape of the vessel is reconstructed by registering IVUS images to angiography data in the 3D space, but still involves x-ray radiation and the use of contrast agents. Reconstruction only based on IVUS requires assumptions on pose of the transducer [7]. Recently, Simultaneous Catheter and Environment Modelling (SCEM) [8] has been proposed to reconstruct the 3D vessel shape by fusing IVUS and electromagnetic (EM) sensing data. This framework has been enhanced in [9] with SCEM+ for more robust and real time 3D vessel reconstruction. This has been achieved by formulating the 3D vessel reconstruction as a real-time nonlinear optimisation problem by considering the uncertainty in both the IVUS contour and the EM pose, as well as vessel structure information from pre-operative CT data. Both of the above frameworks rely on accurate pre-registration between the EM and CT data. However, in practice this is challenging as the registration is performed using external markers which can cause large errors [10], and requires updating every time the patient moves.

To this end, this paper proposes a registration-free simultaneous catheter and environment modelling method for endovascular navigation. This framework advances SCEM+ as it does not require any prior information about the EM-CT registration and can deal with global motion (e.g. patient motion) and periodic vessel deformation caused by the cardiac cycle. A novel optimisation framework has been formulated which incorporates the relative pose between the EM and CT data and allows this pose to be updated online. The uncertainty of the EM-CT registration is reduced incrementally and accurate 3D vessel reconstructions are estimated. In addition, the proposed algorithm can deal with global motion by introducing an anchored EM sensor and periodic deformation is overcome by gating the IVUS images and EM data to the same phase of the cycle using electrocardiogram (ECG) signal. Detailed validation on phantom and in-vivo datasets has been performed to compare the performance of the proposed framework to SCEM and SCEM+ and demonstrate its accuracy and robustness to global motion.

2 Methods

Similarly to SCEM+ [9], in this work, the 3D vessel reconstruction is formulated as a real-time nonlinear optimisation problem by considering the uncertainty in both the IVUS contour and the EM pose, as well as vessel structure information from pre-operative CT data. The transformation between the EM and CT coordinate systems is required. Advancing SCEM+, to avoid the need for pre-registering the EM and CT coordinate systems, the relative pose between the EM and CT data is incorporated in the optimisation framework and updated online. Robustness to global motion is achieved by including in the optimisation the pose of an externally anchored 6DoF EM sensor.

At the $i^{th}$ frame, suppose $\mathbf P ^E_i$ is the 6DoF pose reported from the EM sensor on the catheter tip and $\varSigma _{E}$ its corresponding uncertainty, $\mathbf C ^I_i = [(\mathbf c ^I_1)^T, \ldots , (\mathbf c ^I_n)^T]^T$ is the contour of the inner vessel wall extracted from the IVUS image which consists of a set of n boundary points and $\varSigma _{I} = \text{ diag }(\varSigma _1, \ldots , \varSigma _n)$ is its corresponding uncertainty, and $(\mathbf P ^A_i, \varSigma _{E})$ is the observation from the anchored EM sensor. Then the proposed algorithm can be mathematically formulated as the following nonlinear optimisation framework:

$$\begin{aligned} \begin{aligned} \underset{\mathbf{R , \mathbf P _i, \mathbf A _i}}{\arg \!\min } \sum _{j=1}^n&\Vert \mathbf c ^I_j - R_\mathbf{P _i}(R_\mathbf{A _i}^T(R_\mathbf{R }^T \; \mathbf c ^C_j + \mathbf T _\mathbf{R })+ \mathbf T _\mathbf{A _i} - \mathbf T _\mathbf{P _i}) \Vert ^2_{\varSigma ^{-1}_j} \\ +&\Vert \mathbf P ^E_i - \mathbf P _i \Vert ^2_{\varSigma ^{-1}_{E}} + \Vert \hat{\mathbf{R }}_{i-1} - \mathbf R \Vert ^2_{\varSigma ^{-1}_{R_{i-1}}} + \Vert \mathbf P ^A_i - \mathbf A _i \Vert ^2_{\varSigma ^{-1}_{E}}. \end{aligned} \end{aligned}$$

(1)

In the state vector, $\mathbf P _i = \{R_\mathbf{P _i},\mathbf T _\mathbf{P _i}\}$ is the current catheter pose in the EM coordinate frame, in which $R_\mathbf{P _i}$ and $\mathbf T _\mathbf{P _i}$ are the rotation matrix and the translation vector respectively, $\mathbf A _i = \{R_\mathbf{A _i},\mathbf T _\mathbf{A _i}\}$ is the pose of the anchored EM sensor and $\mathbf R = \{R_\mathbf{R },\mathbf T _\mathbf{R }\}$ is the relative pose from the anchored EM sensor to the CT coordinate frame, which corresponds to the EM-CT registration. $\mathbf C ^C_i = [(\mathbf c ^C_1)^T, \ldots , (\mathbf c ^C_n)^T]^T$ is the vessel contour computed from the pre-operative data as the cross section of the CT model and the plane defined by the catheter pose $\mathbf P _i$ transformed with $\mathbf A _i$ and the registration pose $\mathbf R$. Here, the IVUS images and EM data used in the optimisation are gated to be at the same phase of the cardiac cycle by the ECG signal. The anchored EM sensor is introduced only to deal with global motion and we assume that at each phase of the cardiac cycle, the relative pose between the anchor sensor and the vessel does not change.

The first term in (1) transforms the contour $\mathbf C ^C_i$ from the CT to the IVUS coordinate frame and minimises the difference between the contour extracted from the IVUS image and the contour computed from the pre-operative data, weighted by the uncertainty of the IVUS contour $\varSigma _{I}$. The second term in (1) minimises the difference between the catheter pose and the pose reported by the EM sensor, weighted by the uncertainty of the EM pose $\varSigma _{E}$. The first two terms in (1) are similar to SCEM+ [9], but with $\mathbf R$ and $\mathbf A _i$ included in the state vector.

The third term in the objective function (1) aims to minimise the difference between the EM-CT registration pose $\mathbf R$ in the state vector and the optimal solution of the registration pose $\hat{\mathbf{R }}_{i-1}$ from the previous frame, weighted by the corresponding covariance matrix $\varSigma _{R_{i-1}}$ computed from the proposed algorithm. Here, $(\hat{\mathbf{R }}_{i-1}, \varSigma _{R_{i-1}})$ from the $(i-1)^{th}$ frame is used as the observation in the optimisation of the $i^{th}$ frame. The fourth term in the objective function is to minimise the difference between the anchored EM pose in the state vector and the observation of the EM pose reported from the anchored sensor, weighted by the uncertainty of the EM sensor.

The optimal solution of the optimisation formulated in (1) can be obtained iteratively by using the Gauss-Newton method, where in the $k^{th}$ iteration

$$\begin{aligned} \left\{ \begin{aligned} \mathbf R ^{k+1}&= \mathbf R ^k+\varDelta ^k_R \\ \mathbf P ^{k+1}_i&= \mathbf P ^k_i+\varDelta ^k_P \\ \mathbf A ^{k+1}_i&= \mathbf A ^k_i+\varDelta ^k_A \\ \end{aligned} \right. , ~~ \text{ where } ~ J^T \varSigma ^{-1} J \left[ \begin{aligned} \varDelta ^k_R \\ \varDelta ^k_P \\ \varDelta ^k_A \\ \end{aligned} \right] = J^T \varSigma ^{-1} \varepsilon . \end{aligned}$$

(2)

Here J is the linear mapping represented by the Jacobian matrix of the observation functions evaluated at $\mathbf R ^k$, $\mathbf P ^k_i$ and $\mathbf A ^k_i$, $\varSigma$ is the covariance matrix containing the uncertainties of all the observations, and $\varepsilon$ is the residual vector of all the observations as

$$\begin{aligned} J = \!\! \begin{bmatrix} \frac{\partial \mathbf C ^I_i}{\partial \mathbf R }&\frac{\partial \mathbf C ^I_i}{\partial \mathbf P _i}&\frac{\partial \mathbf C ^I_i}{\partial \mathbf A _i}\\ 0&\frac{\partial \mathbf P ^E_i}{\partial \mathbf P _i}&0\\ \frac{\partial \mathbf R _{i-1}}{\partial \mathbf R }&0&0\\ 0&0&\frac{\partial \mathbf P ^A_i}{\partial \mathbf A _i} \\ \end{bmatrix}\!\!, \varSigma = \!\! \begin{bmatrix} \varSigma _{I}&0&0&0\\ 0&\varSigma _{E}&0&0\\ 0&0&\varSigma _{R_{i-1}}&0\\ 0&0&0&\varSigma _{E}\\ \end{bmatrix}\!\!, \varepsilon = \!\! \begin{bmatrix} \mathbf C ^I_i - f(\mathbf R , \mathbf P _i, \mathbf A _i)\\ \mathbf P ^E_i - \mathbf P _i\\ \hat{\mathbf{R }}_{i-1} - \mathbf R \\ \mathbf P ^A_i - \mathbf A _i\\ \end{bmatrix} \end{aligned}$$

(3)

where $f(\cdot )$ combines all the observation functions in the first term in (1)

$$\begin{aligned} f(\mathbf R , \mathbf P _i, \mathbf A _i) = [\ldots ,(((\mathbf c ^C_j)^T R_\mathbf{R } + \mathbf T _\mathbf{R }^T) R_\mathbf{A _i} + \mathbf T _\mathbf{A _i}^T - \mathbf T _\mathbf{P _i}^T) R_\mathbf{P _i}^T,\ldots ]^T, j=1:n \end{aligned}$$

(4)

and $\frac{\partial \mathbf P ^E_i}{\partial \mathbf P _i} = \frac{\partial \mathbf R _{i-1}}{\partial \mathbf R } = \frac{\partial \mathbf P ^A_i}{\partial \mathbf A _i} = E_6$, where $E_6$ is the $6 \times 6$ identity matrix.

For real-time implementation, the residual in the first term of (1) can be replaced by the shortest distances to the pre-operative CT model. Thus, the objective function and its Jacobians related to the first term can be pre-calculated as the distance space and its gradient from the pre-operative data [9]. By the formulation of the optimisation problem, the state vector can be simply initialised by using the observations $\mathbf R ^0 = \hat{\mathbf{R }}_{i-1}$, $\mathbf P ^0_i = \mathbf P ^E_i$ and $\mathbf A ^0_i = \mathbf P ^A_i$.

After the optimal solutions of the EM-CT registration $\hat{\mathbf{R }}_i$, the current catheter pose $\hat{\mathbf{P }}_i$ and the anchored sensor pose $\hat{\mathbf{A }}_i$ are obtained, their corresponding covariance matrices $\varSigma _{R_i}$, $\varSigma _{P_i}$ and $\varSigma _{A_i}$ which present their uncertainty can also be computed by using the Schur complement

$$\begin{aligned} \left\{ \begin{aligned} \varSigma _{R_i}^{-1} = I_{RR} - \begin{bmatrix} I_{PR}\\ I_{AR}\\ \end{bmatrix}^T \begin{bmatrix} I_{PP}&I_{PA}\\ I_{AP}&I_{AA}\\ \end{bmatrix}^{-1} \begin{bmatrix} I_{PR}\\ I_{AR}\\ \end{bmatrix}\\ \varSigma _{P_i}^{-1} = I_{PP} - \begin{bmatrix} I_{RP}\\ I_{AP}\\ \end{bmatrix}^T \begin{bmatrix} I_{RR}&I_{RA}\\ I_{AR}&I_{AA}\\ \end{bmatrix}^{-1} \begin{bmatrix} I_{RP}\\ I_{AP}\\ \end{bmatrix}\\ \varSigma _{A_i}^{-1} = I_{AA} - \begin{bmatrix} I_{RA}\\ I_{PA}\\ \end{bmatrix}^T \begin{bmatrix} I_{RR}&I_{RP}\\ I_{PR}&I_{PP}\\ \end{bmatrix}^{-1} \begin{bmatrix} I_{RA}\\ I_{PA}\\ \end{bmatrix}\\ \end{aligned} \right. ,\text{ where } \begin{aligned} I&= J^T \varSigma ^{-1} J\\&= \begin{bmatrix} I_{RR}&I_{RP}&I_{RA}\\ I_{PR}&I_{PP}&I_{PA}\\ I_{AR}&I_{AP}&I_{AA}\\ \end{bmatrix}. \end{aligned} \end{aligned}$$

(5)

Here $I_{RR}$, $I_{PP}$, $I_{AA}$ and $I_{RP} = I_{PR}^T$, $I_{RA} = I_{AR}^T$, $I_{PA} = I_{AP}^T$ are the parts of the information matrix I, which correspond to the variables $\mathbf R$, $\mathbf P _i$, $\mathbf A _i$ and their correlations, respectively.

The vessel reconstruction can be performed by transforming the IVUS contour $\mathbf C ^I_i$ into the CT coordinate frame $\mathbf C _i = [\mathbf c _1^T, \ldots , \mathbf c _n^T]^T$ using the optimal $\hat{\mathbf{R }}_i$, $\hat{\mathbf{P }}_i$ and $\hat{\mathbf{A }}_i$, with the corresponding covariance matrix $\varSigma _{C_i}$ as uncertainty:

$$\begin{aligned} \mathbf c _j = \hat{R}_\mathbf{R } (\hat{R}_\mathbf{A _i} (\hat{R}_\mathbf{P _i}^T \mathbf c ^I_j + \hat{\mathbf{T }}_\mathbf{P _i} - \hat{\mathbf{T }}_\mathbf{A _i})-\hat{\mathbf{T }}_\mathbf{R }), ~~ \varSigma _{C_i} = J_C \varSigma _S J_C^T \end{aligned}$$

(6)

where $J_C$ is the Jacobian matrix of $\mathbf C _i$ w.r.t the registration pose $\mathbf R$, catheter pose $\mathbf P _i$, anchor pose $\mathbf A _i$ as well as the IVUS contour $\mathbf C ^I_i$ respectively, and $\varSigma _S$ contains their covariance matrices on its diagonal

$$\begin{aligned} J_C = \begin{bmatrix} \frac{\partial \mathbf C _i}{\partial \mathbf R }~ \frac{\partial \mathbf C _i}{\partial \mathbf P _i}~ \frac{\partial \mathbf C _i}{\partial \mathbf A _i}~ \frac{\partial \mathbf C _i}{\partial \mathbf C ^I_i} \end{bmatrix},~~ \varSigma _S = \text{ diag }(\varSigma _{R_i}, \varSigma _{P_i}, \varSigma _{A_i}, \varSigma _{I}). \end{aligned}$$

(7)

At the end of the $i^{th}$ frame, the optimal solution of the EM-CT registration pose together with the corresponding uncertainty $(\hat{\mathbf{R }}_i, \varSigma _{R_i})$ computed by (5) are used as one of the observations in the $(i+1)^{th}$ frame.

In the proposed algorithm, the uncertainty of the EM-CT registration pose is initialised with zero information as $\varSigma ^{-1}_{R_0} = 0_6$ (where $0_6$ is a $6 \times 6$ zero matrix) at the first frame to ensure that the proposed algorithm only uses the information from IVUS, EM and the pro-operative data. Since the EM-CT registration is incrementally estimated from IVUS and EM data, the result of the registration will not be very accurate at the very beginning. As more parts of the vessel are observed by IVUS, the EM-CT registration is updated intra-operatively and becomes more accurate. By using the formulation above, the information from both the IVUS contour $(\mathbf C ^I_i, \varSigma _{I})$ and the EM pose $(\mathbf P ^E_i, \varSigma _{E})$ at the $i^{th}$ frame are transferred and accumulated in the covariance matrix $\varSigma _{R_i}$ of $\hat{\mathbf{R }}_i$, which means all the information of IVUS and EM from the $1^{st}$ to the $i^{th}$ frame is summarised in $\varSigma _{R_i}$, and is used in the $(i+1)^{th}$ frame as an integrated observation $(\hat{\mathbf{R }}_i, \varSigma _{R_i})$.

3 Results

3.1 Monte-Carlo Simulation

First, simulated data generated from a CT model with known EM-CT registration, and perfect EM poses and IVUS contours as ground truth were used to assess the accuracy of the proposed algorithm w.r.t the observation noise. Different levels of zero mean Gaussian noise were added to the ground truth EM poses and to the IVUS contours and were used as observations to the proposed algorithm. For each noise level, 25 runs were performed and the mean pose and reconstruction errors are shown in Fig. 1(left). In Fig. 1(right), the changes of the error of the EM-CT registration pose during the vessel reconstruction are shown with the $2\sigma$ bound from the corresponding uncertainty estimation, when 0.1 rad noise is added to the rotation and 1 mm noise to the translation of the EM pose, and 1 mm noise to the IVUS contour. It can be seen that the reconstruction errors remain small in the presence of noise and the error and uncertainty of EM-CT registration reduced quickly.

Fig. 1.

Monte-Carlo simulation: (left) the accuracy of catheter pose, vessel reconstruction and EM-CT registration pose w.r.t different levels of noise on the observations of EM poses and IVUS contours, (right) the reduction of error (black lines) and uncertainty ($2\sigma$ bounds shown in blue lines) of the EM-CT registration pose.

3.2 Phantom Experiments

The Static Case: A static HeartPrint$^{\textregistered }$ (Materialise, Leuven, Belgium) aortic phantom was used to compare the proposed algorithm against SCEM and SCEM+. Three setups with different EM-CT registrations were used and for each setup 5 datasets with catheter insertions and pullbacks were generated. The mean errors of the vessel reconstruction are shown in Fig. 2(left). Since the ground truth of EM-CT registrations cannot be obtained, the registrations estimated by the proposed algorithm were compared to the ones computed by using CT markers which were used in the SCEM and SCEM+ algorithms. As shown in Fig. 2(left), the proposed algorithm can achieve the similar accuracy of vessel reconstruction (around 0.3 mm) to SCEM+, but without EM-CT registration.

Fig. 2.

Accuracy of phantom experiments: (left) the static case using HeartPrint phantom, (right) with global motion and periodic deformation using the silicone phantom.

Global Motion and Periodic Deformation: A Silicone aortic phantom connected to a pump was used and periodic deformation and different global motions were simulated. For the global motion, the catheter was first inserted, and the EM field generator was moved before pulling the catheter back. For the periodic deformation, the pump simulated the cardiac motion, and its signal was used to gate the IVUS images in the proposed algorithm. For each cardiac cycle, the 3D shape of the vessel was reconstructed at the same phase of the cycle. Experiments with static phantom, global motion, periodic deformation, and global motion+periodic deformation were performed and for each case five experiments were done. The result of the proposed algorithm with global motion is shown in Fig. 3, and quantitative evaluation of the accuracy of the vessel reconstruction is presented in Fig. 2(right). From the results we can see that the proposed algorithm can successfully deal with global motion and periodic deformation motion achieving about 0.45 mm accuracy.

Fig. 3.

Vessel reconstruction results of the silicone phantom with global motion: (a) pre-operative CT model, (b) result of SCEM shows the changes of the EM-CT registration, (c) result of the proposed algorithm coloured by the error of reconstruction in mm, and (d) the catheter tip poses found using SCEM (red) and the proposed algorithm (black).

3.3 In-vivo Experiments

In-vivo experiments in a swine model with global motions were also performed to validate the proposed algorithm. A segmented CT scan provided the triangular surface mesh of the aorta. Seven CT markers were attached to the body of the swine, but as shown in Fig. 4(b), the EM-CT registration with CT markers has large error. In total 4 pullbacks were performed. The IVUS was gated by the ECG to deal with cardiac motion, and the results of the proposed algorithm are shown in Fig. 4(c), (d) and Table 1. For the 4 pullbacks, the mean errors of vessel reconstruction are 0.80, 0.83, 0.71, 0.68 mm, respectively.

Fig. 4.

Results of in-vivo experiments in swine model: (a) CT and IVUS image, (b) SCEM results show the large error of EM-CT registration by using CT markers and the global motion between Pullback-1 and Pullback-2, the results of Pullback-1 (c) and Pullback-2 (d) by the proposed algorithm.

Table 1. Vessel reconstruction error of in-vivo experiments (in mm)

4 Conclusion

This paper presents an intra-operative, real-time, registration-free 3D vessel reconstruction approach based on nonlinear optimisation using IVUS, EM and pre-operative data. Phantom and in-vivo experiments show that the proposed algorithm can achieve accurate vessel reconstruction, and, unlike the SCEM and SCEM+ methods, the explicit registration between the EM system and pre-operative data is not required. The use of external CT markers for registration can cause large errors and by removing this requirement, the proposed method can be easily integrated clinically, without interrupting the workflow. The algorithm runs in real-time with around 200 fps (on one core of an Intel i7-2600 CPU @3.4GHz) and is also robust to global motion and periodic deformation. In conclusion, the algorithm proposed in the paper can be deployed to improve endovascular navigation without the need of explicit EM-CT registration.