Abstract
Purpose
Fusion of pre/perioperative images and intraoperative images may add relevant information during imageguided procedures. In abdominal procedures, respiratory motion changes the position of organs, and thus accurate image guidance requires a continuous update of the spatial alignment of the (pre/perioperative) information with the organ position during the intervention.
Methods
In this paper, we propose a method to register in real time perioperative 3D rotational angiography images (3DRA) to intraoperative singleplane 2D fluoroscopic images for improved guidance in TACE interventions. The method uses the shape of 3D vessels extracted from the 3DRA and the 2D catheter shape extracted from fluoroscopy. First, the appropriate 3D vessel is selected from the complete vascular tree using a shape similarity metric. Subsequently, the catheter is registered to this vessel, and the 3DRA is visualized based on the registration results. The method is evaluated on simulated data and clinical data.
Results
The first selected vessel, ranked with the shape similarity metric, is used more than 39 % in the final registration and the second more than 21 %. The median of the closest corresponding points distance between 2D angiography vessels and projected 3D vessels is 4.7–5.4 mm when using the brute force optimizer and 5.2–6.6 mm when using the Powell optimizer.
Conclusion
We present a catheterbased registration method to continuously fuse a 3DRA roadmap arterial tree onto 2D fluoroscopic images with an efficient shape similarity.
Introduction
Transcatheter arterial chemoembolization (TACE) is a minimally invasive procedure to treat liver cancer (mostly hepatocellular carcinoma). In this procedure, a catheter is navigated toward a tumor via the femoral and hepatic artery, after which chemotherapeutic agents are injected. Currently, the interventionalist guides the catheter using singleplane 2D Xray (fluoroscopy), visualizing only the catheter (Fig. 1). Frequently, contrast is injected to visualize the arteries. Computed tomography angiography (CTA) or 3D rotational angiography (3DRA) are used pre/perioperatively to visualize the tumor and feeding arteries. The navigation of the catheter using only 2D fluoroscopy is hampered by the inability to continuously visualize the arterial tree.
Purpose of our work is to integrate information of the vasculature from pre/perioperative 3D images by fusing it with the intraoperative 2D Xray images. Such an approach enables a continuous uptodate roadmap and thus may improve the guidance during the procedure and consequently has the potential to reduce intervention time, radiation dose and contrast agent use.
2D–3D registration for improving image guidance has been studied in cardiac, cranial, abdominal and orthopedic procedures. An overview of 2D–3D registration methods is presented by Markelj et al. [13] and Liao et al. [10]. Following [13], 2D–3D registration methods can be classified as extrinsic, intrinsic and calibration based. Extrinsic methods use markers to register and update the registration [17]. Usually objects visible on Xray (e.g., small beads) are inserted close to the region of interest before 3D image acquisition. Intrinsic methods rely on anatomical structures such as bones or the vasculature and are generally intensity, gradient, feature based or a combination of them [13]. In abdominal interventions, the vasculature and catheters are mostly the only structures visible on 2D Xray images that can be used for registration. In cardiac [3, 4, 14, 21, 22], cranial [7, 15, 26, 27] and abdominal [5, 8, 9] interventions, vesselbased registration have been used between pre or perioperative 3D/4D CTA (computed tomography angiography), MRA (magnetic resonance angiography) or CBCT (conebeam computed tomography) and 2D DSA (digital subtraction angiography) or 2D fluoroscopies. The Xray acquisition can either be single plane or biplane. Rigid as well as nonrigid registration approaches for aligning the vessels from 3D/4D pre or perioperative images with those from DSA or fluoroscopy have been described. These approaches update the 3D vessels position with regard to the Carm but do not enable a continuous roadmap of the 3D vessels because continuous contrast agent injection during the intervention would be harmful to the patient. Calibrationbased methods can be used when the 3D perioperative image and the 2D images are acquired with the same device. For example, if the 3D position of the Carm is known accurately, it allows alignment of intraoperative 2D Xray images with perioperative 3D images. Atasoy et al. [2] and Ruijters et al. [23] use Carm information to update the registration between perioperative 3DRA (or CBCT) and 2D Xray. This approach has been demonstrated to work accurately in cranial procedures with no head movement. Utilization in abdominal interventions, however, is hampered by the respiratory motion, which invalidates the initial alignment. Atasoy et al. [2] proposed a semiautomatic method to follow one moving region of interest selected by a physician during the intervention (a part of a catheter) and to update the registration with this information. The transformation model contains inplane translation to correct for shifts caused by respiratory motion. In cardiac interventions, Ma et al. [12] used manual calibrationbased methods to achieve an initial alignment and then used features such as diaphragm/heart border, tracheal bifurcation or the catheter to correct for breathing motion. Another method was proposed by Luan et al. [11] for oral cancer treatment. They track the catheter tip with an electromagnetic sensor, reconstruct the catheter path and then register it with a preoperative image. Although tracking the 3D catheter tip tracking is valuable, breathing motion may hamper the reconstruction of the path in, e.g., the abdomen. Unlike most of the other methods, our previous method [1] performs a 2D/3D catheterbased registration using a 3DRA and the complete catheter visible in the 2D Xray images. It does not require 2D angiographic images nor user interaction for the initial alignment. However, computation times were not interactive, hampering interventional use.
The major contribution of our current work is to propose a method for generating an automatic continuous roadmap during abdominal catheterization using 2D/3D registration with singleplane 2D Xray images and perioperative 3DRA. This paper is an extension of our previous work: The metric for alignment has been improved, the registration is faster, and the evaluation has been performed on a larger set of data, containing synthetic images, clinical images and additional evaluation metrics.
Methods
The method is based on the registration of a 3D vessel tree with a 2D catheter shape. Therefore, in a preprocessing step, the arterial tree is extracted from the 3DRA image and the catheter shape position is determined from the singleplane fluoroscopic images. The extraction of the vessel tree itself is relatively straightforward for highcontrast 3DRA images. The segmentation of the catheter in the fluoroscopic images, albeit more challenging, has been subject of other studies [6, 16, 18, 19, 25, 28, 29]. These steps are not addressed in this paper.
Given the 3D vascular model and the 2D catheter centerline, the method consists of two steps (Fig. 2). First a shape similarity metric is used to find the vessel centerlines from the 3DRA that are most similar to the 2D catheter shape. Subsequently, a constrained 2D–3D registration is applied to find the corresponding rigid transformation between the 2D catheter and the 2D projections of the bestranked vessel centerlines. In the following, we first define our coordinate systems and transformations, and then we describe each registration steps.
Definitions
We define the following coordinate systems (CS) for our setup in the intervention room (Fig. 3):

\(\hbox {CS}_{\mathrm {w}}\), denotes the world 3D CS, with the origin at the isocenter of the Carm, and oriented along the Carm in its default position

\(\hbox {CS}_{\mathrm {det}}\), the detector 3D CS (Xray image plane)

\(\hbox {CS}_{\mathrm {fluoro}}\), the 2D CS of the fluoroscopic image

\(\hbox {CS}_{\mathrm {3DRA}}\), 3D CS of the 3DRA
Accordingly, the following coordinate transformations are defined:

\(T_{\mathrm {det\leftarrow w}}\), transformation from \(\hbox {CS}_{\mathrm {w}}\) to \(\hbox {CS}_{\mathrm {det}}\)

\(T_{\mathrm {proj}}\), conebeam projection from \(\hbox {CS}_{\mathrm {det}}\) to \(\hbox {CS}_{\mathrm {fluoro}}\)

\(T_{\mathrm {w\leftarrow 3DRA}}\), transformation that aligns the 3DRA space with the patient, from \(\hbox {CS}_{\mathrm {3DRA}}\) to \(\hbox {CS}_{\mathrm {w}}\)
\(T_{\mathrm {det\leftarrow w}}\) and \(T_{\mathrm {proj}}\) are known for each Xray image because the geometry and orientation of the Carm are known. \(T_{\mathrm {w\leftarrow 3DRA}}\) is unknown and is the result of our registration.
With the projection function \(F_\mathrm{proj}\) (in homogeneous coordinates):
we have a 3D point in the 3DRA space, \(p_{\mathrm {CS_{3DRA}}}\), which can be projected on the fluoroscopic image space \(\hbox {CS}_{\mathrm {fluoro}}\) using the following equation:
The catheter centerline extracted from a 2D fluoroscopic image is defined as an ordered set of \(n_C\) points:
where \(c_{\mathrm {i}} \in \mathbb {R}^{\mathrm {2}}\) are 2D points at the center of the catheter in \(\hbox {CS}_{\mathrm {fluoro}}\) and \(c_{\mathrm {1}}\) denotes the tip of the catheter.
The blood vessel tree centerline extracted from the 3DRA is represented as a directed tree:
where \(\mathcal {P}\) is the set of 3D points on the centerlines of the vessels in \(\hbox {CS}_{\mathrm {3DRA}}\) and \(\mathcal {E}\) is the set of directed edges between points. The root of \(G_{\mathrm {3D}}\) is in the aorta, and the branches of the tree are in the liver.
We define a vessel centerline \(V(p)\) as an ordered set of points in \(G_{\mathrm {3D}}\), from any point \(p \in \mathcal {P}\) along the directed edges to the root (Fig. 4):
where \(p_i \in \mathbb {R}^{\mathrm {3}}\) in \(\hbox {CS}_{\mathrm {3DRA}}\) and \(p_{n_P}\) is the root of \(G_{\mathrm {3D}}\).
Similarly, we define the 2D projection of the 3D vessel centerline \(V(p)\):
Additionally, we define \(V_{proj,T}(p,u)\) with \(u \in [0, V_l]\), a linearly interpolated version of the projected centerline, with \(V_l\) the length of the projected vessel centerline \(V(p)\).
Shapebased vessel centerline selection
The registration is performed on a vessel centerline running from a leaf to the root (Fig. 4). Before performing the registration, the vessels that are the most likely to contain the catheter are selected. This selection is based on the shape similarity metric between the catheter and the projected 3D vessel. The metric quantifies the alignment of the tangent vectors of the catheter and the projected vessel. Therefore, it is not sensitive to the distance between centerlines. The underlying assumption is that the orientation of the vasculature changes little between the 3DRA and the fluoroscopy acquisition, which is valid for our application. The shape similarity for a vessel from a point \(p \in \mathcal {P}\) is defined as:
where \(C_l\) is the length of the 2D catheter, \(I_{4}\) is the \(4\times 4\) identity matrix, and \(\overrightarrow{C}_{\mathrm {2D}}(u)\) (resp. \(\overrightarrow{V}_{\mathrm{proj},I_{4}}(p, u)\)) is the tangent of \(C_{\mathrm {2D}}\) (resp. \(V_{\mathrm{proj},I_{4}}(p)\)) at the position \(u\). \(C_{\mathrm {2D}}(0)\) is the tip of the catheter, and \(V_{\mathrm{proj},I_{4}}(p,0)\) is the possible location of the tip in the tree \(G_{\mathrm {3D}}\). \(S(p) \in [0, C_l]\) with \(C_l\) denoting the maximum similarity. As the catheter centerline is represented as a set of points, the integral over S is approximated by summing the dot products over all catheter positions, thereby interpolating the corresponding vessel positions (Fig. 5).
In order to select the leaf vessel centerline for which the registration needs to be performed, for each leaf \(l\), the maximum similarity over all points in V(l) is determined:
Based on the values of \(S_\mathrm{max}\), we selected the \(k\) leafs with largest \(S_\mathrm{max}\) for the registration. When several leafs share the same common part with the catheter, only one is kept.
Rigid 2D/3D registration with forward projection
To register the 2D catheter with the vessel centerline, we need to find the rigid transform \(T_{\mathrm {w\leftarrow 3DRA}}\) that yields the best match with the 2D catheter in \(\hbox {CS}_{\mathrm {fluoro}}\). We decompose the transformation as follows:
where \(T_{\mathrm {rot}}\) is a rotation matrix with three unknowns (Euler angles, \(\alpha \), \(\beta \) and \(\gamma \)), and \(T_{\mathrm {trans}}\) is a translation matrix with three unknowns (\(x\), \(y\), \(z\)) with the translations aligned in \(\hbox {CS}_{\mathrm {det}}\). A translation along the projection axis in \(\hbox {CS}_{\mathrm {det}}\) will only have a very minor effect in the projection. We therefore exclude \(z\) from the registration parameters, leaving us with a five degrees of freedom transformation.
Our registration metric is the sum of distances between points on the catheter and closest points on the projected leaf vessel centerline. The catheter tip \(c_1\) thereto is matched to the closest point of the projected vessel \(V(l^\mathrm{sel})\), where \(l^\mathrm{sel}\) is a leaf selected thanks to the shape similarity. The distance between the catheter tip and the vessel centerline \(V(l^\mathrm{sel})\), given a rigid transformation T, is given by:
Each next point of the catheter is similarly matched with a point of the projected vessel. To ensure continuity of the vessel (and simultaneously reducing computation time), the search range is limited to only a few points proximal to the point closest to the previous catheter point. Thus, let \(p_\mathrm{prev} \in V(l^\mathrm{sel})\) be the point matched with \(c_{i1}\), then the distance to the subsequent catheter point \(c_i\) is defined as:
where \([p_\mathrm{prev}, \ldots p_{\mathrm{prev} + h}]\) are the \(h + 1\) consecutive points in \(V(l^\mathrm{sel})\), starting at \(p_\mathrm{prev}\), and \(h\) is determined such that all points in that range are within a distance \(d_\mathrm{max}\) of \(p_\mathrm{prev}\).
Given these definitions, the final registration metric \(M\) of our registration is a weighted sum of these distances (Fig. 6):
where \(W(x)\) is a weight function \(\in [0,1]\) and \(c_i, c_1\) is the length of the catheter between \(c_1\) and \(c_i\). As the registration accuracy close to the tip is more important than at the proximal part of the catheter, we use a weight to decrease the distance values that are further from the tip. We use a Gaussian with an offset:
where \(\sigma \) is a parameter to control how fast the weight decrease (Fig. 7).
This metric \(M\) has two advantages: first, it is fast because we only look for the closest point in a specific neighborhood; second, by only matching points that are locally connected the continuity of the vessel centerline is respected.
Lastly, the final transformation is the one with the smallest cumulative distance:
where \(T\) represents the 5 degrees of freedom rigid transformation matrix.
Every selected leaf vessel centerline \(V(l^\mathrm{sel})\) is registered, and the pair (\(V(l^\mathrm{best})\), \(T_{\mathrm {w\leftarrow 3DRA}}\)) with the optimal similarity \(M\) is kept.
Experiments
We performed two series of experiments. In the first one, we used clinical data from TACE interventions. As we do not have a ground truth available in these data, we rather evaluate the registration on the alignment of the vessels distal to the catheter tip using angiography and we investigate the effect of varying parameter settings. In the second experiment, registration has been performed on the same clinical data, but with the catheter position simulated. The simulation allows us to have a ground truth for the catheter position. The error in the localization of the registered catheter tip position was used for evaluation.
Data
We retrospectively acquired anonymized data of 19 TACE interventions (Table 1). The 16 first sets were acquired in the Erasmus MC, University Medical Center, Rotterdam, the Netherlands, between 2012 and 2014 in two different intervention rooms with angiographic Carm systems (Xper Allura, Philips Healthcare, Best, the Netherlands). The last three sets were acquired in the Hôpitaux Universitaires Henri Mondor, Créteil, Paris, France, and the Ospedale di Circolo e Fondazione Macchi, Varese, Italy. For each intervention, we have a set of images consisting of one 3DRA image where the catheter was inside the hepatic artery, a set of fluoroscopic sequences with contrast agent and a set of digital subtraction angiographies (DSA) (Fig. 8). In these sequences, both the catheter and a part of the vasculature distal to the catheter tip are visible (by using the contrast agent). For each sequence, we selected the image with most of the vasculature visible and we manually annotated both the 2D catheter centerline and the 2D vasculature centerlines. The 3D arterial tree from 3DRA is segmented with a semiautomatic method based on thresholding and skeletonization [24].
We divided our data in two different groups depending on the 3DRA acquisition: complete and incomplete acquisition. Incomplete acquisition occurs when the patient’s liver is not aligned with the Carm rotation isocenter. In that case, the aorta and the hepatic artery are not visible in the 3DRA which hampers the registration.
Implementation
The method described was implemented in C++ and run on a computer with a 3.4 Ghz Intel Core i7. We set \(k\), the number of selected leaf vessel centerline to register, to 5.
In order to minimize our metric, we evaluated two different optimizers: a brute force and the Powell optimizer [20]. The brute force is exhaustive and is more likely to find the global minimum, whereas Powell is faster but because of its local search is more likely to converge to local minimum.
Our brute force optimizer has \(n=7\) iterations, and for each iteration \(i\), the search space is centered at the minimum found in iteration \(i  1\) with an interval size \(s_i = c*s_{i1}\) whereby coefficient \(c\) was set to 0.5 (Fig. 9). Each dimension in the search space interval is subdivided in \(d=7\) steps. We set the initial intervals \(s_0\) of our brute force search to \(\pm \)50 mm for \(x\) and \(y\) and \(\pm \)7\(^\circ \) for \(\alpha \), \(\beta \) and \(\gamma \). These intervals are sufficiently large to capture breathing motion.
For the Powell optimizer, we use a twostage approach. We first optimize the inplane translation and subsequently use that translation to initialize the full 5 degrees of freedom registration.
Clinical data and parameter optimization
In the first experiment, we investigated optimal parameter settings for the method and evaluated how well the resulting registration aligns the vasculature distal to the catheter tip. To determine optimal parameter settings and evaluate the effect of changing parameters, we applied the method with a large set of different settings (\(\lambda \), \(\sigma \) and \(d_\mathrm{max}\)) in a leaveoneout crossvalidation scheme (determine the optimal parameter values over the set containing all patients except the one on which the evaluation is done). We tested the following settings; \(\lambda \): \(0\), \(0.1\), \(0.2\), \(0.3\); \(\sigma \): \(20\), \(40\), \(60\), \(80\), \(100\); \(d_\mathrm{max}\): \(10\), \(20\), \(30\), \(40\), \(50\) mm. Because the 2D catheter centerline and 3D blood vessel tree are discretized, we also investigated the effect of using different samplings: \(1.5\), \(3\) and \(6\) mm between each point. Finally, computation time is recorded.
As we do not have a ground truth for the registration, we used the vasculature visible on the angiographies as a reference. Thus, for validating, we compared how well the projected 3DRA matches the arteries visible in the projection images. To this end, we projected the 3DRA vasculature on the 2D image with the registered \(T_{\mathrm {w\leftarrow 3DRA}}\) and we computed the closest corresponding points distance between the projected 3D vasculature and the 2D vasculature. The most relevant region for the roadmapping is the area close to the catheter tip; we therefore only evaluated in a circular region (3 cm radius) around the catheter tip. Before computing the distance, we manually labelled the vessels such that distances are computed between corresponding vessels (Fig. 10). To prevent bias in this assignment, the manual annotation was done without registration, thus only using the initial projection of the 3DRA. To aid in the annotation, the observer could manually register 3DRA and 2D vasculature by changing translations and rotations of the 3DRA. Vessels that cannot be manually adequately linked were not used in the evaluation. Using the labelled corresponding vessels, we computed the closest corresponding points distance for each pair of vessels (excluding distances to endpoints). The distance was computed both from the 2D angiovessel and from its registered projected 3D vessel pair (Fig. 11). Our evaluation metric for one image is the average of distances over all pairs of vessels.
Clinical data with a simulated catheter
When evaluating the registration on patient data, no accurate ground truth registration is available. We therefore also evaluated our method on simulated data with a known ground truth. To stay as close as possible to the reality, our simulation was based on the clinical data (Table 1) where we used the 3D extracted vessel tree in the 3DRA registered space (using \(T_{\mathrm {w\leftarrow 3DRA}}\)) as well as the fluoroscopic sequences with their projection information. We choose the position of the 3D simulated catheter tip in the 3D vessel tree such that it matches the catheter tip in the fluoroscopic image after a registration of the 3D vessel tree to the fluoroscopic image. Next, we extract a 3D simulated catheter centerline following the 3D vessel path from the tip to the root and project it on the fluoroscopic image using the angles from the fluoroscopic sequences acquired. Those ground truth projections are used to quantify the accuracy of the registration results. To simulate a smooth catheter that may be stretching the vessel and that may be partially outside the vasculature, we smooth the 2D projection centerline with a Gaussian kernel (with a standard deviation \(\sigma _\mathrm{simu}\)).
We applied random transformations \(T_{\mathrm {w\leftarrow 3DRA}}\) to the 3DRA volume, divided over three sets depending on the magnitude of the transformations and the Gaussian smoothing parameter (Table 2).
In this experiment, we used the algorithm parameters obtained in the previous leaveoneout crossvalidation. We quantified the Euclidean distance \(e_d\) between the known projected 3D tip and the registered projected 3D tip. We also computed the longitudinal \(l_d\) and orthogonal \(o_d\) distances from the point of view of the known tip (Fig. 12).
Results
Clinical data and parameter optimization
The results of the leaveoneout crossvalidation with the two optimizers Powell and brute force, 3 mm sampling and all images are presented in Table 3. The optimal parameter settings are consistent over the leaveoneout experiments. Based on these results, unless noted otherwise, we used 3 mm sampling and with Powell: \(\lambda = 0.2\), \(\sigma = 20\), \(d_\mathrm{max} = 40\) mm and with brute force: \(\lambda = 0.1\), \(\sigma = 80\), \(d_\mathrm{max} = 20\) mm.
The average paired vessels distance results are summarized in Fig. 13. Results are grouped depending on the 3DRA acquisition. Compared with our previous method [1], using the same sampling, the new method has a median that is smaller. Also the brute force optimizer performs better than Powell and is more robust. With the complete 3DRA set, medians are around 5 mm for brute force and 6 mm for Powell. Varying the sampling density between 1.5 and 6 mm does not clearly affect the accuracy.
In Table 4, we present the distribution of the best registered leaf vessel centerline \(V(l^{\mathrm {best}})\) among the \(k\) ranked and selected leaf vessel centerlines. This table shows that the best registration result is generally obtained with the leaf vessel centerlines that ranked best using the shapebased metric \(S_\mathrm{max}\). This demonstrates that the metric can effectively be used to reduce the number of potential vessels to register. The low percentage for the fifth ranked vessel also suggests that the choice of \(k = 5\) is a good compromise between registration speed and the robustness of the method. The first ranked leaf vessel centerline is also the one giving the best registration for 39–49 % of the images.
Figure 14 shows the paired vessels distance after the registration as function of the distance from the catheter tip. The median becomes less accurate after 3 cm between 1 and 5 mm.
The sampling and the local distance \(d_\mathrm{max}\) in the metric \(M\) are the parameters that affect the computation time. We show in Fig. 15 the relation between accuracy and computation time. Brute force is slower than Powell optimization. Our previous method [1] did the registrations with the samplings: 1.5, 3 and 6 mm in 95, 25 and 10 s, which is at least twice as slow as the brute force of our current approach. After 20 mm, \(d_\mathrm{max}\) does not seem to change the accuracy with brute force. Powell is less stable with both the sampling and \(d_\mathrm{max}\).
Figure 16 shows examples of correct and incorrect registrations. We note that when there is a small part of the catheter visible on the image, the optimizers are more likely to yield misregistrations because of the lack of information. A correct tip position and distance metric \(M\) do not imply a perfect match of the vasculature due to the deformation of the liver and the catheter.
We visually inspected all registrations from the complete 3DRA set with optimal settings and labelled them as correct, visually close and incorrect for both the registered tip and the registered vessels distal to the tip (Table 5). Visually close implies that the registration is sufficient to know where the catheter is in the 3D vasculature, while incorrect is of no use for the intervention. For each incorrect case, we also report the likely cause of failure (Table 6). The two main reasons of wrong registrations are a too small part of the catheter visible in the 2D Xray image and large deformation of both the catheter and the vessels distal to it.
Clinical data with a simulated catheter
Figure 17 shows the distance between the real tip in the simulated catheter (without smoothing) and the tip after registration. Figure 18 shows the results with catheter smoothing. Without catheter smoothing, for the brute force optimizer, the median of the Euclidean distance \(e_d\) is below 1 mm whereas for Powell the distance is below 3 mm. With catheter smoothing, the registered tip is less accurate and less robust with both Powell and brute force optimization. The longitudinal and orthogonal distance are similar with slight, moderate or large transformation. Overall, the longitudinal distance is slightly more robust than the orthogonal.
Discussion and conclusion
We proposed and evaluated a method that enables a continuous roadmap during abdominal catheterization. The method registers a 3D vessel model obtained from 3DRA imaging using a catheter that is extracted from singleplane 2D Xray images with or without contrast agent. The method first selects the vessels using shape similarity and then rigidly registers the selected vessels to the catheter.
With the complete 3DRA set and optimal settings, the median of average paired vessels distances of the roadmap distal to the catheter tip and within a radius of 3 cm from the tip is 5.4 mm for the brute force optimizer and 5.2 mm for the Powell optimizer. The first selected vessel during shape similarity is used more than 39 % in the final registration and the second more than 21 %.
We investigated two optimizers for the registration approach: brute force and Powell. In our setup, with \(<\)200 ms computation time on average, the registration is real time with the Powell optimizer and a 3 or 6mm sample interval. Though the brute force optimizer is slower, it could be improved with parallelization and a dedicated implementation. The brute force optimizer tends to be more accurate and robust than the Powell optimizer. Powell is more sensitive to the initial position of the registration (ends up in local minima) as well as the length and distinct shape of the catheter.
The simulation experiments with catheter deformation demonstrate that the registration is robust for both optimizers with slight deformation. They also show that larger deformation leads to less accurate registration. In the simulated data, the longitudinal distance from the tip shows how well the tip is registered along the catheter direction. This distance is more significative than the orthogonal and is slightly more robust.
An important source of error in our experiments was the lack of vessel information in the 3DRA (especially missing the aorta and the hepatic artery), partly caused by the retrospective nature of our study. Optimization of the 3DRA acquisition protocol could remedy this. Another source of registration errors is the lack of information in the 2D Xray because only very short part of the catheter is visible. This shows the limitation of the method working with no prior knowledge other than the current image. This may be addressed, during the intervention, by slightly increasing the field of view, moving the patient table, or adding more a priori knowledge into the registration such as previous image registration transformations. If we take into account the previous registered transformations and the table motion (which in principle could be obtained from the Carm system, but is not available in our acquired fluoroscopic images), the registration should have a better initialization and thus use a smaller search space and both Powell and brute force optimizers will perform more robustly while reducing computation time.
Most related 2D/3D registration methods register angiography with CTA or 3DRA. As the complete vasculature is visible on both 2D and 3D images and nonrigid registration is performed, they reach submillimeter accuracies. Our method, dealing only with the catheter visible on the 2D image, has lower accuracy. However, we are interested in improving guidance, and the fusion provides a continuous roadmap of sufficient accuracy to the clinician to reliably estimate the catheter tip position in the 3D vasculature. As far as we know, in abdominal studies, the presented method can be compared only with the method proposed by Atasoy et al. [2]. They evaluate their method with the overlap of the 3DRA vessels onto the catheter. In our case, the overlap is our distance metric so a comparison will be biased toward our method.
During abdominal catheterization, knowledge of the position of the tip in the 3D vasculature is of crucial importance. Table 5 shows a small percentage of incorrect registered tip positions. This implies that if we use the presented fusion method as a roadmap, combining any of the optimizers, the resulting fused visualization is sufficient to guide the interventionists in localizing the tip and identifying the subsequent bifurcations, also in case of slight misalignment.
A robust automatic 2D catheter segmentation is required after initialization to integrate our method into the interventional workflow. The accuracy of the segmentation will influence the registration method. For example, Heibel et al. [6] obtain a median error of realtime automatic catheter tracking \(<\)1.5 pixels for abdominal fluoroscopies. Those results are sufficiently accurate for our registration.
Registration studies often lack ground truth for clinical data. In our case, this also prevented us to evaluate the accuracy of the registration method directly. However, both the simulation experiments and the validation with angiographic images demonstrate the good performance of the method.
In our current setup, each registration is independent from previous registrations. During continuous roadmapping, only slight motion should occur between two registrations. In the future, we intend to use previous registration results to further improve the robustness (especially when the visible catheter part is too small to do an accurate independent registration) and to limit the computation time by reducing the space search. A source of registration errors was due to large vessels and catheter deformation. A nonrigid registration to match the catheter deformation could also improve the accuracy close to the catheter tip.
To conclude, we presented a catheterbased registration method to fuse continuously 3DRA roadmap arterial tree onto 2D fluoroscopic images. We evaluated our work with clinical and simulated data demonstrating an efficient shape similarity and a median accuracy, evaluated on close by vessels, of 4.7–6.6 mm and below 4 mm on simulated experiments.
References
 1.
Ambrosini P, Ruijters D, Moelker A, Niessen WJ, van Walsum T (2014) 2d/3d catheterbased registration for image guidance in TACE of liver tumors. In: Information processing in computerassisted interventions. Springer, pp 246–255 (2014)
 2.
Atasoy S, Groher M, Zikic D, Glocker B, Waggershauser T, Pfister M, Navab N (2008) Realtime respiratory motion tracking: roadmap correction for hepatic artery catheterizations. In: Medical imaging. International Society for Optics and Photonics, p 691, 815 (2008)
 3.
Baka N, Metz C, Schultz C, van Geuns R, Niessen W, van Walsum T (2014) Oriented gaussian mixture models for nonrigid 2d/3d coronary artery registration. IEEE Trans Med Imaging 33(5):1023–1034
 4.
Baka N, Metz C, Schultz C, Neefjes L, van Geuns RJ, Lelieveldt BP, Niessen WJ, van Walsum T, de Bruijne M (2013) Statistical coronary motion models for 2d+ t/3d registration of Xray coronary angiography and CTA. Med Image Anal 17(6):698–709
 5.
Groher M, Zikic D, Navab N (2009) Deformable 2d–3d registration of vascular structures in a one view scenario. IEEE Trans Med Imaging 28(6):847–860
 6.
Heibel H, Glocker B, Groher M, Pfister M, Navab N (2013) Interventional tool tracking using discrete optimization. IEEE Trans Med Imaging 32(3):544–555
 7.
Hipwell JH, Penney GP, McLaughlin RA, Rhode K, Summers P, Cox TC, Byrne JV, Noble JA, Hawkes DJ (2003) Intensitybased 2d3d registration of cerebral angiograms. IEEE Trans Med Imaging 22(11):1417–1426
 8.
Jomier J, Bullitt E, Van Horn M, Pathak C, Aylward SR (2006) 3d/2d modeltoimage registration applied to tips surgery. In: Medical image computing and computerassisted interventionMICCAI 2006. Springer, pp 662–669 (2006)
 9.
Liao R, Tan Y, Sundar H, Pfister M, Kamen A (2010) An efficient graphbased deformable 2d/3d registration algorithm with applications for abdominal aortic aneurysm interventions. In: Medical imaging and augmented reality. Springer, pp 561–570 (2010)
 10.
Liao R, Zhang L, Sun Y, Miao S, Chefd’Hotel C (2013) A review of recent advances in registration techniques applied to minimally invasive therapy. IEEE Trans Multimed 15(5):983–1000
 11.
Luan K, Ohya T, Liao H, Kobayashi E, Sakuma I (2013) Vessel bifurcation localization based on intraoperative threedimensional ultrasound and catheter path for imageguided catheter intervention of oral cancers. Comput Med Imaging Graph 37(2): 113–122
 12.
Ma Y, King AP, Gogin N, Gijsbers G, Rinaldi C, Gill J, Razavi R, Rhode KS (2012) Clinical evaluation of respiratory motion compensation for anatomical roadmap guided cardiac electrophysiology procedures. IEEE Trans Biomed Eng 59(1):122–131
 13.
Markelj P, Tomaževič D, Likar B, Pernuš F (2012) A review of 3d/2d registration methods for imageguided interventions. Med Image Anal 16(3):642–661
 14.
Metz C, Schaap M, Klein S, Rijnbeek P, Neefjes L, Mollet N, Schultz C, Serruys P, Niessen W, van Walsum T (2012) Alignment of 4d coronary CTA with monoplane Xray angiography. In: Augmented environments for computerassisted interventions. Springer, pp 106–116 (2012)
 15.
Mitrović U, Spiclin Z, Likar B, Pernuš F (2013) 3d–2d registration of cerebral angiograms: a method and evaluation on clinical images. IEEE Trans Med Imaging 32(8):1550–1563
 16.
Pauly O, Heibel H, Navab N (2010) A machine learning approach for deformable guidewire tracking in fluoroscopic sequences. In: Medical image computing and computerassisted interventionMICCAI 2010. Springer, pp 343–350 (2010)
 17.
Pawiro SA, Markelj P, Pernuš F, Gendrin C, Figl M, Weber C, Kainberger F, NöbauerHuhmann I, Bergmeister H, Stock M, Georg D, Bergmann H, Birkfellner W (2011) Validation for 2d/3d registration I: a new gold standard data set. Med Phys 38(3):1481–1490
 18.
Petkovic T, Loncaric S (2010) Guidewire tracking with projected thickness estimation. In: 2010 IEEE international symposium on biomedical imaging: from Nano to Macro. IEEE, pp 1253–1256 (2010)
 19.
Petkovic T, Loncaric S (2010) Using Xray imaging model to improve guidewire detection. In: 2010 IEEE 10th international conference on signal processing (ICSP). IEEE, pp 805–808 (2010)
 20.
Powell MJ (1964) An efficient method for finding the minimum of a function of several variables without calculating derivatives. Comput J 7(2):155–162
 21.
RivestHenault D, Sundar H, Cheriet M (2012) Nonrigid 2d/3d registration of coronary artery models with live fluoroscopy for guidance of cardiac interventions. IEEE Trans Med Imaging 31(8):1557–1572
 22.
Ruijters D, ter Haar Romeny BM, Suetens P (2009) Vesselnessbased 2d–3d registration of the coronary arteries. Int J Comput Assist Radiol Surg 4(4):391–397
 23.
Ruijters D, Homan R, Mielekamp P, Van de Haar P, Babic D (2011) Validation of 3d multimodality roadmapping in interventional neuroradiology. Phys Med Biol 56(16):5335
 24.
Selle D, Preim B, Schenk A, Peitgen HO (2002) Analysis of vasculature for liver surgical planning. IEEE Trans Med Imaging 21(11):1344–1357
 25.
Slabaugh G, Kong K, Unal G, Fang T (2007) Variational guidewire tracking using phase congruency. In: Medical image computing and computerassisted interventionMICCAI 2007. Springer, pp 612–619 (2007)
 26.
Sundar H, Khamene A, Xu C, Sauer F, Davatzikos C (2006) A novel 2d–3d registration algorithm for aligning fluoro images with 3d preop ct/mr images. In: Medical imaging. International Society for Optics and Photonics, p 61,412K (2006)
 27.
Van der Bom I, Klein S, Staring M, Homan R, Bartels L, Pluim J (2011) Evaluation of optimization methods for intensitybased 2d–3d registration in Xray guided interventions. In: SPIE medical imaging. International Society for Optics and Photonics, p 796,223 (2011)
 28.
Wu X, Housden J, Ma Y, Razavi B, Rhode K, Rueckert D (2014) Fast catheter segmentation from echocardiographic sequences based on segmentation from corresponding Xray fluoroscopy for cardiac catheterization interventions. IEEE Trans Med Imaging PP(99):1
 29.
Yatziv L, Chartouni M, Datta S, Sapiro G (2012) Toward multiple catheters detection in fluoroscopic image guided interventions. IEEE Trans Inf Technol Biomed 16(4):770–781
Acknowledgments
This research is funded by Philips Healthcare, Best, The Netherlands. We thank the Hôpitaux Universitaires Henri Mondor, Créteil, Paris, France, and the Ospedale di Circolo e Fondazione Macchi, Varese, Italy, for providing image datasets.
Conflict of interest
Pierre Ambrosini, Daniel Ruijters, Wiro J. Niessen, Adriaan Moelker and Theo van Walsum declare that they have no conflict of interest.
Author information
Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Ambrosini, P., Ruijters, D., Niessen, W.J. et al. Continuous roadmapping in liver TACE procedures using 2D–3D catheterbased registration. Int J CARS 10, 1357–1370 (2015). https://doi.org/10.1007/s115480151218x
Received:
Accepted:
Published:
Issue Date:
Keywords
 Roadmap
 2D/3D Registration
 Singleplane Xray fluoroscopy
 3DRA
 Abdominal
 Breathing compensation