HEX: a safe research framework for hybrid EMT X-ray navigation

Purpose Navigating with continuous X-ray provides visual guidance, but exposes both surgeon and patient to ionizing radiation, which is associated with serious health risks. Interleaving fluoro snapshots with electromagnetic tracking (EMT) potentially minimizes radiation. Methods We propose hybrid EMT + X-ray (HEX), a research framework for navigation with an emphasis on safe experimentation. HEX is based on several hardware and software components that are orchestrated to allow for safe and efficient data acquisition. Results In our study, hybrid navigation reduces radiation by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$71\%$$\end{document}71% with cubic, and by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$40\%$$\end{document}40% with linear error compensation while achieving submillimeter accuracy. Training points for compensation can be reduced by half while keeping a similar accuracy–radiation trade-off. Conclusion The HEX framework allows to safely and efficiently evaluate the hybrid navigation approach in simulated procedures. Complementing intraoperative X-ray with EMT significantly reduces radiation in the OR, increasing the safety of patients and surgeons.


Introduction
Continuous fluoroscopic imaging is the gold standard for navigation in endovascular surgery [1]. But although continuous fluoroscopic X-ray supports the mental model of the surgeon by providing visual guidance, associated radiation exposure poses a serious health risk [2]. Moving from continuous fluoroscopy to operator-controlled imaging (OCI) can significantly decrease the dose area product (DAP) and fluoroscopy time (FT) [2,3]. However, when no X-ray image is available, the surgeon would need to navigate blindly.
We have previously shown that complementing X-ray with electromagnetic tracking (EMT) has the potential to provide B Henry J. Krumb henry.john.krumb@gris.tu-darmstadt.de Bernhard Dorweiler bernhard.dorweiler@uk-koeln.de Anirban Mukhopadhyay anirban.mukhopadhyay@gris.tu-darmstadt.de 1 Computer Science Department, TU-Darmstadt, Fraunhoferstr. 5, 64283 Darmstadt, Germany 2 Vascular Surgery, UK Köln, Kerpener Str. 62" 50937 Cologne, Germany a fall-back navigation modality when no X-ray is available, while reducing radiation [4]. X-ray snapshots are taken at discrete time intervals, while EMT is used for continuous navigation. Such hybrid setting adheres to the as low as reasonably achievable (ALARA) principle [2], as only a minimal number of X-ray snapshots are taken and EMT is used for continuous navigation in between. However, combining X-ray and EMT requires accurate registration strategies and a framework to automate the image acquisition process in real time.
To our knowledge, a complete hybrid navigation approach with EMT and X-ray to reduce radiation has not yet been demonstrated. The idea itself is already discussed in literature [4,5], but so far only individual components of the hybrid pipeline are evaluated whereas the actual radiation saving of the full pipeline has not yet been investigated.
In this paper, we implement a complete hybrid setup by presenting the hybrid EMT + X-ray (HEX) framework, a safe research environment to investigate hybrid navigation in minimal invasive surgery. One major obstacle that we overcome in this paper is the radiation safety with regards to the researcher who needs to acquire large datasets for retrospec-tive evaluations. In the longer run, the foundational research enabled by HEX will be translated to the operating theater.
Our contributions are threefold: 1. We are the first to analyze the trade-off between radiation exposure and tracking accuracy in hybrid navigation with EMT and X-ray. Further, we analyze the impact of individual parts of the pipeline on the overall accuracy and radiation, evaluating exemplary compensation mechanisms. 2. We present the HEX research framework, which is the first of its kind to assess the feasibility of a hybrid navigation setting in phantom studies. HEX will be published under a free software license after acceptance. 3. The HEX framework features several means of automation that allow for safe but efficient experimentation. Large amounts of data can be recorded without researchers having to expose themselves to radiation.

Related work
Complementing fluoroscopic navigation with EMT is an idea that is discussed in literature [4,5], and several optimizations have been proposed to improve the accuracy of EMT in the hybrid setting. For instance, distortion compensation on the hardware level is investigated by the group around the Anser EMT project, which is intended to be used in the vicinity of the C-arm [5,6]. One optimization involves a field generator that is designed to be radiolucent in order to simplify hybrid navigation under X-ray [7]. These optimizations are intended to be leveraged in hybrid navigation with EMT and X-ray imaging. While integration EMT with ultrasound is common [8], only few related works cover the integration of EMT and X-ray imaging in a hybrid setup, as the registration is more difficult [9] or requires dedicated hardware [10].
Our previous work analyzes tracking accuracy in the hybrid setting in in-silico and in-vitro studies [4,11,12] and discusses techniques for algorithmic EMT distortion compensation. An overview of EMT systems, as well as previous passive and active techniques to mitigate EMT distortion, is found in the comprehensive review by Franz et al. [13].
So far, research has focused on optimizing individual parts of the hybrid setting, but a complete framework for hybrid navigation with EMT and X-ray is not yet available to the research community. Analyzing the impact of such modifications on radiation reduction in the hybrid setting is therefore not possible. With HEX, we introduce the first complete opensource framework that enables an ablation of individual parts of the pipeline on the overall navigation accuracy.

Methods
Ensuring safety for researchers in a laboratory environment requires the interplay of different automated devices in spatially separated rooms (see Fig. 1). With such an automated setup, it is possible to generate large amounts of data for evaluation in retrospective. HEX comprises several hardware components (see Fig. 2) and a software/firmware framework to orchestrate those, which together provide a safe and efficient data acquisition routine. We will first outline our imaging pipeline for EMT to X-ray registration and then elaborate on our automation approach. Finally, we will detail the evaluation protocol and materials and data used in our study. 3D EMT points are registered to X-rays by taking images from three gantry orientations, namely 75 • , 90 • and 105 • . Using the known angular difference between both images, the 3D position of the sensor is deduced in image coordinates (we assume the coordinate frame to be in the center of the 90 • image) by modeling the C-arm as a pinhole camera. This process requires a detection framework to determine the EMT position on X-rays, as well as a calibration routine for the C-arm. Finally, the EMT sensor data are aligned into the Fig. 1 The HEX framework orchestrates the acquisition process used for hybrid navigation in a research setting, allowing for spatial separation of high radiation and safe environments

EMT sensor detection on X-ray
Detecting the EMT sensor on X-ray is challenging due to the noisy acquisition process. We therefore use the YoloV5 detection network [14], as Yolo is successfully employed in both real-world and medical applications (e.g., ultrasound [15]). The weights we use are pre-trained on MSCOCO, re-trained on 154 and validated on 60 handannotated images with EMT sensors for 300 epochs, yielding 0.955 mAP50 and 0.512 mAP50-95 [16]. During detection, we consider the center of the detected bounding box in each image as the sensor position (Fig. 3).

C-arm calibration
For registering 3D EMT points to the X-ray, it is necessary to find the center of rotation in an image depending on the gantry angle and height at which it was captured. In clinical practice, this calibration step would be executed once for all subsequent interventions. We use a polynomial regression approach in order to infer the center of rotation, which we obtain by capturing images of a custom calibration board with metallic orbs, which is shown in Fig. 4. The orbs are positioned on a straight line that is aligned to the C-arm's laser beams. Then, each orb is detected using the OpenCV built-in blob detector. At minimum three orb centers per image are then averaged to find the x-coordinate at which the C-arm gantry pivots with respect to the image taken at 90 • . Finally, we fit a family of polynomials to all pivot centers depending on the gantry height, to obtain a model for the pivot center depending on gantry height and angle.

EMT to image registration
With the detected 2D sensor positions in three images, we estimate the sensor's 3D position by calculating the intersection point between orthogonal rays through the images. We obtain two sets of 3D points: one from the sensor itself and one from the detected positions on X-ray. These two point sets are aligned by rigid registration.

Automated X-ray acquisition protocol
Two prerequisites are necessary to automatically take X-ray snapshots from different angles: C-arm gantry rotation and X-ray acquisition. HEX automates both tasks, ensuring a safe procedure for the researcher and time-efficient collection of data for later evaluation.
In the OR, the rotation is carried out either by manually rotating the gantry or by pressing a virtual button on the resistive touchscreen of the C-arm workstation. Since there is no accessible digital interface available, the easiest solution to automate the rotation is to press the touchscreen with the help of a robot. We have developed a lightweight 3D-printed 3-axis robot to accomplish this task (Fig. 5). To control the gantry, we determined the setup time and angular velocity of the C-arm gantry to be t setup = 1.0 s and ω gantry = 4.0 • s . Once the C-arm gantry is set to the desired orientation, the image acquisition needs to be triggered. The designated way to capture images with the Ziehm Vision RFD is to manually trigger the foot pedal. As the foot pedal merely closes a switch, we replaced it with an Arduino-controlled relay which is triggered for a certain amount of time. We found that 100 ms is the minimum exposure time to obtain a good quality X-ray image. Once triggered, X-rays are taken from the C-arm's analog video output using a USB framegrab-  C-arm with phantom setup (center) are spatially separated by a lead-supported wall to block radiation from the researchers inside the control room ber, averaging over 100 frames to eliminate noise in the analog signal. HEX also provides access to higher resolution DICOM images if the C-arm is connected to a Picture Archiving and Communication System (PACS).
In between X-ray snapshots, the EMT sensor, which is attached to a directed catheter, is moved under the use of a 3D-printed linear guide. The sensor is retracted in steps of 2 mm while taking positional measurements with the EMT system (Fig. 6).

EMT distortion compensation
An EMT distortion compensation scheme is implemented, for which we collect a total number of 132 points on a Lego board (100 training, 32 validation), with the gantry being  rotated to a fixed angle of 90 • . As ground truth, we use the corresponding detected sensor positions on the X-rays. Both linear and cubic compensation schemes are implemented and evaluated.
It must be noted that this kind of compensation is an offline compensation scheme, which would require preoperative data collection in the OR, which typically is barely feasible. Online compensation, which has been proposed earlier [12], eliminates one more calibration step in clinical practice. However, in this work, we only apply exemplary offline compensation mechanisms for the sake of simplicity and brevity [12].

Materials
We use an Ascension/NDI trakSTAR 3D Guidance EMT navigation system (180-type sensor, mid-range field generator). HEX communicates with the tracker via OpenIGTLink, such that other trackers can be plugged in with ease. X-ray snapshots are taken with a Ziehm Vision RFD C-arm. All software modules are written in Python and C++ under the use of Pytorch and OpenCV libraries.

Experimental setup and results
We detail the experiments conducted with our hybrid pipeline and the resulting intermediate findings. Finally, we evaluate our phantom study, which investigates the effect of EMT compensation on overall radiation saving.

Data acquisition protocol
Accuracy is evaluated on a dedicated test measurement with the sensor and catheter inside the healthy phantom (see Fig. 3), with which 3 × 60 snapshots are collected. In this measurement run, the sensor is moved in 60 steps with a step size of 2 mm. To evaluate accuracy subject to radiation exposure, we gradually reduce the number of snapshots inside the dataset in retrospective. Each snapshot-triplet (75 • , 90 • , 105 • ) is then replaced by the corresponding EMT position from the timestamp of the 90 • image. Error is calculated as the difference between the registered EMT point and the detected sensor position on the corresponding X-ray. We estimate the conversion factor from X-ray pixels to mm by taking X-rays of a caliper at defined lengths and measuring the corresponding sizes in pixels.

Datasets
Parts of the HEX pipeline, as well as the final evaluation ("Experimental setup and results" Section), rely on pre-recorded data. A summary of these datasets and their respective purpose can be found in Table 1. In total, we  have collected 600 X-rays and over 100k EMT points for the experiments conducted in this study. We plan to release these datasets together with our code upon acceptance. Tables 2 and 3 show intermediate results for trainable sensor  detection and EMT compensation, respectively. The evaluation of the accuracy vs. radiation trade-off we described earlier uses the best-performing models for each task.

Evaluation protocol
Radiation exposure is evaluated in phantom studies with three 3D-printed anatomical vascular phantoms, which are shown in Fig.3. We state the total FT and DAP [2] for the simulated guidewire retractions to indicate the achieved radiation reduction. It must be noted that these values not do not directly quantify the exposure to surgeon or patientevaluating this would add significantly to the complexity of our setup. The C-arm itself reports an average FT of 0.95 ± 0.39 s and an average DAP of 0.28 ± 0.12 cGy cm 2 per X-ray for the parameters we set (48 kV, 2.6 mA, 57% pulse width, 25 pulses/s). FT and DAP were averaged over 20 X-ray snapshots taken with our setup, and the average val-

Phantom study
The main goal of HEX is to minimize total radiation exposure, while maintaining reasonable accuracy. Figures 7 and   8 visualize this trade-off in a Monte Carlo experiment, in which we leave out M = 0...60 random images from the X-ray set, replacing them with the corresponding EMT point.
For the remaining X-rays, we model the navigation error as a Gaussian with σ = 0.32 mm, corresponding to the pixel quantization noise. We repeat this process for N = 200 iterations and evaluate for uncompensated and compensated (cubic) EMT data. This allows us to visualize total navigation accuracy over the number of X-rays taken during a procedure, the latter being proportional to the total FT or DAP, respectively. This way, we observe that EMT distortion compensation has a beneficial effect on the overall tracking accuracy of the hybrid pipeline, achieving submillimeter accuracy even for a small number of snapshots. If we assume 1 mm to be the upper bound for high precision accuracy, hybrid navigation with cubic offline distortion compensation reduces FT from 60 s down to 17.1 s, limiting overall radiation exposure by 71%. Linear distortion compensation reduces FT by 36 s (40%), whereas submillimeter accuracy is not achieved without compensation. Further, we analyze the impact of the number of training points on the radiation saving, as these are collected manually which is tedious (see "EMT distortion compensation" Section). It is possible to reduce the training points by half, while still saving more than 70% of radiation exposure.

Conclusion
In this paper, we demonstrate how an end-to-end pipeline for hybrid navigation can be realized in a research environment.
Combining custom hardware and off-the-shelf components, we complement fluoroscopy by EMT guidance, which effectively reduces the radiation dose. The automated image acquisition protocol allows us to collect large amounts of data safely and efficiently, with a total of 600 X-ray images being used in the scope of this paper. Efficient data collection enables analyses of the impact of optimizations on the full pipeline before deployment of optimizations in the OR. A phantom study shows that even with a simple compensation scheme, we are able to reduce fluoroscopy time by 71%. This result indicates that HEX can significantly decrease radiation exposure to patients and surgeons and should be investigated toward bench-to-bedside translation in endovascular surgery.
Funding Open Access funding enabled and organized by Projekt DEAL. This authors funded by Deutsche Forschungsgemeinschaft (FE 431/13-2).

Code availability
The code used in this work will be made available via https://github.com/ai-health-care/Hybrid-EMT-Xray upon acceptance.

Declarations
Conflict of interest This research was partially funded by the German Research Foundation. The authors declare that they have no conflict of interest.
Research involving human participants and/or animals This article does not contain any studies with human participants or animals.
Informed consent This article does not contain patient data.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecomm ons.org/licenses/by/4.0/.