Data fusion and 3D visualization for optimized representation of neurovascular relationships in the posterior fossa

Background Reliable 3D visualization of neurovascular relationships in the posterior fossa at the surface of the brainstem is still critical due to artifacts of imaging. To assess neurovascular compression syndromes more reliably, a new approach of 3D visualization based on registration and fusion of high-resolution MR data is presented. Methods A total of 80 patients received MRI data with 3D-CISS and 3D-TOF at 3.0 Tesla. After registration and subsequent segmentation, the vascular information of the TOF data was fused into the CISS data. Two 3D visualizations were created for each patient, one before and one after fusion, which were verified with the intraoperative situation during microvascular decompression (MVD). The reproduction quality of vessels was evaluated with a rating system. Results In all cases, the presented approach compensated for typical limitations in the 3D visualization of neurovascular compression such as the partial or complete suppression of larger vessels, suppression of smaller vessels at the CSF margin, and artifacts from heart pulsation. In more than 95% of the cases of hemifacial spasm and glossopharyngeal neuralgia, accurate assessment of the compression was only possible after registration and fusion. In more than 50% of the cases with trigeminal neuralgia, the presented approach was crucial to finding the actually offending vessel. Conclusions 3D visualization of fused image data allows for a more complete representation of the vessel-nerve situation. The results from this approach are reproducible and the assessment of neurovascular compression is safer. It is a powerful tool for planning MVD.


Introduction
The understanding of the complex nerve-vessel relationships in the posterior fossa on the surface of the brainstem is of high importance for the assessment of neurovascular compression (NVC) syndromes [13,15]. Established sequences for the assessment of NVC are high-spatial-resolution strong 3D gradient echo T2-weighted imaging, i.e., Constructive Interference in Steady State (CISS), and MR-angiography, i.e., Time-of-Flight (TOF) [14,19,22]. In addition to that, the higher signal of 3.0 Tesla can improve the spatial resolution for the assessment of NVC [7].
The introduction of a standardized image processing and visualization method has proven to be a powerful tool for analyzing neurovascular relationships [10,11,16,23]. As an important advantage of this strategy, the nerve and vessel structures are effectively delineated with implicit segmentation based on transfer functions for color and opacity This article is part of the Topical Collection on Neurosurgery general values after explicit segmentation of coarse structures in the beginning.
Major problems limiting the quality of 3D-visualizations are the flow-related signal variability of vessels and pulsation artifacts of the cerebrospinal fluid (CSF) [16]. This leads to a considerably inferior representation of larger vessels, e.g., basilar artery (BA), vertebral artery (VA), and related vascular junctions in CISS data. In addition, vessels at the boundary of the CSF suffer from contour fusion with neighboring structures, thereby preventing their precise delineation [16]. On the other hand, in-plane and small vessels are not consistently reproduced in TOF data due to known restrictions [7]. To solve these problems, the principles of data fusion based on the source data are applied for the first time and the corresponding results of the 3D visualization are presented. Some studies claim to perform image fusion, but limit themselves to combining rendering results of different image data. In contrast, to solve the problems described above, the principles of the fusion of source data are described for the first time and the results of the 3D visualization are evaluated.

Patients
In a prospective study, a total of 80 patients (f:m, 46:34) with trigeminal neuralgia (TN, n = 54), hemifacial spasm (HFS, n = 22), and glossopharyngeal neuralgia (GN, n = 4) were considered in this work. After preoperative diagnostics, microvascular decompression (MVD) was performed on all patients. The ethics committee of the University approved this study. Informed consent was obtained from all patients included in the study.

Image data
All patients obtained image data with a 3.0 Tesla MR system (Magnetom Trio, Siemens Healthcare, Erlangen, Germany). The two sequences CISS and TOF were used. CISS provides images with a high degree of contrast between the hyperintense cerebrospinal fluid (CSF) and the hypointense cranial nerves, brainstem, and vessels. TOF is the most commonly used MR angiography sequence in neurovascular imaging with the highest resolution. Using the signals of floating fluid, it displays the vessels as hyperintense structures without the use of a contrast agent [18]. Both sequences were adjusted to isotropic voxel size (0.4 mm) ensuring equally high resolution in axial, coronal, and sagittal slice images. Details of the sequence parameters were previously published [7,16].

Registration
To improve the representation of the vessels in the 3D visualization, CISS and TOF data were registered. Initial alignment was achieved during imaging by performing both sequences immediately after each other. In order to compensate for motion artifacts and to obtain accurate registration of the data, an automatic entropy-based approach using a rigid transformation was used [24]. After reformatting the TOF data according to the grid of the CISS data, the arrangement, number, and size of the voxels in both data became identical. Figure 1 shows corresponding slices after registration.

Segmentation of CISS data
In CISS data, cranial nerves and vessels are represented in the same range of intensity values as the surrounding tissues. For a clear 3D visualization, an explicit segmentation of the CISS data is mandatory as presented previously [10,16]. In a first step, the CSF area of the basal cisterns ventral and lateral to the brainstem was semi-automatically segmented with volume growing. An upper and lower threshold served to limit the range of intensity values and a bounding box helped to restrict the volume of interest. In the next step, the brainstem was interactively delineated. The segmentation to the CSF area, which was precisely determined in the preceding step, was defined as constant. This served as a major support of rapid processing. Then, the cranial nerves were segmented by manual labeling. As a result, the CISS data is divided into four sub-volumes according to the respective anatomical structures: (1) CSF area including all vascular structures, (2) brainstem, (3) cranial nerves, and (4) all remaining structures.

Segmentation of TOF data
For the TOF data, volume growing was applied to extract the vascular structures. Since vessels in TOF data have higher intensity values than the other tissues, a lower threshold was sufficient to limit the growing process. A bounding box was used to restrict the procedure to the area of interest. Finally, the segmented vessels in the TOF data were labeled with the same tag as the vessels in the CISS data, i.e., identical to sub-volume 1 (CSF region including all vascular structures).

Fusion of CISS and TOF data
The basis of the presented fusion strategy is a direct correspondence of pairs of voxels in the CISS and TOF data, which is ensured by reformatting the TOF data in the grid of the CISS data following registration. As a further prerequisite for fusion, the segmentation of both the CISS and TOF data assigns an identical label to all voxels that describe vessels. In order to match the values of the labeled voxels, the hypertensive values of the voxels marked as vessels in TOF data were inverted and mapped into the hypointense value range of vessels in the CISS data (see Fig. 1). For this inversion, the maximum voxel values in the CISS and TOF data were determined and thus an equation of a line was established. The linear mapping thus described was used to convert the values of all labeled voxels in the TOF data into the value range of the CISS data. Thereafter, for each pair of corresponding voxels, the original CISS values were replaced by the inverted TOF values. As a result, the vascular information of both data was combined and a new dataset with fused TOF and CISS information was obtained (see Fig. 1).

3D visualization
For the 3D visualization, an implementation for direct volume rendering (dVR) was used, which allows using an individual transfer function for every sub-volume [11,12,16]. Thereby, the data values of the four sub-volumes are mapped to different color and opacity values. The following colors were used: red for vessels, yellow for nerves, and light gray for brainstem. All remaining structures (sub-volume 4), which do not provide relevant information for NVC syndromes, were made completely transparent.
All data was visualized both with 2D visualization using slice images and with 3D visualization using direct volume rendering before and after fusion. In this work, the following terms are used for 2D visualizations with slice images: 2D-vis-CISS and 2D-vis-FUSION, based on CISS data only and fused CISS and TOF data, respectively. Similarly, for 3D visualizations with direct volume rendering, the following terms are used: 3D-vis-CISS and 3D-vis-FUSION based on CISS data only and fused CISS and TOF data, respectively.

Evaluation of the fused data
Two 3D visualizations were created for each patient, one before and one after fusion. All were interactively compared and verified with the intraoperative situation during microvascular decompression (MVD). A rating system developed by our working group was used to assess the visualization quality of the vessels (see Fig. 2). Two independent expert observers (neurosurgeon with regular experience in MVD) made the evaluation. Particular attention was paid to the effects of artifacts since they can significantly influence the visualization. Flow-related artifacts are common in vessels with a wide lumen. In general, moving spins do not contribute to the signal formation of a given voxel at MRI and result in vessels not being displayed [18]. Artifacts by pulsation are motion artifacts caused by CSF pulsation and blood flow. In the area of the posterior fossa, especially the basilar artery vibrates in the CSF due to the pulse wave. In the visualization, this appears as a foggy cloud. Finally, contour fusion is caused by similar intensity values of vessels and brainstem in the CISS data [16]. If the course of a vessel is on the surface of the brainstem, the segmentation can lead to an incorrect labeling and thus to a vessel, which in the visualization becomes brainstem.

Results
The presented approach was successfully applied in 80 consecutive cases of NVC syndromes for preoperative analysis. To evaluate the improvements achieved by fusion, all visualizations (for each case: 2D and 3D before and after fusion) were evaluated using the scheme of Fig. 2 and compared to the intraoperative situation during MVD (see Fig. 3).
The results presented in Table 1 demonstrate that the suggested fusion method leads to considerably improved 3D visualization and slightly improved 2D visualization. With regard to the applied assessment scheme, the total score for 3D visualization was improved by more than one level so that the important nerve and vessel structures are presented. A more detailed analysis of Table 1 shows that the smaller vessels (i.e., AICA and SCA) are best represented in 2D-VIS-CISS, whereas the larger vessels (i.e., A. basilaris and A. vertebralis) were usually shown only in a schematic way. In contrast, the slice images of TOF mainly showed the larger vessels (i.e., A. basilaris and A. vertebralis) in an optimal way. After fusion, 3D-vis-FUSION improved the score of the larger vessels by 1 to 2 points compared to 3D-vis-CISS. On the other hand, the smaller vessels after Score 0 Score 1 Score 2 Score 3 Score 4 Score 5 The improved representation of NVC syndromes by the fusion of CISS and TOF data is illustrated in Fig. 4 for a case of hemifacial spasm and in Fig. 5 for a case of trigeminal neuralgia.
Pulsation artifacts could not be avoided despite absolute resting position of the patient during the MRI measurement. They can completely or partially obscure relevant vessels (see Table 2). As shown in Fig. 6, the assessment of the vessel-nerve relationships in the 3D visualizations of CISS data (3D-vis-CISS) was significantly affected. The proposed fusion method helped to complement the vessels in the 3D visualization of the fused data (3D-vis-FUSION), but reduced the artifacts only to a limited extent. If the underlying segmentation was further optimized through expert knowledge, the number of artifacts could be significantly reduced (3D-vis-FUSION optimized). The improvement was less effective and much more time-consuming if the optimization was attempted only with CISS data and without fusion; see Table 2.
Flow artifacts were particularly noticeable in the larger vessels, while they occurred less with smaller vessels. In particular, the basilar and the vertebral artery were strongly affected in the 3D visualization of the CISS data (3D-vis-CISS), as shown in Fig. 6. Their course could not be fully represented in 65-90% of the cases and vessel junctions could not be objectively understood. The quantitative evaluation in Table 2 shows that the problem could not be solved if  the segmentation of only the CISS data had been optimized (3D-vis-CISS optimized). In contrast, all flow artifacts were effectively eliminated by fusion (3D-vis-FUSION). If the segmentation of the fused data was further optimized, the resulting visualization (3D-vis-FUSION optimized) was slightly better but it did not substantially improve the analysis of the NVC. Contour fusion was mainly observed in the smaller vessels and with venous structures. As Table 2 shows, it was observed mainly in the 3D visualization of the CISS data (3D-vis-CISS) in the PICA and AICA and less in the SCA. The affected vessel was interrupted or completely extinguished due to incorrect segmentation (see Fig. 6). With the proposed fusion approach, the problem could be significantly improved (3D-vis-FUS). In consequence, the segmentation is more robust because the inversion of the TOF signal values suppresses the noise in the area of the vessels and allows distinguishing the values of vessels and brainstem. If the segmentation of the fused data was additionally optimized (3D-vis-FUS optimized), it was possible to achieve further improvement.
The processing of the CISS data was straightforward and followed the established procedure previously described by the authors [10,16]. The segmentation of TOF data was unproblematic due to the clear signal intensities of vessels. Only if the noise level was higher, the separation of smaller vessels was restricted in 8 cases. The missing information could be compensated in these cases by the information in the CISS data.
Using the suggested fusion approach, vessels are represented with a clearer signal intensity that has in consequence simplified segmentation in all cases. The time originally required for image processing was ranging between 2 to 4 h [16]. With the proposed fusion approach, it was reduced by 60% to an average of 1 h.
The comparison with the intraoperative situation during MVD revealed that an accurate and safe assessment of NVC was possible in more than 95% of the cases of hemifacial spasm and glossopharyngeal neuralgia (see Fig. 3). In more than 50% of the cases with trigeminal neuralgia, the presented approach was crucial to finding the actually offending vessel.

Discussion
The exact understanding of the relationship of cranial nerves and vascular structures in the posterior fossa at the surface of the brainstem is crucial for preoperative analysis of NVC. This study presents an approach of 3D visualization based on registration and fusion of high-resolution CISS and TOF data, effectively complementing the authors' established visualization method [10,11,16]. In contrast to all previous studies, the presented work quantitatively analyzed data obtained with MR imaging and image processing by comparing 3D-visualization of CISS data (3D-vis-CISS) and 3D-visualization of fused CISS and TOF data (3D-vis-FUSION). Overall, the presented approach leads to a significantly improved assessment of NVC and makes the evaluation of NVC faster and safer. When the technical fundaments of interactive and virtual 3D visualization of NVC syndromes were introduced [1,2,16], it was proposed almost parallel to register and fuse additional vascular information from angiographic data [10,11]. As described in further studies, MR angiography data can effectively complement the information gained from strongly T2-weighted MR data. In this context, Zeng et al. preoperatively analyzed the neurovascular relationships in TN using 2D visualization based on slice images without registration and fusion [27]. Haller et al. presented a study on imaging of NVC syndromes, in which fused CISS and TOF data with 2D-visualizations using slice images were presented without information about the fusion strategy used [9]. Docampo et al. used similar data at 3.0 Tesla, visualized the TOF data with volume rendering and transferred the resulting 3D visualization into slice images to investigate NVC in TN [4]. The technique of fusion is not specified and the subsequent result is not a complete 3D representation. Yao et al. used data primarily at 1.5 Tesla to visualize them in 3D with surface rendering. The information of the different data was only fused in the 3D visualization by selecting the modality that offers the highest contrast of the respective target tissue [26]. In this strategy, major information of structures within the CSF cisterns was not taken into account during image processing and was therefore not provided in the resulting 3D representations. A large proportion of the studies does not refer to the applied methods of image processing, which Further studies about fusion at NVC have considered image data from different sequences or modalities and different visualization techniques. Oishi et al. used high-resolution MR and CT data to generate interactive 3D representations to prepare MVD [17]. Guo et al. combined different T1-and T2-weighted MR data to study neurovascular compression in facial neuralgia [8], and El Refaee et al. compared the MR data with intraoperative high definition endoscopic visualization in HFS [6].
Other studies have correlated the results of imaging and image processing with surgical findings in order to analyze NVC in TN, HFS, and GN [13,14,21]. It turned out that 2D-representations do not give sufficient overview of the spatial relationships of nerves and vessels.
There is general agreement that high-resolution sequences are necessary tools to assess the neurovascular relationships in NVC correctly. The quality of the source data is of paramount importance for the quality of the 3D visualization [16,25]. There is further consensus that patients with NVC may benefit from higher resolution and greater sensitivity of 3.0 Tesla MRI [7]. As a limitation, we observed high signal intensities in the center of the lumen within the basilar artery and vertebral arteries in almost every CISS data, significantly affecting 3D visualization. In 25 cases, the contrast between vessels and CSF was completely lost (see Fig. 1). In consequence, the vessel could not be presented in 3D visualization.
An approach that combines the original source data in such a way that a new dataset is created with fused source information has not been proposed until now. In the presented approach, the original CISS and TOF data are fused on the level of the voxels and a fused dataset is generated which is then used for 3D visualization. As a major advantage of this strategy, only that information is taken from the data that optimally represent a structure or part of it. In this way, the smaller vessels (i.e., AICA and SCA) are primarily taken from the CISS data and the larger vessels (i.e., A. basilaris and A. vertebralis) more from the TOF data. The decision as to which vessel to take from which data is automatically controlled by the segmentation of the CISS and TOF data as well as the technique for inverting the TOF signal values.
In the present work, the quality of the 3D visualizations was assessed objectively by evaluating the imaging and reproduction of the vessels before and after fusion. Our data show that the suggested fusion of CISS and TOF data leads to a reliable 3D representation of the cranial nerves and at the same time to a significantly increased quality of vessel presentation. Thereby, an accurate and reliable identification of the nerve-vessel relationships is available (see Fig. 3, Fig. 4, Fig. 5). We were able to assess a 49% increase in the quality of the vessel presentation for 3D-vis-FUSION compared to 3D-vis-CISS. A detailed analysis showed that the mean display quality was higher for all vessels in 3D-vis-FUSION than in 3D-vis-CISS. Adequate reproduction of vessels in 3D-visualizations has shown to be crucial for the detection of NVC. Poor vessel reproduction due to flow and pulsation artifacts are a major causes of difficult-to-assess 3D-visualizations in which the NVC is obscured (see Fig. 6). For the flow artifacts, the presented fusion approach immediately resulted in improved 3D vascular representations (see Table 2), while the pulsation artifacts could be effectively eliminated after optimizing the segmentation of the fused data (see Table 2). Vessels such as PICA or AICA, which were interrupted or completely extinguished in 3D-vis-CISS due to contour fusion, could be visualized significantly better in 3D-vis-FUSION. By optimizing the segmentation of the fused data, the effect was eliminated in almost all cases (see Table 2).
There were also limitations in our study. Although the evaluation of image data is straightforward and easy to understand, the semi-automatic image processing requires expert knowledge about the complex anatomy of the neurovascular relationship needed. In the future, further automation is necessary.
As a further limitation of the presented work, the analysis of image data focuses on arteries. However, it is important to also consider veins as a possible cause of NVC. In the study of Dumot et al., compression veins found in MVD were precisely described and their relevance as a source of TN was shown [5]. Since our approach aims at further automated processing of the image data, it cannot be used safely for veins due to the currently uncertain differentiability of arteries and veins in T2 and T1 data. It will be necessary in the future to also integrate veins into a safe 3D visualization within our framework.
Finally, the classification of the NVC should be analyzed in more detail. In this context, Brinzeu et al. quantitatively examined the predictability of NVC in TN based on protocolized MR data and findings of MVD revealing high prediction values for high grades of compression and lower values with possible false positives in low-grade compression [3]. With regard to classification, we have used colorencoded distance information to calculate the vessel-nerve contact area and to analyze the probability of its extent in an earlier work [20].

Conclusion
With the presented approach for fusion, the 3D representation of the anatomical course of the vessels and nerves in the posterior fossa is significantly improved. Typical artifacts that affect the assessment of NVC are effectively eliminated. This allows for clearer and more complete 3D visualizations, which are particularly advantageous when using image data at higher field strengths. In summary, we present an efficient and robust fusion strategy that further enhances our established technique for 3D visualization of NVC. It makes the detection and analysis of NVC syndromes more precise and effectively supports microvascular decompression.
Author contribution • Peter Hastreiter designed and led the technical aspects of the research, analyzed the data, produced all figures, and wrote the manuscript.
• Ramin Naraghi designed and led the medical aspects of the research, recruited patients, performed surgery, analyzed the data, and edited the manuscript.
• Barbara Bischoff recruited patients and contributed to data analysis.
• Rudolf Fahlbusch recruited patients, performed surgery, contributed on work design, and edited the manuscript.
• Michael Buchfelder recruited patients, performed surgery, contributed on work design, and edited the manuscript.
• Arnd Doerfler provided the radiological imaging and contributed to data analysis.
Funding Open Access funding enabled and organized by Projekt DEAL.

Data availability
The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request. All raw data and processed image data are stored on servers of the University Hospital of Erlangen.

Declarations
Ethics approval All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Approval was obtained from the ethics committee of the University of Erlangen-Nuremberg (waivers: No. 3767, No. 2833, No. 122_13 B).
Informed consent Informed consent was obtained from all individual participants included in the study.

Conflict of interests The authors declare no competing interests.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.