Illustrative uncertainty visualization of DTI fiber pathways

Abstract

Diffusion Tensor Imaging (DTI) and fiber tracking provide unique insight into the 3D structure of fibrous tissues in the brain. However, the output of fiber tracking contains a significant amount of uncertainty accumulated in the various steps of the processing pipeline. Existing DTI visualization methods do not present these uncertainties to the end-user. This creates a false impression of precision and accuracy that can have serious consequences in applications that rely heavily on risk assessment and decision-making, such as neurosurgery. On the other hand, adding uncertainty to an already complex visualization can easily lead to information overload and visual clutter. In this work, we propose Illustrative Confidence Intervals to reduce the complexity of the visualization and present only those aspects of uncertainty that are of interest to the user. We look specifically at the uncertainty in fiber shape due to noise and modeling errors. To demonstrate the flexibility of our framework, we compute this uncertainty in two different ways, based on (1) fiber distance and (2) the probability of a fiber connection between two brain regions. We provide the user with interactive tools to define multiple confidence intervals, specify visual styles and explore the uncertainty with a Focus+Context approach. Finally, we have conducted a user evaluation with three neurosurgeons to evaluate the added value of our visualization.

1 Introduction

Diffusion is the process of random movement of water mole-cules over time, also called Brownian motion. Diffusion Tensor Imaging (DTI) is an imaging technique based on Magnetic Resonance (MR) that can measure diffusion of water in living tissues. By measuring the amount of diffusion in many different directions, the 3D shape of the diffusion profile can be approximated at each point in the tissue. In pure water, diffusion is unrestricted and has equal magnitude in all directions. This results in a spherical or isotropic diffusion profile. In fibrous tissues however, such as the brain white matter, the diffusion will be restricted in directions perpendicular to the fibers. This results in a more elongated or anisotropic diffusion profile. In DTI, the diffusion profile is modeled as a 3D Gaussian probability distribution using a 2nd-order tensor. In this model, the tensor’s main eigenvector corresponds to the direction of greatest diffusion and is assumed to be aligned with the underlying fiber structure [1]. Based on this assumption, it is possible to do streamline tracing in the main eigenvector field. In the context of DTI, this is called fiber tracking [21, 27, 34] and it allows reconstruction of the fibers in three dimensions. Diffusion-weighted MRI is the only imaging modality that allows to do this noninvasively and in-vivo. For this reason, it has great potential for applications that involve fibrous tissues, such as the brain white matter and muscle.

Despite its potential, the output of DTI fiber tracking contains a significant amount of uncertainty. This uncertainty is accumulated in the various stages of the DTI processing pipeline. A schematic overview of this pipeline is illustrated in Fig. 1. The data acquisition stage suffers from noise, image distortion, movement artifacts, partial volume effects, and scan parameters. The modeling stage can introduce approximation errors depending on the diffusion model that is used. For example, the 2nd-order tensor model produces unreliable results when fibers are crossing, kissing, or diverging within a voxel. The fiber tracking stage attempts to reconstruct 3D pathways from the tensor field. However, different numerical integration schemes produce different results and each fiber tracking algorithm has several user-defined parameters that can significantly affect fiber length and/or shape [5]. Finally, the visualization stage may introduce uncertainty due to the use of different illumination models or simplification of the fiber geometry.

Fig. 1

DTI processing pipeline with sources of uncertainties at each stage

Existing DTI visualizations largely ignore the above mentioned sources of uncertainty and their effect on the reconstructed fiber pathways. This gives an impression of certainty that can be misleading. This is unacceptable in applications such as neurosurgery, where fiber tracking may be used for surgical risk assessment and decision-making. If false positives or false negatives in the fiber tracking output are not correctly identified this can result in suboptimal tumor resection or damage to healthy brain tissue. To deal with risks of healthy tissue damage, neurosurgeons may take into account safety margins around critical brain structures [35]. However, such safety margins are mostly fixed and have no relation to the uncertainty in the underlying data. Furthermore, safety margins enclose only one out of many possible fiber configurations. Figure 2 illustrates that, due to noise and modeling errors, there are many such configurations possible, each one slightly different. It is clear from this example that a fixed safety margin may not adequately cover the possible variations in fiber shape.

Fig. 2

Left: single fiber obtained from original tensor volume. The yellow silhouette represents fixed safety margin. Right: 100 variations of same fiber based on wild bootstrap method [14] with same safety margin. It is clear that it does not cover the possible variations in the tensor data

With the visualization framework described in this paper, we attempt to visually communicate to the neurosurgeon that the fiber tracking algorithm they use may produce a suboptimal reconstruction of the tracts of interest. We are specifically focusing on variations in the output due to noise and modeling errors. Such information can be captured by different probabilistic algorithms but intuitively showing this information is not a trivial task. Even without considering uncertainty, diffusion tensor data presents considerable visualization challenges. For this reason, each tensor is often reduced to a single vector describing the principal direction of diffusion. Streamline visualization can then be used to show pathways through the tensor field. This gives a good impression of the global structure of fiber tracts. However, if there are too many streamlines this approach can lead to highly cluttered visualizations, such as illustrated in Fig. 2 on the right. Probabilistic fiber tracking algorithms generally sample thousands of potential pathways in order to compute connection probabilities between different regions of the brain. In that case, it is no longer feasible, nor informative, to render each individual streamline.

To deal with this issue, we propose Illustrative Confidence Intervals, a rendering technique based on illustrative silhouettes and outlines that provides information about the variation in fiber pathways while at the same time reducing the visual clutter associated with standard streamline visualizations. We define a silhouette to be the interior area of an object’s shape when projected to the viewing plane. An outline is defined as the border of a silhouette. Our framework does not depend on a specific algorithm to compute the variation in fiber pathways. It only requires a set of streamlines, each associated with a so-called confidence value. A confidence value is a scalar-valued, fiber-specific measure that can be computed in different ways depending on the application. The specific measure used in our framework can be freely chosen but to demonstrate the flexibility of our approach we have selected two such measures, (1) based on fiber distance and (2) based on the connection probability between different brain regions. Our contributions are the following:

• 1. A processing framework that allows different methods for constructing sets of streamlines with associated confidence values. We implemented two example methods but other methods can be added easily (see Sect. 3).

• 2. A visualization and interaction framework that uses illustrative silhouettes and outlines to analyze variations in the output of a selected fiber tracking algorithm (see Sect. 4). A set of interaction widgets that allows easy specification of intervals and their visual styles. A Focus+Context uncertainty lens shows confidence intervals only within a user-defined region of interest. Outside this region standard fiber visualization can be used allowing for easy comparison between the two visualizations (see Sects. 4.3 and 4.4).

• 3. A quantitative and qualitative user study based on video demonstrations, questionnaires and interviews with three neurosurgeons.

2 Related work

Visualization of uncertainty is closely related to and dependent on the method of measuring or computing uncertainty. As Fig. 1 illustrates there are many different sources of uncertainty present in the DTI processing pipeline. In our current work, we focus on the uncertainty in the DTI (tensor) data caused by noise and modeling errors. In the next subsections, we discuss related work in the areas of DTI uncertainty analysis, uncertainty visualization, and illustrative rendering for DTI applications.

2.1 Uncertainty analysis

Several approaches are possible to characterize the effects of noise and modeling errors on DTI fiber tracking. They can be roughly subdivided into two types: empirical methods and mathematical modeling methods. There also exist different fiber tracking methods. We divide them into deterministic algorithms, i.e., they give the same result given the same input (such as streamline tracing), and probabilistic algorithms that involve a random process. To characterize the uncertainty, an obvious, empirical approach is to repeat a diffusion-weighted imaging (DWI) scan multiple times. Due to random noise, each scan will be slightly different. Repeating this often enough will give a good approximation of the uncertainty in the data. Of course, this would require hours or days of scanning. A variation of this method, called bootstrap, is to generate new datasets, by taking random samples with replacement from a much smaller set (4 to 5) of DWI scans [24]. However, this approach is still too time-consuming for clinical purposes. Wild bootstrap [14, 17, 36] is a method that generates random variations based on a single DWI scan. This method is often combined with deterministic fiber tracking to obtain information about fiber shape variations (see Sect. 3.1). The second type of approach to characterize uncertainty is based on mathematical modeling [2, 10, 15]. Klein et al. [15] generate variations of a single DWI dataset by adding increasing levels of complex Gaussian noise. A drawback of such methods is that they assume a noise model (e.g., Gaussian) which generally is a simplification of the real noise characteristics. Others take a predictive model, such as the 2nd-order tensor, and use Bayesian statistics to estimate a posterior PDF for the model parameters, including fiber orientation [2, 10]. These methods generally do not use deterministic fiber tracking but instead apply probabilistic algorithms to reconstruct fiber pathways (e.g. based on Monte Carlo simulations). Still, these methods assume a noise model and have to deal with the complexity of having to propagate this noise model through the many, often nonlinear, transformations in the DTI processing pipeline. On the other hand, the explicit mathematical representation of assumptions allows a more formal way to compute uncertainty and can be applied to a wide range of diffusion models. Some authors use a combination of mathematical modeling and empirical techniques to characterize uncertainty in the data [3, 31].

In this paper, we make no statements about which approach for computing uncertainty is superior. Our main focus is to visualize variations in fiber pathways due to noise and modeling errors. To demonstrate our approach, we have chosen the Wild Bootstrap method by Jones et al. [14] and the ConTrack algorithm by Sherbondy et al. [31]. However, our method does not depend in any way on the particular choice of algorithm used to characterize uncertainty.

2.2 Uncertainty and illustrative fiber visualization

The last decade has shown a slow but growing interest in the visualization of uncertainty, especially for scalar and vector fields. Pang et al. [26] provide an extensive overview of techniques, as well as a classification scheme for describing data together with its associated uncertainty. Lodha et al. [18] present a method for visualizing variations in streamline tracing due to different integration schemes. They provide different options for representing the uncertainty, e.g., glyphs, ribbons, and motion-blurred animation. Pang [25] and Wittenbrink et al. [37] use different types of glyphs to display the effect of data uncertainty on height field measurements and the direction and magnitude of 2D vector fields.

The literature concerning uncertainty visualization for tensor fields is very limited. Variations in the tensor main eigenvector were visualized by Jones et al. [14] using cone glyphs and a modification of hyperstreamlines [7]. They compute these variations using the Wild Boostrap method. However, they render all fibers in the same way (as in Fig. 2) without giving an impression of which fibers are more reliable or reproducible than others. Furthermore, their method cannot easily be adapted to more complex fiber shapes. Enders et al. [8] compute a central fiber to create a closed surface hull wrapped around the fiber bundle. However, similar to Jones et al., their method cannot deal with complex fiber shapes and concave bundle cross-sections without significant simplification of the hull geometry. The wrapped hull geometry described by Chen et al. [6] suffers from similar problems. Merhof et al. [19] propose a hull generation algorithm based on smoothed isosurfaces that does not suffer from these limitations. Schultz et al. [29] combine probabilistic tractography with a volume rendering approach to show different probability iso-surfaces.

The visualization methods discussed so far primarily deal with noise and modeling errors. The effect of user-defined parameters on the output of DTI fiber tracking was addressed by Brecheisen et al. [5] who use a combination of color-coding, linked views and interactive parameter space exploration to visualize the threshold sensitivity of streamline tracing. To improve the visualization of fiber structures, either to reduce visual clutter or provide surrounding context, a number of researchers have used illustrative techniques [4, 9, 11, 30, 32]. Weiler et al. [35] attempt to highlight fiber tract uncertainty by means of a fixed, user-defined safety margin. However, this safety margin does not take data uncertainty into account.

In this paper, we propose a method that uses the output of probabilistic fiber tracking methods to generate Illustrative Confidence Intervals that do take uncertainty of the data into account. Each fiber is assigned a confidence value based on a chosen confidence measure (to be explained in the next section). We define a confidence interval [C _i ,C _i+1] to contain a subset of all fibers with a confidence between C _i and C _i+1. An illustrative confidence interval is the visual representation of these fibers and shows the variation in fiber pathways due to noise and modeling error. The rendering algorithm uses a silhouette and outline representation that is based on the work of Otten et al. [23], but we extend it with additional visual styles, interaction features, Focus+Context views, and anatomical context. Furthermore, we apply and evaluate our approach in the context of neurosurgical risk assessment and decision-making.

3 Computing fiber confidence

The visualization framework we propose requires a set of fibers (represented by streamlines) where each fiber is associated with a confidence value. Figure 3 gives an overview of the processing pipeline to compute fiber confidence from a set of diffusion-weighted images. Confidence values are stored in a table, normalized, and sorted from high (1.0) to low (0.0) confidence. How we use the table to visualize fiber confidence intervals is described in the next section. As described in the related work section, there are different ways to obtain a confidence value for each streamline. Each approach has its advantages and disadvantages. Our framework is general and does not depend on a specific approach for computing fiber confidence. To demonstrate this flexibility we have chosen two methods, (1) the Wild Bootstrap algorithm [14] combined with a fiber distance measure and (2) the ConTrack fiber tracking algorithm [31] which assigns a connection probability to each pathway. Both methods rely on a random process, either in the fiber tracking itself (ConTrack) or in the generation of the underlying tensor field from which the fibers are reconstructed (Wild Bootstrap). For this reason, we consider both algorithms to be probabilistic in nature. The output of both algorithms is a set of fiber pathways. Often these pathways are converted to a probability volume by counting the number of times each voxel is intersected by a pathway. This approach however is dependent on the chosen resolution of the data volume. We prefer to work with the streamlines directly because it preserves the visual continuity of the fiber tract of interest. This is especially important for the neurosurgeon who expects a certain correspondence between the visualization of a fiber tract and his or her knowledge of white matter anatomy.

Fig. 3

Pipeline for computing fiber confidence using (1) Wild Bootstrap and (2) ConTrack. The Wild Bootstrap method generates a large collection of unlabeled fibers which are assigned a confidence value by means of a fiber distance metric. The fibers produced by ConTrack already have a confidence value assigned to them

3.1 Wild bootstrap and fiber distance

The Wild Bootstrap method has been described by different authors [17, 36] but was first combined with fiber tracking by Jones et al. [14]. It attempts to simulate repeated diffusion-weighted imaging (DWI) scans by generating random variations of the tensor data from a single DWI dataset. For this, the diffusion tensor model is fit to the data in each voxel. For more than 6 diffusion directions, this results in a set of model errors or residuals, one for each measurement direction. The residuals are randomly flipped (by multiplying them with either −1 or 1), after which the tensor model is fit again. By repeating this procedure many times a large number of tensor volumes is generated. For details on the exact procedure to compute the Wild Bootstrap, we refer the reader to Jones et al. [14]. In each generated tensor volume, we perform deterministic streamline tracing [33] starting from a preselected seed region. The same seed region is used for all tensor volumes and tracing is terminated when the anisotropy drops below a predefined anisotropy threshold or the angle of two consecutive main eigenvectors exceeds a given angular threshold. For each seed point this results in a set of fibers, one for each tensor volume. Each fiber describes one possible result of the streamline tracing algorithm. Because we keep the seed region fixed, we consider fiber variations for each seed point separately. To be able to assign a confidence value to each fiber we assume that the distribution of fibers in each seed point is unimodal. In that case, we can choose a distance measure between pairs of fibers and use it to define a fiber which is most central, given the distance measure, in the set of fibers at each seed point. We subsequently use each fiber’s distance to the central fiber as a representation of confidence. A confidence interval can then be expressed as: (1) all fibers with a confidence higher than a given threshold confidence (e.g., 0.95) or (2) a percentage (e.g., 50 %) of the most confident fibers. Which option is most appropriate depends on the user task and the distribution of confidence values in the total set of fibers. Option (1) could be more informative in studies of brain connectivity. Users generally have more knowledge about the underlying details of the tractography algorithm and understand the meaning of specific confidence values or probabilities. Such background knowledge cannot be expected from a neurosurgeon who is primarily interested in the spatial extent of fiber bundles and how it varies with uncertainty. In this case, specific confidence values or probabilities are probably less meaningful and, for this reason, the percentages provided by option (2) could be more intuitive. In any case, our framework supports both options.

Fiber distance measures have been extensively described in earlier work on fiber clustering [20] and comparison of fiber tracking algorithms [13]. The choice of distance measure is entirely dependent on the application within which it is used. Each measure has its advantages and disadvantages. For this reason, our framework is not dependent on the particular choice of distance measure in any way. To demonstrate its flexibility in this respect, we implemented two widely used measures: (1) the Minimum End-Point Distance and (2) the Mean of Closest-Point Distances [20]. The minimum end-point distance d _E between two fibers F _i and F _j is defined as follows:

$$ d_E(F_i,F_j) = \text{min}(d_1,d_2) $$

(1)

where,

(2)

(3)

where F _i,1 and F _i,end refer to the first and last points on fiber F _i . The mean of closest-point distances d _M is defined as follows:

$$ d_M(F_i,F_j) = \text{mean} \bigl(d_m(F_i,F_j),d_m(F_j,F_i) \bigr) $$

(4)

where, given that p _r and p _s are the points on each fiber,

$$ d_m(F_i,F_j) = \mathop{\text{mean}\ \text{min}}_{\mathbf{p}_r \in F_i, \mathbf{p}_s \in F_j} \Vert \mathbf{p}_r - \mathbf{p}_s\Vert $$

(5)

Given the assumptions described previously, we define, for each seed point, a medoid fiber which is the fiber with the smallest sum of distances to all other fibers at that seed point. This definition will result in an existing fiber and, given the chosen distance measure, this will be the most central fiber of the set. Our approach is similar to O’Donnell et al. [22] in that they also select an existing fiber instead of trying to compute an exact center line. The latter approach has been attempted in previous work [6, 8, 14]; however, this can lead to ill-defined results when streamlines strongly diverge or even turn back on themselves. Merhof et al. [19] circumvent this problem by reverting to a smoothed, voxel-based representation of the fibers which allows them to generate a tight fitting surface mesh around the tracts. However, their approach does not take uncertainty into account.

Our definition of fiber distance assumes a unimodal distribution of fibers originating from a seed point. However, this may not always be the case. If the distribution is multimodal, e.g., bimodal, then the medoid fiber ends up at the boundary of one mode, in between the two modes. Rendering a single, narrow confidence interval may not immediately show that there are two modes instead of only one. In this case, it may be necessary to preprocess the fibers using a fiber clustering algorithm which can detect the different modes in the fiber set. A survey of such algorithms was recently published by Schultz et al. [28]. After clustering, our illustrative rendering method can be used to visualize the different modes. Using the above-described fiber distances represents only one possible method of computing fiber confidence. Our framework is sufficiently general that it allows easy integration of more complex methods to compute fiber confidence.

3.2 ConTrack

ConTrack is a probabilistic fiber tracking algorithm proposed by Sherbondy et al. [31]. It is specifically designed to take into account knowledge that a fiber connection exists between two regions. For example, in a normally sighted individual, we know that there exists a functional connection between the lateral geniculate nucleus (LGN) and the primary visual cortex (V1 and V2). However, most probabilistic FT algorithms would estimate the connection probability between LGB and V1/V2 to be substantially less than one. The reason for this is that the probability of a connection between A and B, that is P(A→B), depends on the probability of connections between A and all other points. A consequence of this dependence is that in most cases connections are not symmetric, that is P(A→B)≠P(B→A). In contrast, ConTrack ensures both independence and symmetry. The algorithm outputs a large set of streamlines, each associated with a pathway score representing the probability of connection between the source and target regions. This fits the requirements of our visualization framework very well because the pathway score can be used directly as a measure of confidence. We do not claim that ConTrack is superior to other probabilistic methods such as proposed by Behrens et al. [2] or Friman et al. [10]. These methods also sample many pathways but do not always explicitly store the pathways as streamlines. Instead, the pathways are used to compute a probability density volume where each voxel value represents the number of pathways intersecting it.

4 Visualizing fiber confidence

In the current work, we have chosen to describe the uncertainty in fiber pathways in terms of discrete confidence intervals and represent the fibers in each interval by means of silhouettes and outlines. This approach has several advantages over traditional volume rendering where intervals might be defined using transfer functions. The quality of a volumetric representation would depend on data resolution. Also, interval surfaces defined by a transfer function can easily lead to visual clutter. Our silhouette and outline representation preserves the visual continuity of the original fibers and allows to deal with confidence intervals directly, without using a TF editor. It provides a high-level overview of the distribution of fibers in the set, similar to confidence intervals used in descriptive statistics. Some detail is lost by aggregating multiple fibers in a single interval, but if multiple confidence intervals are shown, which our framework allows, the user can obtain a good impression of which parts of the fiber tract are most reliable or reproducible and which parts are highly variable. Showing the fibers by means of streamlines, even if each confidence interval would be colored differently, will result in a large amount of visual clutter. Also, users are often not interested in individual fibers but rather in fiber bundles. A silhouette and outline representation fulfills these requirements by reducing visual clutter while preserving the overall shape of the fiber bundle. As explained before, we define a silhouette as the interior area of an object’s shape when projected to the viewing plane. An outline is defined as the border of a silhouette. Otten et al. [23] presented an approach to render silhouettes and outlines for sets of clustered fibers using a GPU-accelerated algorithm. We extend this approach in the following ways:

• Extending the set of visual styles (e.g., transparency, color schemes, blurring, etc.)

• Making each visual style adaptive to interval confidence

• Combining an arbitrary number of confidence intervals in multiple render passes

• Adding anatomical context by means of 3D orthogonal slices, volume rendering and surface models (e.g., tumor, brain ventricles, and cortical surface) and other types of streamline visualizations

Figure 4 gives an overview of the different stages in our visualization pipeline.

Fig. 4

Pipeline for visualizing our illustrative confidence intervals

4.1 Generating silhouettes and outlines

To create a silhouette and outline representation, we render the streamlines directly into an off-screen buffer image. Next, we apply a dilation operators to each nonempty pixel of the streamline image. The dilation operator is based on a circular structuring element with a user-defined silhouette radius R _s . This widens the area covered by the streamlines and closes holes between neighboring streamlines. A second, larger structuring element with a user-defined outline radius R _o is used to add an outline of a given thickness to the silhouette. The general mechanism is illustrated in Fig. 5.

Fig. 5

Creating a silhouette and outline. Blue pixels represent the added silhouette. Black pixels represent the added outline

4.2 Rendering confidence intervals

To visualize the illustrative confidence intervals, we transfer the streamlines and associated confidence table to our rendering pipeline (see Fig. 4). We then specify a confidence interval Δ=[a,b], where [a,b]⊆[0,1], to control which streamlines are rendered to screen as silhouettes and outlines. If the confidence interval is [0,1], we render all streamlines at once and the resulting silhouette represents all possible variations of streamlines. If we have N intervals, then each interval Δ _i where 0≤i<N, can be rendered with a different visual style. Figure 4 illustrates the different steps in our visualization pipeline. For each interval Δ _i these steps are executed as follows:

Step 1: :

We start with the interval of highest confidence, that is Δ _N−1 and render the corresponding streamlines to a color buffer C _STR and depth buffer D _STR using OpenGL framebuffer objects. Depth testing is set to GL_LESS.

Step 2: :

In this step, stencil testing is enabled (function: (GL_EQUAL,0,1), operation: GL_INCR) to ensure that subsequent intervals (with lower confidence) do not overwrite the current one (with higher confidence). We could have solved this by setting the depth test to GL_ALWAYS and rendering the low confidence intervals first. However, reducing the opacity of a given silhouette would then result in unwanted color mixing of the different intervals. If the opacity is reduced, we wish to see the anatomy lying behind it and not other confidence intervals. The buffers C _STR and D _STR from the previous step are passed to a GPU shader program that creates the silhouette and outline representation described previously. The output result of the shader program is written to the color buffer C _SILH and depth buffer D _SILH. In this step the visual style parameters are applied (see also Sect. 4.3).

Step 3 (Optional)::

This step is only performed if silhouette blurring is enabled. This allows the user to show intervals of lower confidence with increased blurring. The stencil test is now performed at this step instead of Step 2. The buffers C _SILH and D _SILH from Step 2 are passed to a second GPU shader program that applies Gaussian blurring to the color buffer. The shader program takes a blurring radius and brightness offset as parameters (see also Sect. 4.3). The brightness offset allows the user to adjust brightness in case it is too low after the blurring operation.

4.3 Confidence histogram widget

Depending on the distribution of confidence values within the fiber set, the user may wish to choose the confidence intervals differently. To make this possible, we introduce the Confidence Histogram Widget that shows a histogram of confidence values and allows the user to define and manipulate confidence intervals graphically. Additionally, it is possible to identify deviations from a unimodal distribution in the fibers. In Sect. 3, we explained how we use distance to compute fiber confidence. This approach assumes that the distribution of fibers originating from a seed point is uni-modal. If this is not the case, we have explained in Sect. 3.1 that this fact will easily show up in the visualization. Furthermore, if a deviation from unimodality occurs in multiple seed points, we are likely dealing with multiple fiber bundles instead of only one and the histogram will show multiple peaks. Figure 6 on the left illustrates an example histogram computed from output scores of the ConTrack algorithm. The confidence intervals are displayed as a set of semitransparent rectangles overlaid on top of the histogram. Each rectangle represents an interval. Rectangle colors map to silhouette or outline colors, depending on the current selection mode. Rectangle width maps to interval width. Rectangle height maps to a selected visual property value, such as opacity or blurring radius. The full list of visual properties available in our framework is given in Table 1.

Fig. 6

Left: Confidence Histogram Widget with confidence values increase from left to right. White arrows indicate possible manipulation direction. Selected active property is “Opacity.” Right: Corresponding confidence intervals of optic radiation fibers looping around ventricles

Table 1 User-adjustable properties

Visual properties can be set for each confidence interval separately or they can be assigned automatically through a number of presets. For example, the user can assign a number of standard color scales to the different intervals, such as warm to cool, light to dark, or decreasing saturation. Scalar properties such as opacity can be assigned in a staircase or inverted staircase pattern. Intervals can be automatically subdivided in equal widths, e.g., $[0,\frac{1}{2}]$ and $[\frac {1}{2},1]$. Alternatively, a subdivision can be selected where each interval contains the same amount of fibers. These two options reflect the two different perspectives we discussed in Sect. 3.1 where a confidence interval can be expressed as either (1) a percentage of the most confident fibers or (2) all fibers with a confidence between a certain range. As explained previously, which options is most appropriate depends on the user task. Finally, the confidence histogram widget allows saving and loading of visual styles. From a practical point of view, we do not expect the neurosurgeon to set these visual properties each time he or she uses the tool. The automatic presets, saving and loading of visual properties are specifically designed to make this process easier and less repetitive. However, to be able to properly evaluate the benefits and drawbacks of many different visual styles (as described in Sect. 5.2), we also provide low-level control of these properties.

4.4 Uncertainty lens

Uncertainty in DTI fiber tracking algorithms is an important issue in neurosurgical applications. However, some fiber tracts can be more reliably reconstructed than others, even with deterministic approaches. Also, not all regions of the brain are affected by the surgical approach. In such regions, showing uncertainty may be unnecessary or even confusing. For this reason, we provide the Uncertainty Lens as a Focus + Context approach to show uncertainty only within a user-defined region-of-interest (ROI). Outside the lens, standard streamline visualization can be used. For example, the surgeon can place the uncertainty lens over the tumor and its immediate surroundings and see the fiber variations due to the uncertainty within a relevant context. If necessary, the user can interactively move and resize the uncertainty lens. Figure 7(c) gives an example visualization.

Fig. 7

(a) Optic radiation, (b) pyramidal tract with tumor, (c) uncertainty lens, (d–i) different visual styles applied to optic radiation (warm-to-cool outlines, warm-to-cool surfaces, decreasing opacity, light-to-dark, increasing dilation, increasing blur)

5 Results and discussion

Figure 7 illustrates uncertainty visualizations created with our framework. Figure 7(a) depicts a transversal view of the optic radiation (running from the thalamus to the visual cortex) with brain ventricles rendered using DVR. Figure 7(b) provides a sagittal view of the corticospinal tract surrounding a tumor. Figure 7(c) illustrates our Uncertainty Lens together with a standard streamtube visualization, tumor and semi-transparent cortical surface. Figures 7(d–i) illustrate a close-up of the optic radiation with most of the visual styles we support (warm-to-cool outlines, warm-to-cool surfaces, decreasing opacity, light-to-dark, increasing dilation, increasing blur).

5.1 Datasets and performance

These visualizations were created with datasets from three suppliers. Kempenhaeghe Epilepsy Center: DTI (112×112×60, 2×2×2mm³, b-value 1000, 30 grad. dirs.). St. Elisabeth hospital: DTI (128×128×60, 1.75×1.75×2 mm³, b-value 800, 30 grad. dirs.), T1 MRI (288×288×175, 1×1×1 mm³). VisContest 2010: DTI (128×128×72, 1.8×1.8×1.8 mm³, b-value 1000, 30 grad. dirs.), T1 MRI (512×512×176, 0.488×0.488×1mm³). We measured the framerate performance of our rendering pipeline using a GeForce 8800 GTX graphics card from NVIDIA and a screensize of 720×760 pixels: Fig. 7(a) = 6.6 fps, Fig. 7(b) = 10.2 fps, and Fig. 7(c) = 17.5 fps.

5.2 User evaluation

To evaluate our visualization approach, we conducted an informal user study with three neurosurgeons. Before we presented our visualization method, we wanted to know their general opinion about the need and benefits of uncertainty visualization for their work. These prior opinions are recorded in Table 2. After this, we evaluated the specific rendering options that our visualization framework provides. We presented the neurosurgeons with a set of screen-shots, videos, and live demonstrations. Furthermore, we asked them to fill out a questionnaire with respect to the visual material. Three clinical scenarios were presented: (1) anterior temporal lobe resection (ATLR) for the treatment of focal epilepsy, (2) resection of a low-grade glioma close to motor cortex, and (3) resection of an intracerebral metastasis with surrounding edema (fluid), also close to motor cortex. For each scenario, we showed a standard fiber visualization (stream tubes) and our illustrative confidence intervals. We used three intervals each containing 10 %, 25 %, and 100 % of the most confident fibers. Table 3 shows the specific ratings for each question. Ratings are given on a [1–5] Likert scale where 1 means either “not useful at all” or “low” and 5 means “highly useful” or “high,” depending on the context of the question.

Q1.1 :

What is the risk of a visual field deficit in anterior temporal lobe resection?

Q1.2 :

How important is uncertainty visualization for anterior temporal lobe resections?

Q1.3 :

What is the risk of resecting this particular low-grade glioma?

Q1.4 :

How important is uncertainty visualization for low-grade glioma resection?

Q1.5 :

What is the risk of resecting this particular intracerebral metastasis?

Q1.6 :

To what extent would you discuss the uncertainty with the patient?

Q2.1 :

To what extent does the standard fiber visualization give you more confidence in assessing risk of anterior temporal lobe resections?

Q2.2 :

To what extent does the standard visualization give more confidence in assessing risk of resecting the given low-grade glioma?

Q2.3 :

To what extent does the standard visualization give more confidence in assessing risk of resecting the given intracerebral metastasis?

Q2.4 :

Which two visual styles do you prefer?

Q2.5 :

Does our representation of uncertainty give you more confidence?

Q2.6 :

How useful is it to show uncertainty only in a selected ROI?

Q2.7 :

What is the overall rating of the potential use of our visualization?

Table 2 General opinion about DTI uncertainty visualization. Ranking of visual styles rating is done on a [1–5] Likert scale

Table 3 Opinions about the proposed uncertainty visualization. Except for the ranking of visual styles rating is done on a [1–5] Likert scale. Scores are averages over all the three scenarios

The standard fiber visualization was considered somewhat helpful for scenarios 1 and 2. It gives at least a rough indication of the location of the fiber tract. For scenario 3, all users gave a low rating because fibers seemed to be missing inside the edema that, in their opinion, should have been present. This is caused by a failure of the fiber tracking algorithm to trace into low anisotropy regions such as edematous fluid. We presented the users with different visual styles for the illustrative silhouettes and asked them which style or combination of styles most clearly communicated the different levels of confidence. Initially, two out of three users rated decreasing opacity (with decreasing confidence) to be most intuitive. Light-to-dark, nongray coloring came second. After they finished the questionnaire, we discussed this first choice with the users and explained that reducing opacity actually removes information instead of showing it. They agreed but indicated that they selected decreasing opacity mainly because it prevents occlusion of the underlying anatomical slices (which is important for context). After some discussion, they proposed that a combination of light-to-dark coloring and a reduced, but fixed, opacity would be a good alternative for representing fiber confidence. The third user, interestingly, preferred equal colors for all intervals but with decreasing outline thickness (going from fat to thin to zero). All other visual styles (silhouette dilation, blurring, and warm-to-cool colors) were less appealing to the users. Our Focus+Context uncertainty lens was considered to be useful. The initial uncertainty visualization (without uncertainty lens) looked rather intimidating to them. The lens reduces visual clutter and allows attention to be focused on the tumor and its immediate surroundings. Also, the fully opaque visual styles were considered to be much more acceptable this way.

5.3 General discussion

As noted in the user evaluation, the neurosurgeons were skeptical about the fiber tracking results for the intracerebral metastasis. The edematous fluid surrounding the tumor causes the tracking algorithm to prematurely terminate thereby resulting in false negatives. Our particular implementation of the Wild Bootstrap method will also suffer from such false negatives (failing to show something that is actually there) because it uses the same fiber tracking algorithm with the same termination criteria. If diffusion in a voxel, because of edematous fluid, is almost isotropic, the tensor model will fit the data well and have very small residuals. Random perturbation of the residuals will not result in large shape variations, so on average the tensors will remain isotropic. Because the Wild Bootstrap method uses the same thresholds as standard streamline tracing, none of the generated streamlines will get past these isotropic tensors. With crossing fibers, however, the tensor can become disk-like while the underlying diffusion profile might actually be cross-shaped. In this case, the fractional anisotropy of the tensor remains high which may cause standard algorithms to continue tracking even though the model fit is bad (large residuals). The Wild Bootstrap will expose this problem by showing a large amount of variation in fiber pathways originating from the crossing fiber region. This indicates that the standard tracking algorithm may be generating random results. In order to deal with false negatives, alternative methods for probabilistic fiber tracking can be used. These are often based on Monte Carlo sampling of pathways and less dependent on stopping criteria [2, 10]. Even in regions of low anisotropy these algorithms can detect the presence of fiber pathways.

In this paper, we have applied the Wild Bootstrap approach using the 2nd-order tensor model and deterministic streamline tracing. However, these choices are not specific to the Wild Bootstrap. It is a general technique that can be applied to any model that tries to fit data and results in nonzero residuals. Even with higher-order models that are able to detect crossing fibers, you can still use the Wild Bootstrap to investigate the sensitivity of these models to noise in the data [12]. Our choice to use streamline tracing is also not the only choice. Other fiber tracking algorithms, such as tensor deflection (TEND) [16], can be used in combination with the Wild Bootstrap as well.

With respect to the computation of confidence based on fiber distance we wish to point out that the choice of distance measure may have a large effect on the confidence intervals. As Jiao et al. [13] point out distance measures that are based on averages, such as the mean of closest-point distances, may over or underestimate the true distance due to streamline discretization problems or complex streamline configurations. The closest end-points distance may surely over-simplify the situation. However, for certain cases it may provide relevant information. For example, if a priori knowledge is available that fibers start and end in the same anatomical regions, they could still be considered “close” even though they may take wildly different routes to get to their destination. In general we regard distance measures to be application-specific without any particular measure being the best one for all situations.

The user evaluation, although not conclusive, has provided us with useful information. Whether DTI fiber tracking, and therefore DTI uncertainty, is relevant for neurosurgical planning depends on many factors, such as tumor type. High-grade gliomas have a bad prognosis and are commonly associated with functional deficits in the patient. If there is any risk of additional damage due to resection, tumor tissue is simply left in place. Low-grade gliomas, on the other hand, have a relatively good prognosis. The tumor grows slowly allowing brain functions to reorganize if they get invaded (brain plasticity). In these cases neurosurgeons are willing to use all the information available in order to maximize resection while at the same time minimizing damage to healthy tissue. DTI fiber tracking and uncertainty visualization are especially relevant here. The discrepancy in risk assessment between user A (score 4.0) and users B and C (both 1.0) may be explained by a difference in interpretation of surgical risk. Since the tumor is close to the supplementary motor area, user A considered the risk (i.e., likelihood) of postoperative deficits quite high. However, after discussion, user A admitted that these deficits are almost always temporary and will disappear after a few weeks. Users B and C already took this recovery into account, found it to be normal, and saw no reason for a “high risk” classification. For anterior temporal lobe resections, DTI uncertainty information is important because part of the optic radiation, Meyer’s Loop, is often damaged during the procedure, resulting in a partial loss of vision. The extent of Meyer’s Loop varies significantly between patients (between 3 and 6 cm’s from the temporal pole). This makes it difficult to predict in advance whether visual defects will result or not. Meyer’s Loop, however, is a difficult fiber tract to reconstruct due to strong curvature and proximity to other fiber tracts (resulting in partial volume effects at the tract boundaries). Uncertainty visualization can provide helpful information in this case.

6 Conclusions and future work

In this paper, we have presented a framework for visualizing uncertainties in DTI fiber tracking due to noise and modeling errors. Our contribution consists of a processing pipeline for computing fiber confidence and an algorithm for visualizing fiber confidence intervals based on illustrative silhouettes and outlines. We also provide an interactive histogram widget to view the distribution of confidence values within a fiber set and adjust the visual parameters of each interval. Our uncertainty lens allows a Focus+Context view of both illustrative silhouettes and tumor (focus) and standard fiber visualization techniques with anatomical information (context). Finally, we have evaluated the added value of our uncertainty visualization for surgical risk assessment with three neurosurgeons. We concluded that our illustrative confidence intervals are a potentially useful approach to display uncertainty. Furthermore, we confirmed that uncertainty visualization is of great value for surgical risk assessment and may play an important role in patient counselling.

For future work, we would like to experiment with more visual styles and test these in a larger group of users. In this paper, we have focused our user evaluation on neurosurgeons who are actually involved in diagnosis and surgical risk assessment. We would like to continue and expand our collaboration with these medical experts to find ways to apply DTI uncertainty visualization in a clinical context for specific surgical procedures, such as tumor resection or epilepsy surgery. This requires dedicated visualization and specialized interaction paradigms that are fast, intuitive, and easy to use without requiring a detailed understanding of the underlying algorithms. Also, we would like to investigate alternatives for computing fiber confidence that do not rely on a unimodal distribution of fibers originating from a single seed point.