1 Introduction

Image registration is a cornerstone of brain imaging applications. After registration of the data of two brains, homologous anatomical structures are expected to overlap better than before registration. Typically, brain images are registered to atlases, or between different subject, for different purposes, such as: conducting group-studies, guiding segmentation, or for building new atlases [9].

Diffusion magnetic resonance imaging (dMRI) techniques measure the orientation of diffusion of water molecules in biological tissues. By means of tractography techniques on dMRI data from the brain, it is possible to obtain a representation of the anatomical connections in the white matter, as hundreds of thousands of streamlines, forming the tractogram.

The linear or nonlinear transformation to register the tractogram of one subjects to that of another subject can be estimated in two main ways: from volumetric data, such as T1-weighted, fractional anisotropy (FA), or orientation distribution functions (ODFs), see [3, 4, 11], or directly from streamlines. In this last case, the literature on nonlinear methods addresses the registration of bundles, which are just a portion of the tractogram, usually because of the very high computational cost of the algorithms, see [5, 17, 18, 20, 21]. Differently, in the case of linear methods, solutions to compute an affine transformation from whole tractograms are available, see [7, 9].

Recently, a new concept has been proposed to accurately align tractograms, based on the idea of streamline correspondence [10] and formalized as a graph matching (GM) problem. There, the building block of the methodology is to compute which streamline of the first subject corresponds to which streamline of the second subject, as in a combinatorial optimization problem. The principle of streamline correspondence has also been used for the problem of bundle segmentation [6, 8, 12, 13, 19].

In this work, we present two contributions: first, we propose the use of the linear assignment problem (LAP) as computational building block to align entire brain tractograms from streamline information, extending the work of [12], which addressed only bundles and segmentation. The alignment obtained with LAP acts locally, as in a nonlinear transformation. Second, we quantitatively compare the proposed method against some methods in the state of the art for tractogram alignment, for which the implementation is freely-available. To the best of our knowledge, it is pretty infrequent to find quantitative comparisons between tractogram alignment methods, in the literature.

In order to carry out the quantitative comparison, we designed an experiment where, given the tractograms of two subjects, we computed the transformation of the first in order to match the second one. Similarly to [10], we quantified the accuracy of the whole tractogram alignment by estimating the degree of overlap between homologous bundles. The reason behind it is that, after registration, the anatomy of the white matter of the two subjects should match more accurately than before. On 90 pairs of subjects from the dMRI dataset of the Human Connectom Project [14], we compared the registration obtained with 5 different methods, using 10 main bundles as landmarks for the quantification.

The results show that the proposed method outperforms voxel-based methods and streamline-based methods in almost all cases. Moreover, as expected, linear methods are outperformed by nonlinear methods. The Symmetric Normalization (SyN) algorithm of the Advanced Normalization Tools (ANTs, see [1]), despite being designed for volumetric images, shows excellent results against streamline based methods, outperforming them in some cases. The main limitation of streamline-based methods is the high computational cost, which requires to resort to approximations.

In the following, we briefly describe the proposed method. Then, in Sect. 3, we describe the details of the experiments. In Sect. 4 we discuss the claims in the light of the results.

2 Methods

Let \(s = \{\mathbf {x}_1, \dots , \mathbf {x}_n\}\) be a streamline, i.e. a sequence of points in 3D space, i.e. \(\mathbf {x}_i = [x_i, y_i, z_i] \in \mathbb {R}^3\), \(\forall i\). Let \(T = \{s_1, \dots , s_N\}\) be a tractogram and \(b \subset T\) the set of streamlines corresponding to a white matter bundle of interest, e.g. the cortico-spinal tract (cst). With \(|\cdot |\) we indicate the number of objects in a set. Typically, |T| is in the order of \(10^5\)–\(10^6\). Several distances has been defined between streamlines, see [9] In this work, we adopt the commonly adopted mean of closest distances, \(d(s_a, s_b) = \frac{d_m(s_a, s_b) + d_m(s_b, s_a)}{2}\), where \(d_m(s_a, s_b) = \frac{1}{|s_a|} \sum _{\mathbf {x}_i \in s_a} \min _{\mathbf {x}_j \in s_b} ||\mathbf {x}_i - \mathbf {x}_j||_2\).

2.1 Streamline Correspondence

Given two tractograms, \(T_A\) and \(T_B\), the problem of aligning them can be framed as finding the correspondences between each streamline in \(T_A\) the corresponding one in \(T_B\), see [10]. If \(s^A_i \in T_A\) corresponds to \(s^B_j \in T_B\), the transformation obtained from such correspondence is the one that returns \(s^B_j\) when given \(s^A_i\). The set of correspondences will provide a good alignment of tractograms if, after the transformation, homologous anatomical structures will match. Previously, the task of finding a good set of correspondences has been formulated as graph matching [10].

2.2 Linear Assignment Problem

Given two sets of objects, e.g. \(T_A\) and \(T_B\), of the same size N, and a \(N \times N\) cost matrix C, whose element \(c_{ij} \in \mathbb {R}\) is the cost of assigning \(s^A_i \in T_A\) to \(s^B_j \in T_B\), e.g. \(c_{ij} = d(s^A_i,s^B_j)\), then the linear assignment problem (LAP) is the combinatorial optimization problem that attempt to optimally assign each element of \(T_A\) to each element of \(T_B\), with a one-to-one assignment, minimizing the total cost:

$$\begin{aligned} P^* = \mathop {{{\,\mathrm{argmin}\,}}}\limits _{{P \in \mathcal {P}}} \sum _{i=1}^N \sum _{j=1}^N c_{ij}p_{ij} \end{aligned}$$
(1)

where \(\mathcal {P}\) is the set of all possible one-to-one assignments, each represented as a \(N \times N\) permutation matrix, i.e. \(P \in \mathcal {P}\) is a binary matrix where each row and column sum up to 1 and the element \(p_{ij}=1\) if \(s^A_i\) is assigned to \(s^B_j\) and 0 otherwise. Notice that LAP for streamlines, the minimization of the total cost is the minimization of the distances of corresponding streamlines and, in many cases, the one-to-one constraint forces the correspondences to follow the local differences between the anatomical structures, see [13]. When \(|T_B| > |T_A|\), the problem is called rectangular LAP (RLAP), which seeks the best assignment of \(\{s^A_1,\dots ,s^A_N\}\) to subset of size N of \(T_B\). The most efficient algorithm to find the optimal solution of LAP and RLAP is LAPJV, see [2], whose time complexity is \(\mathcal {O}(N^3\)) and space complexity is \(\mathcal {O}(N^2)\). In practice, even LAPJV is unfeasible to be executed on problems where \(N > 10^4\), both in terms of time and memory required.

2.3 Large-Scale Approximation

The correspondence between entire tractograms cannot be computed even with LAPJV, because of the excessive computational cost. Here, we adopt a hierarchical two-steps procedure that exploits the geometrical structure of tractograms. In the first step, both \(T_A\) and \(T_B\) are clustered into k clusters, named \(\{\alpha _i\}_{i=1 \dots k}\) for \(T_A\) and \(\{\beta _j\}_{j=1 \dots k}\) for \(T_B\). Then, each tractogram is simplified with k streamlines, i.e. the centroids of the respective clusters. Then, LAP is computed between the two simplified tractograms, as explained above, i.e. a \(k \times k\) LAP is solved. This first step aims at finding corresponding clusters across the two tractograms, e.g. \(\alpha _i\) corresponds to \(\beta _j\). In the second step, given two corresponding clusters, the correspondence of streamlines is computed by solving the RLAP between the streamlines of the two clustersFootnote 1. Details are given below. In total, the two-steps procedure requires to solve 1 LAP with a \(k \times k\) cost matrix and k RLAPs each with, approximately, a \(\frac{N}{k} \times \frac{N}{k}\) cost matrix. For this reason, the resulting time complexity is reduced from \(\mathcal {O}(N^3)\) to \(\mathcal {O}\left( k^3 + \frac{N^3}{k^2}\right) \) and space complexity from \(\mathcal {O}(N^2)\) to \(\mathcal {O}(k^2 + \frac{N^2}{k})\).

In the second step of the procedure, assuming cluster \(\alpha _i \subset T_A\) to correspond to cluster \(\beta _j \subset T_B\), there are two possible scenarios: either (i) \(|\alpha _i| \le |\beta _j|\), for which finding the corresponding streamlines of \(\alpha _i\) in \(\beta _j\) is a straightforward RLAP, or (ii) \(|\alpha _i| > |\beta _j|\), for which there are not enough streamlines in \(\beta _j\) to set up a LAP or RLAP. In this last case, we propose to compute the corresponding streamlines of \(\alpha _i\) by violating the one-to-one constraint and assigning one or streamlines of \(\alpha _i\) to each of \(\beta _j\). The procedure is the following: first we solve the reverse RLAP, i.e. we compute the optimal assignment of all the streamlines in \(\beta _j\) to (some of) those in \(\alpha _i\). In this way, a subset of the streamlines in \(\alpha _i\) will obtain their corresponding ones in \(\beta _j\). We denote such subset of assigned streamlines as \(\alpha _i^a \subset \alpha _i\) and that of the remaining ones, i.e. the non assigned streamlines, as \(\alpha _i^{na} \subset \alpha _i\). Then, for each non assigned streamline \(s \in \alpha _i^{na}\), we compute its nearest neighbor in \(\alpha _i^a\). Finally, we define the corresponding streamline of \(s \in \alpha _i^{na}\) as the one corresponding to its nearest neighbor in \(\alpha _i^{a}\).

3 Experiments

We selected 30 healthy subjects at random from the publicly available Human Connectome Project (HCP) dMRI dataset [14] (90 gradients; b = 2000; voxel size = 1.25 mm isotropic). For each subject, tractograms of 400–500 thousands streamlines were obtained using the constrained spherical deconvolution (CSD) algorithm [15] and the local deterministic tracking algorithm implemented in DiPyFootnote 2 (step size = 0.625 mm, 1 seed/voxel from the white matter). We segmented 14 major bundles from each tractogram, using the TractQuerier/white matter query language (WMQL, see [16]) and we used some of them as ground truth. In order to reduce the impact of poor segmentations obtained in some cases, we jointly selected 10 subjects and 10 bundles in order to minimize the differences in number of streamlines for each bundles across the subjects. These bundles are (both left and right): cingulum (cb), cortico-spinal tract (cst), inferior fronto-occipital fasciculus (ifof), thalamo prefrontal (thpref) and uncinate fasciculus (uf). We visually inspected the resulting bundles to avoid outliers. The experiments were then conducted on the 90 pairs of different tractograms that can be obtained from the selected 10 subjects.

3.1 Comparison

We quantified the quality of alignment between two tractograms as the degree of overlap between the voxel masks of homologous bundles, after registration, see [7, 10]. The degree of overlap was quantified as dice similarity coefficient (DSC): \(DSC = \frac{ 2 \times (|v(\hat{b}_A) \cap v(b_B)|)}{|v(\hat{b}_A)| + |v(b_B)|}\), where \(v(\hat{b}_A)\) is the voxel mask of the bundle \(b_A \in T_A\) after the alignment of the entire tractogram \(T_A\) to \(T_B\). In other words, \(v(\hat{b}_A)\) attempts to approximate the voxel mask of the homologous bundle of the target subject, \(v(b_B)\), considered as ground truth. In the comparison, we considered the following methods to align tractograms: 1) registration based on anterior and posterior commissures (AC-PC), directly provided within the HCP dataset, used as baseline. 2) Streamline linear registration (SLR, from DiPy, see [7]): in [10], SLR has shown slightly superior quality of registration with respect to other linear methods, so we considered it as a good representative of the linear methods. 3) The voxel-based nonlinear registration method of ANTsFootnote 3, see [1], used with default values. As reference volume, we considered the T1w images of the two subjects and the fractional anisotropy (FA) volumes. 4) DeformetricaFootnote 4 [5], a diffeomorphic streamline-based registration method for bundles. Streamlines were modeled as varifolds and we used 7 mm and 15 mm for the varifolds and diffeomorphic kernel bandwidths respectively. 5) Correspondence between streamlines as graph matching (GMFootnote 5, see [10]). 6) Correspondence between streamlines as linear assignment problem (LAP) (Sect. 2). Some of the methods have too high computational cost when computed on whole tracotgrams. For this reason, tractograms were simplified for such methods following the simplification step described in Sect. 2.3, using the fast mini-batch k-means algorithm, as described in [10]. For each tractogram, we computed the approximate k-means clustering on streamlines, with \(k=1000\) and \(k=5000\). The simplified tractogram consisted of the k centroid streamlines. Note that such values of k ensure an extensive coverage of the brain, which we can assume to be enough to guarantee reasonable one-to-one assignments. We provide code and datasets of all experiments under a Free/OpenSource license here: https://github.com/FBK-NILab/WBIR2020_experiments.

3.2 Results

In Table 1 we report degree of overlap (higher is better) between homologous bundles after whole tractogram registration, with different methods. The DSC value is averaged over 90 pairs of subjects. The standard deviation of the means is always below 0.01. All computations were executed on a modern desktop computer, i.e. Intel Xeon E5 8 cores, 3.50 GHz, 16 Gb RAM, always using only CPUFootnote 6. In Fig. 1, we show an example of matching between homologous bundles (IFOF left) after whole tractogram alignment for some of the methods.

Table 1. For each of the 10 bundles considered in this study (one per column), the table reports the degree of voxel-overlap after whole brain tractogram alignment, quantified as DSC (higher is better) and averaged over 90 pairs of subjects, across different methods - one per row. In each cell, the standard deviation of the mean is always below 0.01. In bold face are reported the highest values for each bundle, as well as those within 0.01 from them. The last column reports the computational time in minutes for aligning a pair of tractograms.
Full size table
Fig. 1.
figure 1

Example of homologous bundles after tractogram registration of \(T_A\) (HCP subject ID: 199655) to \(T_B\) (HCP subject ID: 599671). In green, on the left, the IFOF left of the static \(T_B\), as segmented by WMQL. In yellow, the IFOF left of the moving \(T_A\), after tractogram registration with four different methods. Red circle indicates the location of major differences with respect to the (green) IFOF left in \(T_B\). (Color figure online)

Full size image

4 Discussion and Conclusions

In this work, we describe the use of the linear assignment problem (LAP) to align entire tractograms of two different subjects, by introducing approximations as computational shortcuts. The LAP acts locally, as a nonlinear registration method. In Table 1, we compare the proposed method with some linear and nonlinear methods in the literature. The results show that LAP (\(k=5000\)) outperforms other methods on almost all bundles. The only exception is the cingulum (cbL, cbR) for which ANTs provides significantly better DSC, with LAP second in the ranking.

The results also show a number of other interesting facts and confirm basic sanity checks: despite the limited quality of the ground truth provided by the WMQL, the values of DSC increase steadily from AC-PC registration, to linear registration and to nonlinear methods. 90 pairs of subjects selected as described in Sect. 3 are enough to keep the standard deviation of the means low enough (\(\le \)0.01) to clearly see differences between the methods. It is also reassuring that, for each bundle, the results are sufficiently similar across the two hemispheres, e.g. ifofL and ifofR obtain almost the same score, for all methods. Nonlinear methods outperform linear methods in all cases, with the exception of Deformetrica, most probably because we could not perform a large model selection for the user-selected parameters due to the high computational time. Furthermore, the introduced approximations might interfere too much with the underlying method, which is tailored to register bundles. ANTs provided excellent results, given the fact that it does not operate on streamline information. Most probably this occurs because the grid on which ANTs operates, i.e. the voxel grid, is much more dense than what the simplified tractograms offer. The results between graph matching (GM) and LAP are not very different, for the same level of approximation (\(k=1000\)), with LAP in advantage. This advantage can be explained by the fact that LAPJV computes the exact solution of the RLAPs, while in the case of GM the underlying algorithm, DSPFP (see [10]), provides only an approximate solution. Notably, LAP is 8 times faster than GM for this size of tractograms, see Table 1 last column, which allowed us to run LAP with \(k=5000\) in a reasonable amount of time and to obtain substantially superior scores.

All this evidence supports the hypothesis that the results are limited also by the level of approximation and that, by improving algorithms and implementations to reduce computational cost, some of the methods may reach even better results.