Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The human cerebral cortex is a highly convoluted structure of gray matter. Geometrically, its surface is topologically equivalent to a sphere (without holes and handles), when artificially closing the midline hemispheric connections. Reconstruction of topologically correct and accurate cortical surfaces from MR images plays a fundamental role in neuroimaging studies [1]. However, due to the highly folded nature of the cortex and limitations in the MRI acquisition process, it is inevitable to have errors in brain tissue segmentation, which is a prerequisite for cortical surface reconstruction. This situation is especially severe in infant brain MR images, which typically have extremely low tissue contrast, severe partial volume effects, and regionally-heterogeneous, dynamically-changing imaging appearance patterns across time, due to the rapid brain growth and ongoing myelination [2]. For example, at birth, T2-weighted images typically have much better contrast than T1-weighted images; at 1 year of age, T1-weighted images have much better contrast than T2-weighted images; at 6 months of age, both T1- and T2-weighted images exhibit extremely low contrast. Although some infant-dedicated tissue segmentation method [3] was proposed and achieved reasonable segmentation results, they do not guarantee the topological correctness of the reconstructed infant cortical surface, and topological errors caused by inaccurate segmentation are frequently seen during surface reconstruction, as shown in Fig. 1. Of note, even a very small error in segmentation could lead to a significant topological defect (the enlarged view in Fig. 1), thus bringing errors to the cortical surface-based analysis, e.g., measuring and processing structural and functional signals based on the geodesic (e.g., the red dotted curve in Fig. 1) in the cortical surface.

Fig. 1.
figure 1

Illustration of topological errors in infant cortical surfaces.

Topological correction typically involves two sequential tasks, i.e., (1) locating topologically defected regions and (2) correcting them. For the former task, methods largely rely on the priori knowledge that each cortical hemisphere has a simple spherical topology. Based on this, cyclic graph loops [46] or overlapping surface meshes after remapping [79] are used as hints to locate regions with topological defects. The latter task is much more challenging, as the two types of topological errors, i.e., holes and handles, are essentially only different in terms of their inconsistency with the cortex anatomy. Typically, holes incorrectly perforate the cortical surface, while handles erroneously bridge the nonadjacent points in the cortical surface, as shown in Fig. 1. In this context, topological correction methods have to make a choice between the two correction types: filling a hole or breaking a handle. However, since the difference between holes and handles actually lies in the sense of anatomical correctness, they are hard to distinguish solely using geometric information. So heuristics were usually made to address this issue. For example, a minimal correction criterion was adopted by assuming that the change for correction should be as small as possible [4, 5, 7, 10]. As this criterion is not reliable enough, several ad hoc rules based on MRI appearance patterns were proposed [6, 8, 11] to help determine the correction type. Although these methods achieve good performance on adult cortical surfaces, they have major limitations in processing infant cortical surfaces for two reasons. First, the minimal correction criterion typically cuts the handles of large topological defects frequently occurring in infant images, thus leading to anatomical regions missing or inconsistent. Second, the ad hoc rules designed based on adult MRIs (typically with clear contrast) are invalid for the infant MRIs, which have longitudinally changing and regionally heterogeneous intensity patterns. Hence, methods for handling infant MR images at a variety of developmental stages are highly desired.

In this paper, we propose a novel learning-based method for correcting the topological defects in infant cortical surfaces, without requiring predefined rules as in the existing methods. Specifically, we first locate topologically defected regions by using a topology-preserving level set method. Then, by leveraging rich information of the corresponding patches from anatomical reference images with correct and accurate topology, we build region-specific dictionaries and infer the correct tissue labels using sparse representation. Notably, we further integrate these two steps as an iterative framework to gradually correct large topological errors that frequently occur in infant MR images and cannot be completely corrected in one-shot sparse representation. Extensive experiments demonstrate the feasibility and effectiveness of our method.

2 Method

Given a tissue segmentation image \( {\mathbf{V}} \), labeled as white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF), our method includes two stages: extracting candidate voxels (Sect. 2.1) and inferring their new tissue labels (Sect. 2.2). Then, these two stages are further integrated into an iterative framework (Sect. 2.3).

2.1 Extracting Candidate Voxels

To locate candidate voxels involved in topological defects, we propose to leverage a topology-preserving level set method [12]. Specifically, for each hemisphere, a level set function with a spherical topology is first initialized by a large ellipsoid containing all WM and GM voxels. Then the level set function is gradually shrunk towards the WM surface, and meanwhile preserves its initial topology by carefully checking the “simple” voxels in topology during the evolution. Briefly, for a binary volume, a voxel is called “simple” if its addition or removal from the volume does not change the target’s topology. On the contrary, a voxel is called “non-simple” if these manipulations change its topology [12]. During the level set evolution, the judgment on a voxel’s simpleness is regarded as the topology-aware constraint, which always enables the evolving surface to keep the genus-zero-topology. Therefore, the converged volume of the level set (LS) evolution \( {\mathbf{V}}_{\text{LS}} \) can be considered as the result of a pure hole-filling process on \( {\mathbf{V}}_{\text{WM}} \), where all the hole errors are successfully fixed, while all the handle errors are failed. However, this level set evolution process is still very useful as it facilitates us in extracting candidates (CAN) of topological defects \( {\mathbf{V}}_{\text{CAN}} \) by a simple XOR operation, \( {\mathbf{V}}_{\text{CAN}} = {\mathbf{V}}_{\text{LS}} \oplus {\mathbf{V}}_{\text{WM}} \).

However, \( {\mathbf{V}}_{\text{CAN}} \) only provides all the hole positions in \( {\mathbf{V}}_{WM} \) in a pure topological sense. Considering anatomical correctness, these holes actually have different origins. Some of them are caused by erroneous perforations of WM (holes in anatomy, such as blue arrows in Fig. 1), while others are spin-off products of the erroneous connection of WM (handles in anatomy, such as red arrows in Fig. 1). Hence, the obtained \( {\mathbf{V}}_{\text{CAN}} \) only covers all the hole voxels, but does not contain any handle voxels. So we further enroll the neighboring voxels by a morphological dilation that is adaptive to the shape of the local structures in \( {\mathbf{V}}_{\text{CAN}} \). Specifically, for each connected region in \( {\mathbf{V}}_{\text{CAN}} \), we first compute its three PCA coefficients. We then deform the originally isotropic dilator according to these PCA coefficients and directions. Finally, the connected region is dilated with this region adaptive dilator. By gathering all the dilated results of connected regions, an updated version of the candidate set for topological correction \( {\mathbf{V}}_{\text{CAN}} \) is generated, containing voxels from both holes and handles. In this way, the cardinality of \( {\mathbf{V}}_{\text{CAN}} \) is smaller than simply using an isotropic dilator.

2.2 Inferring New Labels of Candidate Voxels Using Anatomical References

As topological defects stem from mislabeled brain tissue voxels, the task of topological correction can be considered as a classification problem, with the goal of accurately inferring new labels for the topologically defected voxels. To this end, we leverage a sparse representation model based on multiple reference volumes \( \left\{ {{\mathbf{R}}^{k} } \right\}(k = 1, \ldots ,K) \), which are the manually corrected volumes by experts and thus are free of topological errors. Given the fact that the common morphological patterns of brain anatomy exist across subjects, the new labels of voxels involved in topological defects in a subject can be learned from the reference volumes \( \left\{ {{\mathbf{R}}^{k} } \right\} \), thus improving the topological correctness and anatomical accuracy. The details are described as follows.

For a candidate voxel \( v \in {\mathbf{V}}_{\text{CAN}} \), we collect the tissue labels from v and its \( d_{c} \times d_{c} \times d_{c} \) neighbors, and represent v by a cubic patch \( {\mathbf{c}}(v) \), which encodes information of local anatomical structure. Topologically defected voxels can be considered as noises that contaminate the patch’s morphology. For \( {\mathbf{c}}(v) \), we build a region-specific dictionary \( {\mathbf{D}}(v) \) composed of “clean” patches from \( \left\{ {{\mathbf{R}}^{k} } \right\} \) without topological errors. Considering the inter-subject variability, each volume in \( \left\{ {{\mathbf{R}}^{k} } \right\} \) is firstly non-linearly aligned to the input \( {\mathbf{V}} \) (the tissue-segmented infant MR image) using the Diffeomorphic Demons method [13]. In this way, the morphed \( \left\{ {{\mathbf{R}}^{k} } \right\} \) play a better role as the anatomical reference, since more accurate correspondence can be established between \( {\mathbf{V}} \) and \( \left\{ {{\mathbf{R}}^{k} } \right\} \). \( {\mathbf{D}}(v) \) is then built as follows. Considering possible errors in image registration, for the corresponding voxel \( r_{v}^{k} \) in each morphed volume in \( {\mathbf{R}}^{k} \), we gather patches of \( d_{c} \times d_{c} \times d_{c} \) size from \( r_{v}^{k} \) and its \( d_{n} \times d_{n} \times d_{n} \) neighbors. Hence,\( {\mathbf{D}}(v) \) is formed as a \( (d_{c} )^{3} \times (K \cdot (d_{n} )^{3} ) \) matrix, where each column (atom) is the vectorized patch and \( K \cdot (d_{n} )^{3} \) is the number of the atoms collected from the morphed \( \left\{ {{\mathbf{R}}^{k} } \right\} \).

Based on the dictionary \( {\mathbf{D}}(v),{\mathbf{c}}(v) \) is then linearly reconstructed by a weighting vector \( {\mathbf{w}} \) using a sparse representation model, with the following motivations. First, we intend to exclude the irrelevant patches in \( {\mathbf{D}}(v) \) during the reconstruction, which adversely affect the representation results. So an \( \ell_{1} \) LASSO penalty term is adopted to keep their weights close to zero. Second, as the atoms are densely extracted in \( d_{n} \times d_{n} \times d_{n} \) neighborhood for each morphed \( {\mathbf{R}}^{k} \), some of them can be highly similar to each other and should be either jointly selected or ignored. As the \( \ell_{1} \) LASSO term tends to select only one atom from a group and ignore others, we add an \( \ell_{2} \) penalty. Thus the model is finally formulated as a non-negative Elastic-Net problem [14]:

$$ { \hbox{min} }_{{{\mathbf{w}} \ge 0}} \left\| {{\mathbf{c}}(v) - {\mathbf{D}}(v){\mathbf{w}}} \right\| + \lambda_{1} \left\| {\mathbf{w}} \right\|_{1} + \lambda_{2} \left\| {\mathbf{w}} \right\|_{2}^{2} $$
(1)

The element \( w_{i} \) in the weight vector w indicates the appearance similarity between \( {\mathbf{c}}(v) \) and the i-th atom in \( {\mathbf{D}}(v) \). Herein, the center of the i-th atom in \( {\mathbf{D}}(v) \) is a voxel \( r_{i} \) in the reference image. Based on the assumption that the appearance similarity \( w_{i} \) also reveals the likelihood that v in subject shares the same label as \( r_{i} \), we can infer the new label of v with a weighted nearest neighbor model. Denoting \( l_{v} \) as the tissue label of v, we can compute the probability of \( l_{v} = j \), where \( j \in \left\{ {{\text{WM}}, {\text{GM}}, {\text{CSF}}} \right\} \).

$$ p\left( {l_{v} = j} \right) = \sum\nolimits_{i = 1}^{{K \cdot (d_{n} )^{3} }} {w_{i} p(l_{v} = j|r_{i} )} $$
(2)
$$ p\left( {l_{v} = j |r_{i} } \right) = \left\{ {\begin{array}{*{20}c} 1 & { l _{{r_{i} }} = j} \\ 0 & {otherwise} \\ \end{array} } \right. $$
(3)

The new label of \( v \) is finally obtained by the MAP criteria, i.e., \( { \arg }\max_{j} p\left( {l_{v} = j} \right) \).

2.3 Iterative Framework

As infant cortical surfaces often contain large handles or holes, which generally cannot be completely corrected using one-shot sparse representation, we further propose to integrate the above two steps in an iterative framework to gradually refine the topological correction results. The whole algorithm is summarized as follow:

This framework brings two benefits. First, large topological defects in infant cortical surfaces are gradually corrected, as the algorithm updates candidate voxels in each iteration. Second, the cardinality of the candidate voxels decreases during the iterations, because successfully fixed defects are no longer included in the next iteration. The computational cost is mainly determined by the dictionary size, the cardinality of \( {\mathbf{V}}_{\text{CAN}} \), and the iteration number.

3 Experiments

To validate our method, brain MR images with the resolution of 1 × 1 × 1 mm3 from 100 infants at 6 months of age were used in experiments. As our method only relies on tissue segmentation results, we note that it is generic, and can also be applied to adult brains and infant brains at other developmental stages, such as neonates and 1-year-old. The main motivation of using 6-month-old infants for validation is that, among all stages during early brain development, MR images at 6 months exhibit the lowest tissue contrast and thus the most severe topological errors in tissue segmentation. Herein, the tissue segmentation was conducted by the state-of-the-art method in [3]. After segmentation, experts manually corrected the topological errors in the cortical surfaces of WM for all subjects, by using ITK-SNAP. Among the 100 pairs of uncorrected and manually corrected volumes, 20 manually corrected volumes are randomly selected as \( \left\{ {{\mathbf{R}}^{k} } \right\}(k = 1, \ldots ,20) \). One half of the rest 80 pairs were randomly selected for adjusting parameters and the other half were for performance evaluation.

We use the successful rate \( S_{c} \) to quantitatively evaluate our method:

$$ S_{c} = \frac{{\# ({\text{successfully}}\,{\text{corrected}}\,{\text{topological}}\,{\text{defects}})}}{{\# ({\text{topological}}\,{\text{defects}})}} $$
(4)

Here the successfully corrected topological defects indicate that holes are correctly filled or handles are correctly broken. However, \( S_{c} \) is limited in reflecting the anatomical consistency between the resulting surface and the ground truth. So we also adopt the Dice Ratio (DR) and average Surface Distance (SD) as the evaluation measures:

$$ DR = \frac{{2 \times \left| {{\mathbf{V}}_{1}^{'} \mathop {\bigcap }\nolimits {\mathbf{V}}_{2}^{'} } \right|}}{{\left| {{\mathbf{V}}_{1}^{'} } \right| + \left| {{\mathbf{V}}_{2}^{'} } \right|}} $$
(5)
$$ SD({\mathbf{V}}_{1} ,{\mathbf{V}}_{2} ) = \frac{1}{2}(\frac{1}{{n_{1} }}\sum\nolimits_{{v_{1} \in surf({\mathbf{V}}_{1} )}} {d(v_{1} ,surf({\mathbf{V}}_{2} ))} + \frac{1}{{n_{2} }}\sum\nolimits_{{v_{2} \in surf({\mathbf{V}}_{2} )}} {d(v_{2} ,surf({\mathbf{V}}_{1} )))} $$
(6)

where \( {\mathbf{V}}_{1} \) is the output of a topological correction method, and \( {\mathbf{V}}_{2} \) is the manually corrected WM volume. Of note, to better reflect the performance of topology correction using DR, we only use those regions enclosing the candidate voxels and their adjacent voxels in \( {\mathbf{V}}_{1} \) and \( {\mathbf{V}}_{2} \), i.e., \( {\mathbf{V}}_{1}^{'} \) and \( {\mathbf{V}}_{2}^{'} \), obtained by dilation of the set of the candidate voxels. In Eq. 6, \( d( \cdot , \cdot ) \) is the Euclidean distance, and \( n_{1} \) and \( n_{2} \) are cardinalities of \( surf({\mathbf{V}}_{1} ) \) and \( surf({\mathbf{V}}_{2} ) \), respectively.

Based on the validation set, we found the best \( S_{c} \) was achieved by setting \( \lambda_{1} = 0.2,\lambda_{2} = 0.01,d_{c} = 11,d_{n} = 5 \), T = 4, which were then applied to the testing set. Figure 2 shows an example of topological correction result by our method. We can see that our method can effectively fix topological defects and meanwhile ensure the anatomical consistency and correctness. In the iterative framework (visually validated in Fig. 3), four iterations (T = 4) are empirically enough for all the cases in our experiments.

Fig. 2.
figure 2

Examples of topological correction results by our method.

Fig. 3.
figure 3

Examples of iterative correction of topological defects.

As there is no available software specifically designed for correcting infant cortical surfaces, we compared our method with two popular software BrainSuite [5] and FreeSurfer [8], which are designed for processing the adult brain and achieve the state-of-the-art performance in the field. We show typical results in Fig. 4 and quantitative results in Table 1. Due to the minimal correction criterion, BrainSuite does not fully remove the handle regions, e.g., the red ellipses in Fig. 4. More importantly, it erroneously breaks too many holes that should be filled, e.g., the blue ellipses in Fig. 4, leading to a low \( S_{c} \). In contrary, our learning-based method and FreeSurfer achieve much better \( S_{c} \) than BrainSuite. However, FreeSurfer has low accuracy in terms of DR and SD, indicating poor anatomical consistency and correctness. For example, in Fig. 4, the gyral structures highlighted by the ellipses in FreeSurfer’s results are missing, compared with the ground truth. After checking all experimental results, we found that the similar problem of missing large gyral structures occurred in over half of the FreeSurfer’s results, resulting in a clear drop in DR and significant increase in SD in Table 1. In contrast, our method produces more balanced results. Its \( S_{c} \) is generally comparable with FreeSurfer, and its DR and SD are much better than FreeSurfer, indicating that our method not only effectively corrects topological defects, but also better ensure the anatomical accuracy.

Fig. 4.
figure 4

Comparison with other topological correction methods.

Table 1. Quantitative comparison with other topological correction methods.

4 Conclusion

In this paper, we proposed a learning-based method for correcting topological defects in infant brain cortical surfaces. Our contribution is threefold. First, based on the sparse representation model, for the first time, we correct topological errors in infant cortical surfaces by learning rich information from manually-corrected volumes by experts. Second, to locate the regions with topological errors, we leverage a topology-preserving level set method. Third, we formulate an iterative framework to facilitate the correction of large topological errors frequently occurred in infant cortical surfaces. Experiments demonstrate the effectiveness and accuracy of our method.