Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Surface mapping plays an important role in brain imaging research by enabling a localized comparison of anatomical structures [1, 2]. With the advance of MRI techniques, there is an avalanche of large scale brain imaging data that focus on mapping the connectome of human brains [3] and thus an increase of interests in mapping brain surfaces with connectivity [4, 5]. However, they also pose significant challenges for existing surface mapping techniques that have largely depended upon customized geometric features for the mapping of specific brain structures such as the cortex. For general connectivity-driven surface mapping, we develop in this work a novel computational framework for intrinsic and diffeomorphic surface mapping in the Laplace-Beltrami (LB) embedding space via the optimization of Riemannian metrics.

For intrinsic shape analysis, there have been growing interests in using the spectrum of the LB operator in computer vision and medical image analysis [6, 7]. Graph-based approaches were proposed in [8] for surface mapping with LB eigenfunctions. An isometry invariant embedding space was proposed in [7] using the LB spectrum. Based on the equivalence of isometry and the minimization of a spectral-\(l_2\) distance in the LB embedding space, a novel surface mapping algorithm was developed recently via conformal metric optimization on surfaces (CMOS) [9]. The CMOS approach, however, only computes conformal maps and cannot incorporate rich connectivity features.

To overcome this limitation, we propose in this paper a more general computational framework based on the Riemannian metric optimization on surfaces (RMOS). Given any diffeomorphism between two surfaces, the pullback metric defines the isometry between two surfaces. Since the LB eigen-system is completely determined by the Riemannian metric, we can thus pose the computation of diffeomorphism as a problem of finding the proper Riemannian metric that minimizes the spectral-\(l_2\) distance in the LB embedding space, which ensures an isometry is achieved with the resulting diffeomorphism. In this general framework, we can easily incorporate the matching of desirable connectivity features during the RMOS process. For numerical implementation, it was established that the Riemannian metrics on triangular meshes are weights defined on the edges and they fully determine the heat kernel on the triangular meshes [10]. Thus the goal of our RMOS is to compute the optimal weights on the mesh edges to realize diffeomorphic mapping of connectivity features in the LB embedding space. In our experimental results, we apply RMOS for connectivity-driven mapping of the thalamic surfaces, which have well-established rich connectivity to cortical regions [11]. In comparisons with the CMOS method, we demonstrate that the proposed RMOS method can achieve better alignment of anatomical features and improved sensitivity in detecting thalamic atrophy due to normal aging.

The rest of the paper is organized as follows. In Sect. 2, we first introduce the mathematical background of LB embedding and Riemannian metric optimization. After that, we propose the RMOS framework and develop the numerical algorithms for energy minimization. Experimental results on surface mapping with connectivity features are presented in Sect. 3. Finally, conclusions will be made in Sect. 4.

2 Riemannian Metric Optimization on Surfaces (RMOS)

In this section, we first review the computation of the LB embedding on triangular meshes. Then we develop a numerical algorithm to compute its gradient w.r.t. the Riemannian metrics, and the metric optimization algorithm for connectivity-driven surface mapping.

Fig. 1.
figure 1

Notations for Riemannian metric.

LB Embedding and Riemannian Metric. Let \(\mathcal {M}=(\mathcal {V},\mathcal {T})\) denote a triangular mesh where \(\mathcal {V}\) and \(\mathcal {T}\) are the set of vertices and triangles, respectively. For a triangular mesh, the Riemannian metric is denoted as the set of weights \( W = [w_1, w_2,\cdots ]\) on all edges of the mesh [10]. These weights are non-negative and satisfy the triangular inequality on each triangle. With the standard metric induced from \(\mathbb {R}^3\), the metric on the edge \(V_iV_j\) is the edge length \(d_{ij}\) as shown in Fig. 1. For metric optimization, we will optimize these weights to achieve desirable surface mapping results. To compute the LB eigen-system of \(\mathcal {M}\), we solve a generalized matrix eigenvalue problem: \(Qf=\lambda Uf\), where \(\lambda \) is an eigenvalue, f is an eigenfunction, and the matrices Q and U are defined as [9]:

$$\begin{aligned} Q_{ij} = {\left\{ \begin{array}{ll} \sum \limits _{\mathcal {V}_j\in \mathcal {N}(\mathcal {V}_i)} \sum \limits _{\mathcal {T}_l\in \mathcal {N}(\mathcal {V}_i,\mathcal {V}_j)} \frac{\cot \theta _{l}^{ij}}{2}, \\ \sum \limits _{\mathcal {T}_l\in \mathcal {N}(\mathcal {V}_i,\mathcal {V}_j)} \frac{-\cot \theta _{l}^{ij}}{2}, \\ 0, \end{array}\right. } U_{ij} = {\left\{ \begin{array}{ll} \sum \limits _{\mathcal {T}_l\in \mathcal {N}(\mathcal {V}_i)} \frac{\mathcal {A}_l}{6}, &{} \text {if}\ i=j \\ \sum \limits _{\mathcal {T}_l\in \mathcal {N}(\mathcal {V}_i,\mathcal {V}_j)} \frac{\mathcal {A}_l}{12}, &{} \text {if}\ \mathcal {V}_j\in \mathcal {N}(\mathcal {V}_i) \\ 0, &{} \text {otherwise}, \end{array}\right. } \end{aligned}$$
(1)

where \(\mathcal {N}(\mathcal {V}_i)\) is the set of vertices in the 1-ring neighborhood of \(\mathcal {V}_i\), \(\mathcal {N}(\mathcal {V}_i,\mathcal {V}_j)\) is the set of triangle sharing the edge \((\mathcal {V}_i,\mathcal {V}_j)\), \(\theta _l^{ij}\) is the angle in the triangle \(\mathcal {T}_l\) opposite to the edge \((\mathcal {V}_i,\mathcal {V}_j)\), and \(\mathcal {A}_l= \frac{1}{2} d_{ik}d_{jk}\sin \theta _l^{ij}\) is the area of the l-th triangle \(\mathcal {T}_l\). The set of eigenfunctions \(\varPhi = \{ f_0, f_1, f_2,\cdots \}\) form an orthonormal basis on the surface. Using the LB eigen-system, an embedding \(I_\mathcal {M}^\varPhi : \mathcal {M}\rightarrow \mathbb {R}^\infty \) was proposed in [7]:

$$\begin{aligned} I_\mathcal {M}^\varPhi (x) = (\frac{f_1(x)}{\sqrt{\lambda _1}},\frac{f_2(x)}{\sqrt{\lambda _2}} \cdots ,\frac{f_n(x)}{\sqrt{\lambda _n}},\cdots ) ~~~\forall x \in \mathcal {M}, \end{aligned}$$
(2)

where \(\lambda _n\) and \(f_n\) denote the n-th eigenvalue and eigenfunction. To compute the gradient of the LB embedding w.r.t. the Riemannian metric, we derive the gradient of Q and U w.r.t. the metric on each edge, i.e., the length \(d_{ij}\), and list them in Table 1. We can compute the gradient of \(f_n\) w.r.t. a metric element \(w_i \in W\) by taking the derivative on both sides of the eigen-system equation and solving it as in [9].

Table 1. Non-zero gradient elements of Q and U w.r.t. the metric (length) of an edge \(\mathcal {V}_i\mathcal {V}_j\), i.e., \(d_{ij}\). \(\partial Q / \partial d_{ij}\) and \(\partial U / \partial d_{ij}\) are symmetric. Each edge has two neighboring triangles: \(\mathcal {T}_{l_1}\) and \(\mathcal {T}_{l_2}\), where \(\mathcal {A}_{l_1}\) and \(\mathcal {A}_{l_2}\) are their areas, and \(\mathcal {B}_{l_1}\) and \(\mathcal {B}_{l_2}\) are the product of their edge lengths. The third vertex in these two triangles are \(\mathcal {V}_{k_1}\) and \(\mathcal {V}_{k_2}\).

Diffeomorphism and LB Embedding. Let us consider two brain surfaces as Riemannian manifolds: \((\mathcal {M}_1,g_1)\) and \((\mathcal {M}_2,g_2)\), where \(g_i\) is the metric on \(\mathcal {M}_i (i=1,2)\). For any point \(p\in \mathcal {M}_1\), the metric \(g_1(p)\) defines the inner product of vectors on the tangent plane at p. Let \(u:\mathcal {M}_1\rightarrow \mathcal {M}_2\) be a diffeomorphic map from \(\mathcal {M}_1\) to \(\mathcal {M}_2\). Following the definition of diffeomorphism in differential geometry, we can map two vectors \(x_1\) and \(x_2\) on the tangent plane of \(p \in \mathcal {M}_1\) onto the tangent plane of \(u(p) \in \mathcal {M}_2\) and denote them as \(du(x_1)\) and \(du(x_2)\). We can then define the inner product of \(x_1\) and \(x_2\) as the inner product of \(du(x_1)\) and \(du(x_2)\) using the metric \(g_2(u(p))\) at \(u(p) \in \mathcal {M}_2\), which is called the pullback metric for \(\mathcal {M}_1\) induced by the map u. For every possible diffeomorphism from \(\mathcal {M}_1\) to \(\mathcal {M}_2\), we can thus induce an isometry between \(\mathcal {M}_1\) and \(\mathcal {M}_2\) via the pullback metric. Since the LB spectrum of a surface is completely determined by its Riemannian metric, the LB spectrum of \(\mathcal {M}_1\) generated by the pullback metric will match the LB spectrum of \(\mathcal {M}_2\). For any desired diffeomorphic map, this shows mathematically the existence of a Riemannian metric on \(\mathcal {M}_1\) that will ensure its perfect alignment with \(\mathcal {M}_2\) in the LB embedding space. For the problem of surface mapping, the diffeomorphism we want to compute is unknown. In the RMOS framework, our goal is to search for the Riemannian metric that can minimize their distance in the LB embedding space while matching connectivity features.

Let \(W_1\) and \(W_2\) denote their Riemannian metrics, i.e., the edge weights of \(\mathcal {M}_1\) and \(\mathcal {M}_2\), respectively. The eigen-systems of \(\mathcal {M}_1\) and \(\mathcal {M}_2\) are denoted as \((\lambda _{1,n}, f_{1,n})\) and \((\lambda _{2,n}, f_{2,n}) (n=1,2,\cdots )\), respectively. We denote \(u_1:\mathcal {M}_1\rightarrow \mathcal {M}_2\) as the map from \(\mathcal {M}_1\) to \(\mathcal {M}_2\) and \(u_2:\mathcal {M}_2\rightarrow \mathcal {M}_1\) as the map from \(\mathcal {M}_2\) to \(\mathcal {M}_1\). As shown in Fig. 2, we compute the LB eigen-system and construct their embeddings as: \(\widetilde{\mathcal {M}}_1=I_{\mathcal {M}_1}^{\varPhi _1}(\mathcal {M}_1)\) and \(\widetilde{\mathcal {M}}_2=I_{\mathcal {M}_2}^{\varPhi _2} (\mathcal {M}_2)\). In the embedding space, the maps are \(\tilde{u}_1: \widetilde{\mathcal {M}}_1 \rightarrow \widetilde{\mathcal {M}}_2\) and \(\tilde{u}_2: \widetilde{\mathcal {M}}_2 \rightarrow \widetilde{\mathcal {M}}_1\). The final maps between the two surfaces are obtained via composition of the embeddings and the maps in the embedding space.

Fig. 2.
figure 2

Symmetric RMOS mapping process.

Energy Function for Surface Mapping. Let \(\xi _1^j:\mathcal {M}_1\rightarrow \mathbb {R}\) and \(\xi _2^j:\mathcal {M}_2\rightarrow \mathbb {R}\) (\(j = 1,2,\cdots ,L\)) denote L connectivity feature functions on each surface. In our experiments, we will define each feature as the normalized fiber count to a specific cortical region for thalamic surfaces, but our framework and numerical algorithm are general for both geometric and other forms of connectivity features. We define an energy function for connectivity-driven surface mapping with RMOS: \(E = E_{F} + \gamma E_R\), where \(E_F\) is the data fidelity term for matching given features, \(E_R\) is the regularization term, and \(\gamma \) is the weight between the two terms. We define the data fidelity term with an L2 energy:

$$\begin{aligned} E_F= \sum _{j=1}^L \biggl [ \int _{\mathcal {M}_1}(\xi _1^j-\xi _2^j\circ u_1)^2 d\mathcal {M}_1 + \int _{\mathcal {M}_2}(\xi _2^j-\xi _1^j\circ u_2)^2 d\mathcal {M}_2 \biggl ]. \end{aligned}$$
(3)

This energy is symmetric w.r.t. both surfaces. It penalizes the mismatch between the original and mapped features. We define the regularization term as:

$$\begin{aligned} E_R = \sum _{w_{1,i}\in W_1} \biggl ( \frac{w_{1,i}}{\hat{w}_{1,i}}-\frac{1}{n_{1,i}}\sum _{j\in \mathcal {N}_{1,i}}\frac{w_{1,j}}{\hat{w}_{1,j}}\biggl )^2 + \sum _{w_{2,i}\in W_2} \biggl ( \frac{w_{2,i}}{\hat{w}_{2,i}}-\frac{1}{n_{2,i}}\sum _{j\in \mathcal {N}_{2,i}}\frac{w_{2,j}}{\hat{w}_{2,j}}\biggl )^2, \end{aligned}$$
(4)

where \(\mathcal {N}_{1,i}\) and \(\mathcal {N}_{2,i}\) are the sets of edges in the neighborhood (directly connected to the edge i), \(n_{1,i}\) and \(n_{2,i}\) are the total numbers of the neighbor edges, and \(w_{1,i}\) and \(w_{2,i}\) (\(\hat{w}_{1,i}\) and \(\hat{w}_{2,i}\)) are the metric (the standard metric), respectively, on an edge of \(\mathcal {M}_1\) and \(\mathcal {M}_2\). This term constrains the changes of metric ratios to be smooth.

Optimization Algorithm. To minimize the energy function using metric optimization, we first construct a coarse correspondence, which we call a \(\beta \)-map, that transforms the energy into distance measurements in the embedding space. Let \(\tilde{u}_1^\beta : \widetilde{\mathcal {M}}_1 \rightarrow \widetilde{\mathcal {M}}_2\) denote the \(\beta \)-map from \(\widetilde{\mathcal {M}}_1\) to \(\widetilde{\mathcal {M}}_2\). For each point \(x \in \widetilde{\mathcal {M}}_1\), \(\tilde{u}_1^\beta (x)\) can be discretized as a linear combination of vertex positions in \(\widetilde{\mathcal {M}}_2\). Thus we can represent the \(\beta \)-maps: \(\tilde{u}_1^\beta : \widetilde{\mathcal {M}}_1 \rightarrow \widetilde{\mathcal {M}}_2\) and \(\tilde{u}_2^\beta :\widetilde{\mathcal {M}}_2 \rightarrow \widetilde{\mathcal {M}}_1\) as linear operators A and B, respectively, as shown in Fig. 2. To construct the \(\beta \)-maps for the minimization of \(E_F\), we start from the nearest point maps and move the points along the gradient descent direction in the tangent space of the meshes as:

$$\begin{aligned} \frac{\partial E_F}{\partial \tilde{u}_1^\beta } = -2\sum _{j=1}^L \left( \xi _1^j - \xi _2^j\circ \tilde{u}_1^\beta \right) \nabla _{\widetilde{\mathcal {M}}_2}\xi _2^j(\tilde{u}_1^\beta )&, \frac{\partial E_F}{\partial \tilde{u}_2^\beta } = -2\sum _{j=1}^L \left( \xi _2^j - \xi _1^j\circ \tilde{u}_2^\beta \right) \nabla _{\widetilde{\mathcal {M}}_1}\xi _1^j(\tilde{u}_2^\beta ),\nonumber \end{aligned}$$

where \(\nabla _{\widetilde{\mathcal {M}}_1}\) and \(\nabla _{\widetilde{\mathcal {M}}_2}\) are the intrinsic gradients on the surfaces \(\widetilde{\mathcal {M}}_1\) and \(\widetilde{\mathcal {M}}_2\). The \(\beta \)-maps are obtained by updating the maps for a fixed number of time steps. Given the \(\beta \)-map, we convert the data fidelity term \(E_F\) into the distance energy in the embedding space, which we call \(\widetilde{E}_F\), and compute its gradient descent direction \(\partial \widetilde{E}_F / \partial W_i\) (\(i=1,2\)) using Eqs. 11 and 12 in [9].

To minimize the energy \(E_R\) w.r.t. the metric \(W_1\) and \(W_2\), we rewrite Eq. 4 in a matrix form: \(E_R = \Vert D_1W_1\Vert ^2 + \Vert D_2W_2\Vert ^2\), and compute the gradients of \(E_R\) as:

$$\begin{aligned} \frac{\partial E_R}{\partial W_1} = 2D_1^TD_1W_1, \quad \frac{\partial E_R}{\partial W_2} = 2D_2^TD_2W_2, \end{aligned}$$
(5)

where \(D_1\) and \(D_2\) are used to calculate the difference of the metrics between neighboring edges. They are initially given and fixed because the mesh connectivity does not change during the optimization process. We redefine the energy function as:

$$\begin{aligned} \widetilde{E} = \widetilde{E}_F+\tilde{\gamma }E_R, \end{aligned}$$
(6)

that is directly differentiable w.r.t. the metric \(W_i\), and finally form the gradient as: \(\partial \widetilde{E} / \partial W_i = \partial \widetilde{E}_F / \partial W_i + \tilde{\gamma }(\partial E_R/\partial W_i) (i=1,2)\). By minimizing this energy using gradient descent, we deform the embedding of a surface toward its \(\beta \)-map, thus achieving the goal of minimizing the original energy E. When it is minimized, the LB embeddings of \(\mathcal {M}_1\) and \(\mathcal {M}_2\) are perfectly aligned and the nearest point map gives the final maps that are diffeomorphic.

As mentioned above, the Riemannian metrics defined on the edges of triangular meshes need to satisfy the triangular inequality. By incorporating these convex conditions, we have an optimization problem for the minimization of the energy \(\widetilde{E}\) with the linear constraints. We use Rosen’s gradient projection method [12] that projects the search direction onto the subspace tangent to any active constraints. Every iteration in the optimization process, we compute the projection matrix P, apply it to the gradient: \(P(\partial \widetilde{E}/\partial W_i)\) (\(i=1,2\)), and finally update the metric with the projected gradient.

3 Results

In this section, we present experimental results to demonstrate the value of the RMOS framework in connectivity-based brain mapping. MRI data from 212 subjects of the Q1–Q3 release of the Human Connectome Project (HCP)[3] and 18 subjects from the LifeSpan pilot project of HCP were used in our experiments. We use the left thalamus surfaces from these subjects to compare the performance of RMOS and CMOS in aligning connectivity features and detecting group differences. All thalamic surfaces are represented as triangular meshes with 1000 vertices and 2994 edges. For CMOS-based experiments, we use its implementation in the publicly distributed MOCA softwareFootnote 1.

To define the connectivity features, we use probabilistic tractography with fiber orientation distributions (FODs) reconstructed from the multi-shell diffusion MRI data from HCP [13]. For each thalamic surface, 100,000 fiber tracts are generated. For each vertex, we define a neighborhood with a radius of 2 mm. Given a cortical region, the connectivity from this vertex to the cortical region is defined as the number of tracts that pass through the vertex neighborhood and reach the cortical region. By repeating this process for each vertex, we obtain a connectivity map for this cortical region. After that, we divide the connectivity map by its maximum value to generate a normalized connectivity map, which we use in our surface mapping. Overall we compute the connectivity maps to ten cortical regions: orbital-frontal, superior-frontal, middle/inferior-frontal, motor, sensory, superior-parietal, inferior-parietal, insular, temporal, and occipital cortices of the same hemisphere.

Fig. 3.
figure 3

RMOS mapping of two thalamic surfaces. Plots of (a) \(\widetilde{E}_f\), (b) \(E_F\) and (c) \(E_R\) over a range of the parameter \(\bar{\gamma }\) after RMOS on (d) the source and (e) target surface (lateral view). Projection of the source to target surface with \(\bar{\gamma }=\) (f) 0.01, (g) 0.24. (h) the final optimized metric on the source surface with \(\bar{\gamma }=0.24\) (lateral and medial views).

As a first experiment, we demonstrate a robust approach of selecting the regularization parameter \(\tilde{\gamma }\) in our energy function Eq. 6. Instead of using a fixed value during the whole iterative optimization process, we adaptively change \(\tilde{\gamma }\) in every iteration so that the normalized maximum gradient magnitudes of \(\widetilde{E}_F\) and \(E_R\) have the constant ratio \(\bar{\gamma }\). For a pair of thalamus surfaces shown in Fig. 3(d) and (e), the effect of the regularization term can be clearly observed in Fig. 3(f) and (g), where the source mesh is projected onto the target surface using the RMOS maps computed with two different \(\bar{\gamma }\) values. For a wide range of \(\bar{\gamma }\) values, we run the RMOS map and plot the the optimized total energy \(\widetilde{E}\), data fidelity term \(E_F\) and regularization term \(E_R\) as a function of \(\bar{\gamma }\) in Fig. 3(a), (b) and (c), respectively. With the increase of \(\bar{\gamma }\) until the turning point of the L-shape curve in (a), we have relatively large decrease of the regularization energy without much increase of the data fidelity term. Thus we consider it as the sweet spot of our energy minimization problem and choose the parameter \(\bar{\gamma }=10^{-0.625}=0.24\) for our large scale experiments. We follow the multi-scale strategy in [9] that starts with the first 10 eigenfunctions, iteratively increases the number of eigenfunctions by 5 to 20, and set a maximum of 500 iterations at the final eigen-order. The optimiazed metric was plotted on the source surface in Fig. 3(h). The RMOS computational process takes around 2 h on a 16-core 2.6-GHz Intel Xeon CPU (multi-threading enabled) with maximal memory consumption around 900 MB.

Fig. 4.
figure 4

Mapping the connectivity features of thalamic surfaces. To highlight the differences between RMOS and CMOS, only connectivity features to two cortical regions: superior-frontal (left) and sensory (right) cortices are shown in each subfigure (a)–(f) from the lateral view.

As an illustration, the connectivity features to the superior-frontal and sensory cortices of the source and target surfaces are shown in Fig. 4(a) and (b). Using the maps computed by RMOS and CMOS, we pull back the connectivity features from the target surface onto the source surface, and the results are shown in Fig. 4(c) and (d). Clearly a better match with the source connectivity features is achieved by the RMOS method. This is not surprising but emphasizes the need of integrating connectivity features into diffeomorphic surface mapping. We then apply both RMOS and CMOS to the 212 thalamic surfaces from the HCP data and constructed average connectivity maps to the ten cortical regions. The results of the maps to the superior-frontal and sensory cortices are shown in Fig. 4(e) and (f), where we can see the atlas from the RMOS method appears to be more concentrated, i.e., less variable, than the CMOS atlas. This demonstrates the potential of connectivity-based mapping with RMOS for the construction of more anatomically meaningful atlases.

Fig. 5.
figure 5

Log-scale p-value (\(-\log p\)) maps of the thickness for the 9 young (14–35 yrs) vs. 9 old (45–75 yrs). Each subfigure shows the superior (left) and inferior (right) views.

In the last experiment, we examine localized thickness changes of the left thalamus between two groups from the LifeSpan pilot project of HCP. Group one consists of 9 subject in the age range 14–35 yrs. Group two consists of 9 subjects in the age range 45–75 yrs. The thickness map of each surface is computed for statistical analysis [2]. Using the surface maps generated by RMOS and CMOS we run vertex-wise t-test, and the p-value maps from these two methods are shown in Fig. 5. Clearly the RMOS maps generate more significant results about thalamic atrophy due to normal aging.

4 Conclusion

In this paper, we developed a novel method for mapping surface connectivity based on the optimization of the Riemannian metric in the Laplace-Beltrami embedding space. We demonstrated the value of our method by applying it to compute connectivity-driven maps of the thalamic surfaces. In comparisons with a state-of-the-art method, we showed that our method can achieve better alignment of connectivity features and higher sensitivity in detecting thalamic atrophy in normal aging. For future work, we will validate our method on more general anatomical surfaces with both geometric and connectivity features.