Joint Representation of Connectome-Scale Structural and Functional Profiles for Identification of Consistent Cortical Landmarks in Human Brains

Abstract

There have been significant interests in the representation of structural or functional profiles for establishment of structural/functional correspondences across individuals and populations in the brain mapping field. For example, from the structural perspective, previous studies have identified hundreds of consistent cortical landmarks across human individuals and populations, each of which possess consistent DTI-derived fiber connection patterns. From the functional perspective, a large collection of well-characterized functional brain networks based on sparse coding of whole-brain fMRI signals have been identified. However, due to the considerable variability of structural and functional architectures in human brains, it is challenging for the earlier studies to jointly represent the connectome-scale profiles to establish a common cortical architecture which can comprehensively encode both brain structure and function. In order to address this challenge, in this paper, we proposed an effective computational framework to jointly represent the structural and functional profiles for identification of a set of consistent and common cortical landmarks with both structural and functional correspondences across different human brains based on multimodal DTI and fMRI data. Experiments on the Human Connectome Project (HCP) data demonstrated the promise of our framework.

1 Introduction

Representation of structural and/or functional profiles for establishment of a common structural and/or functional cortical architecture across individuals and populations has been of significant interest in the brain mapping field. With the help of advanced multimodal neuroimaging techniques for quantitatively representing the whole-brain structural profiles (e.g., mapping fiber connections using diffusion tensor imaging (DTI) [1]) or functional profiles (e.g., mapping functional localizations using functional MRI (fMRI) [2]) of the same brain, a variety of studies have attempted to construct a connectome-scale and common representation of human brain based on either structural or functional profiles. For example, from a structural perspective, previous studies have identified hundreds of consistent cortical landmarks across human individuals and populations, each of which possesses consistent DTI-derived fiber connection patterns. (e.g., [3]). From a functional perspective, connectome-scale well-characterized functional brain networks are effectively and robustly reconstructed by using sparse coding method applied on the fMRI data [4]. However, due to the considerable variability of structural and functional architectures in human brain [5], it is challenging to jointly represent the connectome-scale structural and functional profiles to establish a common cortical architecture which can comprehensively encode both brain structure and function [6].

As an attempt to address the abovementioned challenge, in this paper, we propose a novel computational framework to jointly represent connectome-scale functional and structural profiles for the identification of a set of consistent and common cortical landmarks with both reasonably accurate structural and functional correspondences across different human brains based on multimodal DTI and fMRI data. In total, 116 structurally and functionally consistent cortical landmarks (SFCCL) are identified from 32 functional consistent networks, 69 of which are demonstrated to show both functional and structural consistence across all of the HCP Q1 subjects examined. Moreover, this set of 116 SFCCLs can be effectively and accurately estimated in a new subject brain via the proposed prediction step.

2 Materials and Methods

2.1 Overview

As shown in Fig. 1, in the proposed framework, joint representation of connectome-scale structural and functional profiles for the identification of consistent landmarks includes three major steps (marked as 1–3 in Fig. 1): 1. Representation of connectome-scale functional profiles for landmark location initialization. 2. Joint constraint of connectome-scale structural and functional profiles based on MRI/DTI data for landmark location optimization. 3. The prediction is used to validate the framework and results.

Fig. 1.

The proposed computational framework of joint representation of connectome-scale structural and functional profiles for landmark identification. The three major steps mentioned above are labeled as 1–3, respectively. (a) Identified connectome-scale group-wise consistent functional networks across different subjects, similar as existing study [4]. Axial slices of spatial maps of 2 example networks in the template space are shown for illustration. (b) Identified peak points in the major components of each functional network. The peak points of selected illustration networks are shown in red dots. All identified peak points are mapped to individual cortical surfaces as the initial locations of landmarks (represented as red bubbles). (c) Optimization of landmark locations on cortical surfaces based on structural fiber connection pattern and functional constraints. (d) Finalized consistent and common cortical landmarks (shown as red bubbles) across individual human brains which encode joint connectome-scale structural and functional profiles. (e) Finalized consistent and common cortical landmarks (shown as red bubbles) by using prediction methods in Sect. 2.5.

2.2 Data Acquisition and Pre-processing

The dataset in this work comes from the Human Connectome Project (HCP) Q1 release included task fMRI (tfMRI), resting-state fMRI (rsfMRI), T1-weighted MRI data and diffusion tensor imaging (DTI) data. Preprocessing pipeline includes motion correction, spatial smoothing, temporal pre-whitening, slice time correction, and global drift removal. Please referred to [7, 8] for more details.

2.3 Representation of Connectome-Scale Functional Profiles for Landmark Location Initialization

Two steps for representation of the connectome-scale functional profiles for landmark location initialization are introduced. First, we obtained 32 existing group-wise consistent and meaningful functional networks across different human brains via dictionary learning and sparse coding of preprocessed HCP Q1 data via similar methods in the literature [4], and examples of such networks are shown in Fig. 2a. Second, we identify the connectome-scale functional peak points (voxels) with the highest functional activities in each component of each discovered functional network. As illustrated in Figs. 1a–b, firstly, group-wise functional networks are linearly transformed into individual spaces, and then we automatically identify the functional components in each functional network by labeling the number of components of each functional network pattern using the widely adopted connected component labeling (CCL) algorithm implemented in FSL toolbox in each individual space (http://fsl.fmrib.ox.ac.uk). The basic idea is that by searching the neighborhood of all voxels involved in a specific functional network, those connected voxels involved in the functional networks are assigned to the same component. In this way, each functional network may have one or more components (e.g., each network in Fig. 1a has two components). In order to obtain meaningful, stable and consistent functional components across different subjects, we only consider those major components with more than 100 connected voxels. After aggregating all the peak points into each subjects, two examples are shown in Fig. 2b. Those clusters with nodes less than 100 connected voxels will be discarded in this work.

Fig. 2.

Identified consistent common functional brain networks and the functional peak points used as initialized landmarks. (a) 9 examples from 32 functional networks. Three axial slices in a row represent one functional network. (b) Initial landmarks are obtained by aggregating all the peak points into each subjects, and 2 subjects are shown here as examples.

2.4 Joint Constraint of Connectome-Scale Structural and Functional Profiles for Landmark Location Optimization

In this paper, we represent the structural profile as the ‘trace-map’ of DTI-derived axonal fiber bundles (e.g., as similar to those in the literature [3, 9, 10]). Here, we briefly demonstrate the ‘trace-map’ representation and comparison of the DTI-derived structural fiber connection pattern. The “trace-map” method is shown in Fig. 3 by projecting each beginning and ending point for each fiber from fiber bundles (Fig. 3(b)) onto the uniform sphere surface. Then we divide the surface into 48 equally areas and construct histogram for each area, and list them as the vectors. A 48 dimensional histogram vector tr = [d₁, d₂…d₄₈] containing 48 density values, namely ‘trace-map’ (Fig. 3d), is finally obtained as the structural connectivity profile of a landmark.

Fig. 3.

Pipeline of ‘trace-map’ representation of the fiber bundle of the landmark for representation of structural profile. (a) An example of fiber bundle and cortical surface. (b) Points distribution by projection of the principal orientation of each fiber in the fiber bundle on the unit sphere. (c) 48 equally areas from one uniform sphere are represented. (d) 48 vectors are used to represent one fiber bundle.

Based on initial landmarks derived from the identified connectome-scale consistent functional brain networks, we optimize their locations via integrating structural fiber connection patterns and functional activities. These two constraints are jointly modeled as an energy minimization problem. Note that we perform landmark optimization for each corresponding landmark separately.

Specifically, we assume $v_{i}^{p}$ is the initial location of landmark $p$ in subject $i$ ($i$ = 1…$N$), $c_{i}^{p}$ is the set of candidate locations within the morphological neighborhood $N_{{v_{i}^{p} }}$ of $v_{i}^{p}$ ($c_{i}^{p} \in N_{{v_{i}^{p} }}$), the functional activity of $v_{i}^{p}$ is $Z_{{v_{i}^{p} }}$ (peak value), and the functional activity value of $c_{i}^{p}$ is $Z_{{c_{i}^{p} }}$. In this paper, we consider 3-ring neighbors of $v_{i}^{p}$, i.e., about 20 mesh vertices as the candidate locations for optimization of landmark $p$ in subject $i$. First, for the functional constraint, shown as $E_{f} (p)$, we use the ratio of change between $Z_{{v_{i}^{p} }}$ and each $Z_{{c_{i}^{p} }}$. We assume that $E_{f} (p)$ should be large enough to retain the functional consistency.

$$ E_{f} \left( p \right) = 1 - ( Z_{{v_{i}^{p} }} - Z_{{c_{i}^{p} }} )/ Z_{{v_{i}^{p} }} $$

(1)

Second, the structural fiber connection pattern similarity constraint for landmark $p$ as

$$ E_{s} \left( p \right) = \frac{{\mathop \sum \nolimits_{i,j = 1 \ldots N,i \ne j} corr(\varvec{tr}\left( {v_{i}^{p} } \right),\varvec{tr}(v_{j}^{p} ))}}{N*(N - 1)} $$

(2)

where corr(.) is the Pearson’s correlation value between the ‘trace-map’ vectors of vertices $v_{i}^{p}$ and $v_{j}^{p}$ in subject $i$ and $j$, respectively. $N$ is the number of subjects. Then by combining these two constraints together, we can measure the group-wise variance of jointly modeled constraints, and it is mathematically represented as the energy E:

$$ E\left( p \right) = 1 - (E_{f} \left( p \right) + E_{s} \left( p \right))/2 $$

(3)

Our aim is to minimize the energy $E(p)$. By using Eq. (3), for each iteration, we search all possible combinations of candidate landmark locations across all subjects for landmark p, and find an optimal combination of landmark locations which has the minimum $E(p)$. In this paper, we performed optimization for two groups separately (one is the validation experiment), due to the computational limitation, and N is set as 4 for each group. An example can be seen in Fig. 5(a). After going over all the iterations, all the landmarks for each subject will be finalized. These 8 sets of SFCCLs will be set as templates for Sect. 2.5. Then, we determine those common consistent landmarks which are reproducible across the two groups via both quantitative and qualitative measurements similar to that in the literature [3]. If the value has statistically significant difference (two-sample t-test, p = 0.05) between two groups, this landmark will be considered instable and discarded. Moreover, we visually examined and confirmed the consistency of corresponding fiber connection patterns across all subjects in the two groups by two experts.

2.5 Prediction of SFCCLs

The prediction of SFCCLs is akin to the optimization procedure in Sects. 2.3 and 2.4. We will transform a new subject (on DTI/fMRI image via FSL FLIRT) to be predicted to the template brain which was used for discovering the SFCCLs and perform the optimization procedure following the Eq. (2). However, there is still slight difference when comparing with the steps in Sect. 2.4, because we already have the template SFCCLs. So in this step, we will keep the template SFCCLs unchanged and only search the suitable location of the landmark in the new subject to achieve the goal that we obtain the maximum values among the templates and the newly added subjects for each correspondence landmark. Here are the algorithmic steps that we used for prediction.

• 1. Linearly register the 116 initial landmarks from the standard space onto individual cortical surface. For each landmark, find all the candidate landmarks around the neighborhood.

• 2. Since we have many landmarks candidates for each landmark we plan to predict, the work is to calculate the “trace-map” value among fiber connection pattern across the 8 landmarks from model and another landmark comes from new subject.

• 3. Search all the combinations and then find the combination with the largest trace-map value. Repeat it for each landmark we would like to predict.

• 4. Extract the fiber bundles that cross each correspondence landmark obtained from step 3. And check the similarity manually.

3 Experimental Results

3.1 Consistent Cortical Landmarks via Joint Representation of Connectome-Scale Structural and Functional Profiles

We jointly represented the connectome-scale structural and functional profiles for the identification of consistent cortical landmarks, as demonstrated in Sects. 2.3 and 2.4. Figure 4 shows all 116 SFCCLs across 8 subjects in the two groups, and those red landmarks (69 of which) demonstrated both functional and structural consistence across all the subjects. Blue ones (22 of which) are those landmarks with 87.5% probability of success to show both functional and structural consistence across all the subjects, which reflects only 1 subject among 8 cannot obtain the consistent shape of the fiber bundle when compared with other 7 subjects (confirmed by experts). The left (25) are white ones, which have 75% probability of success. From the results, we can conclude that all the landmarks we obtained from consistent functional networks in Sect. 2 have good potential to be the SFCCL, which demonstrate that our strategy is effective and efficient. Here, good potential means although not all the shapes of fiber bundles are consistent, most of them are similar, like 7 out of 8 have similar shapes. Thus we believe it has the possibility to be SFCCL, but need to be further confirmed in future studies. To show more details, we randomly selected six example landmarks (Fig. 5) and visualized their fiber connection patterns in Figs. 5a–c, f–h.

Fig. 4.

Overview of 116 SFCCLs with their consistent probability. (a–c) are the x-y-z direction of the cortical surface. Red ones are consistent 100% accuracy, blue ones have 87.5% probability to be consistent across all the subjects, and white ones have 75% probability.

Fig. 5.

Examples to show those consistent landmarks with their fiber bundles. 5(a)–(c), (f)–(h) show the fiber connection patterns of each landmark across 8 subjects (templates), respectively. (d) 69 SFCCLs are highlighted here and they achieve both functional and structural consistencies across all the human brains. (e) The locations of those examples on the cortical surface, and the color represents the correspondence landmark with (a)–(c), (f)–(h), respectively.

We quantitatively examine the effectiveness of the proposed joint representation of connectome-scale structural and functional profiles, as shown in Table 1. For fiber connection pattern similarity, 0.635 is a relatively high similarity according to our existing knowledge and experience from visual check. $E_{f} (p)$ reaches as high as 0.89, which clearly represents high functional activities. Another important statistical result which also demonstrated the effectiveness of our result is the mean movement when comparing locations of SFCCLs before and after the optimization procedure. 2.2 mm is a relatively small change on the cortical surface. That is, those locations for the final SFCCLs are meaningful since their locations are not far from the peak points in the functional networks.

Table 1. The average percentage of three parameters from all SFCCLs in the model.

3.2 Prediction of SFCCLs

By applying the algorithms in Sect. 2.5 onto another 8 different test subjects separately, we successfully obtained 116 SFCCLs on each subject. The results remain the same, and the average structural fiber connection pattern similarity is 62.59% when compared with the templates. It is quite similar to the value of what we obtained from the template models. In total, 69 of them have both functional and structural consistence with the results obtained from Sect. 3.1. Six corresponding examples are provided here in Fig. 6. Note that the color dots in Figs. 6a–f have the correspondence with those in Figs. 5a–c, f–h. The prediction results demonstrated that our SFCCLs are very consistent and reproducible across the subjects.

Fig. 6.

Examples to show those consistent landmarks with their fiber bundles on prediction data. (a)–(f): The fiber connection patterns of each example landmark across another 8 subjects, respectively. (g) 69 SFCCLs which exhibit both functional and structural consistencies across all the human brains.

4 Conclusion

In this study, we jointly represent the connectome-scale structural and functional profiles via a computational framework for the identification of consistent cortical landmarks in human brains. Finally, we have identified 116 SFCCLs which have the potential to represent common structural/functional cortical architecture. Our experimental results demonstrate that there is reasonable regularity and agreement among the brain’s function and structural fiber connection patterns. In this study, we focused on the methodology development of joint representation of connectome-scale structural and functional profiles. The potential applications of our methods on clinical neuroimaging datasets are left to our future studies.