Integrated Analysis of EEG and fMRI Using Sparsity of Spatial Maps

Abstract

Integration of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) is an open problem, which has motivated many researches. The most important challenge in EEG-fMRI integration is the unknown relationship between these two modalities. In this paper, we extract the same features (spatial map of neural activity) from both modality. Therefore, the proposed integration method does not need any assumption about the relationship of EEG and fMRI. We present a source localization method from scalp EEG signal using jointly fMRI analysis results as prior spatial information and source separation for providing temporal courses of sources of interest. The performance of the proposed method is evaluated quantitatively along with multiple sparse priors method and sparse Bayesian learning with the fMRI results as prior information. Localization bias and source distribution index are used to measure the performance of different localization approaches with or without a variety of fMRI-EEG mismatches on simulated realistic data. The method is also applied to experimental data of face perception of 16 subjects. Simulation results show that the proposed method is significantly stable against the noise with low localization bias. Although the existence of an extra region in the fMRI data enlarges localization bias, the proposed method outperforms the other methods. Conversely, a missed region in the fMRI data does not affect the localization bias of the common sources in the EEG-fMRI data. Results on experimental data are congruent with previous studies and produce clusters in the fusiform and occipital face areas (FFA and OFA, respectively). Moreover, it shows high stability in source localization against variations in different subjects.

Appendices

Appendix 1: Referenced-Based Source Separation

The objective is to extract the sources related to a reference activation model. We consider two states, denoted $C^{1}$ and $C^{2}$, which correspond to the reference and non-reference activations, respectively. Denoting ${\mathscr{T}}^{\ell }$, $\ell =1,2,$ the set of time samples related to each state, we can build the corresponding segment matrix, ${\mathbf{X}} ^{\ell }\in \mathbb {R}^{N\times M^{\ell }}$. The correlation matrix of data for each state can be estimated as:

$$\widehat{\mathbf{R }}^{{\ell }}=\frac{1}{M^{\ell }}{} {\mathbf{X}} ^{\ell } \mathbf{X ^{\ell }}^{\prime }.$$

(16)

The spatial filters, ${\mathbf{W}}$ (whose columns are generically denoted ${\mathbf{w}}$), for which the temporal sources, ${\mathbf{S}} = {\mathbf{W}}^{\prime } {\mathbf{X}}$ have maximum similarity with the reference activation state, i.e., maximum variance in the reference state compared to the other state, is computed as:

$$\max _{{\mathbf{w}}} \frac{{\mathbf{w}}^{\prime } \widehat{{\mathbf{R}}}^{1} {\mathbf{w}}}{{\mathbf{w}}^{\prime } \widehat{{\mathbf{R}}}^{2} {\mathbf{w}}}$$

(17)

Solving (17) leads to generalized eigenvalue decomposition (GEVD) of $(\widehat{\mathbf{R }}^{1} , \widehat{\mathbf{R }}^{2})$:

$$\widehat{\mathbf{R }}^{1}{} {\mathbf{W}}=\widehat{\mathbf{R }}^{2}{} {\mathbf{W}}\varvec{\Lambda }$$

(18)

Using ${\mathbf{W}}$, the spatial patterns, ${\mathbf{A}} = ({\mathbf{W}}^{\prime })^{-1}$, and the temporal sources, ${\mathbf{S}} = {\mathbf{W}}^{\prime } {\mathbf{X}}$, are extracted. The maximum eigenvalue in (18) is related to the maximum power ratio in (17). We rank the eigenvalues in decreasing order. This implies ranking of the estimated temporal sources, according to their resemblance to the reference activation state. Here, we propose a simple method to define the number of sources. Remember that the simplicity of this method is due to two facts; (1) the usage of experiment information in the source separation step, and (2) a special interpretation of the eigenvalues in the source selection step, which will be explained below.

After obtaining the discriminative sources (${\mathbf{s}} _i$) between the reference and non-reference states, sources are ranked according to their similarity to the reference state. Now, we need to select the sources, which are similar enough to the reference, for being considered as belonging to the reference class. To this end, we propose the following procedure.

The probability of the reference class ($\omega _1$) membership is calculated as follows:

$$p({\mathbf{s}} _i^\prime \in \omega _1)=\frac{\lambda _i}{max(\lambda _{j=1,\ldots ,N})}$$

(19)

where $\lambda _i$, $i=1,\ldots ,N$ indicate the eigenvalues or the diagonal elements of $\varvec{\Lambda }$ in (18). For the classification of the sources, the error probability using Bayes rule is defined as:

$$p_{{\textit{error}}}=\sum \limits _{j=1}^{N} \left( {p({\mathbf{s}} _{j}^\prime \in \omega _2 \left| \omega _1 \right. ) p(\omega _1)}\right) + \sum \limits _{j=1}^{N} \left( {p({\mathbf{s}} _j^\prime \in \omega _1 \left| \omega _2 \right. )p(\omega _2)}\right)$$

where $p({\mathbf{s}} _{j} \in \omega _2 \left| \omega _1 \right. )=1-p({\mathbf{s}} _j \in \omega _1)$ and $p({\mathbf{s}} _j \in \omega _1 \left| \omega _2 \right. )=1-p({\mathbf{s}} _j \in \omega _2)$ as $\omega _1$ and $\omega _2$ constitute a partition. $p(\omega _1)$ and $p(\omega _2)$ are the prior probabilities of the reference and non-reference classes, respectively. We remind that GEVD sorts the separated sources in the decreasing order of similarity with respect to the reference. Therefore, if we assume that only the first i sources belong to the reference class (and consequently the $N-i$ others belong to the non-reference class), then the total error probability (false positive plus false negative errors) can be written as follows:

$$\begin{array}{ll} p_{{\textit{error}}}(i)=\sum \limits _{j=1}^{i} {p({\mathbf{s}} _{j}^\prime \in \omega _2 \left| \omega _1 \right. ) p(\omega _1)} + \sum \limits _{j=i+1}^{N} {p({\mathbf{s}} _j^\prime \in \omega _1 \left| \omega _2 \right. )p(\omega _2)} \end{array}$$

(20)

where $p(\omega _1)=\frac{i}{N}$ and $p(\omega _2)=\frac{N-i}{N}$. Thus, the minimum of $p_{{\textit{error}}}(i)$ provides the number $i^{*}$ of the sources in the reference class, i.e., $i^{*}={{\mathrm{arg\,min}}}_i p_{{\textit{error}}}(i)$.

Appendix 2: Pareto Optimization

${\mathbf{B}} ^s$ is a $M \times i^{*}$ matrix whose columns present the contributions of the related sources in each mesh vertex. All the $i^{*}$ sources are involved in the neural activation. Therefore, all of them are important, and if we use the sum or the square sum of the contributions the weak sources would be hidden in the shadow of the sources with higher power. The remedy of this problem is to use multi-objective optimization method, called Pareto method (Deb 1999). Therefore, we are able to take into account the individual effect of each source. Pareto optimization finds the optimum without any threshold, that obviates the need for arbitrary user-defined threshold.

A multi-objective optimization problem, in Pareto sense, has the following form:

$$\begin{array}{lll} \text {maximize } &{} ({\mathbf{b}} ^s_i)^\prime &{} \text {for }\, i=1, \ldots ,M \\ \text {subject to }&{} ({\mathbf{b}} ^s_i)^\prime \in P \subset \mathfrak {R}^{i^{*}} &{} \\ \end{array}$$

(21)

It then consists of $i^{*}$ objective functions that are aimed to be maximized simultaneously. $({\mathbf{b}} ^s_i)^\prime$ is the i-th row of the matrix ${\mathbf{b}} ^s$ which shows the contributions of the sources at the ith vertex. From the geometrical point of view, each $({\mathbf{b}} ^s_i)^\prime$ can be considered as a point in a $i^{*}$-dimensional space. In Pareto optimization the non-dominated points should be chosen as the optimum points (Deb 1999): a point is non-dominated if either it dominates the others, or there is no other point dominating it. Point $({\mathbf{b}} ^s_i)^\prime$ dominates point $({\mathbf{b}} ^s_k)^\prime$, if $\forall l$, ${\mathbf{b}} ^s_i(l) \ge {\mathbf{b}} ^s_k(l)$, and $\exists l^{*}$, ${\mathbf{b}} ^s_i(l) > {\mathbf{b}} ^s_k(l^{*})$,³ where ${\mathbf{b}} ^s_i(l)$ is the l-th component of the vector $b_i^s$. The set of all non-dominated points is called non-dominated layer. Let us consider M $i^{*}$-dimensional decision vectors, $({\mathbf{b}} ^s_i)^\prime$, as M points in the search space P. The non-dominated layer, denoted by D(P), is obtained using the following Pareto optimization algorithm (Deb 1999):

• 1. Initialize D(P) with the first point ($i=1$) with the value of $({\mathbf{b}} _1)^\prime$. This can be any point.

• 2. Choose a new point ($i=i+1$):

  • (a) If any node in D(P) dominates point i go to Step 3.

  • (b) Else add point i to D(P) and remove any points of D(P) that point i dominates.

• 3. If i is not equal to M go to Step 2.