# EECoG-Comp: An Open Source Platform for Concurrent EEG/ECoG Comparisons—Applications to Connectivity Studies

- 535 Downloads
- 1 Citations

## Abstract

Electrophysiological Source Imaging (ESI) is hampered by lack of “gold standards” for model validation. Concurrent electroencephalography (EEG) and electrocorticography (ECoG) experiments (EECoG) are useful for this purpose, especially primate models due to their flexibility and translational value for human research. Unfortunately, there is only one EECoG experiments in the public domain that we know of: the Multidimensional Recording (MDR) is based on a single monkey (www.neurotycho.org). The mining of this type of data is hindered by lack of specialized procedures to deal with: (1) Severe EECoG artifacts due to the experimental produces; (2) Sophisticated forward models that account for surgery induced skull defects and implanted ECoG electrode strips; (3) Reliable statistical procedures to estimate and compare source connectivity (partial correlation). We provide solutions to the processing issues just mentioned with EECoG-Comp: an open source platform (https://github.com/Vincent-wq/EECoG-Comp). EECoG lead fields calculated with FEM (Simbio) for MDR data are also provided and were used in other papers of this special issue. As a use case with the MDR, we show: (1) For real MDR data, 4 popular ESI methods (MNE, LCMV, eLORETA and SSBL) showed significant but moderate concordance with a usual standard, the ECoG Laplacian (standard partial \( AUC = 0.65 \pm 0.05 \)); (2) In both monkey and human simulations, all ESI methods as well as Laplacian had a significant but poor correspondence with the true source connectivity. These preliminary results may stimulate the development of improved ESI connectivity estimators but require the availability of more EECoG data sets to obtain neurobiologically valid inferences.

## Keywords

EEG ECoG Forward modeling Electrophysiological source imaging Connectivity Open science Monkey electrophysiology## Introduction

Among the non-invasive technologies used to probe normal and pathological brain function in both humans and animals, Electroencephalography (EEG) (Schomer and Da Silva 2012) is the most affordable, popular and widespread. It also provides the highest temporal resolution. Alas, like any other imaging modality, EEG is not a straightforward proxy for neural activity. In fact, it is a spatially filtered and sub-sampled reflection of hidden brain states, corrupted as well by noise and artifacts. Indeed, at the EEG electrodes, the signals are spatially filtered by volume conduction (van den Broek et al. 1998)—owing to the very low conductivity of the skull, and they are the observation of mixed EEG source activities. This mixing also introduces spurious brain coherence, leading to false brain connectivity estimations. For this reason, EEG connectivity should be, in principle, assessed from estimates of source activity. Thus, what is required is the solution of the inverse problem (Pascual-Marqui 1999), i.e. the estimation of brain sources activities from EEG recordings—a technique also called electrophysiological source imaging (ESI).

ESI has been long studied in the attempt to localize epileptic foci (Ding et al. 2007; He et al. 2011). In addition, it has been proposed to study the localization of different brain functions as well as the dynamics of neural connectivity. Nevertheless, ESI methods have important methodological issues and requires external validation (Nunez et al. 2019).

Electrocorticography (ECoG) or intracranial electroencephalography (iEEG) is considered by many as a “gold standard” in clinical settings, since it provides more accurate information of the cortical brain states (Nunez et al. 2019; Frauscher et al. 2018). ECoG is a type of invasive monitoring technology with electrodes placed directly on the exposed surface of the cortex to record electrical activity of the brain. It should be noted that, even though the ECoG electrodes are directly placed on the cortex, the activity at the surface of the cortex is not equivalent to the true “sources” of the “physiological cortical macro columns” proposed by Buxhoeveden and Casanova (2002). This is the reason why we need to solve the inverse problem for ECoG as well (Nunez et al. 2019).

Solving the inverse problem first requires the solution of the forward problem (Hallez et al. 2007) for EEG and ECoG (EECoG), i.e., the calculation of the so-called lead field matrix (LF), a mapping matrix from known sources to EECoG measurements. Such mappings demand the definition of a head model specifying the spatial distribution of tissue conductivity, a source space model and the location the EECoG electrodes with respect to the source space. The more realistic the head model, the more accurate the ESI solutions. Nowadays, high detailed structural magnetic resonance images (MRI) are used to define the head model and either boundary element method (BEM) or finite element method (FEM) are used to calculate the LF. These methods differ in accuracy.

The EECoG inverse problem is ill-posed in the Hadamard sense: the null space of the lead field has finite dimension, sabotaging the uniqueness of the solution. Thus, it is necessary to impose priors on the sources such as minimum energy, sparsity, smoothness and independence; and to restrict the source space with anatomical information, e.g. by forcing the sources to be in the gray matter or by imposing the orientation of the dipoles to be perpendicular to the cortical layer. This is another reason to provide accurate forward head models. Abundant inverse solutions using of these constraints have been developed and implemented, examples being weighted minimum-norm estimation (Hämäläinen and Ilmoniemi 1994), beamforming (BF) (Grech et al. 2008; Van Veen et al. 1997), Blind Source Separation (BSS) (Oosugi et al. 2017), exact Low Resolution Electromagnetic Tomography (eLORETA) (Pascual-Marqui 2007), sparse structural bayesian learning (SSBL) (Paz-Linares et al. 2017), and so on. This problem is even more difficult for source connectivity estimation, as an aspect that we will later address more closely.

It follows, due to the complexity of the forward modeling procedure and the diversity of possible inverse solutions, we can arrive at completely different source activity and connectivity estimations from different ESI methods. In view of the proliferation of ESI methods, it is quite challenging to determine which is the best and to quantify its properties reliably. There is general agreement that simulations are not enough to make this decision—experimental verification is necessary.

Unfortunately, there is a paucity of data sets that can serve as a “gold standard” to evaluate ESI. This is unsurprising given the obvious difficulty in recording concurrent scalp EEG signals in a normal intact head simultaneously with direct brain activity measurements. This problem was only tackled with a few unrealistic (though exceptionally useful) attempts in the past. That is the case of Leahy et al. (1998) who designed and conducted a multiple dipole phantom consisting of 32 independently programmable and isolated dipoles inserted in a skull mount filled with conductive gelatin. They used this phantom to collect EEG and MEG data to evaluate the performance of current dipole localization methods. In Marin et al. (1998) another realistic conductive skull phantom was used to evaluate the accuracy of forward FEM methods. Later, this phantom was used to evaluate the performance of different ESI methods in Baillet et al. (2001).^{1} Recently, Peterson and Ferris (2018) evaluated the performance of ICA-based connectivity in a phantom in motion. A much more ideal “phantom” to study the effect of the accuracy of ESI methods on assessing brain connectivity and source localization is the one providing scalp EEG signals concurrently with actual and direct whole brain measures of brain activity, such as Local Field Potential (LFP) and Multi-Unit Activity (MUA) (Mattia et al. 2010). However, coverage of all brain areas simultaneously with these techniques is impossible. On the other hand, optical imaging (Wang et al. 2003) provides a direct measurement of hemodynamic responses at the cortical level, but all these techniques require the removal of the skull during the experiment.

A better candidate is a simultaneous EECoG experiment which allows recording electrical activity directly on the cortex and obtaning simultaneously the scalp EEG after closing the skull. This can achieve comparatively large coverage for both types of recordings. However, the invasiveness makes it available only in human clinical and preclinical settings, mainly restricted to pathologies like epilepsy—only targeting the seizure onset zones and thus usually precluding the recording of larger spatial extensions of the cortex (Frauscher et al. 2018).

A better situation is achievable in primate preparations. For example, the Multidimensional Recording (MDR) database (http://neurotycho.org) (Nagasaka et al. 2011) offers such a kind of simultaneous EECoG data, with ECoG covering the left hemisphere of the cortex and EEG covering the whole head. To our knowledge, this is the only open macaque simultaneous EECoG data set currently available. Despite the complexity of setting up this type of experiment, EECoG has the unique advantage of allowing cross validation of two types of recordings originated from the same neural sources. Testing the ESI methods has also been possible with this dataset (Oosugi et al. 2017). Even though the electrophysiological and imaging data is limited to a single monkey, it is the only publicly available dataset with the required information for detailed forward EEG and ECoG modeling.

- (1)
The artifacts induced by the very complicated primate experimental configurations, which hinders the correct estimation of brain sources activity and connectivity patterns;

- (2)
The difficulty in obtaining a realistic head model that takes into consideration the implanting of very low conductivity ECoG silicone stripes, and the difficulty of computing the lead fields for EEG and ECoG with the same source space;

- (3)
The lack of tools to compare results in terms of connectivity estimation with the same level of statistical significance.

- (1)
Customized preprocessing algorithms, including synchronization and model-based artifact removal algorithms;

- (2)
Realistic MRI-based biophysical head model which takes into consideration of the implanted ECoG stripes, and provides FEM-based lead fields for both EEG and ECoG with the same source space;

- (3)
Standard source connectivity simulation and statistical analysis to guarantee the reliability of comparisons.

This platform is shared via https://github.com/Vincent-wq/EECoG-Comp, and the data will be available upon request. For a detailed description of the content shared, refer to Table 2 in next section. The lead fields from our platform have been used to research directed connectivity measures (Papadopoulou et al. 2015) and for testing the resolution properties of different inverse solutions (Todaro et al. 2018).

In this paper, we describe the simultaneous EECoG platform in Sect. 2. As an illustration of the usefulness of this platform, we take ESI methods connectivity comparison as an example in Sect. 3 and 4. Our preliminary results yield interesting results, indicating that the performance of the tested ESI methods seem to detect connectivity slightly above chance but with very low accuracy.

## Concurrent EEG/ECoG Comparison Open Source Platform

*EECoG*-

*Comp*) is composed of four main modules as depicted in Fig. 1, and they are: (1)

*EECoG_Preproc*(Fig. 1 upper right) for preprocessing; (2)

*EECoG_FWM*(Fig. 1 lower left) for realistic forward modeling; (3)

*EECoG_scSim*(Fig. 1 upper left) for EECoG source connectivity simulation; (4)

*EECoG_conComp*(Fig. 1 lower right) for inverse solution, graph model estimation and statistical comparison. They are designed respectively for: (1) Rejecting the artifacts induced by the very complicated experimental configurations; (2) Solving the EEG forward problem in presence of the very thin silicone ECoG subdural array; (3) Create simulations as null model for testing of the methods (for example ESI methods); (4) Solving the inverse problem, estimating and comparing the source level connectivity of EECoG at the same significance level.

In what follows, we describe the modules and submodules relevant to specific challenges of EECoG connectivity analysis. We take the simultaneous EECoG experiment from neurotycho.org (http://neurotycho.org/eeg-and-ecog-simultaneous-recording) as an example to illustrate how EECoG-Comp works. This is shown in the following subsections. The following mathematical notations are used to explain the theory and methods behind all these modules: \( x \in {\mathbb{R}} \) (lower case, Italic, not bold) is a function or a variable, \(\varvec{x} \in {\mathbb{R}}^{N}\) (lower case, Italic, and bold) is a vector, \(\varvec{X} \in {\mathbb{R}}^{M \times N}\) (upper case, Italic, and bold) is a matrix or matrix operator, refer to Table 3 in the Appendix 1 for all details of the notations.

### MDR Recording

This dataset is partly described in Nagasaka et al. (2011) and Oosugi et al. (2017). All the experimental and surgical procedures were performed in accordance with protocols approved by the RIKEN ethical committee (No. H22-2-202(4)) and the recommendations of the Weatherall report, “the use of non-human primates in research”. During the experiment, a macaque monkey (*Macaca fuscata*) was seated in a primate chair with hands tied and head movement restricted. The eyes of the monkey were covered to avoid evoked visual responses during the entire experimental period.

The electrophysiological data (EEG and ECoG) was recorded during two conditions: (1) awake resting state with eyes folded and movements restricted; and (2) drug induced anesthesia: 1.15 ml (8.8 mg/Kg) ketamine hydrochloride and 0.35 ml (0.05 mg/Kg) medetomidine injection. Two five-min trials of simultaneous resting state EEG and ECoG activities were recorded for each condition.

After placing the ECoG electrodes, the dura, the skull and the skin were carefully placed back (see Fig. 2). ECoG was filtered and recorded by a Cerebus recording system (Blackrock Microsystems, Utah, USA). Plastic connectors from the array to the ECoG system were attached to the top of the head with dental cement and small titanium screws. The sampling rate was 1000 Hz, and the recordings were Butterworth bandpass filtered within 0.3–400 Hz.

The EEG was recorded with a NeuroPRAX system (eldith GmbH, Ilmenau, Germany) of sampling rate 4096 Hz, using 18 electrodes which follows the standard 19-electrode 10–20 system. Electrode “Cz” has been removed to place the ECoG plastic connector. The recordings were down-sampled to 1000 Hz. An external TTL pulse was used to define the beginning and the end of the EEG and ECoG recordings interval of interest for synchronization.

### Preprocessing Module: EECoG_Preproc

The first challenge of analyzing the EECoG (MDR) data is the high level of artifacts present in the recordings. We therefore addressed this attention to assimilating procedures to alleviate this interference.

By introducing TARA, we were able to clean most of the above two types of artifacts as shown in Fig. 4. After preprocessing, several channels in the EEG (F3, C3, and Fz) and ECoG (channel 38, 50, and 93) have been removed from further analyzing due to bad quality, as visually judged by an expert neurophysiologist.

It is to be noted that, surprisingly, we did not observe a clear alpha peak, not even from the occipital recordings of both EEG and ECoG, despite our careful artifact rejection and pre-processing.

### Forward Modeling Module: EECoG_FWM

The tetrahedral meshes in Fig. 5a were created from the surfaces of the head model using TetGen 1.5.0 (Si 2015), an open source software in Linux repositories. During the meshing process the nodes in the source space surface (cortical grid) were forced to be nodes of the tetrahedral mesh to guarantee the Venant’s condition. Both EEG and ECoG lead fields were calculated using NeuroFEM. This is a program for computing lead fields using FEM which is part of the SimBio software package (Wolters et al. 2007) (index page: https://www.mrt.uni-jena.de/simbio/index.php/Main\_Page). This module can provide average referenced LFs based on 3d cortical grid source space.

It should be noted that only 15 channels of EEG left after removing the bad channels, and 125 channels of ECoG left. Finally, we only took the left hemisphere of the cortical grid (source space) and study its connectivity due to the limited coverage of ECoG in real data analysis.

### Source Connectivity Comparison Module: EECoG_conComp

Brain connectivity has received much attention in recent years (Bassett and Sporns 2017). However, the estimation and comparison of brain connectivity at source level is still very challenging (Zalesky et al. 2010). The next challenge for EECoG-Comp platform is to compare the source level connectivity with sound statistics. EECoG_conComp is designed to fulfill this goal, and it has the following submodules: (1) ESI methods submodule, which provide the unified interface for all the ESI methods to estimate the source activities; (2) Graphical model estimation submodule, which estimate the graphical model from the estimated source activities; and (3) The statistical comparison submodule, which compare the estimated graph model parameters at the same statistical significance level. In addition to all these submodules, EECoG_conComp also provides the ECoG Laplacian submodule, which estimates the Laplacian of ECoG over the cortical grid as another way of exploring the “pseudo source connectivity”. All these submodules will be detailed below.

### ECoG Laplacian Submodule

With its wide acceptance, we shall use the ECoG Laplacian over the cortical grid as a surrogate measure for the primary current density \( {\varvec{J}} \). Note that, this estimator is very similar to the “spline Laplacian/dura images” discussed by Nunez and Srinivasan (2006).

The hyper parameter \( \lambda \) is selected with generalized cross validation (GCV). The ECoG Laplacian estimator is calculated for each point of the cortical grid at each time instant, thus also yielding a “cortical grid time series” like the ESI method.

### Electrophysiological Source Imaging Methods Submodule

The inverse methods implemented in ESI submodule

Acronym | Name | Reference |
---|---|---|

MNE | Minimum norm estimation | Hämäläinen and Ilmoniemi (1994) |

LCMV | Linearly constrained minimum variance spatial filtering | Van Veen et al. (1997) |

eLORETA | Exact low resolution electromagnetic tomography | Pascual-Marsqui (2007) |

SSBL | Sparse structural bayesian learning | Paz-Linares et al. (2017) |

To compare the source connectivity, we need to ensure that all ESI methods reached their best performance. For MNE, LCMV and eLORETA, we use GCV to select the hyper parameters. For SSBL, the best performance is guaranteed by selecting the hyper parameters with its marginal likelihood stopping criteria.

Each type of inverse solution is calculated for each point on the cortical grid at each time instant. As mentioned before, we need to solve the inverse problem for both EEG and ECoG, then we can compare them on the same source space.

### Connectivity Estimation Submodules

- 1.
The high dimensionality of the covariance matrices to be studied;

- 2.
The lack of standardized statistical criteria for the selections of significant partial correlations.

We here take advantage of recently developed theory for inference of high dimensional sparse precision matrices from Jankova and van de Geer (2018) to address these challenges. The assumption of sparsity seems to be natural and widely adopted for estimators of brain connectivity and is relaxed with recent “debiasing” techniques explained below.

We leverage the recent theoretical results from Jankova and van de Geer (2018) in two ways:

This result allows us to find the thresholds for all the PCMs with the same statistical significance level.

### Connectivity Comparison Submodules

The connectivity comparison submodule offers mainly two methods to compares the thresholded estimated PCMs against a “base connectivity” matrix (either the truth known from simulations or any of the ESI based source connectivity estimators).

The first measure is based on ROC curves, which shows the true positive rate (TPR) against false positive rate (FPR) at all possible FPR values. In order to evaluate the significance of ROC curve, we also calculate the pointwise confidence bound for all the possible FPRs by bootstrap. We correct the bias introduced from the bootstrap distribution with bias-corrected and accelerated bootstrap interval (BCa interval) method, which is a second-order accurate interval that addresses these problems (Efron 1987; Huang et al. 2007). All the results reported in this paper have been confirmed by bootstrap permutation test (n = 200) with significance level of 0.05, which shows consistency with very small variances.

The ROC curves allow us to explore the effects of different pFDR choices on the false positive and false negative rates. ROC curves are usually summarized by the area under curve (AUC). Since we are only interested in low FDRs, we summarize the ROC curves with the partial area under the ROC curve.

\( spAUC \) varies from 0 to 1, with 0.5 and 1 are the random and perfect diction. We use \( spAUC(0.1) \) in latter summary of results.

PIR can be positive or negative. Positive \( {\text{PIR}} \) indicates that M1 outperforms M2, and vice versa.

### Source Connectivity Simulation Module: EECoG_scSim

Simulation is useful when we need to select the best available methods in a certain condition. For example, we have four ESI methods plus ECoG Laplacian, but we never know the “ground truth connectivity” beneath the real data or which is the most suitable method for our particular data. The only way to answer this question is to create a simulation as similar as possible to the experimental condition where the data is recorded. EECoG_scSim is designed to do so by simulating a structured partial correlation matrix as a simulated true source connectivity, and then generate the source activities with such connectivity. By using the EECoG lead fields generated from EECoG_FWM, we are able to simulate the same experiment with known source connectivity. Together with EECoG_conComp, EECoG-Comp provides tools for both testing methods and exploring data.

EECoG_scSim provides human based simulation for validation as well. A standard human forward model based on the MNI152 template (http://www.bic.mni.mcgill.ca) and standard bioSemi electrodes layouts is calculated with BEM from BrainStorm (Tadel et al. 2011), and the corresponding lead field is provided for human simulation. EECoG_scSim allows the users to change the standard EEG electrode layouts from 10–20 system to 256 electrodes of full head coverage as well as the size of the cortical grid.

- 1.
Sparse random PCM \( {\varvec{\varTheta}}_{rand} \), where a small number of the sources on the cortical grids are connected randomly;

- 2.
Sparse block diagonal PCM \( {\varvec{\varTheta}}_{block} \), where a small number of patches of cortical grid points are active and densely connected between them.

The lead fields used in the simulation module are \( {{\varvec{K}}_{H,eeg}} \) (Human EEG), \( {{\varvec{K}}_{M,eeg}} \) (Monkey EEG), \( {{\varvec{K}}_{M,ecog}} \) (Monkey ECoG) and \( {{\varvec{K}}_{Lap}}\) (Monkey ECoG Laplacian “pseudo lead field”).

### Specification of Simulations for the MDR dataset

We first evaluate the performance of all these methods in EECoG_scSim for both standard human EEG and monkey EECoG by means of simulations (with different number of sources, sensors and partial correlation matrix (PCM) structures); Then, we choose the best method from the monkey EECoG simulation to compared against; At last, we use all these ESI methods in exploring the source connectivity of the real data and give the final ranking of all these tested ESI methods as results.

\({\varvec{V}_{HS,EEG}}\) is the simulated human EEG with dimension \( N_{HS,chan} \times N_{tSample} \), \( N_{HS,chan} \in (19,32,64,128) \) is the number of human EEG channels, and \( N_{tSample} = 10800 \) is the number of time samples. The sampling frequency is 400 Hz for all simulations, which is kept the same as that in the preprocessed monkey recordings. \( {\varvec{K}_{HS,EEG}}\) is the Human EEG lead field with dimension \( N_{HS,chan} \times N_{HS,sour} \), \( N_{HS,chan} \) is the number of human EEG sensors and \( N_{HS,sour} \in (150,200,300,350) \). \( \varepsilon_{HS,EEG} \) is the human EEG sensor noise which follows I.I.D. Gaussian distribution \( \varepsilon_{HS,EEG} \sim N(0,\xi \times I) \) with dimension of \( N_{HS,chan} \times N_{tSample} \). \( {\varvec{V}_{MS,EEG}} \) and \({\varvec{V}_{MS,ECoG}} \) are generated in the similar way but with monkey EEG lead field \( {\varvec{K}}_{MS,EEG} \) (\( N_{MS,EEG} \times N_{MS,sour} \)) and monkey ECoG lead field \( K_{MS,ECoG} \) (\( N_{MS,ECoG} \times N_{MS,sour} \)). In order to select the best method to use on the actual monkey EECoG data (\( {\varvec{V}}_{MD,EEG} \) and \( {\varvec{V}}_{MD,ECoG} \)), we use the same number of the available monkey EEG channels \( N_{MS,EEG} = 15 \) and ECoG channels \( N_{MS,ECoG} = 125 \), as in the simulation, but changing the number of sources \( N_{MS,sour} \in (150,200,300,350) \), note that, \( N_{MS,ECoG} < N_{MS,sour} \) and \( N_{MS,EEG} < N_{MS,sour} \) always hold.

Then, we apply the four well-known ESI methods, namely MNE, eLORETA, LCMV and SSBL in human simulation (EEG), and MNE, eLORETA, LCMV and ECoG Laplacian (this last is viewed as another “pseudo inverse method”) in monkey simulations to reconstruct the source activities. The source activity estimator can be denoted as \({\varvec{\hat{J}_{{HS,EEG,\varTheta}}}}_{rand,ESI}\) for example, among all the sun-notations: (1) \( HS \) stands for human simulation, which can also be \( MS \) for monkey simulation and \( MD \) for monkey data, this is a notation for experimental condition; (2) \( EEG \) is the modality, and it can be \( ECoG \) as well; (3) \( {\varvec{\varTheta}_{rand}}\) is the underlying true random source connectivity matrix, which can alternatively be a blocked structured source connectivity matrix \({\varvec{\varTheta}_{block}} \), and it is unknown in the real data analysis; (4) \( ESI \) stands for different ESI methods, like MNE, eLORETA, LCMV, SSBL and ECoG Laplacian—specified when necessary.

With true source activities in theory * J* and estimated \( {\varvec{\hat{J}}}\), we are able to further estimate the underlying source level connectivity in terms of PCM. We take an example of \({\varvec{\hat{\varTheta}}}_{{HS,EEG,{\hat{\varvec{J}}_{{{\varvec{HS,EEG,\varTheta}}_{rand}}},ESI}}} \) to describe our notations for the connectivity estimators: (1) The first suffix indicates the type of experimental condition, which can be \( HS \) for Human Simulation, \( MS \) for Monkey Simulation and \( MD \) for Monkey Data analysis; (2) The second suffix is modality, which can be EEG or ECoG; (3) The third suffix is the source activities based on which this source connectivity matrix is estimated, for example it can simply be \( {\varvec{J_{{\varTheta_{rand}}}}} \), which means the direct estimation from the true source activities with underlying connectivity \( {\varvec{\varTheta}}_{rand} \), or like \({\varvec{\hat{J}_{{HS,EEG,\varTheta}}}}_{rand,ESI}\) in this example, which is the estimator from the reconstructed source activities obtained with a specific ESI method \( ESI \) for human EEG simulation (\( HS \)) with the underlying true source connectivity \( {\varvec{\varTheta}_{rand}} \).

We clarify that with this simulation we produce the true source activities \( {\varvec{J_{\varTheta}}} \) (no volume conduction) and its associated underlying true source connectivity \( {\varvec{\varTheta}} \). Before further analysis, we bandpass all the true source activities from 8–12 Hz to limit the connectivity estimations to alpha band. We then proceed to use the graphical LASSO (Hsieh et al. 2014) to find the “estimated true source connectivity” \({\varvec{\hat{\varTheta }_{{J_{\varTheta } }}}} \) which will help us quantify the error due to the partial correlation estimation procedure, without additional errors due to the source reconstruction.

With this machinery in place, we can generate simulated recordings of human EEG, monkey EEG and monkey ECoG based on models with different number of sources and different number of sensors. After band pass filtering, we apply different ESI methods to determine the “estimated source activities” \( \hat{J}_{\varTheta ,ESI} \) and finally apply the graphical LASSO to estimate the “ESI source connectivity” \( {\varvec{\hat{\varTheta }_{{\hat{J}_{\varTheta ,ESI} }}}} \).

To summarize the ESI methods performances in simulation, where we know the true source activities and connectivity, we use Performance Improvement Ratio (\( \text{PIR}_{ESI} \)) for a specific ESI method as a metric. For this case, M1 is the ESI estimated source connectivity and M2 is the estimated true source connectivity. For the analysis of the real MDR data, where we do not have true source activities and connectivity, we only report \( spAUC(0.1)_{ESI} \), which compares the ESI estimated source connectivity against the best ESI source connectivity estimator from the monkey simulations.

## Results

### Human EEG Simulation

In the human simulation, we are testing the performance of different ESI methods on the standard human head model (MNI152 template http://www.bic.mni.mcgill.ca) with different electrode layouts and different sizes of cortical grids. In this simulation, the number of sensors is \( N_{chan} \in (19,32,64,128) \) for 10–20 system, 32,64, and 128 channel system with whole head coverage, the number of sources is \( N_{sour} \in (150,200,300,350) \) over the whole cortex. Due to the computational load of graphical model estimation and ROC evaluation (bootstrapping for ROC confidence band), we are only simulating small cortical grids, therefore we have 16 results in total (four number of sensors times four grid sizes) for each type of source connectivity setting.

The results showed the ESI methods performance improvements in terms of \( \text{PIR}_{ESI} \) for both random and block sources for various sensor and source configurations. In summary, \( \text{PIR}_{ESI} \) of all the ESI methods improved less than 10%. The best performance was always achieved when the source connectivity had a block structure.

### Simultaneous Monkey EECoG Simulation

EECoG-Comp platform allows us to simulate simultaneous monkey EECoG data to test not only the ESI methods but also ECoG Laplacian described in Sect. 2.4. Due to the fixed number of EECoG channels in the real experiment, we only tested the performance of ESI methods across different sizes of cortical grids: \( N_{sour} \in (150,\;200,\;300,\;350) \), the methods tested in this simulation are MNE, LCMV, eLORETA and ECoG Laplacian. This means for each data (EEG or ECoG), we have four groups of results for each type of structured source connectivity.

The results from this simulation showed that the ECoG Laplacian almost always had the largest improvement, and the only exceptions were EEG and ECoG with 150 sources. In total, the performance differences between the ESI methods and the Laplacian are within 5%. Though ECoG Laplacian improves the source connectivity estimation the most, the results were not impressive and might not have practical implications. Like the human simulation, the best performance was always achieved when the source connectivity had a block structure.

### ESI methods Comparison with the Real Simultaneous EECoG MDR Data

Even though the ECoG Laplacian connectivity estimation did not yield huge performance improvement in the simulations, it did provide the highest \( \text{PIR}_{ESI} \). Therefore, we choose the ECoG Laplacian as a yardstick (even though we know it is an imperfect “ground truth”) when exploring the real data and compare the ESI methods against ECoG Laplacian.

It is also interesting to see that all the methods are giving “higher \( spAUC \) for the awake state data, and ECoG outperforms EEG more than 10%. On the contrary, in anesthesia state, all the methods give equally low accuracy compared with ECoG Laplacian, there is even no clear performance difference between EEG and ECoG in the anesthesia state data.

## Discussion

In this paper, we provide EECoG-Comp, an open source EEG and ECoG (EECoG) analysis platform. We illustrate its use with the only publicly available simultaneous EECoG recordings, the MDR dataset (Nagasaka et al. 2011). To our knowledge, this is the only public dataset that combines large coverage ECoG with moderate coverage EEG and high-quality structural imaging data, thus allowing the computation of the lead fields required for evaluation of ESI.

- 1.
For real MDR data, four popular ESI methods (MNE, LCMV, eLORETA and SSBL) showed significant but moderate concordance with a usual standard, the ECoG Laplacian (standard partial \( {\text{AUC}} = 0.65\,\; \pm \;0.05 \));

- 2.
The results from both simulation and empirical data analysis agree with several other papers (Mahjoory et al. 2017; Palva et al. 2018) in which most ESI connectivity estimators suffer from very high level of false positive and false negative connections when compared to the true source connectivity.

- 3.
In simulations, sparse random source connectivity estimation is shown to be very challenging. None of the ESI methods or ECoG Laplacian achieve high levels of accuracy of source connectivity estimation. By contrast, there is a slight improvement in performance for source connectivity estimation that have a “block” type of activations and partial correlations, which may be less sensitive to the effects of “leakage” (Paz-Linares et al. 2018). This is consistent with the results described by Todaro (2018) which quantified the effect of leakage for this same preparation.

- 4.
In simulations, the ECoG Laplacian (Nunez and Srinivasan 2006), which does not depend on the solution of the forward modeling, is only marginally better in performance than any of the four ESI connectivity estimators. But it still does not provide high accuracy when compared to the true source connectivity. This result points to the need for both better forward models (Piastra et al. 2018) as well as better methods to deal specifically with accuracy of source connectivity estimations (Palva et al. 2018; Gonzalez-Moreira et al. 2018).

- 5.
We suggest that the ECoG might not be the best tool to investigate the actual primary current density, that future experiments might require other types of measurements like Local Field Potential (Bimbi et al. 2018). Current technological advances may enable this type of wide coverage joint EEG-LFP experiment soon.

While analyzing the data, we need to stress that due to the complexity of the simultaneous EEG and ECoG experimental setup, the data quality is not that ideal. It is striking that careful artifact control and signal processing procedures failed to show a spectral peak in the alpha range (which is commonly observed in monkey experiments such as described in van Kerkoerle et al. (2014), Haegens et al. (2015) and Bollimunta et al. (2011)). This may be due to the circumstances of the recording. The monkey studied was blindfolded but may have been very alert due to unrolled factors. It also could have been due to the peculiarities of this monkey, which sometimes, as in some humans, alpha activity is not present (Charles Schroeder, personal communication). Also, as pointed out by one of the reviewers, and mentioned in the introduction, there is only one data set included in the analyses presented. Therefore, general neurobiological principles cannot be generalized from the results obtained. In addition, the relative merits of one ESI over the other are not conclusive. The small differences observed might not in practice indicate the superiority of one ESI technique over another.

- (1)
We have shown that the EEG channels that have to be rejected are all close to the ECoG connector. Only 15 EEG channels were then left, which is very challenging for inverse estimations, thus we may suggest a higher density recording system (at least 32 channel EEG).

- (2)
The subject is a male monkey, who has very thick muscles on both sides of head (see Fig. 5), which not only makes the forward modeling more difficult (may need to model the muscle as another independent counterpart with different electrical properties) but also introduces more artifacts, thus we may suggest to choose a female subject for the similar experiments.

Thus, the analysis of the MDR data set with this EECoG-Comp platform cannot only serve to compare different ESI methods in both simulations and real data, but also provide suggestions for future experiments. We share this platform via https://github.com/Vincent-wq/EECoG-Comp so that ESI researchers can access, extend and customize it for their own EECoG analysis, and help the neuroscientists to interpreted their data better. As indicated before, for valid neurobiological conclusions future experiments should be carried out with improved experimental design as well as a larger sample of monkeys. It is impossible to obtain general physiological conclusions with such a limited dataset. Nevertheless, the MDR dataset was useful to test our software platform. We certainly hope that the present work will stimulate further high-density concurrent EEG-intracranial recordings in monkey preparations.

## Conclusion

In this paper, we first provide the open source EECoG-Comp platform, which aims to help researchers analyze the source level connectivity of simultaneous EEG and ECoG experimental data, to achieve this goal, this platform offers four modules: (1) EECoG_Preproc to clean the recordings and reject artifacts, (2) EECoG_FWM to build the realistic bisophysical forward model for EEG and ECoG, (3) EECoG_conComp for inverse solutions, connectivity estimation, and statistical comparison, and (4) EECoG_scSim for source level connectivity simulation and server as the null model for self-validation and exploring the real data. This platform has great potential in solving the following problems: (1) evaluate the effectiveness of EEG analytical methods (compare to ECoG), (2) evaluate different ESI methods (the example we use to demonstrate the usage of this platform), (3) evaluate different connectivity estimators. In order to demonstrate the application of this platform, we take the simultaneous EEG/ECoG data of a single *macaca fuscata* from neurotycho.org as an example, build the head model with FEM, clean the data, and test different ESI methods in both simulations and real data.

The results from the demonstration show the capability of our EECoG-Comp platform in analyzing the such simultaneous EECoG data, and a striking result is that none of the tested ESI methods is able to give accurate source connectivity reconstructions in terms of source partial correlation matrix. This work provides practical suggestions for future protocols. In summary, this platform might help the researchers to explore this type of simultaneous EEG and ECoG data and obtain more reliable and confident conclusions. It may well stimulate further concurrent EEG-ECoG experiments in macaques.

## Open Source and Data Sharing

Table of shared contents from EECoG-Comp platform

Folders | Files | Description |
---|---|---|

1. EECoG-Csomp platform | EECoG_FWM.zip | The forward model module of EECoG-Comp platform |

EECoG_Preproc.zip | The preprocess module of EECoG-Comp platform | |

EECoG_scSim.zip | The source connectivity simulation module of EECoG-Comp platform | |

EECoG_conComp.zip | The source connectivity comparison module of EECoG-Comp platform | |

2. Lead fields | EcoG-leadfield.mat | Lead field for the EcoG data (MATLAB matrix) |

EEG-leadfield.mat | Lead field for the EEG data | |

3. Nii images | mcSu_t1[bias corrected]-eq.nii | T1 MRI of the specimen (NIFTI file) |

Svbrain_cSu_t1[bias_corrected].nii | Brain mask | |

Masked_mcSu_t1[bias_corrected]-eq.nii | MRI with non-brain tissue masked out | |

4. BrainSuite segmentations | Csf_Su.nii | Binary cerebrospinal fluid mask |

Gray_Su.nii | Binary gray matter mask | |

White_Su.nii | Binary white matter mask | |

5. SPM results | c1masked_mcSu_t1[bias corrected]-eq.nii | Probabilistic gray matter mask |

c2masked_mcSu_t1[bias corrected]-eq.nii | Probabilistic white matter mask | |

c3masked_mcSu_t1[bias corrected]-eq.nii | Probabilistic cerebrospinal fluid mask | |

Masked_mcSu_t1[bias corrected]-eq_seg_inv_sn.mat | Inverse transformation template to individual | |

Masked_mcSu_t1[bias corrected]-eq_seg_sn.mat | Transformation from template to individual | |

6. Head model | EcoG-elecs_Su-silicone_sheet.mat | Surface representing the set of silicone stripes |

Hybrid3.nii | Image labeling the three head compartments | |

Inskull_Su-corrected.mat | Inner surface of the skull | |

Outskull_Su-corrected.mat | Outer surface of the skull | |

Scalp_Su.mat | Surface of the scalp (MATLAB patch format) | |

EcoG-elecs_Su-embbed.mat | ECoG electrodes in the set of silicone stripes | |

EEG-elecs_Su.mat | EEG electrodes on the scalp | |

MESHmodel.2.ele | Elements of the FEM model (generated using TetGen) | |

MESHmodel.2.face | Modes of the FEM model (generated using TetGen) | |

Cortex-mid_Su.mat | Surface representing cortex | |

7. Atlas and template | c1masked_macaque_25_model-MNI-eq.nii | Gray matter prior for the template |

c2masked_macaque_25_model-MNI-eq.nii | White matter prior for the template | |

c3masked_macaque_25_model-MNI-eq.nii | Cerebrospinal fluid prior for the template | |

Macaque_25_model-MNI-eq.nii | Template MRI of |

## Footnotes

## Notes

### Acknowledgements

The authors would like to thank the helpful discussions with Carsten Wolters, Marios Antonakakis, Maria Carla Piastra, and Laura Marzetti in forward modeling, the discussions with Prof. Peng Ren, Mayrim Vega Hernandez, and Eduardo Martinez Montes with simulation and inverse solutions, and thanks Shiang Hu with the help in reference selection and spectrum decomposition. The authors also would like to thank for the support from the Tri-national axis in normal and pathological cognitive aging project with Grant No. 81861128001, and the funds from National Nature and Science Foundation of China (NSFC) with funding Grant Nos. 61871105, 61673090, and 81330032, and CNS Program of UESTC with Grant No. Y03111023901014005.

## References

- Baillet S, Riera JJ, Marin G, Mangin JF, Aubert J, Garnero L (2001) Evaluation of inverse methods and head models for EEG source localization using a human skull phantom. Phys Med Biol 46(1):77–96Google Scholar
- Bassett DS, Sporns O (2017) Network neuroscience. Nat Neurosci 20(3):353Google Scholar
- Bimbi M, Festante F, Coudé G, Vanderwert RE, Fox NA, Ferrari PF (2018) Simultaneous scalp recorded EEG and local field potentials from monkey ventral premotor cortex during action observation and execution reveals the contribution of mirror and motor neurons to the mu-rhythm. NeuroImage 175:22–31Google Scholar
- Bollimunta A, Mo J, Schroeder CE, Ding M (2011) Neuronal mechanisms and attentional modulation of corticothalamic alpha oscillations. J Neurosci 31(13):4935–4943Google Scholar
- Buxhoeveden DP, Casanova MF (2002) The minicolumn hypothesis in neuroscience. Brain 125(5):935–951Google Scholar
- Collier TJ, Kynor DB, Bieszczad J, Audette WE, Kobylarz EJ, Diamond SG (2012) Creation of a human head phantom for the testing of electroencephalography equipment and techniques. IEEE Trans Biomed Eng 59:2628–2643Google Scholar
- Dawson DA, Lam J, Lewis LB, Carbonell F, Mendola JD, Shmuel A (2016) Partial correlation-based retinotopically organized resting-state functional connectivity within and between areas of the visual cortex reflects more than cortical distance. Brain Connect 6(1):57–75Google Scholar
- Delorme A, Scott M (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134(1):9–21Google Scholar
- Ding L, Wilke C, Xu B, Xu X, van Drongelene W, Kohrman M, He B (2007) EEG source imaging: correlate source locations and extents with ECoG and surgical resections in epilepsy patients. J Clin Neurophysiol 24(2):130Google Scholar
- Efron B (1987) Better bootstrap confidence intervals. J Am Stat Assoc 82(397):171–185Google Scholar
- Frauscher B, von Ellenrieder N, Zelmann R et al (2018) Atlas of the normal intracranial electroencephalogram: neurophysiological awake activity in different cortical areas. Brain 141(4):1130–1144Google Scholar
- Friston KJ (2011) Functional and effective connectivity: a review. Brain Connect 1(1):13–36Google Scholar
- Gonzalez-Moreira E, Paz-Linares D, Martinez-Montes E, Valdes-Hernandez P, Bosch-Bayard J, Bringas-Vega ML, Valdes-Sosa P (2018) Populational super-resolution sparse M/EEG sources and connectivity estimation. bioRxiv. https://doi.org/10.1101/346569 Google Scholar
- Grech R, Cassar T et al (2008) Review on solving the inverse problem in EEG source analysis. J Neuroeng Rehabil 5(1):25Google Scholar
- Haegens S, Barczak A, Musacchia G, Lipton ML, Mehta AD, Lakatos P, Schroeder CE (2015) Laminar profile and physiology of the α rhythm in primary visual, auditory, and somatosensory regions of neocortex. J Neurosci 35(42):14341–14352Google Scholar
- Hallez H, Vanrumste B, Grech R, Muscat J, De Clercq W, Vergult A, Asseler AD, Camilleri KP, Fabri SG, Van Huffel S et al (2007) Review on solving the forward problem in EEG source analysis. J Neuroeng Rehab 4(1):46Google Scholar
- Hämäläinen MS, Ilmoniemi RJ (1994) Interpreting magnetic fields of the brain: minimum norm estimates. Med Biol Eng Comput 32(1):35–42Google Scholar
- He B, Yang L, Wilke C, Yuan H (2011) Electrophysiological imaging of brain activity and connectivity—challenges and opportunities. IEEE Trans Biomed Eng 58(7):1918–1931Google Scholar
- Hsieh CJ, Sustik MA, Dhillon IS, Ravikumar P (2014) QUIC: quadratic approximation for sparse inverse covariance estimation. J Mach Learn Res 15(1):2911–2947Google Scholar
- Huang Y, Sullivan Pepe M, Feng Z (2007) Evaluating the predictiveness of a continuous marker. Biometrics 63(4):1181–1188Google Scholar
- Jankova J, van de Geer S (2018) Inference in high-dimensional graphical models. arXiv. https://arxiv.org/abs/1801.08512
- Leahy RM, Mosher JC et al (1998) A study of dipole localization accuracy for MEG and EEG using a human skull phantom. Electroencephalogr Clin Neurophysiol 107(2):159–173Google Scholar
- Mahjoory K, Nikulin VV, Botrel L, Linkenkaer-Hansen K, Fato MM, Haufe S (2017) Consistency of EEG source localization and connectivity estimates. Neuroimage 152:590–601Google Scholar
- Marin G, Guerin C, Baillet S et al (1998) Influence of skull anisotropy for the forward and inverse problem in EEG: simulation studies using FEM on realistic head models. Hum Brain Mapp 6(4):250–269Google Scholar
- Mattia M, Ferraina S, Del Giudice P (2010) Dissociated multi-unit activity and local field potentials: a theory inspired analysis of a motor decision task. Neuroimage 52(3):812–823Google Scholar
- McClish DK (1989) Analyzing a portion of the ROC curve. Med Decis Mak 9(3):190–195Google Scholar
- Nagasaka Y, Shimoda K, Fujii N (2011) Multidimensional recording (MDR) and data sharing: an ecological open research and educational platform for neuroscience. PLoS ONE 6(7):e22561Google Scholar
- Nunez PL, Srinivasan R (2006) Electric fields of the brain: the neurophysics of EEG. Oxford University Press, New YorkGoogle Scholar
- Nunez PL, Nunez MD, Srinivasan R (2019) Multi-Scale Neural Sources of EEG: genuine, Equivalent, and Representative A Tutorial Review. Brain Topogr. https://doi.org/10.1101/391318 Google Scholar
- Oostenveld R, Fries P, Maris E, Schoffelen JM (2011) FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput Intell Neurosci 2011:156869Google Scholar
- Oosugi N, Kitajo K, Hasegawa N, Nagasaka Y, Okanoya K, Fujii N (2017) A new method for quantifying the performance of EEG blind source separation algorithms by referencing a simultaneously recorded ECoG signal. Neural Netw 93:1–6Google Scholar
- Palva JM, Wang SH, Palva S, Zhigalov A, Monto S, Brookes MJ, Schoffelen JM, Jerbi K (2018) Ghost interactions in MEG/EEG source space: a note of caution on inter-areal coupling measures. Neuroimage 173:632–643Google Scholar
- Papadopoulou M, Friston KJ, Marinazzo D (2015) Estimating directed connectivity from cortical recordings and reconstructed sources. Brain Topogr. https://doi.org/10.1007/s10548-015-0450-6 Google Scholar
- Pascual-Marqui RD (1999) Review of methods for solving the EEG inverse problem. Int J Bioelectromagn 1(1):75–86Google Scholar
- Pascual-Marqui RD (2007) Discrete, 3d distributed, linear imaging methods of electric neuronal activity. part 1: exact, zero error localization. arXiv. https://arxiv.org/abs/0710.3341
- Paz-Linares D et al. (2018) Caulking the Leakage Effect in MEEG Source Connectivity Analysis. ArXiv. https://arxiv.org/abs/1810.00786
- Paz-Linares D, Vega-Hernandez M, Rojas-Lopez PA, Valdes-Hernandez PA, Martinez-Montes E, Valdes-Sosa PA (2017) Spatio temporal EEG source imaging with the hierarchical Bayesian elastic net and elitist LASSO models. Front. Neurosci. 11:635Google Scholar
- Peterson SM, Ferris DP (2018) Differentiation in theta and beta electrocortical activity between visual and physical perturbations to walking and standing balance. eNeuro. https://doi.org/10.1523/ENEURO.0207-18.2018 Google Scholar
- Piastra MC, Nüßing A, Vorwerk J, Bornfleth H, Oostenveld R, Engwer C, Wolters CH (2018) The Discontinuous Galerkin Finite Element Method for Solving the MEG and the Combined MEG/EEG Forward Problem. Front Neurosci 12:30Google Scholar
- Poline JB, Breeze JL, Ghosh SS, Gorgolewski K, Halchenko YO, Hanke M, Ashburner J (2012) Data sharing in neuroimaging research. Front Neuroinform 6:1–9Google Scholar
- Schomer DL, Da Silva FL (2012) Niedermeyer’s electroencephalography: basic principles, clinical applications, and related fields. Lippincott Williams & WilkinsGoogle Scholar
- Selesnick IW, Graber HL, Ding Y, Zhang T, Barbour RL (2014) Transient artifact reduction algorithm (TARA) based on sparse optimization. IEEE Trans Signal Process 62(24):6596–6611Google Scholar
- Shattuck DW, Sandor-Leahy SR, Schaper KA, Rottenberg DA, Leahy RM (2001) Magnetic resonance image tissue classification using a partial volume model. NeuroImage 13(5):856–876Google Scholar
- Si H (2015) TetGen, a Delaunay-based quality tetrahedral mesh generator. ACM Trans Math Softw (TOMS) 41(2):11Google Scholar
- Storey JD (2002) A direct approach to false discovery rates. J Royal Stat Soc 64(3):479–498Google Scholar
- Tadel F, Baillet S, Mosher JC, Pantazis D, Leahy RM (2011) Brainstorm: a user-friendly application for MEG/EEG analysis. Comput Intell Neurosci 1:8Google Scholar
- Todaro C, Marzetti L, Valdés-Sosa PA, Valdés-Hernandez PA, Pizzella V (2018) Mapping brain activity with electrocorticography: resolution properties and robustness of inverse solutions. Brain Topogr. https://doi.org/10.1007/s10548-018-0623-1 Google Scholar
- van den Broek SP, Reinders F, Donderwinkel M, Peters M (1998) Volume conduction effects in EEG and MEG. Electroencephalogr Clin Neurophysiol 106(6):522–534Google Scholar
- Van Kerkoerle T, Self MW, Dagnino B, Gariel-Mathis MA, Poort J, Van Der Togt C, Roelfsema PR (2014) Alpha and gamma oscillations characterize feedback and feedforward processing in monkey visual cortex. Proc Natl Acad Sci 111(40):14332–14341Google Scholar
- Van Veen BD, Van Drongelen W, Yuchtman M, Suzuki A (1997) Localization of brain electrical activity via linearly constrained minimum variance spatial filtering. IEEE Trans Biomed Eng 44(9):867–880Google Scholar
- Wang X, Pang Y, Ku G et al (2003) Noninvasive laser-induced photoacoustic tomography for structural and functional in vivo imaging of the brain. Nat Biotechnol 21(7):803Google Scholar
- Wolters CH, Anwander A, Berti G, Hartmann U (2007) Geometry-adapted hexahedral meshes improve accuracy of finite element method based EEG source analysis. IEEE Trans Biomed Eng 54(8):1446–1453Google Scholar
- Zalesky A, Fornito A, Bullmore ET (2010) Network-based statistic: identifying differences in brain networks. Neuroimage 53(4):1197–1207Google Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.