Statistical significance testing is a necessary step in connectivity analysis. Several statistical test methods have been employed to assess the significance of functional connectivity, but the performance of these methods has not been thoroughly evaluated. In addition, the effects of the intrinsic brain connectivity and background couplings on performance of statistical test methods in task-based studies have not been investigated yet. The background couplings may exist independent of cognitive state and can be observed on both pre- and post-stimulus time intervals. The background couplings may be falsely detected by a statistical test as task-related connections, which can mislead interpretations of the task-related functional networks. The aim of this study was to investigate the relative performance of four commonly used non-parametric statistical test methods—surrogate, demeaned surrogate, bootstrap resampling, and Monte Carlo permutation methods—in the presence of background couplings and noise, with different signal-to-noise ratios (SNRs). Using simulated electrocorticographic (ECoG) datasets and phase locking value (PLV) as a measure of functional connectivity, we evaluated the performances of the statistical test methods utilizing sensitivity, specificity, accuracy, and receiver operating curve (ROC) analysis. Furthermore, we calculated optimal p values for each statistical test method using the ROC analysis, and found that the optimal p values were increased by decreasing the SNR. We also found that the optimal p value of the bootstrap resampling was greater than that of other methods. Our results from the simulation datasets and a real ECoG dataset, as an illustrative case report, revealed that the bootstrap resampling is the most efficient non-parametric statistical test for identifying the significant PLV of ECoG data, especially in the presence of background couplings.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Ashtari M, Lencz T, Zuffante P, Bilder R, Clarke T, Diamond A et al (2004) Left middle temporal gyrus activation during a phonemic discrimination task. NeuroReport 15:389–393
Astolfi L, Fallani FDV, Cincotti F, Mattia D, Marciani M, Salinari S et al (2009) Estimation of effective and functional cortical connectivity from neuroelectric and hemodynamic recordings. IEEE Trans Neural Syst Rehabil Eng 17:224–233
Babajani-Feremi A, Holder CM, Narayana S, Fulton SP, Choudhri AF, Boop FA et al (2018a) Predicting postoperative language outcome using presurgical fMRI, MEG, TMS, and high gamma ECoG. Clin Neurophysiol 129:560–571
Babajani-Feremi A, Noorizadeh N, Mudigoudar B, Wheless JW (2018b) Predicting seizure outcome of vagus nerve stimulation using MEG-based network topology. Neuroimage Clin 19:990–999
Barbey AK (2018) Network neuroscience theory of human intelligence. Trends Cogn Sci 22:8–20
Bastos AM, Schoffelen J-M (2016) A tutorial review of functional connectivity analysis methods and their interpretational pitfalls. Front Syst Neurosci 9:175
Binney RJ, Embleton KV, Jefferies E, Parker GJ, Ralph MA (2010) The ventral and inferolateral aspects of the anterior temporal lobe are crucial in semantic memory: evidence from a novel direct comparison of distortion-corrected fMRI, rTMS, and semantic dementia. Cereb Cortex 20:2728–2738
Chen Z, Caprihan A, Damaraju E, Rachakonda S, Calhoun V (2018) Functional brain connectivity in resting-state fMRI using phase and magnitude data. J Neurosci Methods 293:299–309
Cohen MX (2015) Effects of time lag and frequency matching on phase-based connectivity. J Neurosci Methods 250:137–146
Cohen JR (2017) The behavioral and cognitive relevance of time-varying, dynamic changes in functional connectivity. NeuroImage 180:515–525
Cole MW, Bassett DS, Power JD, Braver TS, Petersen SE (2014) Intrinsic and task-evoked network architectures of the human brain. Neuron 83:238–251
Dimitriadis SI, Zouridakis G, Rezaie R, Babajani-Feremi A, Papanicolaou AC (2015) Functional connectivity changes detected with magnetoencephalography after mild traumatic brain injury. Neuroimage Clin 9:519–531
Efron B (1982) The jackknife, the bootstrap, and other resampling plans, vol 38. SIAM, Philadelphia
Elahian B, Yeasin M, Mudigoudar B, Wheless JW, Babajani-Feremi A (2017) Identifying seizure onset zone from electrocorticographic recordings: a machine learning approach based on phase locking value. Seizure 51:35–42
Flinker A, Korzeniewska A, Shestyuk AY, Franaszczuk PJ, Dronkers NF, Knight RT et al (2015) Redefining the role of Broca’s area in speech. Proc Natl Acad Sci 112:2871–2875
Fox KCR, Foster BL, Kucyi A, Daitch AL, Parvizi J (2018) Intracranial electrophysiology of the human default network. Trends Cogn Sci 22:307–324
Fries P (2005) A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cogn Sci 9:474–480
Gordon SM, Franaszczuk PJ, Hairston WD, Vindiola M, McDowell K (2013) Comparing parametric and nonparametric methods for detecting phase synchronization in EEG. J Neurosci Methods 212:247–258
Greenblatt RE, Pflieger ME, Ossadtchi AE (2012) Connectivity measures applied to human brain electrophysiological data. J Neurosci Methods 207:1–16
Guthrie D, Buchwald JS (1991) Significance testing of difference potentials. Psychophysiology 28:240–244
Hagiwara K, Ogata K, Okamoto T, Uehara T, Hironaga N, Shigeto H et al (2014) Age-related changes across the primary and secondary somatosensory areas: an analysis of neuromagnetic oscillatory activities. Clin Neurophysiol 125:1021–1029
He BJ, Zempel JM, Snyder AZ, Raichle ME (2010) The temporal structures and functional significance of scale-free brain activity. Neuron 66:353–369
Hickok G, Poeppel D (2007) The cortical organization of speech processing. Nat Rev Neurosci 8:393
Hope TMH, Price CJ (2016) Why the left posterior inferior temporal lobe is needed for word finding. Brain 139:2823–2826
Hramov AE, Koronovskii AA, Kurovskaya MK, Moskalenko OI (2005) Synchronization of spectral components and its regularities in chaotic dynamical systems. Phys Rev E Stat Nonlinear Soft Matter Phys 71:056204
Koutsoukos E, Maillis A, Papageorgiou C, Gatzonis S, Stefanis C, Angelopoulos E (2015) The persistent and broadly distributed EEG synchronization might inhibit the normal processing capability of the human brain. Neurosci Lett 609:137–141
Krienen FM, Yeo BT, Buckner RL (2014) Reconfigurable task-dependent functional coupling modes cluster around a core functional architecture. Philos Trans R Soc Lond B Biol Sci 369:20130526
Kucyi A, Schrouff J, Bickel S, Foster BL, Shine JM, Parvizi J (2018) Intracranial electrophysiology reveals reproducible intrinsic functional connectivity within human brain networks. J Neurosci 38:4230–4242
Lachaux J, Rodriguez E, Martinerie J, Varela FJ (1999) Measuring phase synchrony in brain signals. ma.utexas.edu
Liebenthal E, Binder JR, Spitzer SM, Possing ET, Medler DA (2005) Neural substrates of phonemic perception. Cereb Cortex 15:1621–1631
Maris E, Oostenveld R (2007) Nonparametric statistical testing of EEG- and MEG-data. J Neurosci Methods 164:177–190
Micheli C, Kaping D, Westendorff S, Valiante TA, Womelsdorf T (2015) Inferior-frontal cortex phase synchronizes with the temporal–parietal junction prior to successful change detection. NeuroImage 119:417–431
Miller KJ, Sorensen LB, Ojemann JG, den Nijs M (2009) Power-law scaling in the brain surface electric potential. PLoS Comput Biol 5:e1000609
Nobre AC, Allison T, McCarthy G (1994) Word recognition in the human inferior temporal lobe. Nature 372:260–263
Nolte G, Bai O, Wheaton L, Mari Z, Vorbach S, Hallett M (2004) Identifying true brain interaction from EEG data using the imaginary part of coherency. Clin Neurophysiol 115:2292–2307
Nolte G, Ziehe A, Nikulin VV, Schlögl A, Krämer N, Brismar T et al (2008) Robustly estimating the flow direction of information in complex physical systems. Phys Rev Lett 100:234101
Phillips JM, Vinck M, Everling S, Womelsdorf T (2014) A long-range fronto-parietal 5- to 10-Hz network predicts “top-down” controlled guidance in a task-switch paradigm. Cereb Cortex 24:1996–2008
Porcaro C, Coppola G, Pierelli F, Seri S, Di Lorenzo G, Tomasevic L et al (2013) Multiple frequency functional connectivity in the hand somatosensory network: an EEG study. Clin Neurophysiol 124:1216–1224
Rimol LM, Specht K, Hugdahl K (2006) Controlling for individual differences in fMRI brain activation to tones, syllables, and words. Neuroimage 30:554–562
Sadaghiani S, Poline J-B, Kleinschmidt A, D’Esposito M (2015) Ongoing dynamics in large-scale functional connectivity predict perception. Proc Natl Acad Sci USA 112:8463–8468
Sekihara K, Sahani M, Nagarajan S (2004) Bootstrap-based statistical thresholding for MEG source reconstruction images. In: 26th annual international conference of the IEEE on engineering in medicine and biology society, 2004. IEMBS’04, pp 1018–1021
Slepian D (1978) Prolate spheroidal wave functions, Fourier analysis, and uncertainty—V: the discrete case. Bell Labs Tech J 57:1371–1430
Srinivasan R, Winter WR, Ding J, Nunez PL (2007) EEG and MEG coherence: measures of functional connectivity at distinct spatial scales of neocortical dynamics. J Neurosci Methods 166:41–52
Stam C, Nolte G, Daffertshofer A (2007) Phase lag index: assessment of functional connectivity from multi channel EEG and MEG with diminished bias from common sources. Wiley Online Library, New York
Stephen EP, Lepage KQ, Eden UT, Brunner P, Schalk G, Brumberg JS et al (2014) Assessing dynamics, spatial scale, and uncertainty in task-related brain network analyses. Front Comput Neurosci 8:31
Takahashi T, Goto T, Nobukawa S, Tanaka Y, Kikuchi M, Higashima M et al (2018) Abnormal functional connectivity of high-frequency rhythms in drug-naïve schizophrenia. Clin Neurophysiol 129:222–231
Theiler J, Eubank S, Longtin A, Galdrikian B, Doyne Farmer J (1992) Testing for nonlinearity in time series: the method of surrogate data. Physica D 58:77–94
Tomasello R, Garagnani M, Wennekers T, Pulvermüller F (2017) Brain connections of words, perceptions and actions: a neurobiological model of spatio-temporal semantic activation in the human cortex. Neuropsychologia 98:111–129
Varela F, Lachaux J-P, Rodriguez E, Martinerie J (2001) The brainweb: phase synchronization and large-scale integration. Nat Rev Neurosci 2:229–239
Vinck M, Oostenveld R, Wingerden MV, Battaglia F (2011) An improved index of phase-synchronization for electrophysiological data in the presence of volume-conduction, noise and sample-size bias. Elsevier, Amsterdam
Yang H, Lu K, Lyu X, Hu F (2017) Two-way partial AUC and its properties. Stat Methods Med Res 28:184–195
This study was supported by the Children’s Foundation Research Institute at Le Bonheur Children’s Hospital and the Le Bonheur Associate Board, Memphis, TN.
Conflict of interest
None of the authors have any conflicts of interest to disclose.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Handling Editor: Kevin Whittingstall.
About this article
Cite this article
Mostame, P., Moharramipour, A., Hossein-Zadeh, GA. et al. Statistical Significance Assessment of Phase Synchrony in the Presence of Background Couplings: An ECoG Study. Brain Topogr 32, 882–896 (2019). https://doi.org/10.1007/s10548-019-00718-8
- Functional connectivity
- Phase locking value (PLV)
- Background couplings
- Statistical test
- Electrocorticography (ECoG)