Skip to main content
SpringerLink
Log in

Reproducible Inter-Personal Brain Coupling Measurements in Hyperscanning Settings With functional Near Infra-Red Spectroscopy

  • Original Article
  • Published:
Neuroinformatics Aims and scope Submit manuscript

A Correction to this article was published on 13 January 2022

This article has been updated

Abstract

Despite a huge advancement in neuroimaging techniques and growing importance of inter-personal brain research, few studies assess the most appropriate computational methods to measure brain-brain coupling. Here, we focus on the signal processing methods to detect brain-coupling in dyads. From a public dataset of functional Near Infra-Red Spectroscopy signals (N=24 dyads), we derived a synthetic control condition by randomization, we investigated the effectiveness of four most used signal similarity metrics: Cross Correlation, Mutual Information, Wavelet Coherence and Dynamic Time Warping. We also accounted for temporal variations between signals by allowing for misalignments up to a maximum lag. Starting from the observed effect sizes, computed in terms of Cohen’s d, the power analysis indicated that a high sample size (\(N> 150\)) would be required to detect significant brain-coupling. We therefore discuss the need for specialized statistical approaches and propose bootstrap as an alternative method to avoid over-penalizing the results. In our settings, and based on bootstrap analyses, Cross Correlation and Dynamic Time Warping outperform Mutual Information and Wavelet Coherence for all considered maximum lags, with reproducible results. These results highlight the need to set specific guidelines as the high degree of customization of the signal processing procedures prevents the comparability between studies, their reproducibility and, ultimately, undermines the possibility of extracting new knowledge.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Change history

References

  • Abibullaev, B., An, J., Jin, S.-H., Lee, S. H., & Moon, J. I. (2013). Minimizing inter-subject variability in fnirs-based brain-computer interfaces via multiple-kernel support vector learning. Medical Engineering & Physics, 35(12), 1811–1818.

    Article  Google Scholar 

  • Ayrolles, A., Brun, F., Chen, P., Djalovski, A., Beauxis, Y., Delorme, R., Bourgeron, T., Dikker, S., & Dumas, G. (2020). HyPyP: a hyperscanning python pipeline for inter-brain connectivity analysis. Social Cognitive and Affective Neuroscience, 16(1–2).

  • Azhari, A., Gabrieli, G., Bizzego, A., Bornstein, M. H., & Esposito, G. (2020a). Probing the association between maternal anxious attachment style and mother-child brain-to-brain coupling during passive co-viewing of visual stimuli. Attachment & Human Development, pages 1–16.

  • Azhari, A., Leck, W., Gabrieli, G., Bizzego, A., Rigo, P., Setoh, P., et al. (2019). Parenting stress undermines mother-child brain-to-brain synchrony: A hyperscanning study. Scientific Reports, 9(1), 1–9.

    Article  CAS  Google Scholar 

  • Azhari, A., Lim, M., Bizzego, A., Gabrieli, G., Bornstein, M. H., & Esposito, G. (2020). Physical presence of spouse enhances brain-to-brain synchrony in co-parenting couples. Scientific Reports, 10(1), 1–11.

    Article  Google Scholar 

  • Azhari, A., Rigo, P., Bornstein, M. H., & Esposito, G. (2020c). Where sounds occur matters: Context effects influence processing of salient vocalisations. Brain Sciences, 10(7).

  • Berndt, D. J. & Clifford, J. (1994). Using dynamic time warping to find patterns in time series. In Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining, page 359-370. AAAI Press.

  • Bigliassi, M., León-Domínguez, U., & Altimari, L. R. (2015). How does the prefrontal cortex listen to classical and techno music? a functional near-infrared spectroscopy (fNIRS) study. Psychology & Neuroscience, 8(2), 246–256.

    Article  Google Scholar 

  • Bilek, E., Ruf, M., Schäfer, A., Akdeniz, C., Calhoun, V. D., Schmahl, C., et al. (2015). Information flow between interacting human brains: Identification, validation, and relationship to social expertise. Proceedings of the National Academy of Sciences, 112(16), 5207–5212.

    Article  CAS  Google Scholar 

  • Bizzego, A., Azhari, A., Campostrini, N., Truzzi, A., Ng, L. Y., Gabrieli, G., et al. (2020). Strangers, friends, and lovers show different physiological synchrony in different emotional states. Behavioral Sciences, 10(1), 11.

    Article  Google Scholar 

  • Bizzego, A., Balagtas, J. P. M., & Esposito, G. (2020). Commentary: Current status and issues regarding pre-processing of fnirs neuroimaging data: An investigation of diverse signal filtering methods within a general linear model framework. Frontiers in Human Neuroscience, 14, 247.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bizzego, A., Gabrieli, G., Azhari, A., Setoh, P., & Esposito, G. (2021). Computational methods for the assessment of empathic synchrony. In Esposito, A., Faundez-Zanuy, M., Morabito, F. C., and Pasero, E., editors, Progresses in Artificial Intelligence and Neural Systems. Springer Singapore.

  • Carpenter, J., & Bithell, J. (2000). Bootstrap confidence intervals: when, which, what? a practical guide for medical statisticians. Statistics in Medicine, 19(9), 1141–1164.

    Article  CAS  PubMed  Google Scholar 

  • Chatel-Goldman, J., Schwartz, J.-L., Jutten, C., & Congedo, M. (2013). Non-local mind from the perspective of social cognition. Frontiers in Human Neuroscience, 7, 107.

    Article  PubMed  PubMed Central  Google Scholar 

  • Chen, Y., Zhang, Q., Yuan, S., Zhao, B., Zhang, P., & Bai, X. (2020). The influence of prior intention on joint action: an fnirs-based hyperscanning study. Social Cognitive and Affective Neuroscience, 15(12), 1351–1360.

    Article  PubMed  Google Scholar 

  • Chong, J. S., Lu, C. K., & Tang, T. B. (2019). Study of emotional state effect on decision making by using fNIRS. 2019 IEEE International Circuits and Systems Symposium (ICSyS).

  • Czeszumski, A., Eustergerling, S., Lang, A., Menrath, D., Gerstenberger, M., Schuberth, S., et al. (2020). Hyperscanning: a valid method to study neural inter-brain underpinnings of social interaction. Frontiers in Human Neuroscience, 14, 39.

    Article  PubMed  PubMed Central  Google Scholar 

  • de Pasquale, F., Della Penna, S., Snyder, A. Z., Lewis, C., Mantini, D., Marzetti, L., et al. (2010). Temporal dynamics of spontaneous meg activity in brain networks. Proceedings of the National Academy of Sciences, 107(13), 6040–6045.

    Article  Google Scholar 

  • Dwivedi, A. K., Mallawaarachchi, I., & Alvarado, L. A. (2017). Analysis of small sample size studies using nonparametric bootstrap test with pooled resampling method. Statistics in Medicine, 36(14), 2187–2205.

    PubMed  Google Scholar 

  • Efron, B. & Tibshirani, R. J. (1994). An introduction to the bootstrap. CRC Press.

  • Faes, L., Nollo, G., Jurysta, F., & Marinazzo, D. (2014). Information dynamics of brain-heart physiological networks during sleep. New Journal of Physics, 16(10).

    Article  Google Scholar 

  • Fallani, F. D. V., Nicosia, V., Sinatra, R., Astolfi, L., Cincotti, F., Mattia, D., et al. (2010). Defecting or not defecting: how to human behavior during cooperative games by eeg measurements. PloS one, 5(12), 1–9.

    Google Scholar 

  • Fisher, N. I., & Hall, P. (1991). Bootstrap algorithms for small samples. Journal of Statistical Planning and Inference, 27(2), 157–169.

    Article  Google Scholar 

  • Funane, T., Kiguchi, M., Atsumori, H., Sato, H., Kubota, K., & Koizumi, H. (2011). Synchronous activity of two people’s prefrontal cortices during a cooperative task measured by simultaneous near-infrared spectroscopy. Journal of Biomedical Optics, 16(7).

    Article  PubMed  Google Scholar 

  • Gashi, S., Di Lascio, E., & Santini, S. (2018). Using students’ physiological synchrony to quantify the classroom emotional climate. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, page 698-701.

  • Giorgino, T. (2009). Computing and visualizing dynamic time warping alignments in R: The dtw package. Journal of Statistical Software, 31(7), 1–24.

    Article  Google Scholar 

  • Golland, Y., Arzouan, Y., & Levit-Binnun, N. (2015). The mere co-presence: Synchronization of autonomic signals and emotional responses across co-present individuals not engaged in direct interaction. PloS one, 10(5).

    Article  PubMed  PubMed Central  Google Scholar 

  • Golland, Y., Keissar, K., & Levit-Binnun, N. (2014). Studying the dynamics of autonomic activity during emotional experience. Psychophysiology, 51(11), 1101–1111.

    Article  PubMed  Google Scholar 

  • Golland, Y., Mevorach, D., & Levit-Binnun, N. (2019). Affiliative zygomatic synchrony in co-present strangers. Scientific Reports, 9(1), 1–10.

    Article  CAS  Google Scholar 

  • Grinsted, A., Moore, J. C., & Jevrejeva, S. (2004). Application of the cross wavelet transform and wavelet coherence to geophysical time series. Nonlinear Processes in Geophysics, 11(5–6), 561–566.

    Article  Google Scholar 

  • Handwerker, D. A., Roopchansingh, V., Gonzalez-Castillo, J., & Bandettini, P. A. (2012). Periodic changes in fMRI connectivity. NeuroImage, 63(3), 1712–1719.

    Article  PubMed  Google Scholar 

  • Hasson, U., & Frith, C. D. (2016). Mirroring and beyond: coupled dynamics as a generalized framework for modelling social interactions. Philosophical Transactions of the Royal Society B: Biological Sciences, 371(1693), 20150366.

    Article  Google Scholar 

  • Hasson, U., Ghazanfar, A. A., Galantucci, B., Garrod, S., & Keysers, C. (2012). Brain-to-brain coupling: a mechanism for creating and sharing a social world. Trends in Cognitive Sciences, 16(2), 114–121.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., & Malach, R. (2004). Intersubject synchronization of cortical activity during natural vision. Science, 303(5664), 1634–1640.

    Article  CAS  PubMed  Google Scholar 

  • Hinrichs, H., Heinze, H.-J., & Schoenfeld, M. A. (2006). Causal visual interactions as revealed by an information theoretic measure and fMRI. NeuroImage, 31(3), 1051–1060.

    Article  CAS  PubMed  Google Scholar 

  • Huppert, T. J. (2016). Commentary on the statistical properties of noise and its implication on general linear models in functional near-infrared spectroscopy. Neurophotonics, 3(1).

    Article  PubMed  PubMed Central  Google Scholar 

  • Huppert, T. J., Hoge, R. D., Diamond, S. G., Franceschini, M. A., & Boas, D. A. (2006). A temporal comparison of bold, asl, and nirs hemodynamic responses to motor stimuli in adult humans. Neuroimage, 29(2), 368–382.

    Article  CAS  PubMed  Google Scholar 

  • Ikeda, S., Nozawa, T., Yokoyama, R., Miyazaki, A., Sasaki, Y., Sakaki, K., & Kawashima, R. (2017). Steady beat sound facilitates both coordinated group walking and Inter-Subject neural synchrony. Frontiers in Human Neuroscience, 11.

  • Jäncke, L., Loose, R., Lutz, K., Specht, K., & Shah, N. J. (2000). Cortical activations during paced finger-tapping applying visual and auditory pacing stimuli. Brain Research. Cognitive Brain Research, 10(1–2), 51–66.

    Article  PubMed  Google Scholar 

  • Jiang, J., Dai, B., Peng, D., Zhu, C., Liu, L., & Lu, C. (2012). Neural synchronization during face-to-face communication. Journal of Neuroscience, 32(45), 16064–16069.

    Article  CAS  PubMed  Google Scholar 

  • Karamzadeh, N., Medvedev, A., Azari, A., Gandjbakhche, A., & Najafizadeh, L. (2013). Capturing dynamic patterns of task-based functional connectivity with EEG. NeuroImage, 66, 311–317.

    Article  PubMed  Google Scholar 

  • Kawasaki, M., Yamada, Y., Ushiku, Y., Miyauchi, E., & Yamaguchi, Y. (2013). Inter-brain synchronization during coordination of speech rhythm in human-to-human social interaction. Scientific Reports, 3(1), 1–8.

    Article  Google Scholar 

  • Kettunen, J., Ravaja, N., Näätänen, P., Keskivaara, P., & Keltikangas-Järvinen, L. (1998). The synchronization of electrodermal activity and heart rate and its relationship to energetic arousal: A time series approach. Biological Psychology, 48(3), 209–225.

    Article  CAS  PubMed  Google Scholar 

  • Kinreich, S., Djalovski, A., Kraus, L., Louzoun, Y., & Feldman, R. (2017). Brain-to-Brain synchrony during naturalistic social interactions. Scientific Reports, 7(1).

  • Kiviniemi, V., Vire, T., Remes, J., Elseoud, A. A., Starck, T., Tervonen, O., & Nikkinen, J. (2011). A sliding Time-Window ICA reveals spatial variability of the default mode network in time. Brain Connectivity, 1(4), 339–347.

    Article  PubMed  Google Scholar 

  • Konvalinka, I., Vuust, P., Roepstorff, A., & Frith, C. D. (2010). Follow you, follow me: continuous mutual prediction and adaptation in joint tapping. The Quarterly Journal of Experimental Psychology, 63(11), 2220–2230.

    Article  PubMed  Google Scholar 

  • Liu, D., Liu, S., Liu, X., Zhang, C., Li, A., Jin, C., et al. (2018). Interactive brain activity: Review and progress on EEG-Based hyperscanning in social interactions. Frontiers in Psychology, 9, 1862.

    Article  PubMed  PubMed Central  Google Scholar 

  • Liu, N., Mok, C., Witt, E. E., Pradhan, A. H., Chen, J. E., & Reiss, A. L. (2016). NIRS-based hyperscanning reveals inter-brain neural synchronization during cooperative jenga game with face-to-face communication. Frontiers in Human Neuroscience, 10, 82.

    Article  PubMed  PubMed Central  Google Scholar 

  • Lizier, J. T. (2014). Jidt: An information-theoretic toolkit for studying the dynamics of complex systems. Frontiers in Robotics and AI, 1, 11.

    Article  Google Scholar 

  • Lombardo, M. V., Auyeung, B., Holt, R. J., Waldman, J., Ruigrok, A. N. V., Mooney, N., et al. (2016). Improving effect size estimation and statistical power with multi-echo fMRI and its impact on understanding the neural systems supporting mentalizing. NeuroImage, 142, 55–66.

    Article  PubMed  Google Scholar 

  • Mauss, I. B., Levenson, R. W., McCarter, L., Wilhelm, F. H., & Gross, J. J. (2005). The tie that binds? coherence among emotion experience, behavior, and physiology. Emotion, 5(2), 175.

    Article  PubMed  Google Scholar 

  • Meszlenyi, R. J., Hermann, P., Buza, K., Gál, V., & Vidnyánszky, Z. (2017). Resting state fMRI functional connectivity analysis using dynamic time warping. Frontiers in Neuroscience, 11, 75.

    Article  PubMed  PubMed Central  Google Scholar 

  • Mønster, D., Håkonsson, D. D., Eskildsen, J. K., & Wallot, S. (2016). Physiological evidence of interpersonal dynamics in a cooperative production task. Physiology & Behavior, 156, 24–34.

    Article  Google Scholar 

  • Mostofian, B., & Zuckerman, D. M. (2019). Statistical uncertainty analysis for small-sample, high log-variance data: Cautions for bootstrapping and bayesian bootstrapping. Journal of chemical theory and computation, 15(6), 3499–3509.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Nam, C. S., Choo, S., Huang, J., & Park, J. (2020). Brain-to-brain neural synchrony during social interactions: A systematic review on hyperscanning studies. Applied Sciences, 10(19), 6669.

    Article  CAS  Google Scholar 

  • Pan, Y., Cheng, X., Zhang, Z., Li, X., & Hu, Y. (2017). Cooperation in lovers: An fNIRS-based hyperscanning study. Human Brain Mapping, 38(2), 831–841.

    Article  PubMed  Google Scholar 

  • Pan, Y., Novembre, G., Song, B., Zhu, Y., & Hu, Y. (2020). Dual brain stimulation enhances interpersonal learning through spontaneous movement synchrony. Social Cognitive and Affective Neuroscience, 16.

  • Pinti, P., Scholkmann, F., Hamilton, A., Burgess, P., & Tachtsidis, I. (2019). Current status and issues regarding pre-processing of fnirs neuroimaging data: an investigation of diverse signal filtering methods within a general linear model framework. Frontiers in Human Neuroscience, 12, 505.

    Article  PubMed  PubMed Central  Google Scholar 

  • Pravitha Ramanand, M. C. B. & Bruce, E. N. (2010). Mutual information analysis of eeg signals indicates age-related changes in cortical interdependence during sleep in middle-aged vs. elderly women. Journal of Clinical Neurophysiology, 27(4):274.

  • Repp, B. H. (2005). Sensorimotor synchronization: a review of the tapping literature. Psychonomic Bulletin & Review, 12(6), 969–992.

    Article  Google Scholar 

  • Sadoun, A., Chauhan, T., Mameri, S., Zhang, Y. F., Barone, P., Deguine, O., & Strelnikov, K. (2020). Stimulus-specific information is represented as local activity patterns across the brain. NeuroImage, 223,.

    Article  PubMed  Google Scholar 

  • Sakoe, H., & Chiba, S. (1978). Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing, 26(1), 43–49.

    Article  Google Scholar 

  • Salvador, R., Martinez, A., Pomarol-Clotet, E., Sarró, S., Suckling, J., & Bullmore, E. (2007). Frequency based mutual information measures between clusters of brain regions in functional magnetic resonance imaging. NeuroImage, 35(1), 83–88.

    Article  CAS  PubMed  Google Scholar 

  • Scholkmann, F., Holper, L., Wolf, U., & Wolf, M. (2013). A new methodical approach in neuroscience: assessing inter-personal brain coupling using functional Near-Infrared Imaging (fNIRI) hyperscanning. Frontiers in Human Neuroscience, 7, 813.

    Article  PubMed  PubMed Central  Google Scholar 

  • Seghouane, A.-K., & Ferrari, D. (2019). Robust hemodynamic response function estimation from fnirs signals. IEEE Transactions on Signal Processing, 67(7), 1838–1848.

    Article  Google Scholar 

  • Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27(3), 379–423.

    Article  Google Scholar 

  • Sun, C., Yang, F., Wang, C., Wang, Z., Zhang, Y., Ming, D., & Du, J. (2018). Mutual information-based brain network analysis in post-stroke patients with different levels of depression. Frontiers in Human Neuroscience, 12, 285.

    Article  PubMed  PubMed Central  Google Scholar 

  • Tang, H., Mai, X., Wang, S., Zhu, C., Krueger, F., & Liu, C. (2016). Interpersonal brain synchronization in the right temporo-parietal junction during face-to-face economic exchange. Social Cognitive and Affective Neuroscience, 11(1), 23–32.

    Article  PubMed  Google Scholar 

  • Taylor, A. J., Kim, J. H., & Ress, D. (2018). Characterization of the hemodynamic response function across the majority of human cerebral cortex. NeuroImage, 173, 322–331.

    Article  PubMed  Google Scholar 

  • Toppi, J., Borghini, G., Petti, M., He, E. J., De Giusti, V., He, B., et al. (2016). Investigating cooperative behavior in ecological settings: an EEG hyperscanning study. PloS one, 11(4).

    Article  PubMed  PubMed Central  Google Scholar 

  • Torrence, C., & Compo, G. P. (1998). A practical guide to wavelet analysis. Bulletin of the American Meteorological Society, 79(1), 61–78.

    Article  Google Scholar 

  • Trendafilov, D., Schmitz, G., Hwang, T.-H., Effenberg, A. O., & Polani, D. (2020). Tilting together: An Information-Theoretic characterization of behavioral roles in rhythmic dyadic interaction. Frontiers in Human Neuroscience, 14.

  • Vesper, C., & Richardson, M. J. (2014). Strategic communication and behavioral coupling in asymmetric joint action. Experimental Brain Research, 232(9), 2945–2956.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wass, S. V., Noreika, V., Georgieva, S., Clackson, K., Brightman, L., Nutbrown, R., et al. (2018). Parental neural responsivity to infants’ visual attention: How mature brains influence immature brains during social interaction. PLOS Biology, 16(12).

    Article  PubMed  PubMed Central  Google Scholar 

  • Zhang, M., Ding, K., Jia, H., & Yu, D. (2018). Brain-to-brain synchronization of the expectation of cooperation behavior: A fNIRS hyperscanning study. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 546–549. IEEE.

  • Zhang, X., Noah, J. A., Dravida, S., & Hirsch, J. (2020). Optimization of wavelet coherence analysis as a measure of neural synchrony during hyperscanning using functional near-infrared spectroscopy. Neurophotonics, 7(1).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Funding

A.B. was supported by a Post-doctoral Fellowship within MIUR programme framework “Dipartimenti di Eccellenza” (DiPSCO, University of Trento). G.E. was supported by NAP SUG 2015, Singapore Ministry of Education ACR Tier 1 (RG149/16 and RT10/19).

Author information

Authors and Affiliations

  1. Department of Psychology and Cognitive Science, University of Trento, Trento, Italy

    Bizzego Andrea & Esposito Gianluca

  2. Psychology Program, School of Social Sciences, Nanyang Technological University, Singapore, Singapore

    Azhari Atiqah & Esposito Gianluca

  3. Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore

    Esposito Gianluca

Authors
  1. Bizzego Andrea
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Azhari Atiqah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Esposito Gianluca
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Esposito Gianluca.

Ethics declarations

Ethics Approval

Not applicable.

Conflict of Interest

The authors declare no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Andrea, B., Atiqah, A. & Gianluca, E. Reproducible Inter-Personal Brain Coupling Measurements in Hyperscanning Settings With functional Near Infra-Red Spectroscopy. Neuroinform 20, 665–675 (2022). https://doi.org/10.1007/s12021-021-09551-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12021-021-09551-6

Keywords

Search

Navigation