Advertisement

Data-driven Koopman operator approach for computational neuroscience

  • Natasza MarrouchEmail author
  • Joanna Slawinska
  • Dimitrios Giannakis
  • Heather L. Read
Open Access
Article
First Online:
  • 107 Downloads

Abstract

This article presents a novel, nonlinear, data-driven signal processing method, which can help neuroscience researchers visualize and understand complex dynamical patterns in both time and space. Specifically, we present applications of a Koopman operator approach for eigendecomposition of electrophysiological signals into orthogonal, coherent components and examine their associated spatiotemporal dynamics. This approach thus provides enhanced capabilities over conventional computational neuroscience tools restricted to analyzing signals in either the time or space domains. This is achieved via machine learning and kernel methods for data-driven approximation of skew-product dynamical systems. The approximations successfully converge to theoretical values in the limit of long embedding windows. First, we describe the method, then using electrocorticographic (ECoG) data from a mismatch negativity experiment, we extract time-separable frequencies without bandpass filtering or prior selection of wavelet features. Finally, we discuss in detail two of the extracted components, Beta (\( \sim \) 13 Hz) and high Gamma (\( \sim \) 50 Hz) frequencies, and explore the spatiotemporal dynamics of high- and low- frequency components.

Keywords

Koopman operator Spectral decomposition Nonlinear Spatiotemporal dynamics ECoG Brain Mismatch negativity 

Mathematics Subject Classification (2010)

37M10 37M25 58C40 30C40 37N25 47A35 92C55 
Download to read the full article text

Notes

Acknowledgements

The authors would like to thank Ian Stevenson, Stephen Herzog, and the journal editorial team and reviewers for their helpful comments. Misako Komatsu, Kana Takura, and Naotaka Fujii from the RIKEN Brain Science Institute generously provided the open-access data used in this article. HLR has ownership interest in Elemind Technologies, Inc.

References

  1. 1.
    Alho, K.: Cerebral generators of mismatch negativity (MMN) and its magnetic counterpart (MMNm) elicited by sound changes. Ear Hear. 16(1), 38–51 (1995)CrossRefGoogle Scholar
  2. 2.
    Alho, K., Woods, D.L., Algazi, A., Knight, R.T., Näätänen, R.: Lesions of frontal cortex diminish the auditory mismatch negativity. Electroencephalogr. Clin. Neurophysiol. 91(5), 353–362 (1994).  https://doi.org/10.1016/0013-4694(94)00173-1 CrossRefGoogle Scholar
  3. 3.
    Aru, J., Aru, J., Priesemann, V., Wibral, M., Lana, L., Pipa, G., Singer, W., Vicente, R.: Untangling cross-frequency coupling in neuroscience. Curr. Opin. Neurobiol. 31, 51–61 (2015).  https://doi.org/10.1016/j.conb.2014.08.002 CrossRefGoogle Scholar
  4. 4.
    Aubry, N., Guyonnet, R., Lima, R.: Spatiotemporal analysis of complex signals: Theory and applications. J. Stat. Phys. 64, 683–739 (1991).  https://doi.org/10.1007/bf01048312 MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Belkin, M., Niyogi, P.: Laplacian Eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 1373–1396 (2003).  https://doi.org/10.1162/089976603321780317 CrossRefzbMATHGoogle Scholar
  6. 6.
    Brunton, B.W., Johnson, L.A., Ojemann, J.G., Kutz, J.N.: Extracting spatial–temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition. J. Neurosci. Methods 258, 1–15 (2016).  https://doi.org/10.1016/j.jneumeth.2015.10.010 CrossRefGoogle Scholar
  7. 7.
    Budisić, M, Mohr, R, Mezić, I.: Applied Koopmanism. Chaos 22, 047510 (2012).  https://doi.org/10.1063/1.4772195 MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Chandrasekaran, C., Turesson, H.K., Brown, C.H., Ghazanfar, A.A.: The Influence of Natural Scene Dynamics on Auditory Cortical Activity. J. Neurosci. 30(42), 13919–13931 (2010).  https://doi.org/10.1523/JNEUROSCI.3174-10.2010 CrossRefGoogle Scholar
  9. 9.
    Coenen, A., Fine, E., Zayachkivska, O.A.: Beck: A forgotten pioneer in electroencephalography. Journal of the History of the Neurosciences: Basic and Clinical Perspectives 23(3), 276–286 (2014)CrossRefGoogle Scholar
  10. 10.
    Coifman, R.R., Lafon, S.: Diffusion maps. Appl. Comput. Harmon. Anal. 21, 5–30 (2006).  https://doi.org/10.1016/j.acha.2006.04.006 MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Cong, F., Kalyakin, I., Li, H., Huttunen-Scott, T., Huang, Y., Lyytinen, H., Ristaniemi, T.: Answering six questions in extracting children’s mismatch negativity through combining wavelet decomposition and independent component analysis. Cogn. Neurodyn. 5(4), 343–359 (2011).  https://doi.org/10.1007/s11571-011-9161-1 CrossRefGoogle Scholar
  12. 12.
    Cong, F., Sipola, T., Huttunen-Scott, T., Xu, X., Ristaniemi, T., Lyytinen, H.: Hilbert-Huang versus Morlet wavelet transformation on mismatch negativity of children in uninterrupted sound paradigm. Nonlinear Biomedical Physics 3(1) (2009).  https://doi.org/10.1186/1753-4631-3-1
  13. 13.
    Corlett, P.R., Marrouch, N.: Social cognitive neuroscience of attitudes and beliefs. In: Albarracín, D., Johnson, B. T. (eds.) Handbook of Attitudes and Attitude Change, vol. 1, pp 480–519. Taylor & Francis, New York (2018)Google Scholar
  14. 14.
    Csépe, V., Karmos, G., Molnár, M.: Evoked potential correlates of stimulus deviance during wakefulness and sleep in cat — animal model of mismatch negativity. Electroencephalogr. Clin. Neurophysiol. 66, 571–578 (1987).  https://doi.org/10.1016/0013-4694(87)90103-9 CrossRefGoogle Scholar
  15. 15.
    Das, S., Giannakis, D.: Delay-coordinate maps and the spectra of Koopman operators. J. Stati. Phy. 14 (6), 1107–1145 (2019).  https://doi.org/10.1007/s10955-019-02272-w MathSciNetCrossRefGoogle Scholar
  16. 16.
    Dellnitz, M., Junge, O.: On the Approximation of Complicated Dynamical Behavior. SIAM J. Numer. Anal. 36, 491 (1999).  https://doi.org/10.1137/S0036142996313002 MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    van Drongelen, W.: Signal processing for neuroscientists. Elsevier, Amsterdam (2007)Google Scholar
  18. 18.
    Dürschmid, S., Edwards, E., Reichert, C., Dewar, C., Hinrichs, H., Heinze, H. -J., Kirsch, H.E., Dalal, S.S., Deouell, L.Y., Knight, R.T.: Hierarchy of prediction errors for auditory events in human temporal and frontal cortex. Proc. Natl. Acad. Sci. 113, 6755–6760 (2016).  https://doi.org/10.1073/pnas.1525030113 CrossRefGoogle Scholar
  19. 19.
    Eisner, T., Farkas, B., Haase, M., Nagel, R.: Operator Theoretic Aspects of Ergodic Theory. Graduate Texts in Mathematics, vol. 272. Springer, Berlin (2015)CrossRefGoogle Scholar
  20. 20.
    Escabí, M. A., Read, H.L., Viventi, J., Kim, D -H, Higgins, N.C., Storace, D.A., Liu, A.S.K., Gifford, A.M., Burke, J.A., Campisi, M., Kim, Y -S, Avrin, A.E., Van der Spiegel, J., Huang, Y., Li, M., Wu, J., Rogers, J.A., Litt, B., Cohen, Y.E.: A high-density, high-channel count, multiplexed μ ECoG array for auditory cortex. J Neurophysiol 112, 1566–1583 (2014).  https://doi.org/10.1152/jn.00179.2013 CrossRefGoogle Scholar
  21. 21.
    Ford, J.M., Hillpard, S.A.: Event–related potentials (ERPs) to Interruptions of a Steady Rhythm. Psychophysiology 18(3), 322–330 (1981).  https://doi.org/10.1111/j.1469-8986.1981.tb03043.x CrossRefGoogle Scholar
  22. 22.
    Giannakis, D.: Dynamics-adapted cone kernels. SIAM J. Appl. Dyn. Sys. 14(2), 556–608 (2015).  https://doi.org/10.1137/140954544 MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Giannakis, D.: Data-driven spectral decomposition and forecasting of ergodic dynamical systems. Appl. Comut. Harmon. Anal 47(2), 338–396 (2019).  https://doi.org/10.1016/j.acha.2017.09.001 MathSciNetCrossRefGoogle Scholar
  24. 24.
    Giannakis, D., Das, S.: Extraction and prediction of coherent patterns in incompressible flows through space-time Koopman analysis. Phys. D. In revision. arXiv:1706.06450 (2017)
  25. 25.
    Giannakis, D., Kolchinskaya, A., Krasnov, D., Schumacher, J.: Koopman analysis of the long-term evolution in a turbulent convection cell. J. Fluid Mech. 847, 735–767 (2018).  https://doi.org/10.1017/jfm.2018.297 MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Giannakis, D., Majda, A.J.: Time series reconstruction via machine learning: Revealing Decadal variability and intermittency in the North Pacific sector of a coupled climate model. In: Conference on Intelligent Data Understanding Proceedings, Mountain View, California (2011)Google Scholar
  27. 27.
    Giannakis, D., Majda, A.J.: Nonlinear Laplacian Spectral Analysis for Time Series with Intermittency and Low-Frequency Variability. Proc. Natl. Acad. Sci. 109 (7), 2222–2227 (2012).  https://doi.org/10.1073/pnas.1118984109 MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Giannakis, D., Majda, A.J.: Nonlinear laplacian spectral analysis: capturing intermittent and low-frequency spatiotemporal patterns in high-dimensional data. Stat. Anal. Data Min. 6(3), 180–194 (2013).  https://doi.org/10.1002/sam.11171 MathSciNetCrossRefGoogle Scholar
  29. 29.
    Giannakis, D., Ourmazd, A., Slawinska, J., Zhao, Z.: Spatiotemporal pattern extraction by spectral analysis of vector-valued observables. J. Nonlinear Sci., 1–61 (2019).  https://doi.org/10.1007/s00332-019-09548-1 MathSciNetCrossRefGoogle Scholar
  30. 30.
    Giannakis, D., Slawinska, J.: Indo-pacific variability on seasonal to multidecadal time scales. Part II: Multiscale Atmosphere-Ocean Linkages. J. Climate 31, 693–725 (2018). https://doi.org/JCLI-D-17- 0031.1CrossRefGoogle Scholar
  31. 31.
    Giannakis, D., Slawinska, J., Ourmazd, A., Zhao, Z.: Vector-valued spectral analysis of space-time data. In: Proceedings of the Time Series Workshop, Neural Information Processing Systems Conference, Long Beach, California (2017)Google Scholar
  32. 32.
    Giannakis, D., Slawinska, J., Zhao, Z.: Spatiotemporal feature extraction with data-driven Koopman operators. J. Mach. Learn. Res. Proceedings 44, 103–115 (2015)Google Scholar
  33. 33.
    Giard, M.H., Perrin, F., Pernier, J., Bouchet, P.: Brain generators implicated in the processing of auditory stimulus deviance: A topographic event-related potential study. Psychophysiology 27(6), 627–640 (1990).  https://doi.org/10.1111/j.1469-8986.1990.tb03184.x CrossRefGoogle Scholar
  34. 34.
    Ghil, M., et al.: Advanced spectral methods for climatic time series. Rev. Geophys. 40(1), 1003 (2002).  https://doi.org/10.1029/2000RG000092 CrossRefGoogle Scholar
  35. 35.
    Grimm, S., Escera, C., Nelken, I.: Early indices of deviance detection in humans and animal models. Biol. Psychol. 116, 23–27 (2016).  https://doi.org/10.1016/j.biopsycho.2015.11.017 CrossRefGoogle Scholar
  36. 36.
    Gumenyuk, V., Roth, T., Korzyukov, O., Jefferson, C., Kick, A., Spear, L., Tepley, N., Drake, C.L.: Shift work sleep disorder is associated with an attenuated brain response of sensory memory and an increased brain response to novelty: An ERP study. Sleep 33(5), 703–713 (2010)CrossRefGoogle Scholar
  37. 37.
    Häenschel, C., Baldeweg, T., Croft, R.J., Whittington, M., Gruzelier, J.: Gamma and beta frequency oscillations in response to novel auditory stimuli: A comparison of human electroencephalogram (EEG) data with in vitro models. Proc. Natl. Acad. Sci. 97(13), 7645—7650 (2000).  https://doi.org/10.1073/pnas.120162397 CrossRefGoogle Scholar
  38. 38.
    Hamilton, L.S., Edwards, E., Chang, E.F.: Parallel streams define the temporal dynamics of speech processing across human auditory cortex. bioRxiv 097485 (2017).  https://doi.org/10.1101/097485
  39. 39.
    Harms, L., Michie, P.T., Näätänen, R.: Criteria for determining whether mismatch responses exist in animal models: Focus on rodents. Biol. Psychol. 116, 28–35 (2016).  https://doi.org/10.1016/j.biopsycho.2015.07.006 CrossRefGoogle Scholar
  40. 40.
    Javitt, D.C.: Intracortical Mechanisms of Mismatch Negativity Dysfunction in Schizophrenia. Audiol. Neurotol. 5, 207–215 (2000).  https://doi.org/10.1159/000013882 CrossRefGoogle Scholar
  41. 41.
    Javitt, D.C., Grochowski, S., Shelley, A.M., Ritter, W.: Impaired mismatch negativity (MMN) generation in schizophrenia as a function of stimulus deviance, probability, and interstimulus/interdeviant interval. Electroencephalogr. Clin. Neurophysiol. 108(2), 143–153 (1998).  https://doi.org/10.1016/s0168-5597(97)00073-7 CrossRefGoogle Scholar
  42. 42.
    Komatsu, M., Takaura, K., Fujii, N.: Mismatch negativity in common marmosets: Whole-cortical recordings with multi-channel electrocorticograms. Sci. Rep. 5, 1005 (2017).  https://doi.org/10.1038/srep15006 CrossRefGoogle Scholar
  43. 43.
    Koopman, B.O.: Hamiltonian systems and transformation in Hilbert space. Proc. Natl. Acad. Sci. 17(5), 315–318 (1931)CrossRefGoogle Scholar
  44. 44.
    Lee, C.M., Osman, A.F., Volgushev, M., Escabí, M. A., Read, H.L.: Neural spike-timing patterns vary with sound shape and periodicity in three auditory cortical fields. J Neurophysiol 115, 1886–1904 (2016).  https://doi.org/10.1152/jn.00784.2015 CrossRefGoogle Scholar
  45. 45.
    MacLean, S.E., Ward, L.M.: Temporo-frontal phase synchronization supports hierarchical network for mismatch negativity. Clin. Neurophysiol. 125(8), 1604–1617 (2014).  https://doi.org/10.1016/j.clinph.2013.12.109 CrossRefGoogle Scholar
  46. 46.
    May, P.J.C., Tiitinen, H.: Mismatch negativity (MMN): The deviance-elicited auditory deflection explained. Psychophysiology 47(1), 66–122 (2010).  https://doi.org/10.1111/j.1469-8986.2009.00856.x CrossRefGoogle Scholar
  47. 47.
    Marrouch, N., Read, H.L., Slawinska, J., Giannakis, D.: Data-driven spectral decomposition of ECoG signal from an auditory oddball experiment in a marmoset monkey: implications for EEG data in humans. International Joint Conference on Neural Networks, 2161–4407 (2018)Google Scholar
  48. 48.
    Meindertsma, T., Kloosterman, N.A., Engel, A.K., Wagenmakers, E.J., Donner, T.H.: Surprise About Sensory Event Timing Drives Cortical Transients in the Beta Frequency Band. J. Neurosci. 38(35), 7600–7610 (2018).  https://doi.org/10.1523/JNEUROSCI.0307-18.2018 CrossRefGoogle Scholar
  49. 49.
    Mezić, I.: Spectral properties of dynamical systems, model reduction and decompositions. Nonlinear Dyn. 41, 309–325 (2005).  https://doi.org/10.1007/s11071-005-2824-x MathSciNetCrossRefzbMATHGoogle Scholar
  50. 50.
    Michie, P.T., Malmierca, M.S., Harms, L., Todd, J.: Understanding the neurobiology of MMN and its reduction in schizophrenia. Biol. Psychol. 116, 1–3 (2016).  https://doi.org/10.1016/j.biopsycho.2016.02.005 CrossRefGoogle Scholar
  51. 51.
    Näätänen, R., Paavilainen, P., Alho, K., Reinikainen, K., Sams, M.: The mismatch negativity to intensity changes in an auditory stimulus sequence. Electroencephalogr. Clin. Neurophysiol. Suppl. 40, 125–131 (1987)Google Scholar
  52. 52.
    Näätänen, R., Paavilainen, P., Reinikainen, K.: Do event-related potentials to infrequent decrements in duration of auditory stimuli demonstrate a memory trace in man? Neurosci. Lett. 107(1-3), 347–352 (1989).  https://doi.org/10.1016/0304-3940(89)90844-6 CrossRefGoogle Scholar
  53. 53.
    Näätänen, R., Todd, J., Schall, U.: Mismatch negativity (MMN) as biomarker predicting psychosis in clinically at-risk individuals. Biol. Psychol. 116, 36–40 (2016).  https://doi.org/10.1016/j.biopsycho.2015.10.010 CrossRefGoogle Scholar
  54. 54.
    Rodgers, C.C., DeWeese, M.R.: Neural correlates of task switching in prefrontal cortex and primary auditory cortex in a novel stimulus selection task for rodents. Neuron 82(5), 1157–1170 (2014).  https://doi.org/10.1016/j.neuron.2014.04.031 CrossRefGoogle Scholar
  55. 55.
    Ross, B., Schneider, B., Snyder, J.S., Alain, C.: Biological markers of auditory gap detection in young middle-aged and older adults. PLoS ONE 5(4), e10101 (2010).  https://doi.org/10.1371/journal.pone.0010101 CrossRefGoogle Scholar
  56. 56.
    Schall, U.: Is it time to move mismatch negativity into the clinic? Biol. Psychol. 116, 41–46 (2016).  https://doi.org/10.1016/j.biopsycho.2015.09.001 CrossRefGoogle Scholar
  57. 57.
    Slawinska, J., Ourmazd, A., Giannakis, D.: A quantum mechanical approach for data assimilation in climate dynamics. In: Workshop on “Climate Change: How can AI help?”, 36th International Conference on Machine Learning (ICML), Long Beach, California (2019). https://www.climatechange.ai/CameraReady/30/CameraReadySubmission/manuscript.pdf
  58. 58.
    Slawinska, J., Ourmazd, A., Giannakis, D.: A new approach to signal processing of spatiotemporal data. In: 2018 IEEE statistical signal processing workshop (SSP), pp. 338–342.  https://doi.org/10.1109/SSP.2018.8450704 (2018)
  59. 59.
    Slawinska, J., Giannakis, D.: Indo-pacific variability on seasonal to multidecadal time scales. Part I: Intrinsic SST Modes in Models and Observations. J. Climate 30 (14), 5265–5294 (2017).  https://doi.org/10.1175/jcli-d-16-0176.1 CrossRefGoogle Scholar
  60. 60.
    von Stein, A., Rappelsberger, P., Sarnthein, J., Petsche, H.: Synchronization between temporal and parietal cortex during multimodal object processing in man. Cereb. Cortex 9, 137—150 (1999).  https://doi.org/10.1093/cercor/9.2.137 CrossRefGoogle Scholar
  61. 61.
    Tia, B., Takemi, M., Kosugi, A., Castagnola, E., Ansaldo, A., Nakamura, T., Ricci, D., Ushiba, J., Fadiga, L., Iriki, A.: Cortical control of object-specific grasp relies on adjustments of both activity and effective connectivity: A common marmoset study. J. Physiol. 595(6), 7203–7221 (2017).  https://doi.org/10.1113/JP274629 CrossRefGoogle Scholar
  62. 62.
    Wacongne, C.: A predictive coding account of MMN reduction in schizophrenia. Biol. Psychol. 116, 68–74 (2016).  https://doi.org/10.1016/j.biopsycho.2015.10.011 CrossRefGoogle Scholar
  63. 63.
    Widmann, A., Schröger, E.: Filter effects and filter artifacts in the analysis of electrophysiological data. Frontiers in psychology 3, 233 (2012).  https://doi.org/10.3389/fpsyg.2012.00233 CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Department of Psychological SciencesUniversity of ConnecticutStorrsUSA
  2. 2.Department of PhysicsUniversity of Wisconsin-MilwaukeeMilwaukeeUSA
  3. 3.Courant Institute of Mathematical SciencesNew York UniversityNew YorkUSA
  4. 4.Department of Psychological Sciences/Department of Biomedical EngineeringUniversity of ConnecticutStorrsUSA

Personalised recommendations