Data-driven Koopman operator approach for computational neuroscience

  • Natasza MarrouchEmail author
  • Joanna Slawinska
  • Dimitrios Giannakis
  • Heather L. Read
Open Access


This article presents a novel, nonlinear, data-driven signal processing method, which can help neuroscience researchers visualize and understand complex dynamical patterns in both time and space. Specifically, we present applications of a Koopman operator approach for eigendecomposition of electrophysiological signals into orthogonal, coherent components and examine their associated spatiotemporal dynamics. This approach thus provides enhanced capabilities over conventional computational neuroscience tools restricted to analyzing signals in either the time or space domains. This is achieved via machine learning and kernel methods for data-driven approximation of skew-product dynamical systems. The approximations successfully converge to theoretical values in the limit of long embedding windows. First, we describe the method, then using electrocorticographic (ECoG) data from a mismatch negativity experiment, we extract time-separable frequencies without bandpass filtering or prior selection of wavelet features. Finally, we discuss in detail two of the extracted components, Beta (\( \sim \) 13 Hz) and high Gamma (\( \sim \) 50 Hz) frequencies, and explore the spatiotemporal dynamics of high- and low- frequency components.


Koopman operator Spectral decomposition Nonlinear Spatiotemporal dynamics ECoG Brain Mismatch negativity 

Mathematics Subject Classification (2010)

37M10 37M25 58C40 30C40 37N25 47A35 92C55 



The authors would like to thank Ian Stevenson, Stephen Herzog, and the journal editorial team and reviewers for their helpful comments. Misako Komatsu, Kana Takura, and Naotaka Fujii from the RIKEN Brain Science Institute generously provided the open-access data used in this article. HLR has ownership interest in Elemind Technologies, Inc.


  1. 1.
    Alho, K.: Cerebral generators of mismatch negativity (MMN) and its magnetic counterpart (MMNm) elicited by sound changes. Ear Hear. 16(1), 38–51 (1995)CrossRefGoogle Scholar
  2. 2.
    Alho, K., Woods, D.L., Algazi, A., Knight, R.T., Näätänen, R.: Lesions of frontal cortex diminish the auditory mismatch negativity. Electroencephalogr. Clin. Neurophysiol. 91(5), 353–362 (1994). CrossRefGoogle Scholar
  3. 3.
    Aru, J., Aru, J., Priesemann, V., Wibral, M., Lana, L., Pipa, G., Singer, W., Vicente, R.: Untangling cross-frequency coupling in neuroscience. Curr. Opin. Neurobiol. 31, 51–61 (2015). CrossRefGoogle Scholar
  4. 4.
    Aubry, N., Guyonnet, R., Lima, R.: Spatiotemporal analysis of complex signals: Theory and applications. J. Stat. Phys. 64, 683–739 (1991). MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Belkin, M., Niyogi, P.: Laplacian Eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 1373–1396 (2003). CrossRefzbMATHGoogle Scholar
  6. 6.
    Brunton, B.W., Johnson, L.A., Ojemann, J.G., Kutz, J.N.: Extracting spatial–temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition. J. Neurosci. Methods 258, 1–15 (2016). CrossRefGoogle Scholar
  7. 7.
    Budisić, M, Mohr, R, Mezić, I.: Applied Koopmanism. Chaos 22, 047510 (2012). MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Chandrasekaran, C., Turesson, H.K., Brown, C.H., Ghazanfar, A.A.: The Influence of Natural Scene Dynamics on Auditory Cortical Activity. J. Neurosci. 30(42), 13919–13931 (2010). CrossRefGoogle Scholar
  9. 9.
    Coenen, A., Fine, E., Zayachkivska, O.A.: Beck: A forgotten pioneer in electroencephalography. Journal of the History of the Neurosciences: Basic and Clinical Perspectives 23(3), 276–286 (2014)CrossRefGoogle Scholar
  10. 10.
    Coifman, R.R., Lafon, S.: Diffusion maps. Appl. Comput. Harmon. Anal. 21, 5–30 (2006). MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Cong, F., Kalyakin, I., Li, H., Huttunen-Scott, T., Huang, Y., Lyytinen, H., Ristaniemi, T.: Answering six questions in extracting children’s mismatch negativity through combining wavelet decomposition and independent component analysis. Cogn. Neurodyn. 5(4), 343–359 (2011). CrossRefGoogle Scholar
  12. 12.
    Cong, F., Sipola, T., Huttunen-Scott, T., Xu, X., Ristaniemi, T., Lyytinen, H.: Hilbert-Huang versus Morlet wavelet transformation on mismatch negativity of children in uninterrupted sound paradigm. Nonlinear Biomedical Physics 3(1) (2009).
  13. 13.
    Corlett, P.R., Marrouch, N.: Social cognitive neuroscience of attitudes and beliefs. In: Albarracín, D., Johnson, B. T. (eds.) Handbook of Attitudes and Attitude Change, vol. 1, pp 480–519. Taylor & Francis, New York (2018)Google Scholar
  14. 14.
    Csépe, V., Karmos, G., Molnár, M.: Evoked potential correlates of stimulus deviance during wakefulness and sleep in cat — animal model of mismatch negativity. Electroencephalogr. Clin. Neurophysiol. 66, 571–578 (1987). CrossRefGoogle Scholar
  15. 15.
    Das, S., Giannakis, D.: Delay-coordinate maps and the spectra of Koopman operators. J. Stati. Phy. 14 (6), 1107–1145 (2019). MathSciNetCrossRefGoogle Scholar
  16. 16.
    Dellnitz, M., Junge, O.: On the Approximation of Complicated Dynamical Behavior. SIAM J. Numer. Anal. 36, 491 (1999). MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    van Drongelen, W.: Signal processing for neuroscientists. Elsevier, Amsterdam (2007)Google Scholar
  18. 18.
    Dürschmid, S., Edwards, E., Reichert, C., Dewar, C., Hinrichs, H., Heinze, H. -J., Kirsch, H.E., Dalal, S.S., Deouell, L.Y., Knight, R.T.: Hierarchy of prediction errors for auditory events in human temporal and frontal cortex. Proc. Natl. Acad. Sci. 113, 6755–6760 (2016). CrossRefGoogle Scholar
  19. 19.
    Eisner, T., Farkas, B., Haase, M., Nagel, R.: Operator Theoretic Aspects of Ergodic Theory. Graduate Texts in Mathematics, vol. 272. Springer, Berlin (2015)CrossRefGoogle Scholar
  20. 20.
    Escabí, M. A., Read, H.L., Viventi, J., Kim, D -H, Higgins, N.C., Storace, D.A., Liu, A.S.K., Gifford, A.M., Burke, J.A., Campisi, M., Kim, Y -S, Avrin, A.E., Van der Spiegel, J., Huang, Y., Li, M., Wu, J., Rogers, J.A., Litt, B., Cohen, Y.E.: A high-density, high-channel count, multiplexed μ ECoG array for auditory cortex. J Neurophysiol 112, 1566–1583 (2014). CrossRefGoogle Scholar
  21. 21.
    Ford, J.M., Hillpard, S.A.: Event–related potentials (ERPs) to Interruptions of a Steady Rhythm. Psychophysiology 18(3), 322–330 (1981). CrossRefGoogle Scholar
  22. 22.
    Giannakis, D.: Dynamics-adapted cone kernels. SIAM J. Appl. Dyn. Sys. 14(2), 556–608 (2015). MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Giannakis, D.: Data-driven spectral decomposition and forecasting of ergodic dynamical systems. Appl. Comut. Harmon. Anal 47(2), 338–396 (2019). MathSciNetCrossRefGoogle Scholar
  24. 24.
    Giannakis, D., Das, S.: Extraction and prediction of coherent patterns in incompressible flows through space-time Koopman analysis. Phys. D. In revision. arXiv:1706.06450 (2017)
  25. 25.
    Giannakis, D., Kolchinskaya, A., Krasnov, D., Schumacher, J.: Koopman analysis of the long-term evolution in a turbulent convection cell. J. Fluid Mech. 847, 735–767 (2018). MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Giannakis, D., Majda, A.J.: Time series reconstruction via machine learning: Revealing Decadal variability and intermittency in the North Pacific sector of a coupled climate model. In: Conference on Intelligent Data Understanding Proceedings, Mountain View, California (2011)Google Scholar
  27. 27.
    Giannakis, D., Majda, A.J.: Nonlinear Laplacian Spectral Analysis for Time Series with Intermittency and Low-Frequency Variability. Proc. Natl. Acad. Sci. 109 (7), 2222–2227 (2012). MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Giannakis, D., Majda, A.J.: Nonlinear laplacian spectral analysis: capturing intermittent and low-frequency spatiotemporal patterns in high-dimensional data. Stat. Anal. Data Min. 6(3), 180–194 (2013). MathSciNetCrossRefGoogle Scholar
  29. 29.
    Giannakis, D., Ourmazd, A., Slawinska, J., Zhao, Z.: Spatiotemporal pattern extraction by spectral analysis of vector-valued observables. J. Nonlinear Sci., 1–61 (2019). MathSciNetCrossRefGoogle Scholar
  30. 30.
    Giannakis, D., Slawinska, J.: Indo-pacific variability on seasonal to multidecadal time scales. Part II: Multiscale Atmosphere-Ocean Linkages. J. Climate 31, 693–725 (2018). 0031.1CrossRefGoogle Scholar
  31. 31.
    Giannakis, D., Slawinska, J., Ourmazd, A., Zhao, Z.: Vector-valued spectral analysis of space-time data. In: Proceedings of the Time Series Workshop, Neural Information Processing Systems Conference, Long Beach, California (2017)Google Scholar
  32. 32.
    Giannakis, D., Slawinska, J., Zhao, Z.: Spatiotemporal feature extraction with data-driven Koopman operators. J. Mach. Learn. Res. Proceedings 44, 103–115 (2015)Google Scholar
  33. 33.
    Giard, M.H., Perrin, F., Pernier, J., Bouchet, P.: Brain generators implicated in the processing of auditory stimulus deviance: A topographic event-related potential study. Psychophysiology 27(6), 627–640 (1990). CrossRefGoogle Scholar
  34. 34.
    Ghil, M., et al.: Advanced spectral methods for climatic time series. Rev. Geophys. 40(1), 1003 (2002). CrossRefGoogle Scholar
  35. 35.
    Grimm, S., Escera, C., Nelken, I.: Early indices of deviance detection in humans and animal models. Biol. Psychol. 116, 23–27 (2016). CrossRefGoogle Scholar
  36. 36.
    Gumenyuk, V., Roth, T., Korzyukov, O., Jefferson, C., Kick, A., Spear, L., Tepley, N., Drake, C.L.: Shift work sleep disorder is associated with an attenuated brain response of sensory memory and an increased brain response to novelty: An ERP study. Sleep 33(5), 703–713 (2010)CrossRefGoogle Scholar
  37. 37.
    Häenschel, C., Baldeweg, T., Croft, R.J., Whittington, M., Gruzelier, J.: Gamma and beta frequency oscillations in response to novel auditory stimuli: A comparison of human electroencephalogram (EEG) data with in vitro models. Proc. Natl. Acad. Sci. 97(13), 7645—7650 (2000). CrossRefGoogle Scholar
  38. 38.
    Hamilton, L.S., Edwards, E., Chang, E.F.: Parallel streams define the temporal dynamics of speech processing across human auditory cortex. bioRxiv 097485 (2017).
  39. 39.
    Harms, L., Michie, P.T., Näätänen, R.: Criteria for determining whether mismatch responses exist in animal models: Focus on rodents. Biol. Psychol. 116, 28–35 (2016). CrossRefGoogle Scholar
  40. 40.
    Javitt, D.C.: Intracortical Mechanisms of Mismatch Negativity Dysfunction in Schizophrenia. Audiol. Neurotol. 5, 207–215 (2000). CrossRefGoogle Scholar
  41. 41.
    Javitt, D.C., Grochowski, S., Shelley, A.M., Ritter, W.: Impaired mismatch negativity (MMN) generation in schizophrenia as a function of stimulus deviance, probability, and interstimulus/interdeviant interval. Electroencephalogr. Clin. Neurophysiol. 108(2), 143–153 (1998). CrossRefGoogle Scholar
  42. 42.
    Komatsu, M., Takaura, K., Fujii, N.: Mismatch negativity in common marmosets: Whole-cortical recordings with multi-channel electrocorticograms. Sci. Rep. 5, 1005 (2017). CrossRefGoogle Scholar
  43. 43.
    Koopman, B.O.: Hamiltonian systems and transformation in Hilbert space. Proc. Natl. Acad. Sci. 17(5), 315–318 (1931)CrossRefGoogle Scholar
  44. 44.
    Lee, C.M., Osman, A.F., Volgushev, M., Escabí, M. A., Read, H.L.: Neural spike-timing patterns vary with sound shape and periodicity in three auditory cortical fields. J Neurophysiol 115, 1886–1904 (2016). CrossRefGoogle Scholar
  45. 45.
    MacLean, S.E., Ward, L.M.: Temporo-frontal phase synchronization supports hierarchical network for mismatch negativity. Clin. Neurophysiol. 125(8), 1604–1617 (2014). CrossRefGoogle Scholar
  46. 46.
    May, P.J.C., Tiitinen, H.: Mismatch negativity (MMN): The deviance-elicited auditory deflection explained. Psychophysiology 47(1), 66–122 (2010). CrossRefGoogle Scholar
  47. 47.
    Marrouch, N., Read, H.L., Slawinska, J., Giannakis, D.: Data-driven spectral decomposition of ECoG signal from an auditory oddball experiment in a marmoset monkey: implications for EEG data in humans. International Joint Conference on Neural Networks, 2161–4407 (2018)Google Scholar
  48. 48.
    Meindertsma, T., Kloosterman, N.A., Engel, A.K., Wagenmakers, E.J., Donner, T.H.: Surprise About Sensory Event Timing Drives Cortical Transients in the Beta Frequency Band. J. Neurosci. 38(35), 7600–7610 (2018). CrossRefGoogle Scholar
  49. 49.
    Mezić, I.: Spectral properties of dynamical systems, model reduction and decompositions. Nonlinear Dyn. 41, 309–325 (2005). MathSciNetCrossRefzbMATHGoogle Scholar
  50. 50.
    Michie, P.T., Malmierca, M.S., Harms, L., Todd, J.: Understanding the neurobiology of MMN and its reduction in schizophrenia. Biol. Psychol. 116, 1–3 (2016). CrossRefGoogle Scholar
  51. 51.
    Näätänen, R., Paavilainen, P., Alho, K., Reinikainen, K., Sams, M.: The mismatch negativity to intensity changes in an auditory stimulus sequence. Electroencephalogr. Clin. Neurophysiol. Suppl. 40, 125–131 (1987)Google Scholar
  52. 52.
    Näätänen, R., Paavilainen, P., Reinikainen, K.: Do event-related potentials to infrequent decrements in duration of auditory stimuli demonstrate a memory trace in man? Neurosci. Lett. 107(1-3), 347–352 (1989). CrossRefGoogle Scholar
  53. 53.
    Näätänen, R., Todd, J., Schall, U.: Mismatch negativity (MMN) as biomarker predicting psychosis in clinically at-risk individuals. Biol. Psychol. 116, 36–40 (2016). CrossRefGoogle Scholar
  54. 54.
    Rodgers, C.C., DeWeese, M.R.: Neural correlates of task switching in prefrontal cortex and primary auditory cortex in a novel stimulus selection task for rodents. Neuron 82(5), 1157–1170 (2014). CrossRefGoogle Scholar
  55. 55.
    Ross, B., Schneider, B., Snyder, J.S., Alain, C.: Biological markers of auditory gap detection in young middle-aged and older adults. PLoS ONE 5(4), e10101 (2010). CrossRefGoogle Scholar
  56. 56.
    Schall, U.: Is it time to move mismatch negativity into the clinic? Biol. Psychol. 116, 41–46 (2016). CrossRefGoogle Scholar
  57. 57.
    Slawinska, J., Ourmazd, A., Giannakis, D.: A quantum mechanical approach for data assimilation in climate dynamics. In: Workshop on “Climate Change: How can AI help?”, 36th International Conference on Machine Learning (ICML), Long Beach, California (2019).
  58. 58.
    Slawinska, J., Ourmazd, A., Giannakis, D.: A new approach to signal processing of spatiotemporal data. In: 2018 IEEE statistical signal processing workshop (SSP), pp. 338–342. (2018)
  59. 59.
    Slawinska, J., Giannakis, D.: Indo-pacific variability on seasonal to multidecadal time scales. Part I: Intrinsic SST Modes in Models and Observations. J. Climate 30 (14), 5265–5294 (2017). CrossRefGoogle Scholar
  60. 60.
    von Stein, A., Rappelsberger, P., Sarnthein, J., Petsche, H.: Synchronization between temporal and parietal cortex during multimodal object processing in man. Cereb. Cortex 9, 137—150 (1999). CrossRefGoogle Scholar
  61. 61.
    Tia, B., Takemi, M., Kosugi, A., Castagnola, E., Ansaldo, A., Nakamura, T., Ricci, D., Ushiba, J., Fadiga, L., Iriki, A.: Cortical control of object-specific grasp relies on adjustments of both activity and effective connectivity: A common marmoset study. J. Physiol. 595(6), 7203–7221 (2017). CrossRefGoogle Scholar
  62. 62.
    Wacongne, C.: A predictive coding account of MMN reduction in schizophrenia. Biol. Psychol. 116, 68–74 (2016). CrossRefGoogle Scholar
  63. 63.
    Widmann, A., Schröger, E.: Filter effects and filter artifacts in the analysis of electrophysiological data. Frontiers in psychology 3, 233 (2012). CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Department of Psychological SciencesUniversity of ConnecticutStorrsUSA
  2. 2.Department of PhysicsUniversity of Wisconsin-MilwaukeeMilwaukeeUSA
  3. 3.Courant Institute of Mathematical SciencesNew York UniversityNew YorkUSA
  4. 4.Department of Psychological Sciences/Department of Biomedical EngineeringUniversity of ConnecticutStorrsUSA

Personalised recommendations