Machine Learning

, Volume 79, Issue 1–2, pp 5–27 | Cite as

Temporal kernel CCA and its application in multimodal neuronal data analysis

  • Felix BießmannEmail author
  • Frank C. Meinecke
  • Arthur Gretton
  • Alexander Rauch
  • Gregor Rainer
  • Nikos K. Logothetis
  • Klaus-Robert Müller
Open Access


Data recorded from multiple sources sometimes exhibit non-instantaneous couplings. For simple data sets, cross-correlograms may reveal the coupling dynamics. But when dealing with high-dimensional multivariate data there is no such measure as the cross-correlogram. We propose a simple algorithm based on Kernel Canonical Correlation Analysis (kCCA) that computes a multivariate temporal filter which links one data modality to another one. The filters can be used to compute a multivariate extension of the cross-correlogram, the canonical correlogram, between data sources that have different dimensionalities and temporal resolutions. The canonical correlogram reflects the coupling dynamics between the two sources. The temporal filter reveals which features in the data give rise to these couplings and when they do so. We present results from simulations and neuroscientific experiments showing that tkCCA yields easily interpretable temporal filters and correlograms. In the experiments, we simultaneously performed electrode recordings and functional magnetic resonance imaging (fMRI) in primary visual cortex of the non-human primate. While electrode recordings reflect brain activity directly, fMRI provides only an indirect view of neural activity via the Blood Oxygen Level Dependent (BOLD) response. Thus it is crucial for our understanding and the interpretation of fMRI signals in general to relate them to direct measures of neural activity acquired with electrodes. The results computed by tkCCA confirm recent models of the hemodynamic response to neural activity and allow for a more detailed analysis of neurovascular coupling dynamics.

Canonical correlation analysis CCA kCCA tkCCA Neurovascular coupling 


  1. Akaho, S. (2001). A kernel method for canonical correlation analysis. In Proceedings of the international meeting of the psychometric society (IMPS). Berlin: Springer. Google Scholar
  2. Anderson, T. W. (1958). An introduction to multivariate statistical analysis. New York: Wiley. zbMATHGoogle Scholar
  3. Bach, F. R., & Jordan, M. I. (2002). Kernel independent component analysis. Journal of Machine Learning Research. Google Scholar
  4. Bach, F. R., & Jordan, M. I. (2006). A probabilistic interpretation of canonical correlation analysis. Technical Report. Google Scholar
  5. Belitski, A., Gretton, A., Magri, C., Murayama, Y., Montemurro, M. A., Logothetis, N. K., & Panzeri, S. (2008). Low-frequency local field potentials and spikes in primary visual cortex convey independent visual information. Journal of Neuroscience. Google Scholar
  6. Blaschko, M. B., Lampert, C. H., & Gretton, A. (2008). Semi-supervised Laplacian regularization of kernel canonical correlation analysis. In W. Daelemans, B. Goethals, & K. Morik (Eds.), 19th European conference on machine learning. Antwerpen: Springer. Google Scholar
  7. Buxton, R. B., Uludag, K., Dubowitz, D. J., & Liu, T. T. (2004). Modeling the hemodynamic response to brain activation. Neuroimage. Google Scholar
  8. Friman, O., Borga, M., Lundberg, P., & Knutsson, H. (2002). Exploratory fMRI analysis by autocorrelation maximization. Neuroimage. Google Scholar
  9. Friston, K. J., Mechelli, A., Turner, R., & Price, C. J. (2000). Nonlinear responses in fMRI: the Balloon model, Volterra kernels, and other hemodynamics. Neuroimage. Google Scholar
  10. Fukumizu, K., Bach, F. R., & Gretton, A. (2007). Statistical consistency of kernel CCA. Journal of Machine Learning Research. Google Scholar
  11. Goense, J. B. M., & Logothetis, N. K. (2008). Neurophysiology of the BOLD fMRI signal in awake monkeys. Current Biology. Google Scholar
  12. Hardoon, D. R., Szedmak, S., & Shawe-Taylor, J. (2004). Canonical correlation analysis: an overview with application to learning methods. Neural Computation. Google Scholar
  13. Hardoon, D. R., Mourao-Miranda, J., Brammer, M., & Shawe-Taylor, J. (2007). Unsupervised analysis of fMRI data using kernel canonical correlation. Neuroimage. Google Scholar
  14. Harris, F. J. (1978). On then use of windows for harmonic analysis with the discrete Fourier transform. Proceedings of the IEEE. Google Scholar
  15. Haynes, J. D., Sakai, K., Rees, G., Gilbert, S., Frith, C., & Passingham, R. E. (2007). Reading hidden intentions in the human brain. Current Biology. Google Scholar
  16. Hotelling, H. (1936). Relations between two sets of variates. Biometrika. Google Scholar
  17. Langleben, D. D., Loughead, J. W., Bilker, W. B., Ruparel, K., Childress, A. R., Busch, S. I., & Gur, R. C. (2005). Telling truth from lie in individual subjects with fast event-related fMRI. Human Brain Mapping. Google Scholar
  18. Logothetis, N. K. (2008). What we can do and what we cannot do with fMRI. Nature. Google Scholar
  19. Logothetis, N. K., & Wandell, B. A. (2004). Interpreting the bold signal. Annual Reviews of Physiology. Google Scholar
  20. Logothetis, N. K., Guggenberger, H., Peled, S., & Pauls, J. (1999). Functional imaging of the monkey brain. Nature Neuroscience. Google Scholar
  21. Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., & Öltermann, A. (2001). Neurophysiological investigation of the basis of the fMRI signal. Nature. Google Scholar
  22. Logothetis, N. K., Merkle, H., Augath, M., Trinath, T., & Ugurbil, K. (2002). Ultra high-resolution fMRI in monkeys with implanted RF coils. Neuron. Google Scholar
  23. Macke, J. H., Zeck, G., & Bethge, M. (2008). Receptive fields without spike-triggering. In J. C. Platt, D. Koller, Y. Singer, & S. Roweis (Eds.), 21th neural information processing systems conference. Cambridge: MIT Press. Google Scholar
  24. Müller, K. R., Mika, S., Rätsch, G., Tsuda, K., & Schölkopf, B. (2001). An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks. Google Scholar
  25. Norman, K. A., Polyn, S. M., Detre, G. J., & Haxby, J. V. (2006). Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends in Cognitive Sciences. Google Scholar
  26. Öltermann, A., Augath, M. A., & Logothetis, N. K. (2007). Simultaneous recording of neuronal signals and functional NMR imaging. Magnetic Resonance Imaging. Google Scholar
  27. Ogawa, S., Lee, T. M.,  Nayak, A. S., & Glynn, P. (1990). Oxygenation-sensitive contrast in magnetic resonance image of rodent brain at high magnetic fields. Magnetic Resonance in Medicine. Google Scholar
  28. Kettenring, J. R. (1971). Canonical analysis of several sets of variables. Biometrika. Google Scholar
  29. Rauch, A., Rainer, G., Augath, M., Öltermann, A., & Logothetis, N. K. (2008a). Pharmacological MRI combined with electrophysiology in non-human primates: effects of lidocaine on primary visual cortex. Neuroimage. Google Scholar
  30. Rauch, A., Rainer, G., & Logothetis, N. K. (2008b). The effect of a serotonin-induced dissociation between spiking and perisynaptic activity on bold functional MRI. Proceedings of the National Academy of Sciences. Google Scholar
  31. Schölkopf, B., & Smola, A. J. (2001). Learning with kernels: support vector machines, regularization, optimization, and beyond. Adaptive computation and machine learning. Cambridge: MIT Press. Google Scholar
  32. Shawe-Taylor, J., & Cristianini, N. (2004). Kernel methods for pattern analysis. Cambridge: Cambridge University Press. Google Scholar
  33. SPM5. (2005). Statistical parametrical mapping toolbox.

Copyright information

© The Author(s) 2009

Open AccessThis is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (, which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Authors and Affiliations

  • Felix Bießmann
    • 1
    Email author
  • Frank C. Meinecke
    • 1
  • Arthur Gretton
    • 2
  • Alexander Rauch
    • 3
  • Gregor Rainer
    • 4
  • Nikos K. Logothetis
    • 3
  • Klaus-Robert Müller
    • 1
  1. 1.Machine Learning GroupTU BerlinBerlinGermany
  2. 2.Dept. Empirical InferenceMPI Biological CyberneticsTübingenGermany
  3. 3.Dept. Physiology of Cognitive ProcessesMPI Biological CyberneticsTübingenGermany
  4. 4.Visual Cognition LaboratoryUniversity of FribourgFribourgSwitzerland

Personalised recommendations