Improving SNR and Reducing Training Time of Classifiers in Large Datasets via Kernel Averaging

  • Matthias S. TrederEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11309)


Kernel methods are of growing importance in neuroscience research. As an elegant extension of linear methods, they are able to model complex non-linear relationships. However, since the kernel matrix grows with data size, the training of classifiers is computationally demanding in large datasets. Here, a technique developed for linear classifiers is extended to kernel methods: In linearly separable data, replacing sets of instances by their averages improves signal-to-noise ratio (SNR) and reduces data size. In kernel methods, data is linearly non-separable in input space, but linearly separable in the high-dimensional feature space that kernel methods implicitly operate in. It is shown that a classifier can be efficiently trained on instances averaged in feature space by averaging entries in the kernel matrix. Using artificial and publicly available data, it is shown that kernel averaging improves classification performance substantially and reduces training time, even in non-linearly separable data.


Kernel Machine learning Big data SVM FDA 


  1. 1.
    Ayres-de Campos, D., Bernardes, J., Garrido, A., Marques-de Sá, J., Pereira-Leite, L.: Sisporto 2.0: a program for automated analysis of cardiotocograms. J. Matern. Fetal Med. 9(5), 311–318 (2000).<311::AID-MFM12>3.0.CO;2-9Google Scholar
  2. 2.
    Chang, C.C, Lin, C.J.: LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 1–27 (2011)CrossRefGoogle Scholar
  3. 3.
    Choudhury, S., Fishman, J.R., McGowan, M.L., Juengst, E.T.: Big data, open science and the brain: lessons learned from genomics. Front. Hum. Neurosci. 8, 239 (2014). Scholar
  4. 4.
    Cichy, R.M., Pantazis, D.: Multivariate pattern analysis of MEG and EEG: a comparison of representational structure in time and space. NeuroImage 158, 441–454 (2017). Scholar
  5. 5.
    Cichy, R.M., Ramirez, F.M., Pantazis, D.: Can visual information encoded in cortical columns be decoded from magnetoencephalography data in humans? NeuroImage 121, 193–204 (2015). Scholar
  6. 6.
    Danziger, S.A., et al.: Predicting positive p53 cancer rescue regions using most informative positive (MIP) active learning. PLoS Comput. Biol. 5(9), e1000498 (2009). Scholar
  7. 7.
    Dima, D.C., Perry, G., Singh, K.D.: Spatial frequency supports the emergence of categorical representations in visual cortex during natural scene perception. NeuroImage 179, 102–116 (2018). Scholar
  8. 8.
    Gonzalez-Moreno, A., et al.: Signal-to-noise ratio of the MEG signal after preprocessing. J. Neurosci. Methods 222, 56–61 (2014). Scholar
  9. 9.
    Hainmueller, J., Hazlett, C., Alvarez, R.M.: Kernel regularized least squares: reducing misspecification bias with a flexible and interpretable machine learning approach. Polit. Anal. 22(2), 143–168 (2014). Scholar
  10. 10.
    Hinton, G.E.: Machine learning for neuroscience. Neural Syst. Circ. 1(1), 12 (2011). Scholar
  11. 11.
    Hwang, H.J., et al.: A gaze independent brain-computer interface based on visual stimulation through closed eyelids. Sci. Rep. 5, 15890 (2015). Scholar
  12. 12.
    Jäkel, F., Schölkopf, B., Wichmann, F.A.: Does cognitive science need kernels? Trends Cogn. Sci. 13, 381–388 (2009). Scholar
  13. 13.
    Orrù, G., Pettersson-Yeo, W., Marquand, A.F., Sartori, G., Mechelli, A.: Using support vector machine to identify imaging biomarkers of neurological and psychiatric disease: a critical review. Neurosci. Biobehav. Rev. 36(4), 1140–1152 (2012). Scholar
  14. 14.
    Schölkopf, B., Smola, A.J.: A short introduction to learning with kernels. In: Mendelson, S., Smola, A.J. (eds.) Advanced Lectures on Machine Learning. Lecture Notes in Computer Science, vol. 2600, pp. 41–64. Springer, Heidelberg (2003). Scholar
  15. 15.
    Schrouff, J., et al.: PRoNTo: pattern recognition for neuroimaging toolbox. Neuroinformatics 11(3), 319–337 (2013). Scholar
  16. 16.
    Schrouff, J., Mourão-Miranda, J., Phillips, C., Parvizi, J.: Decoding intracranial EEG data with multiple kernel learning method. J. Neurosci. Methods 261, 19–28 (2016). Scholar
  17. 17.
    Treder, M.S., Purwins, H., Miklody, D., Sturm, I., Blankertz, B.: Decoding auditory attention to instruments in polyphonic music using single-trial EEG classification. J. Neural Eng. 11(2), 026009 (2014). Scholar
  18. 18.
    Wang, X., Xing, E.P., Schaid, D.J.: Kernel methods for large-scale genomic data analysis. Brief. Bioinf. 16(2), 183–192 (2015). Scholar
  19. 19.
    Weinstein, J.N., et al.: The cancer genome atlas pan-cancer analysis project. Nat. Genet. 45(10), 1113–1120 (2013). Scholar
  20. 20.
    Youssofzadeh, V., McGuinness, B., Maguire, L.P., Wong-Lin, K.: Multi-kernel learning with dartel improves combined MRI-PET classification of Alzheimer’s disease in AIBL data: group and individual analyses. Front. Hum. Neurosci. 11, 380 (2017). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer Science and InformaticsCardiff UniversityCardiffUK

Personalised recommendations