Skip to main content

Characterizing Regularization Techniques for Spatial Filter Optimization in Oscillatory EEG Regression Problems

Guidelines Derived from Simulation and Real-World Data

Abstract

We report on novel supervised algorithms for single-trial brain state decoding. Their reliability and robustness are essential to efficiently perform neurotechnological applications in closed-loop. When brain activity is assessed by multichannel recordings, spatial filters computed by the source power comodulation (SPoC) algorithm allow identifying oscillatory subspaces. They regress to a known continuous trial-wise variable reflecting, e.g. stimulus characteristics, cognitive processing or behavior. In small dataset scenarios, this supervised method tends to overfit to its training data as the involved recordings via electroencephalogram (EEG), magnetoencephalogram or local field potentials generally provide a low signal-to-noise ratio. To improve upon this, we propose and characterize three types of regularization techniques for SPoC: approaches using Tikhonov regularization (which requires model selection via cross-validation), combinations of Tikhonov regularization and covariance matrix normalization as well as strategies exploiting analytical covariance matrix shrinkage. All proposed techniques were evaluated both in a novel simulation framework and on real-world data. Based on the simulation findings, we saw our expectations fulfilled, that SPoC regularization generally reveals the largest benefit for small training sets and under severe label noise conditions. Relevant for practitioners, we derived operating ranges of regularization hyperparameters for cross-validation based approaches and offer open source code. Evaluating all methods additionally on real-world data, we observed an improved regression performance mainly for datasets from subjects with initially poor performance. With this proof-of-concept paper, we provided a generalizable regularization framework for SPoC which may serve as a starting point for implementing advanced techniques in the future.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Notes

  1. 1.

    To simplify the notation, epoched data X(t,e) will further on be written as a matrix X(e).

References

  1. Arvaneh, M., Guan, C., Ang, K.K., Quek, C. (2011). Optimizing the channel selection and classification accuracy in EEG-based BCI. IEEE Transactions on Biomedical Engineering, 58(6), 1865–1873. https://doi.org/10.1109/TBME.2011.2131142.

    Article  PubMed  Google Scholar 

  2. Arvaneh, M., Guan, C., Ang, K.K., Quek, C. (2013). Optimizing spatial filters by minimizing within-class dissimilarities in electroencephalogram-based brain-computer interface. IEEE Transactions on Neural Networks and Learning Systems, 24(4), 610–619. https://doi.org/10.1109/TNNLS.2013.2239310.

    Article  PubMed  Google Scholar 

  3. Bai, Z, & Silverstein, JW. (2009). Spectral analysis of large dimensional random matrices. Springer Science & Business Media.

  4. Bartz, D., & Müller, K.-R. (2014). Covariance shrinkage for autocorrelated data. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (Eds.) Advances in neural information processing systems, (Vol. 27 pp. 1592–1600): Curran Associates Inc.

  5. Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., Müller, K.-R. (2008). Optimizing spatial filters for robust EEG single-trial analysis. Signal Processing Magazine, IEEE, 25(1), 41–56.

    Article  Google Scholar 

  6. Blankertz, B., Sannelli, C., Halder, S., Hammer, E.M., Kübler, A., Müller, K.-R., Curio, G., Dickhaus, T. (2010). Neurophysiological predictor of SMR-based BCI performance. NeuroImage, 51(4), 1303–1309. https://doi.org/10.1016/j.neuroimage.2010.03.022.

    Article  PubMed  Google Scholar 

  7. Blankertz, B., Acqualagna, L., Dähne, S, Haufe, S., Schultze-Kraft, M., Sturm, I., Ušćumlic, M., Wenzel, M.A., Curio, G., Müller, K.-R. (2016). The berlin brain-computer interface: progress beyond communication and control. Frontiers in Neuroscience, 10, 530. https://doi.org/10.3389/fnins.2016.00530.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Castaño-Candamil, J.S., Meinel, A., Dähne, S., Tangermann, M. (2015). Probing meaningfulness of oscillatory EEG components with bootstrapping, label noise and reduced training sets. In 2015 37th Annual international conference of the IEEE engineering in medicine and biology society (EMBC) (pp. 5159–5162). IEEE.

  9. Castaño-Candamil, S., Meinel, A., Tangermann, M. (2017). Post-hoc labeling of arbitrary EEG recordings for data-efficient evaluation of neural decoding methods. arXiv:171108208.

  10. Chen, Y., Wiesel, A., Hero, A.O. (2011). Robust shrinkage estimation of high-dimensional covariance matrices. IEEE Transactions on Signal Processing, 59(9), 4097–4107. https://doi.org/10.1109/TSP.2011.2138698.

    Article  Google Scholar 

  11. Cheng, M., Lu, Z., Wang, H. (2017). Regularized common spatial patterns with subject-to-subject transfer of EEG signals. Cognitive Neurodynamics, 11(2), 173–181. https://doi.org/10.1007/s11571-016-9417-x.

    Article  PubMed  Google Scholar 

  12. Cho, H., Ahn, M., Kim, K., Jun, S.C. (2015). Increasing session-to-session transfer in a brain–computer interface with on-site background noise acquisition. Journal of Neural Engineering, 12(6), 066,009. https://doi.org/10.1088/1741-2560/12/6/066009.

    Article  Google Scholar 

  13. Clerc, M., Bougrain, L., Lotte, F. (2016). Brain-computer interfaces 2: technology and applications. Wiley.

  14. Dähne, S., Meinecke, F.C., Haufe, S., Höhne, J, Tangermann, M., Müller, K.-R., Nikulin, V.V. (2014). SPoC: a novel framework for relating the amplitude of neuronal oscillations to behaviorally relevant parameters. NeuroImage, 86(0), 111–122. https://doi.org/10.1016/j.neuroimage.2013.07.079.

    Article  PubMed  Google Scholar 

  15. de Cheveigné, A., & Parra, L.C. (2014). Joint decorrelation, a versatile tool for multichannel data analysis. NeuroImage, 98(Supplement C), 487–505. https://doi.org/10.1016/j.neuroimage.2014.05.068.

    Article  PubMed  Google Scholar 

  16. De Bie, T., Cristianini, N., Rosipal, R. (2005). Eigenproblems in pattern recognition. In Handbook of geometric computing (pp. 129–167). Springer.

  17. De Vos, M., Riès, S., Vanderperren, K., Vanrumste, B., Alario, F.X., Huffel, V.S., Burle, B. (2010). Removal of muscle artifacts from EEG recordings of spoken language production. Neuroinformatics, 8(2), 135–150. https://doi.org/10.1007/s12021-010-9071-0.

    Article  PubMed  Google Scholar 

  18. Devlaminck, D., Wyns, B., Grosse-Wentrup, M., Otte, G., Santens, P. (2011). Multisubject learning for common spatial patterns in motor-imagery BCI. Intelligence Neuroscience, 2011, 8:8–8:8. https://doi.org/10.1155/2011/217987.

    Article  Google Scholar 

  19. Engemann, D.A., & Gramfort, A. (2015). Automated model selection in covariance estimation and spatial whitening of MEG and EEG signals. NeuroImage, 108, 328–342. https://doi.org/10.1016/j.neuroimage.2014.12.040.

    Article  PubMed  Google Scholar 

  20. Farina, D., Jiang, N., Rehbaum, H., Holobar, A., Graimann, B., Dietl, H., Aszmann, O.C. (2014). The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 22(4), 797–809. https://doi.org/10.1109/TNSRE.2014.2305111.

    Article  PubMed  Google Scholar 

  21. Farquhar, J., & Hill, N.J. (2013). Interactions between pre-processing and classification methods for event-related-potential classification. Neuroinformatics, 11(2), 175–192. https://doi.org/10.1007/s12021-012-9171-0.

    Article  CAS  PubMed  Google Scholar 

  22. Farquhar, J., Hill, N., Lal, T.N., Schölkopf, B. (2006). Regularised CSP for sensor selection in BCI. In Proceedings of the 3rd international BCI workshop.

  23. Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874.

    Article  Google Scholar 

  24. Frey, J., Daniel, M., Hachet, M., Castet. J., Lotte, F. (2016). Framework for electroencephalography-based evaluation of user experience. InProcedings of CHI (pp. 2283–2294).

  25. Grosse-Wentrup, M., Liefhold, C., Gramann, K., Buss, M. (2009). Beamforming in non invasive brain-computer interfaces. IEEE Transactions on Biomedical Engineering, 56(4), 1209–1219.

    Article  PubMed  Google Scholar 

  26. Hyvarinen, A. (1999). Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks, 10(3), 626–634. https://doi.org/10.1109/72.761722.

    Article  CAS  Google Scholar 

  27. Kang, H., Nam, Y., Choi, S. (2009). Composite common spatial pattern for subject-to-subject transfer. IEEE Signal Processing Letters, 16(8), 683–686. https://doi.org/10.1109/LSP.2009.2022557.

    Article  Google Scholar 

  28. Kenney, J.F. (2013). Mathematics of statistics. Toronto: D. Van Nostrand Company Inc. Princeton; New Jersey; London; New York,; Affiliated East-West Press Pvt-Ltd; New Delhi.

    Google Scholar 

  29. Koles, Z.J. (1991). The quantitative extraction and topographic mapping of the abnormal components in the clinical EEG. Electroencephalography and clinical Neurophysiology, 79(6), 440–447.

    Article  CAS  PubMed  Google Scholar 

  30. Krusienski, D., Grosse-Wentrup, M., Galán, F., Coyle, D., Miller, K., Forney, E., Anderson, C. (2011). Critical issues in state-of-the-art brain-computer interface signal processing. Journal of Neural Engineering, 8(2), 025,002.

    Article  Google Scholar 

  31. Ledoit, O., & Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. Journal of Multivariate Analysis, 88(2), 365–411. https://doi.org/10.1016/S0047-259X(03)00096-4.

    Article  Google Scholar 

  32. Lotte, F. (2015). Signal processing approaches to minimize or suppress calibration time in oscillatory activity-based brain-computer interfaces. Proceedings of the IEEE, 103(6), 871–890. https://doi.org/10.1109/JPROC.2015.2404941.

    Article  Google Scholar 

  33. Lotte, F., & Guan, C. (2010). Learning from other subjects helps reducing brain-computer interface calibration time. In 2010 IEEE International conference on acoustics, speech and signal processing (pp. 614–617). https://doi.org/10.1109/ICASSP.2010.5495183.

  34. Lotte, F., & Guan, C. (2011). Regularizing common spatial patterns to improve BCI designs: unified theory and new algorithms. IEEE Transactions on Biomedical Engineering, 58(2), 355–362. https://doi.org/10.1109/TBME.2010.2082539.

    Article  PubMed  Google Scholar 

  35. Lu, H., Eng, H.L., Guan, C., Plataniotis, K.N., Venetsanopoulos, A.N. (2010). Regularized common spatial pattern with aggregation for EEG classification in small-sample setting. IEEE Transactions on Biomedical Engineering, 57(12), 2936–2946. https://doi.org/10.1109/TBME.2010.2082540.

    Article  PubMed  Google Scholar 

  36. Mahmud, M., Kaiser, M.S., Hussain, A., Vassanelli, S. (2018). Applications of deep learning and reinforcement learning to biological data. IEEE Transactions on Neural Networks and Learning Systems, 29(6), 2063–2079. https://doi.org/10.1109/TNNLS.2018.2790388.

    Article  PubMed  Google Scholar 

  37. Makeig, S., Debener, S., Onton, J., Delorme, A. (2004). Mining event-related brain dynamics. Trends in Cognitive Sciences, 8(5), 204–210. https://doi.org/10.1016/j.tics.2004.03.008.

    Article  PubMed  Google Scholar 

  38. Makeig, S., Kothe, C., Mullen, T., Bigdely-Shamlo, N., Zhang, Z., Kreutz-Delgado, K. (2012). Evolving signal processing for brain-computer interfaces. Proceedings of the IEEE, 100(Special Centennial Issue), 1567–1584. https://doi.org/10.1109/JPROC.2012.2185009.

    Article  CAS  Google Scholar 

  39. Mattout, J., Phillips, C., Penny, W.D., Rugg, M.D., Friston, K.J. (2006). MEG source localization under multiple constraints: an extended Bayesian framework. NeuroImage, 30(3), 753–767. https://doi.org/10.1016/j.neuroimage.2005.10.037.

    Article  PubMed  Google Scholar 

  40. Meinel, A., Castaño-Candamil, S, Reis, J., Tangermann, M. (2016). Pre-trial EEG-based single-trial motor performance prediction to enhance neuroergonomics for a hand force task. Frontiers in Human Neuroscience, 10, 170. https://doi.org/10.3389/fnhum.2016.00170.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Meinel, A., Lotte, F., Tangermann, M. (2017). Tikhonov regularization enhances EEG-based spatial filtering for single-trial regression. In Proceedings of the 7th Graz brain-computer interface conference 2017 (pp. 308-313). https://doi.org/10.3217/978-3-85125-533-1-57.

  42. Millán, J.d.R., Rupp, R., Mueller-Putz, G., Murray-Smith, R., Giugliemma, C., Tangermann, M., Vidaurre, C., Cincotti, F., Kübler, A, Leeb, R., Neuper, C., Müller, K.-R, Mattia, D. (2010). Combining brain–computer interfaces and assistive technologies: state-of-the-art and challenges. Frontiers in Neuroscience, 4, 161.

    PubMed  PubMed Central  Google Scholar 

  43. Nicolae, I.E., Acqualagna, L., Blankertz, B. (2017). Assessing the depth of cognitive processing as the basis for potential user-state adaptation. Frontiers in Neuroscience, 11. https://doi.org/10.3389/fnins.2017.00548.

  44. Park, S.H., Lee, D., Lee, S.G. (2017). Filter bank regularized common spatial pattern ensemble for small sample motor imagery classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering, PP(99), 1–1. https://doi.org/10.1109/TNSRE.2017.2757519.

    Article  Google Scholar 

  45. Parra, L.C., Spence, C.D., Gerson, A.D., Sajda, P. (2005). Recipes for the linear analysis of EEG. NeuroImage, 28(2), 326–341.

    Article  PubMed  Google Scholar 

  46. Ramanathan, C., Ghanem, R.N., Jia, P., Ryu, K., Rudy, Y. (2004). Noninvasive electrocardiographic imaging for cardiac electrophysiology and arrhythmia. Nature Medicine, 10(4), nm1011. https://doi.org/10.1038/nm1011.

    Article  CAS  Google Scholar 

  47. Ramoser, H., Muller-Gerking, J., Pfurtscheller, G. (2000). Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Transactions on Rehabilitation Engineering, 8(4), 441–446.

    Article  CAS  PubMed  Google Scholar 

  48. Reuderink, B., & Poel, M. (2008). Robustness of the common spatial patterns algorithm in the BCI-pipeline. Tech rep. HMI, University of Twente.

  49. Samek, W., Vidaurre, C., Müller, K.-R., Kawanabe, M. (2012). Stationary common spatial patterns for brain-computer interfacing. Journal of Neural Engineering, 9(2), 026,013. https://doi.org/10.1088/1741-2560/9/2/026013.

    Article  Google Scholar 

  50. Samek, W., Meinecke, F.C., Müller, K.-R. (2013). Transferring subspaces between subjects in brain-computer interfacing. IEEE Transactions on Biomedical Engineering, 60(8), 2289–2298. https://doi.org/10.1109/TBME.2013.2253608.

    Article  PubMed  Google Scholar 

  51. Samek, W., Kawanabe, M., Müller, K.-R. (2014). Divergence-based framework for common spatial patterns algorithms. IEEE Reviews in Biomedical Engineering, 7, 50–72. https://doi.org/10.1109/RBME.2013.2290621.

    Article  PubMed  Google Scholar 

  52. Schäfer, J., & Strimmer, K. (2005). A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Statistical Applications in Genetics and Molecular Biology, 4, 1.

    Article  Google Scholar 

  53. Schultze-Kraft, M., Dähne, S, Gugler, M., Curio, G., Blankertz, B. (2016). Unsupervised classification of operator workload from brain signals. Journal of Neural Engineering, 13(3), 036,008. https://doi.org/10.1088/1741-2560/13/3/036008.

    Article  Google Scholar 

  54. Tian, T.S., Huang, J.Z., Shen, H., Li, Z. (2013). EEG/MEG source reconstruction with spatial-temporal two-way regularized regression. Neuroinformatics, 11(4), 477–493. https://doi.org/10.1007/s12021-013-9193-2.

    Article  PubMed  Google Scholar 

  55. Tikhonov, A.N. (1963). Regularization of incorrectly posed problems. Soviet Mathematics Doklady, 4, 1624–1627.

    Google Scholar 

  56. Úbeda, A., Azorín, J.M., Chavarriaga, R., Millán, J.d.R. (2017). Classification of upper limb center-out reaching tasks by means of EEG-based continuous decoding techniques. Journal of NeuroEngineering and Rehabilitation, 14, 9. https://doi.org/10.1186/s12984-017-0219-0.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Wang, H., & Li, X. (2016). Regularized filters for L1-norm-based common spatial patterns. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 24(2), 201–211. https://doi.org/10.1109/TNSRE.2015.2474141.

    Article  PubMed  Google Scholar 

  58. Winkler, I., Brandl, S., Horn, F., Waldburger, E., Allefeld, C., Tangermann, M. (2014). Robust artifactual independent component classification for bci practitioners. Journal of Neural Engineering, 11(3), 035,013.

    Article  Google Scholar 

  59. Wu, D., King, J.T., Chuang, C.H., Lin, C.T., Jung, T.P. (2017). Spatial filtering for EEG-based regression problems in brain-computer interface (BCI). IEEE Transactions on Fuzzy Systems, PP(99), 1–1. https://doi.org/10.1109/TFUZZ.2017.2688423.

    Article  Google Scholar 

Download references

Acknowledgements

This work was fully supported by BrainLinks-BrainTools Cluster of Excellence funded by the German Research Foundation (DFG), grant number EXC1086. For the data analysis, the authors acknowledge support by the state of Baden-Württemberg through bwHPC and the German Research Foundation (DFG) by grant no. INST 39/963-1 FUGG. Fabien Lotte received research support from the French National Research Agency with the REBEL project (grant ANR-15-CE23-0013-01) and the European Research Council with the BrainConquest project (grant ERC-2016-STG-714567). For parts of the data analysis, the Matlab-based BBCI toolbox was utilized (Blankertz et al. 2016). The authors declare that they have no conflict of interest.

Author information

Affiliations

Authors

Corresponding authors

Correspondence to Andreas Meinel or Michael Tangermann.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Meinel, A., Castaño-Candamil, S., Blankertz, B. et al. Characterizing Regularization Techniques for Spatial Filter Optimization in Oscillatory EEG Regression Problems. Neuroinform 17, 235–251 (2019). https://doi.org/10.1007/s12021-018-9396-7

Download citation

Keywords

  • EEG bandpower
  • Subspace decomposition
  • Brain-computer interface
  • Single trial analysis
  • Source power comodulation
  • Brain state decoding algorithm