Abstract
Electroencephalography (EEG) based motor imagery (MI) is one of the promising Brain–computer interface (BCI) paradigms enable humans to communicate with the outside world based solely on brain intentions. Although convolutional neural networks have been gradually applied to MI classification task and gained high performance, the following problems still exist to make effective decoding of raw EEG signals challenging: 1) EEG signals are non-linear, non-stationary, and low signal-noise ratio. 2) Most existing end-to-end MI models utilize single scale convolution which limits the result of classification because the best convolution scale varies with different subject (called subject difference). In addition, even for the same subject, the best convolution scale also differs from time to time (called time difference). In this paper, we propose a novel end-to-end model, named Multi-branch Multi-scale Convolutional Neural Network (MMCNN), for motor imagery classification. The MMCNN model effectively decodes raw EEG signals without any pre-processing including filtering. Meanwhile, the multi-branch multi-scale convolution structure successfully addresses the problems of subject difference and time difference based on parallel processing. In addition, multi-scale convolution can realize the characterization of information in different frequency bands, thereby effectively solving the problem that the best convolution scale is difficult to determine. Experiments on two public BCI competition datasets demonstrate that the proposed MMCNN model outperforms the state-of-the-art models. The implementation code is made publicly available https://github.com/jingwang2020/ECML-PKDD_MMCNN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ang, K.K., Chin, Z.Y., Wang, C., Guan, C., Zhang, H.: Filter bank common spatial pattern algorithm on BCI competition iv datasets 2a and 2b. Front. Neurosci. 6, 39 (2012)
Ang, K.K., Chin, Z.Y., Zhang, H., Guan, C.: Filter bank common spatial pattern (FBCSP) in brain-computer interface. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp. 2390–2397. IEEE (2008)
Clevert, D.A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (ELUs). arXiv preprint arXiv:1511.07289 (2015)
Cohen, M.X.: Analyzing Neural Time Series Data: Theory and Practice. MIT Press, Cambridge (2014)
Dose, H., Møller, J.S., Iversen, H.K., Puthusserypady, S.: An end-to-end deep learning approach to MI-EEG signal classification for BCIs. Expert Syst. Appl. 114, 532–542 (2018)
Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press Professional, Inc., Boston (1990)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Lawhern, V.J., Solon, A.J., Waytowich, N.R., Gordon, S.M., Hung, C.P., Lance, B.J.: EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. J. Neural Eng. 15(5), 056013 (2018)
Lu, N., Li, T., Ren, X., Miao, H.: A deep learning scheme for motor imagery classification based on restricted Boltzmann machines. IEEE Trans. Neural Syst. Rehabil. Eng. 25(6), 566–576 (2016)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 807–814 (2010)
Nicolas-Alonso, L.F., Gomez-Gil, J.: Brain computer interfaces, a review. Sensors 12(2), 1211–1279 (2012)
Schirrmeister, R.T., et al.: Deep learning with convolutional neural networks for EEG decoding and visualization. Hum. Brain Mapp. 38(11), 5391–5420 (2017)
Sun, L., Feng, Z., Lu, N., Wang, B., Zhang, W.: An advanced bispectrum features for EEG-based motor imagery classification. Expert Syst. Appl. 131, 9–19 (2019)
Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Wu, H., et al.: A parallel multiscale filter bank convolutional neural networks for motor imagery EEG classification. Front. Neurosci. 13, 1275 (2019)
Zhang, Y., et al.: Multi-kernel extreme learning machine for EEG classification in brain-computer interfaces. Expert Syst. Appl. 96, 302–310 (2018)
Zhao, D., Tang, F., Si, B., Feng, X.: Learning joint space-time-frequency features for EEG decoding on small labeled data. Neural Netw. 144, 67–77 (2019)
Jia, Z., Lin, Y., Jiao, Z., Ma, Y., Wang, J.: Detecting causality in multivariate time series via non-uniform embedding. Entropy 21(12), 1233 (2019)
Jia, Z., Lin, Y., Liu, Y., Jiao, Z., Wang, J.: Refined nonuniform embedding for coupling detection in multivariate time series. Phys. Rev. E 101(6), 062113 (2020)
Acknowledgments
Financial supports by National Natural Science Foundation of China (61603029), the Fundamental Research Funds for the Central Universities (2020YJS025) are gratefully acknowledge.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Jia, Z., Lin, Y., Wang, J., Yang, K., Liu, T., Zhang, X. (2021). MMCNN: A Multi-branch Multi-scale Convolutional Neural Network for Motor Imagery Classification. In: Hutter, F., Kersting, K., Lijffijt, J., Valera, I. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2020. Lecture Notes in Computer Science(), vol 12459. Springer, Cham. https://doi.org/10.1007/978-3-030-67664-3_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-67664-3_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67663-6
Online ISBN: 978-3-030-67664-3
eBook Packages: Computer ScienceComputer Science (R0)
