Abstract
Motor imagery (MI) based brain computer interface significantly oriented the development of neuro-rehabilitation, and the crucial issue is how to accurately detect the changes of cerebral cortex for MI decoding. The brain activity can be calculated based on the head model and observed scalp EEG, providing insights regarding cortical dynamics by using equivalent current dipoles with high spatial and temporal resolution. Now, all the dipoles within entire cortex or partial regions of interest are directly applied to data representation, this may make the key information weakened or lost, and it is worth studying how to choose the most important from numerous dipoles. In this paper, we devote to building a simplified distributed dipoles model (SDDM), which is combined with convolutional neural network (CNN), generating a MI decoding method at source level (called SDDM-CNN). First, all channels of raw MI-EEG signals are subdivided by a series of bandpass filters with width of 1 Hz, the average energies associated with any sub-band signals are calculated and ranked in a descending order to screen the top n sub-bands; then, the MI-EEG signals over each selected sub-band are mapped into source space by using EEG source imaging technology, and for each scout of neuroanatomical Desikan-Killiany partition, a centered dipole is selected as the most relevant dipole and put together to build a SDDM to reflect the neuroelectric activity of entire cerebral cortex; finally, the 4 dimensional (4D) magnitude matrix is constructed for each SDDM and fused into a novel data representation, which is further input to a well-designed 3DCNN with n parallel branches (nB3DCNN) to extract and classify the comprehensive features from time–frequency-space dimensions. Experiments are carried out on three public datasets, and the average ten-fold CV decoding accuracies achieve 95.09%, 97.98% and 94.53% respectively, and the statistical analysis is fulfilled by standard deviation, kappa value and confusion matrix. Experiment results suggest that it is beneficial to pick out the most sensitive sub-bands in sensor domain, and SDDM can sufficiently describe the dynamic changing of entire cortex, improving decoding performance while greatly reducing number of source signals. Also, nB3DCNN is capable of exploring spatial–temporal features from multi sub-bands.
Similar content being viewed by others
Data availability
No new data were created or analyzed in this study. All data comes from the BCI public data set, which can be downloaded and used on this website: http://www.bbci.de/competition/.
References
Bashivan P, Rish I, Yeasin M and Codella N (2015) Learning representations from EEG with deep recurrent-convolutional neural networks arXiv preprint arXiv:1511.06448.
Chaudhary S, Taran S, Bajaj V, Sengur A (2019) Convolutional neural network based approach towards motor imagery tasks EEG signals classification. IEEE Sens J 19:4494–4500
Collazos-Huertas DF, Álvarez-Meza AM, Acosta-Medina CD, Castaño-Duque GA, Castellanos-Dominguez G (2020) CNN-based framework using spatial dropping for enhanced interpretation of neural activity in motor imagery classification. Brain Inform 7(1):1–13
Dai M, Zheng D, Na R, Wang S, Zhang S (2019) EEG classification of motor imagery using a novel deep learning framework. Sensors 19:551
Dai G, Zhou J, Huang J, Wang N (2020) HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification. J Neural Eng 17(1):016025
Dalin Z et al (2019) Making sense of spatio-temporal preserving representations for EEg-based human intention recognition. IEEE Trans Cybern 50:3033–3044
Deng X, Zhu J, Yang S (2021) SFE-Net: EEG-based Emotion Recognition with Symmetrical spatial feature extraction. arXiv preprint arXiv:2104.06308.
Desikan RS, Ségonne F, Fischl B, Quinn BT, Dickerson BC, Blacker D, Albert MS (2006) An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31:968–980
Dose H, Młller JS, Iversen HK, Puthusserypady S (2018) An end-to end deep learning approach to MI-EEG signal classification for BCIs. Expert Syst Appl 114:532–542
Gramfort A, Papadopoulo T, Olivi E, Clerc M (2010) OpenMEEG: opensource software for quasistatic bioelectromagnetics. Biomed Eng Online 9:45
Ha KW, Jeong JW (2019) Motor imagery EEG classification using capsule networks. Sensors 19:2854
Handiru VS, Vinod AP, Guan C (2018) EEG source imaging of movement decoding: the state of the art and future directions. IEEE Syst, Man, Cybernet Mag 4(2):14–23
Hou Y, Zhou L, Jia S, Lun X (2020) A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN. J Neural Eng 17:016048
Huang S, Chen Y, Wang T et al (2020) Spectrum-weighted tensor discriminant analysis for motor imagery-based BCI. IEEE Access 8:93749–93759
Kwak Y, Kong K, Song WJ, Min BK, Kim SE (2020) Multilevel feature fusion with 3d convolutional neural network for eeg-based workload estimation. IEEE Access 8:16009–16021
Li MA, Ruan ZW (2021) A novel decoding method for motor imagery tasks with 4D data representation and 3D convolutional neural networks. J Neural Eng 18(4):046029
Li MA, Han JF, Duan LJ (2019) A Novel MI-EEG imaging with the location information of electrodes. IEEE Access 8:3197–3211
Li F, He F, Wang F, Zhang D, Xia Y, Li X (2020) A novel simplified convolutional neural network classification algorithm of motor imagery EEG signals based on deep learning. Appl Sci 10:1605
Liu X, Lv L, Shen Y, Xiong P, Yang J, Liu J (2021a) Multiscale space-time-frequency feature-guided multitask learning CNN for motor imagery EEG classification. J Neural Eng 18(2):026003
Liu H, Liu Y, Zhang R and Wu X (2021b) Student behavior recognition from heterogeneous view perception in class based on 3-D multiscale residual dense network for the analysis of case teaching. Front Neurorobot, 15.
Lun X, Yu Z, Chen T, Wang F and Hou Y (2020) A simplified CNN classification method for MI-EEG via the electrode pairs signals. Front Human Neurosci, 14.
Maeder CL, Sannelli C, Haufe S, Blankertz B (2012) Pre-stimulus sensorimotor rhythms influence brain–computer interface classification performance. IEEE Trans Neural Syst Rehabil Eng 20(5):653–662
Mammone N, Ieracitano C, Morabito FC (2020) A deep CNN approach to decode motor preparation of upper limbs from time–frequency maps of EEG signals at source level. Neural Netw 124:357–372
Miao M, Hu W, Zhang W (2021) A spatial-frequency-temporal 3D convolutional neural network for motor imagery EEG signal classification. SIViP. https://doi.org/10.1007/s11760-021-01924-3
Pascual-Marqui RD (2002) Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details Methods Find. Exp Clin Pharmacol 24:5–12
Qiao, W., & Bi, X. (2019) Deep spatial-temporal neural network for classification of eeg-based motor imagery Proceedings of the 2019 international conference on artificial intelligence and computer science (pp. 265–272).
Sakhavi S, Guan C, Yan S (2018) Learning temporal information for brain-computer interface using convolutional neural networks. IEEE Trans Neural Netw Learn Syst 29:5619–5629
Salama ES, El-Khoribi RA, Shoman ME, Shalaby MAW (2018) EEG-based emotion recognition using 3D convolutional neural networks. Int J Adv Comput Sci Appl 9(8):329–337
Selim S, Tantawi MM, Shedeed HA, Badr A (2018) ‘A CSPAM-BASVM approach for motor imagery BCI system’. IEEE Access 6:49192–49208
Sun H, Jin J, Kong W et al (2021) Novel channel selection method based on position priori weighted permutation entropy and binary gravity search algorithm[J]. Cogn Neurodyn 15(1):141–156
Tang X, Li W, Li X, Ma W, Dang X (2020) Motor imagery EEG recognition based on conditional optimization empirical mode decomposition and multi-scale convolutional neural network. Expert Syst Appl 149:113285
Tayeb Z, Fedjaev J, Ghaboosi N, Richter C, Everding L, Qu X, Conradt J (2019) Validating deep neural networks for online decoding of motor imagery movements from EEG signals. Sensors 19:210
Wang P, Hu J (2019) A hybrid model for EEG-based gender recognition[J]. Cogn Neurodyn 13(6):541–554
Wang D, Miao D, Blohm G (2012) Multi-class motor imagery EEG decoding for brain-computer interfaces. Front Neurosci 6:151
Xu B, Zhang L, Song A, Wu C, Li W, Zhang D, Zeng H (2018) Wavelet transform time-frequency image and convolutional network-based motor imagery EEG classification. IEEE Access 7:6084–6093
Xu G, Shen X, Chen S, Zong Y, Zhang C, Yue H, Che W (2019) A deep transfer convolutional neural network framework for EEG signal classification. IEEE Access 7:112767–112776
Xu S, Zhu L, Kong W, et al. (2021) A novel classification method for EEG-based motor imagery with narrow band spatial filters and deep convolutional neural network[J]. Cognitive Neurodynamics, 1–11.
Yang J, Yao S, Wang J (2018) Deep fusion feature learning network for mi-eeg classification. IEEE Access 6:79050–79059
Zhang P, Wang X, Zhang W, Chen J (2018) Learning spatial–spectral–temporal EEG features with recurrent 3D convolutional neural networks for cross-task mental workload assessment. IEEE Trans Neural Syst Rehabil Eng 27(1):31–42
Zhang D, Chen K, Jian D, Yao L (2020) Motor imagery classification via temporal attention cues of graph embedded eeg signals. IEEE J Biomed Health Inform 24:2570–2579
Zhao X, Zhang H, Zhu G, You F, Kuang S, Sun L (2019) A multi-branch 3D convolutional neural network for EEG-based motor imagery classification. IEEE Trans Neural Syst Rehabil Eng 27:2164–2177
Acknowledgements
The National Natural Science Foundation of China (Nos. 62173010, No. 11832003) financially supported this work. We would like to thank Brainstorm, BNCI Horizon 2020, and the people who have given us helpful suggestions and advice. We would also like to thank the anonymous reviewers for carefully examining the details of this paper and for useful comments that improved this paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, Ma., Ruan, Zw. Decoding motor imagery with a simplified distributed dipoles model at source level. Cogn Neurodyn 17, 445–457 (2023). https://doi.org/10.1007/s11571-022-09826-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11571-022-09826-x