Skip to main content
SpringerLink
Log in

Decoding motor imagery with a simplified distributed dipoles model at source level

  • Research Article
  • Published:
Cognitive Neurodynamics Aims and scope Submit manuscript

Abstract

Motor imagery (MI) based brain computer interface significantly oriented the development of neuro-rehabilitation, and the crucial issue is how to accurately detect the changes of cerebral cortex for MI decoding. The brain activity can be calculated based on the head model and observed scalp EEG, providing insights regarding cortical dynamics by using equivalent current dipoles with high spatial and temporal resolution. Now, all the dipoles within entire cortex or partial regions of interest are directly applied to data representation, this may make the key information weakened or lost, and it is worth studying how to choose the most important from numerous dipoles. In this paper, we devote to building a simplified distributed dipoles model (SDDM), which is combined with convolutional neural network (CNN), generating a MI decoding method at source level (called SDDM-CNN). First, all channels of raw MI-EEG signals are subdivided by a series of bandpass filters with width of 1 Hz, the average energies associated with any sub-band signals are calculated and ranked in a descending order to screen the top n sub-bands; then, the MI-EEG signals over each selected sub-band are mapped into source space by using EEG source imaging technology, and for each scout of neuroanatomical Desikan-Killiany partition, a centered dipole is selected as the most relevant dipole and put together to build a SDDM to reflect the neuroelectric activity of entire cerebral cortex; finally, the 4 dimensional (4D) magnitude matrix is constructed for each SDDM and fused into a novel data representation, which is further input to a well-designed 3DCNN with n parallel branches (nB3DCNN) to extract and classify the comprehensive features from time–frequency-space dimensions. Experiments are carried out on three public datasets, and the average ten-fold CV decoding accuracies achieve 95.09%, 97.98% and 94.53% respectively, and the statistical analysis is fulfilled by standard deviation, kappa value and confusion matrix. Experiment results suggest that it is beneficial to pick out the most sensitive sub-bands in sensor domain, and SDDM can sufficiently describe the dynamic changing of entire cortex, improving decoding performance while greatly reducing number of source signals. Also, nB3DCNN is capable of exploring spatial–temporal features from multi sub-bands.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

source estimation, establishment of SDDM, construction of 4D amplitude matrix, multi-band fusion data representation, and classification with nB3DCNN

Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

No new data were created or analyzed in this study. All data comes from the BCI public data set, which can be downloaded and used on this website: http://www.bbci.de/competition/.

References

  • Bashivan P, Rish I, Yeasin M and Codella N (2015) Learning representations from EEG with deep recurrent-convolutional neural networks arXiv preprint arXiv:1511.06448.

  • Chaudhary S, Taran S, Bajaj V, Sengur A (2019) Convolutional neural network based approach towards motor imagery tasks EEG signals classification. IEEE Sens J 19:4494–4500

    Article  Google Scholar 

  • Collazos-Huertas DF, Álvarez-Meza AM, Acosta-Medina CD, Castaño-Duque GA, Castellanos-Dominguez G (2020) CNN-based framework using spatial dropping for enhanced interpretation of neural activity in motor imagery classification. Brain Inform 7(1):1–13

    Article  Google Scholar 

  • Dai M, Zheng D, Na R, Wang S, Zhang S (2019) EEG classification of motor imagery using a novel deep learning framework. Sensors 19:551

    Article  PubMed  PubMed Central  Google Scholar 

  • Dai G, Zhou J, Huang J, Wang N (2020) HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification. J Neural Eng 17(1):016025

    Article  PubMed  Google Scholar 

  • Dalin Z et al (2019) Making sense of spatio-temporal preserving representations for EEg-based human intention recognition. IEEE Trans Cybern 50:3033–3044

    Google Scholar 

  • Deng X, Zhu J, Yang S (2021) SFE-Net: EEG-based Emotion Recognition with Symmetrical spatial feature extraction. arXiv preprint arXiv:2104.06308.

  • Desikan RS, Ségonne F, Fischl B, Quinn BT, Dickerson BC, Blacker D, Albert MS (2006) An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31:968–980

    Article  PubMed  Google Scholar 

  • Dose H, Młller JS, Iversen HK, Puthusserypady S (2018) An end-to end deep learning approach to MI-EEG signal classification for BCIs. Expert Syst Appl 114:532–542

    Article  Google Scholar 

  • Gramfort A, Papadopoulo T, Olivi E, Clerc M (2010) OpenMEEG: opensource software for quasistatic bioelectromagnetics. Biomed Eng Online 9:45

    Article  PubMed  PubMed Central  Google Scholar 

  • Ha KW, Jeong JW (2019) Motor imagery EEG classification using capsule networks. Sensors 19:2854

    Article  PubMed  PubMed Central  Google Scholar 

  • Handiru VS, Vinod AP, Guan C (2018) EEG source imaging of movement decoding: the state of the art and future directions. IEEE Syst, Man, Cybernet Mag 4(2):14–23

    Article  Google Scholar 

  • Hou Y, Zhou L, Jia S, Lun X (2020) A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN. J Neural Eng 17:016048

    Article  PubMed  Google Scholar 

  • Huang S, Chen Y, Wang T et al (2020) Spectrum-weighted tensor discriminant analysis for motor imagery-based BCI. IEEE Access 8:93749–93759

    Article  Google Scholar 

  • Kwak Y, Kong K, Song WJ, Min BK, Kim SE (2020) Multilevel feature fusion with 3d convolutional neural network for eeg-based workload estimation. IEEE Access 8:16009–16021

    Article  Google Scholar 

  • Li MA, Ruan ZW (2021) A novel decoding method for motor imagery tasks with 4D data representation and 3D convolutional neural networks. J Neural Eng 18(4):046029

    Article  Google Scholar 

  • Li MA, Han JF, Duan LJ (2019) A Novel MI-EEG imaging with the location information of electrodes. IEEE Access 8:3197–3211

    Article  Google Scholar 

  • Li F, He F, Wang F, Zhang D, Xia Y, Li X (2020) A novel simplified convolutional neural network classification algorithm of motor imagery EEG signals based on deep learning. Appl Sci 10:1605

    Article  CAS  Google Scholar 

  • Liu X, Lv L, Shen Y, Xiong P, Yang J, Liu J (2021a) Multiscale space-time-frequency feature-guided multitask learning CNN for motor imagery EEG classification. J Neural Eng 18(2):026003

    Article  Google Scholar 

  • Liu H, Liu Y, Zhang R and Wu X (2021b) Student behavior recognition from heterogeneous view perception in class based on 3-D multiscale residual dense network for the analysis of case teaching. Front Neurorobot, 15.

  • Lun X, Yu Z, Chen T, Wang F and Hou Y (2020) A simplified CNN classification method for MI-EEG via the electrode pairs signals. Front Human Neurosci, 14.

  • Maeder CL, Sannelli C, Haufe S, Blankertz B (2012) Pre-stimulus sensorimotor rhythms influence brain–computer interface classification performance. IEEE Trans Neural Syst Rehabil Eng 20(5):653–662

    Article  PubMed  Google Scholar 

  • Mammone N, Ieracitano C, Morabito FC (2020) A deep CNN approach to decode motor preparation of upper limbs from time–frequency maps of EEG signals at source level. Neural Netw 124:357–372

    Article  PubMed  Google Scholar 

  • Miao M, Hu W, Zhang W (2021) A spatial-frequency-temporal 3D convolutional neural network for motor imagery EEG signal classification. SIViP. https://doi.org/10.1007/s11760-021-01924-3

    Article  Google Scholar 

  • Pascual-Marqui RD (2002) Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details Methods Find. Exp Clin Pharmacol 24:5–12

    Google Scholar 

  • Qiao, W., & Bi, X. (2019) Deep spatial-temporal neural network for classification of eeg-based motor imagery Proceedings of the 2019 international conference on artificial intelligence and computer science (pp. 265–272).

  • Sakhavi S, Guan C, Yan S (2018) Learning temporal information for brain-computer interface using convolutional neural networks. IEEE Trans Neural Netw Learn Syst 29:5619–5629

    Article  PubMed  Google Scholar 

  • Salama ES, El-Khoribi RA, Shoman ME, Shalaby MAW (2018) EEG-based emotion recognition using 3D convolutional neural networks. Int J Adv Comput Sci Appl 9(8):329–337

    Google Scholar 

  • Selim S, Tantawi MM, Shedeed HA, Badr A (2018) ‘A CSPAM-BASVM approach for motor imagery BCI system’. IEEE Access 6:49192–49208

    Article  Google Scholar 

  • Sun H, Jin J, Kong W et al (2021) Novel channel selection method based on position priori weighted permutation entropy and binary gravity search algorithm[J]. Cogn Neurodyn 15(1):141–156

    Article  PubMed  Google Scholar 

  • Tang X, Li W, Li X, Ma W, Dang X (2020) Motor imagery EEG recognition based on conditional optimization empirical mode decomposition and multi-scale convolutional neural network. Expert Syst Appl 149:113285

    Article  Google Scholar 

  • Tayeb Z, Fedjaev J, Ghaboosi N, Richter C, Everding L, Qu X, Conradt J (2019) Validating deep neural networks for online decoding of motor imagery movements from EEG signals. Sensors 19:210

    Article  PubMed  PubMed Central  Google Scholar 

  • Wang P, Hu J (2019) A hybrid model for EEG-based gender recognition[J]. Cogn Neurodyn 13(6):541–554

    Article  PubMed  PubMed Central  Google Scholar 

  • Wang D, Miao D, Blohm G (2012) Multi-class motor imagery EEG decoding for brain-computer interfaces. Front Neurosci 6:151

    Article  PubMed  PubMed Central  Google Scholar 

  • Xu B, Zhang L, Song A, Wu C, Li W, Zhang D, Zeng H (2018) Wavelet transform time-frequency image and convolutional network-based motor imagery EEG classification. IEEE Access 7:6084–6093

    Article  Google Scholar 

  • Xu G, Shen X, Chen S, Zong Y, Zhang C, Yue H, Che W (2019) A deep transfer convolutional neural network framework for EEG signal classification. IEEE Access 7:112767–112776

    Article  Google Scholar 

  • Xu S, Zhu L, Kong W, et al. (2021) A novel classification method for EEG-based motor imagery with narrow band spatial filters and deep convolutional neural network[J]. Cognitive Neurodynamics, 1–11.

  • Yang J, Yao S, Wang J (2018) Deep fusion feature learning network for mi-eeg classification. IEEE Access 6:79050–79059

    Article  Google Scholar 

  • Zhang P, Wang X, Zhang W, Chen J (2018) Learning spatial–spectral–temporal EEG features with recurrent 3D convolutional neural networks for cross-task mental workload assessment. IEEE Trans Neural Syst Rehabil Eng 27(1):31–42

    Article  PubMed  Google Scholar 

  • Zhang D, Chen K, Jian D, Yao L (2020) Motor imagery classification via temporal attention cues of graph embedded eeg signals. IEEE J Biomed Health Inform 24:2570–2579

    Article  PubMed  Google Scholar 

  • Zhao X, Zhang H, Zhu G, You F, Kuang S, Sun L (2019) A multi-branch 3D convolutional neural network for EEG-based motor imagery classification. IEEE Trans Neural Syst Rehabil Eng 27:2164–2177

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The National Natural Science Foundation of China (Nos. 62173010, No. 11832003) financially supported this work. We would like to thank Brainstorm, BNCI Horizon 2020, and the people who have given us helpful suggestions and advice. We would also like to thank the anonymous reviewers for carefully examining the details of this paper and for useful comments that improved this paper.

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

    Ming-ai Li & Zi-wei Ruan

  2. Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing, 100124, China

    Ming-ai Li

  3. Engineering Research Center of Digital Community, Ministry of Education, Beijing, 100124, China

    Ming-ai Li

Authors
  1. Ming-ai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zi-wei Ruan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ming-ai Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Ma., Ruan, Zw. Decoding motor imagery with a simplified distributed dipoles model at source level. Cogn Neurodyn 17, 445–457 (2023). https://doi.org/10.1007/s11571-022-09826-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11571-022-09826-x

Keywords

Search

Navigation