Abstract
Electroencephalographic (EEG) recordings can be of great help in decoding the open/close hand’s motion preparation. To this end, cortical EEG source signals in the motor cortex (evaluated in the 1-s window preceding movement onset) are extracted by solving inverse problem through beamforming. EEG sources epochs are used as source-time maps input to a custom deep convolutional neural network (CNN) that is trained to perform 2-ways classification tasks: pre-hand close (HC) versus resting state (RE) and pre-hand open (HO) versus RE. The developed deep CNN works well (accuracy rates up to \(89.65 \pm 5.29\%\) for HC versus RE and \(90.50 \pm 5.35\%\) for HO versus RE), but the core of the present study was to explore the interpretability of the deep CNN to provide further insights into the activation mechanism of cortical sources during the preparation of hands’ sub-movements. Specifically, occlusion sensitivity analysis was carried out to investigate which cortical areas are more relevant in the classification procedure. Experimental results show a recurrent trend of spatial cortical activation across subjects. In particular, the central region (close to the longitudinal fissure) and the right temporal zone of the premotor together with the primary motor cortex appear to be primarily involved. Such findings encourage an in-depth study of cortical areas that seem to play a key role in hand’s open/close preparation.
Similar content being viewed by others
References
Bengio Y (2012) Practical recommendations for gradient-based training of deep architectures. In: Neural networks: tricks of the trade, pp 437–478
Bi X, Zhao X, Huang H, Chen D, Ma Y (2020) Functional brain network classification for alzheimer’s disease detection with deep features and extreme learning machine. Cogn Comput 12(3):513–527
Brodmann K (2007) Brodmann’s: localisation in the cerebral cortex. Springer, Berlin
Cai Z, Shao L (2019) Rgb-d scene classification via multi-modal feature learning. Cogn Comput 11(6):825–840
Catani M (2017) A little man of some importance. Brain 140(11):3055–3061
Chen H, Ding G, Lin Z, Guo Y, Shan C, Han J (2019) Image captioning with memorized knowledge. Cogn Comput 1–14
Chen X, Xie H (2020) A structural topic modeling-based bibliometric study of sentiment analysis literature. Cogn Comput 1–33
Craik A, He Y, Contreras-Vidal JL (2019) Deep learning for electroencephalogram (EEG) classification tasks: a review. J Neural Eng 16(3):031001
Doborjeh ZG, Doborjeh MG, Kasabov N (2018) Attentional bias pattern recognition in spiking neural networks from spatio-temporal EEG data. Cogn Comput 10(1):35–48
Edelman BJ, Baxter B, He B (2015) EEG source imaging enhances the decoding of complex right-hand motor imagery tasks. IEEE Trans Biomed Eng 63(1):4–14
Fellous JM, Sapiro G, Rossi A, Mayberg HS, Ferrante M (2019) Explainable artificial intelligence for neuroscience: behavioral neurostimulation. Front Neurosci 13:1346
Gao F, Huang T, Sun J, Wang J, Hussain A, Yang E (2019) A new algorithm for sar image target recognition based on an improved deep convolutional neural network. Cogn Comput 11(6):809–824
Goshvarpour A, Goshvarpour A (2019) A novel approach for EEG electrode selection in automated emotion recognition based on lagged poincare’s indices and sloreta. Cogn Comput 1–17
Grech R, Cassar T, Muscat J, Camilleri KP, Fabri SG, Zervakis M, Xanthopoulos P, Sakkalis V, Vanrumste B (2008) Review on solving the inverse problem in EEG source analysis. J Neuroeng Rehabilit 5(1):25
Grosse-Wentrup M, Liefhold C, Gramann K, Buss M (2009) Beamforming in noninvasive brain–computer interfaces. IEEE Trans Biomed Eng 56(4):1209–1219
Hallez H, Vanrumste B, Grech R, Muscat J, De Clercq W, Vergult A, D’Asseler Y, Camilleri KP, Fabri SG, Van Huffel S et al (2007) Review on solving the forward problem in EEG source analysis. J Neuroeng Rehabilit 4(1):46
Haufe S, Ewald A (2016) A simulation framework for benchmarking EEG-based brain connectivity estimation methodologies. Brain Topogr, pp 1–18
Haufe S, Huang Y, Parra LC (2015) A highly detailed FEM volume conductor model based on the ICBM152 average head template for EEG source imaging and TCS targeting. In: Conference poceedings of IEEE engineering medicine and biological society
Hermes D, Vansteensel MJ, Albers AM, Bleichner MG, Benedictus MR, Orellana CM, Aarnoutse E, Ramsey N (2011) Functional MRI-based identification of brain areas involved in motor imagery for implantable brain–computer interfaces. J Neural Eng 8(2):025007
Ieracitano C, Mammone N, Bramanti A, Hussain A, Morabito FC (2019) A convolutional neural network approach for classification of dementia stages based on 2d-spectral representation of EEG recordings. Neurocomputing 323:96–107
Ieracitano C, Mammone N, Hussain A, Morabito FC (2020) A novel multi-modal machine learning based approach for automatic classification of EEG recordings in dementia. Neural Netw 123:176–190
Kantak SS, Stinear JW, Buch ER, Cohen LG (2012) Rewiring the brain: potential role of the premotor cortex in motor control, learning, and recovery of function following brain injury. Neurorehabilit Neural Repair 26(3):282–292
Kasabov NK (2014) Neucube: a spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data. Neural Netw 52:62–76
Kasabov NK (2019) Time-space, spiking neural networks and brain-inspired artificial intelligence. Springer, Berlin
Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Lancaster JL, Tordesillas-Gutiérrez D, Martinez M, Salinas F, Evans A, Zilles K, Mazziotta JC, Fox PT (2007) Bias between MNI and Talairach coordinates analyzed using the ICBM-152 brain template. Hum Brain Map 28(11):1194–1205
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436
Liang Z, Shao J, Zhang D, Gao L (2019) Traffic sign detection and recognition based on pyramidal convolutional networks. Neural Comput Appl, 1–11
Liu L, Chen S, Zhang F, Wu FX, Pan Y, Wang J (2019) Deep convolutional neural network for automatically segmenting acute ischemic stroke lesion in multi-modality MRI. Neural Comput Appl, pp 1–14
Lotte F, Bougrain L, Cichocki A, Clerc M, Congedo M, Rakotomamonjy A, Yger F (2018) A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update. J Neural Eng 15(3):031005
Lotte F, Jeunet C (2017) Online classification accuracy is a poor metric to study mental imagery-based BCI user learning: an experimental demonstration and new metrics. In: 7th international BCI conference, pp hal–01519478
Ma X, Liu W, Tao D, Zhou Y (2019) Ensemble p-laplacian regularization for scene image recognition. Cogn Comput 11(6):841–854
Malmivuo J, Plonsey R (1995) Bioelectromagnetism, principles and applications of bioelectric and biomagnetic fields. Oxford University Press, New York
Mammone N, Ieracitano C, Morabito FC (2020) A deep CNN approach to decode motor preparation of upper limbs from time-frequency maps of EEG signals at source level. Neural Netw 124:357–372
Mammone N, Morabito FC (2014) Enhanced automatic wavelet independent component analysis for electroencephalographic artifact removal. Entropy 16(12):6553–6572
Montavon G, Samek W, Müller KR (2018) Methods for interpreting and understanding deep neural networks. Digit Signal Proc 73:1–15
Müller-Putz GR, Schwarz A, Pereira J, Ofner P (2016) From classic motor imagery to complex movement intention decoding: the noninvasive Graz-BCI approach. Prog Brain Res 228:39–70 Elsevier
Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 807–814
Nunez PL, Srinivasan R (2006) Electric fields of the brain, the neurophysics of EEG. Oxford University Press, New York
Ofner P, Schwarz A, Pereira J, Müller-Putz GR (2017) Upper limb movements can be decoded from the time-domain of low-frequency EEG. PLoS ONE 12(8):e0182578
Ofner P, Schwarz A, Pereira J, Wyss D, Wildburger R, Müller-Putz GR (2019) Attempted arm and hand movements can be decoded from low-frequency EEG from persons with spinal cord injury. Sci Rep 9(1):7134
Powers DM (2011) Evaluation: from precision, recall and f-measure to ROC, informedness, markedness and correlation. J Mach Learn Technol 2(1):37–63
Ramirez-Quintana JA, Madrid-Herrera L, Chacon-Murguia MI, Corral-Martinez LF (2020) Brain–computer interface system based on p300 processing with convolutional neural network, novel speller, and low number of electrodes. Cogn Comput, pp 1–17
Roy Y, Banville H, Albuquerque I, Gramfort A, Falk TH, Faubert J (2019) Deep learning-based electroencephalography analysis: a systematic review. J Neural Eng 16(5):051001
Scherer D, Müller A, Behnke S (2010) Evaluation of pooling operations in convolutional architectures for object recognition. In: International conference on artificial neural networks. Springer, pp 92–101
Shakeel A, Navid MS, Anwar MN, Mazhar S, Jochumsen M, Niazi IK (2015) A review of techniques for detection of movement intention using movement-related cortical potentials. Comput Math Methods Med, p 346217
Sun X, Lv M (2019) Facial expression recognition based on a hybrid model combining deep and shallow features. Cogn Comput 11(4):587–597
Tadel F, Baillet S, Mosher JC, Pantazis D, Leahy RM (2011) Brainstorm: a user-friendly application for MEG/EEG analysis. Comput Intell Neurosci, p 8
Tang W, Zou D, Yang S, Shi J, Dan J, Song G (2020) A two-stage approach for automatic liver segmentation with faster R-CNN and deeplab. Neural Comput Appl, pp 1–10
Vato A, Bonzano L, Chiappalone M, Cicero S, Morabito F, Novellino A, Stillo G (2004) Spike manager: a new tool for spontaneous and evoked neuronal networks activity characterization. Neurocomputing 58:1153–1161
Vecchiato G, Del Vecchio M, Ascari L, Antopolskiy S, Deon F, Kubin L, Ambeck-Madsen J, Rizzolatti G, Avanzini P (2019) Electroencephalographic time-frequency patterns of braking and acceleration movement preparation in car driving simulation. Brain Res 1716:16–26
Wagstaff K, Cardie C, Rogers S, Schrödl S et al (2001) Constrained k-means clustering with background knowledge. Icml 1:577–584
Wang Z, Healy G, Smeaton AF, Ward TE (2020) Use of neural signals to evaluate the quality of generative adversarial network performance in facial image generation. Cognit Comput 12(1):13–24
Yue Z, Gao F, Xiong Q, Wang J, Huang T, Yang E, Zhou H (2019) A novel semi-supervised convolutional neural network method for synthetic aperture radar image recognition. Cognit Comput, pp 1–12
Zama T, Shimada S (2015) Simultaneous measurement of electroencephalography and near-infrared spectroscopy during voluntary motor preparation. Sci Rep 5(1):1–9
Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: European conference on computer vision. Springer, pp 818–833
Zeng H, Song A (2016) Optimizing single-trial EEG classification by stationary matrix logistic regression in brain–computer interface. IEEE Trans Neural Netw Learn Syst 27(11):2301–2313
Zheng J, Cai F, Chen W, Feng C, Chen H (2019) Hierarchical neural representation for document classification. Cogn Comput 11(2):317–327
Funding
This work was co-funded by the European Commission, the European Social Fund and the Calabria Region (code: C39B18000080002). The authors are the only responsible for this publication and the European Commission and the Calabria Region decline any responsibility for the use that may be made of the information in it held. This work was also supported by the UK Engineering and Physical Sciences Research Council (EPSRC) (EP/M026981/1, EP/T021063/1, EP/T024917/1).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Availability of data and materials
Data are publicly available from the BNCI Horizon 2020 database at http://bnci-horizon-2020.eu/database/data-sets (Accession Number 001-2017).
Code availability
Custom code will be available on request to the corresponding author.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ieracitano, C., Mammone, N., Hussain, A. et al. A novel explainable machine learning approach for EEG-based brain-computer interface systems. Neural Comput & Applic 34, 11347–11360 (2022). https://doi.org/10.1007/s00521-020-05624-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-020-05624-w