Skip to main content
SpringerLink
Log in

A novel explainable machine learning approach for EEG-based brain-computer interface systems

  • S.I. : Healthcare Analytics
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Electroencephalographic (EEG) recordings can be of great help in decoding the open/close hand’s motion preparation. To this end, cortical EEG source signals in the motor cortex (evaluated in the 1-s window preceding movement onset) are extracted by solving inverse problem through beamforming. EEG sources epochs are used as source-time maps input to a custom deep convolutional neural network (CNN) that is trained to perform 2-ways classification tasks: pre-hand close (HC) versus resting state (RE) and pre-hand open (HO) versus RE. The developed deep CNN works well (accuracy rates up to \(89.65 \pm 5.29\%\) for HC versus RE and \(90.50 \pm 5.35\%\) for HO versus RE), but the core of the present study was to explore the interpretability of the deep CNN to provide further insights into the activation mechanism of cortical sources during the preparation of hands’ sub-movements. Specifically, occlusion sensitivity analysis was carried out to investigate which cortical areas are more relevant in the classification procedure. Experimental results show a recurrent trend of spatial cortical activation across subjects. In particular, the central region (close to the longitudinal fissure) and the right temporal zone of the premotor together with the primary motor cortex appear to be primarily involved. Such findings encourage an in-depth study of cortical areas that seem to play a key role in hand’s open/close preparation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Bengio Y (2012) Practical recommendations for gradient-based training of deep architectures. In: Neural networks: tricks of the trade, pp 437–478

  2. Bi X, Zhao X, Huang H, Chen D, Ma Y (2020) Functional brain network classification for alzheimer’s disease detection with deep features and extreme learning machine. Cogn Comput 12(3):513–527

    Article  Google Scholar 

  3. Brodmann K (2007) Brodmann’s: localisation in the cerebral cortex. Springer, Berlin

    Google Scholar 

  4. Cai Z, Shao L (2019) Rgb-d scene classification via multi-modal feature learning. Cogn Comput 11(6):825–840

    Article  Google Scholar 

  5. Catani M (2017) A little man of some importance. Brain 140(11):3055–3061

    Article  Google Scholar 

  6. Chen H, Ding G, Lin Z, Guo Y, Shan C, Han J (2019) Image captioning with memorized knowledge. Cogn Comput 1–14

  7. Chen X, Xie H (2020) A structural topic modeling-based bibliometric study of sentiment analysis literature. Cogn Comput 1–33

  8. Craik A, He Y, Contreras-Vidal JL (2019) Deep learning for electroencephalogram (EEG) classification tasks: a review. J Neural Eng 16(3):031001

    Article  Google Scholar 

  9. Doborjeh ZG, Doborjeh MG, Kasabov N (2018) Attentional bias pattern recognition in spiking neural networks from spatio-temporal EEG data. Cogn Comput 10(1):35–48

    Article  Google Scholar 

  10. Edelman BJ, Baxter B, He B (2015) EEG source imaging enhances the decoding of complex right-hand motor imagery tasks. IEEE Trans Biomed Eng 63(1):4–14

    Article  Google Scholar 

  11. Fellous JM, Sapiro G, Rossi A, Mayberg HS, Ferrante M (2019) Explainable artificial intelligence for neuroscience: behavioral neurostimulation. Front Neurosci 13:1346

    Article  Google Scholar 

  12. Gao F, Huang T, Sun J, Wang J, Hussain A, Yang E (2019) A new algorithm for sar image target recognition based on an improved deep convolutional neural network. Cogn Comput 11(6):809–824

    Article  Google Scholar 

  13. Goshvarpour A, Goshvarpour A (2019) A novel approach for EEG electrode selection in automated emotion recognition based on lagged poincare’s indices and sloreta. Cogn Comput 1–17

  14. Grech R, Cassar T, Muscat J, Camilleri KP, Fabri SG, Zervakis M, Xanthopoulos P, Sakkalis V, Vanrumste B (2008) Review on solving the inverse problem in EEG source analysis. J Neuroeng Rehabilit 5(1):25

    Article  Google Scholar 

  15. Grosse-Wentrup M, Liefhold C, Gramann K, Buss M (2009) Beamforming in noninvasive brain–computer interfaces. IEEE Trans Biomed Eng 56(4):1209–1219

    Article  Google Scholar 

  16. Hallez H, Vanrumste B, Grech R, Muscat J, De Clercq W, Vergult A, D’Asseler Y, Camilleri KP, Fabri SG, Van Huffel S et al (2007) Review on solving the forward problem in EEG source analysis. J Neuroeng Rehabilit 4(1):46

    Article  Google Scholar 

  17. Haufe S, Ewald A (2016) A simulation framework for benchmarking EEG-based brain connectivity estimation methodologies. Brain Topogr, pp 1–18

  18. Haufe S, Huang Y, Parra LC (2015) A highly detailed FEM volume conductor model based on the ICBM152 average head template for EEG source imaging and TCS targeting. In: Conference poceedings of IEEE engineering medicine and biological society

  19. Hermes D, Vansteensel MJ, Albers AM, Bleichner MG, Benedictus MR, Orellana CM, Aarnoutse E, Ramsey N (2011) Functional MRI-based identification of brain areas involved in motor imagery for implantable brain–computer interfaces. J Neural Eng 8(2):025007

    Article  Google Scholar 

  20. Ieracitano C, Mammone N, Bramanti A, Hussain A, Morabito FC (2019) A convolutional neural network approach for classification of dementia stages based on 2d-spectral representation of EEG recordings. Neurocomputing 323:96–107

    Article  Google Scholar 

  21. Ieracitano C, Mammone N, Hussain A, Morabito FC (2020) A novel multi-modal machine learning based approach for automatic classification of EEG recordings in dementia. Neural Netw 123:176–190

    Article  Google Scholar 

  22. Kantak SS, Stinear JW, Buch ER, Cohen LG (2012) Rewiring the brain: potential role of the premotor cortex in motor control, learning, and recovery of function following brain injury. Neurorehabilit Neural Repair 26(3):282–292

    Article  Google Scholar 

  23. Kasabov NK (2014) Neucube: a spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data. Neural Netw 52:62–76

    Article  Google Scholar 

  24. Kasabov NK (2019) Time-space, spiking neural networks and brain-inspired artificial intelligence. Springer, Berlin

    Book  Google Scholar 

  25. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  26. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  27. Lancaster JL, Tordesillas-Gutiérrez D, Martinez M, Salinas F, Evans A, Zilles K, Mazziotta JC, Fox PT (2007) Bias between MNI and Talairach coordinates analyzed using the ICBM-152 brain template. Hum Brain Map 28(11):1194–1205

    Article  Google Scholar 

  28. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  29. Liang Z, Shao J, Zhang D, Gao L (2019) Traffic sign detection and recognition based on pyramidal convolutional networks. Neural Comput Appl, 1–11

  30. Liu L, Chen S, Zhang F, Wu FX, Pan Y, Wang J (2019) Deep convolutional neural network for automatically segmenting acute ischemic stroke lesion in multi-modality MRI. Neural Comput Appl, pp 1–14

  31. Lotte F, Bougrain L, Cichocki A, Clerc M, Congedo M, Rakotomamonjy A, Yger F (2018) A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update. J Neural Eng 15(3):031005

    Article  Google Scholar 

  32. Lotte F, Jeunet C (2017) Online classification accuracy is a poor metric to study mental imagery-based BCI user learning: an experimental demonstration and new metrics. In: 7th international BCI conference, pp hal–01519478

  33. Ma X, Liu W, Tao D, Zhou Y (2019) Ensemble p-laplacian regularization for scene image recognition. Cogn Comput 11(6):841–854

    Article  Google Scholar 

  34. Malmivuo J, Plonsey R (1995) Bioelectromagnetism, principles and applications of bioelectric and biomagnetic fields. Oxford University Press, New York

    Book  Google Scholar 

  35. Mammone N, Ieracitano C, Morabito FC (2020) A deep CNN approach to decode motor preparation of upper limbs from time-frequency maps of EEG signals at source level. Neural Netw 124:357–372

    Article  Google Scholar 

  36. Mammone N, Morabito FC (2014) Enhanced automatic wavelet independent component analysis for electroencephalographic artifact removal. Entropy 16(12):6553–6572

    Article  Google Scholar 

  37. Montavon G, Samek W, Müller KR (2018) Methods for interpreting and understanding deep neural networks. Digit Signal Proc 73:1–15

    Article  MathSciNet  Google Scholar 

  38. Müller-Putz GR, Schwarz A, Pereira J, Ofner P (2016) From classic motor imagery to complex movement intention decoding: the noninvasive Graz-BCI approach. Prog Brain Res 228:39–70 Elsevier

    Article  Google Scholar 

  39. Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 807–814

  40. Nunez PL, Srinivasan R (2006) Electric fields of the brain, the neurophysics of EEG. Oxford University Press, New York

    Book  Google Scholar 

  41. Ofner P, Schwarz A, Pereira J, Müller-Putz GR (2017) Upper limb movements can be decoded from the time-domain of low-frequency EEG. PLoS ONE 12(8):e0182578

    Article  Google Scholar 

  42. Ofner P, Schwarz A, Pereira J, Wyss D, Wildburger R, Müller-Putz GR (2019) Attempted arm and hand movements can be decoded from low-frequency EEG from persons with spinal cord injury. Sci Rep 9(1):7134

    Article  Google Scholar 

  43. Powers DM (2011) Evaluation: from precision, recall and f-measure to ROC, informedness, markedness and correlation. J Mach Learn Technol 2(1):37–63

    MathSciNet  Google Scholar 

  44. Ramirez-Quintana JA, Madrid-Herrera L, Chacon-Murguia MI, Corral-Martinez LF (2020) Brain–computer interface system based on p300 processing with convolutional neural network, novel speller, and low number of electrodes. Cogn Comput, pp 1–17

  45. Roy Y, Banville H, Albuquerque I, Gramfort A, Falk TH, Faubert J (2019) Deep learning-based electroencephalography analysis: a systematic review. J Neural Eng 16(5):051001

    Article  Google Scholar 

  46. Scherer D, Müller A, Behnke S (2010) Evaluation of pooling operations in convolutional architectures for object recognition. In: International conference on artificial neural networks. Springer, pp 92–101

  47. Shakeel A, Navid MS, Anwar MN, Mazhar S, Jochumsen M, Niazi IK (2015) A review of techniques for detection of movement intention using movement-related cortical potentials. Comput Math Methods Med, p 346217

  48. Sun X, Lv M (2019) Facial expression recognition based on a hybrid model combining deep and shallow features. Cogn Comput 11(4):587–597

    Article  Google Scholar 

  49. Tadel F, Baillet S, Mosher JC, Pantazis D, Leahy RM (2011) Brainstorm: a user-friendly application for MEG/EEG analysis. Comput Intell Neurosci, p 8

  50. Tang W, Zou D, Yang S, Shi J, Dan J, Song G (2020) A two-stage approach for automatic liver segmentation with faster R-CNN and deeplab. Neural Comput Appl, pp 1–10

  51. Vato A, Bonzano L, Chiappalone M, Cicero S, Morabito F, Novellino A, Stillo G (2004) Spike manager: a new tool for spontaneous and evoked neuronal networks activity characterization. Neurocomputing 58:1153–1161

    Article  Google Scholar 

  52. Vecchiato G, Del Vecchio M, Ascari L, Antopolskiy S, Deon F, Kubin L, Ambeck-Madsen J, Rizzolatti G, Avanzini P (2019) Electroencephalographic time-frequency patterns of braking and acceleration movement preparation in car driving simulation. Brain Res 1716:16–26

    Article  Google Scholar 

  53. Wagstaff K, Cardie C, Rogers S, Schrödl S et al (2001) Constrained k-means clustering with background knowledge. Icml 1:577–584

    Google Scholar 

  54. Wang Z, Healy G, Smeaton AF, Ward TE (2020) Use of neural signals to evaluate the quality of generative adversarial network performance in facial image generation. Cognit Comput 12(1):13–24

    Article  Google Scholar 

  55. Yue Z, Gao F, Xiong Q, Wang J, Huang T, Yang E, Zhou H (2019) A novel semi-supervised convolutional neural network method for synthetic aperture radar image recognition. Cognit Comput, pp 1–12

  56. Zama T, Shimada S (2015) Simultaneous measurement of electroencephalography and near-infrared spectroscopy during voluntary motor preparation. Sci Rep 5(1):1–9

    Article  Google Scholar 

  57. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: European conference on computer vision. Springer, pp 818–833

  58. Zeng H, Song A (2016) Optimizing single-trial EEG classification by stationary matrix logistic regression in brain–computer interface. IEEE Trans Neural Netw Learn Syst 27(11):2301–2313

    Article  MathSciNet  Google Scholar 

  59. Zheng J, Cai F, Chen W, Feng C, Chen H (2019) Hierarchical neural representation for document classification. Cogn Comput 11(2):317–327

    Article  Google Scholar 

Download references

Funding

This work was co-funded by the European Commission, the European Social Fund and the Calabria Region (code: C39B18000080002). The authors are the only responsible for this publication and the European Commission and the Calabria Region decline any responsibility for the use that may be made of the information in it held. This work was also supported by the UK Engineering and Physical Sciences Research Council (EPSRC) (EP/M026981/1, EP/T021063/1, EP/T024917/1).

Author information

Authors and Affiliations

  1. DICEAM University Mediterranea of Reggio Calabria, Via Graziella Feo di Vito, 89124, Reggio Calabria, Italy

    Cosimo Ieracitano, Nadia Mammone & Francesco Carlo Morabito

  2. School of Computing, Edinburgh Napier University, Edinburgh, EH10 5DT, Scotland, UK

    Amir Hussain

Authors
  1. Cosimo Ieracitano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nadia Mammone
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amir Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francesco Carlo Morabito
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cosimo Ieracitano.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Availability of data and materials

Data are publicly available from the BNCI Horizon 2020 database at http://bnci-horizon-2020.eu/database/data-sets (Accession Number 001-2017).

Code availability

Custom code will be available on request to the corresponding author.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ieracitano, C., Mammone, N., Hussain, A. et al. A novel explainable machine learning approach for EEG-based brain-computer interface systems. Neural Comput & Applic 34, 11347–11360 (2022). https://doi.org/10.1007/s00521-020-05624-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05624-w

Keywords

Search

Navigation