Deep Learning for EEG Motor Imagery-Based Cognitive Healthcare

  • Syed Umar Amin
  • Mansour Alsulaiman
  • Ghulam Muhammad
  • M. Shamim HossainEmail author
  • Mohsen Guizani


Electroencephalography (EEG) motor imagery signals have recently gained significant attention due to its ability to encode a person’s intent to perform an action. Researchers have used motor imagery signals to help disabled persons control devices, such as wheelchairs and even autonomous vehicles. Hence, the accurate decoding of these signals is important to brain–computer interface (BCI) systems. Such motor imagery-based BCI systems can become an integral part of cognitive modules that are increasingly being used in smart city frameworks. However, the classification and recognition of EEG have consistently been a challenge due to its dynamic time series data and low signal-to-noise ratio. Deep learning methods, such as the convolution neural network (CNN), have achieved remarkable success in computer vision tasks. Considering the limited applications of deep learning for motor imagery EEG classification, this work focuses on developing CNN-based deep learning methods for such purpose. We propose a multiple-CNN feature fusion architecture to extract and fuse features by using subject-specific frequency bands. CNN has been designed with variable filter sizes and split convolutions for the extraction of spatial and temporal information from raw EEG data. A feature fusion technique based on autoencoders is applied. Cross-encoding technique has been proposed and is successfully used to train autoencoders for a novel cross-subject information transfer and augmenting EEG data. This proposed method outperforms the state-of-the-art four-class motor imagery classification methods for subject-specific and cross-subject data. Autoencoder cross-encoding helps to learn subject invariant and generic features for EEG data and achieves more than 10% increase on cross-subject classification results. The fusion approaches show the potential of applying multiple CNN feature fusion techniques for the advancement of EEG-related research.


Motor imagery EEG classification Deep learning Convolution neural network Multi-CNNs feature fusion 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



The authors extend their appreciation to the International Scientific Partnership Program ISPP at King Saud University for funding this research work through ISPP-121.


  1. 1.
    Z. Emami, T. Chau, Investigating the effects of visual distractors on the performance of a motor imagery brain-computer interface. Clin. Neurophysiol. 129(6), 1268–1275 (2018)CrossRefGoogle Scholar
  2. 2.
    F. Lotte et al., A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update. J. Neural Eng. 15(3), 031005 (2018)CrossRefGoogle Scholar
  3. 3.
    A.M. Chiarelli, P. Croce, A. Merla, F. Zappasodi, Deep learning for hybrid EEG-fNIRS brain–computer interface: application to motor imagery classification. J. Neural Eng. 15(3), 036028 (2018)CrossRefGoogle Scholar
  4. 4.
    M.-P. Hosseini, D. Pompili, K. Elisevich, H. Soltanian-Zadeh, Optimized deep learning for EEG big data and seizure prediction BCI via internet of things. IEEE Transactions on Big Data 3(4), 392–404 (2017)CrossRefGoogle Scholar
  5. 5.
    J.R. Wolpaw, N. Birbaumer, D.J. McFarland, G. Pfurtscheller, T.M. Vaughan, Brain–computer interfaces for communication and control. Clin. Neurophysiol. 113(6), 767–791 (2002)CrossRefGoogle Scholar
  6. 6.
    T.M. Vaughan et al., Brain-computer interface technology: a review of the Second International Meeting. IEEE Trans. Neural Syst. Rehabil. Eng. 11(2), 94–109 (2003)CrossRefGoogle Scholar
  7. 7.
    M.S. Hossain et al., Applying Deep Learning for Epilepsy Seizure Detection and Brain Mapping Visualization. ACM Trans. Multimedia Comput. Commun. Appl. (ACM TOMM) 14(5), 10 (2018). 16 pagesGoogle Scholar
  8. 8.
    M. Alhussein et al., Cognitive IoT-cloud integration for smart healthcare: case study for epileptic seizure detection and monitoring. Mobile Netw. Appl., 1–12 (2018)Google Scholar
  9. 9.
    S.U. Amin et al., Cognitive smart healthcare for pathology detection and monitoring. IEEE Access 7, 10745–10753 (2019). CrossRefGoogle Scholar
  10. 10.
    L.J. Greenfield, J.D. Geyer, P.R. Carney, Reading EEGs: A practical approach (Lippincott Williams & Wilkins, Philadelphia, PA, 2012)Google Scholar
  11. 11.
    G. Pfurtscheller, F.L. Da Silva, Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin. Neurophysiol. 110(11), 1842–1857 (1999)CrossRefGoogle Scholar
  12. 12.
    K.K. Ang, Z.Y. Chin, C. Wang, C. Guan, H. Zhang, Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b. Front. Neurosci. 6, 39 (2012)CrossRefGoogle Scholar
  13. 13.
    L. Tonin, T. Carlson, R. Leeb, J. d. R. Millán, Brain-controlled telepresence robot by motor-disabled people, in 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE, Honolulu, HI, 2011), pp. 4227–4230Google Scholar
  14. 14.
    A. Ramos-Murguialday et al., Brain–machine interface in chronic stroke rehabilitation: a controlled study. Ann. Neurol. 74(1), 100–108 (2013)CrossRefGoogle Scholar
  15. 15.
    G. Muhammad et al., Automatic Seizure Detection in a Mobile Multimedia Framework. IEEE Access 6, 45372–45383 (2018)MathSciNetCrossRefGoogle Scholar
  16. 16.
    F. Nijboer et al., A P300-based brain–computer interface for people with amyotrophic lateral sclerosis. Clin. Neurophysiol. 119(8), 1909–1916 (2008)CrossRefGoogle Scholar
  17. 17.
    A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Proces. Syst., 1097–1105 (2012)Google Scholar
  18. 18.
    Y. LeCun and Y. Bengio, Convolutional networks for images, speech, and time series, in MA Arbib The Handbook of Brain Theory and Neural Networks, MIT PressCambridge, MA 3361, 10, p. 1995, 1995Google Scholar
  19. 19.
    H.-C. Shin et al., Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35(5), 1285–1298 (2016)CrossRefGoogle Scholar
  20. 20.
    A. Ghoneim et al., Medical Image Forgery Detection for Smart Healthcare. IEEE Commun. Mag. 56(4), 33–37 (2018). CrossRefGoogle Scholar
  21. 21.
    Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, Greedy layer-wise training of deep networks. Adv. Neural Inf. Proces. Syst., 153–160 (2007)Google Scholar
  22. 22.
    M.S. Hossain et al., Improving consumer satisfaction in smart cities using edge computing and caching: A case study of date fruits classification. Futur. Gener. Comput. Syst. 88, 333–341 (2018)CrossRefGoogle Scholar
  23. 23.
    A. Antoniades et al., Detection of interictal discharges with convolutional neural networks using discrete ordered multichannel intracranial EEG. IEEE Trans. Neural Syst. Rehabil. Eng. 25(12), 2285–2294 (2017)CrossRefGoogle Scholar
  24. 24.
    M. Rawashdeh et al., Reliable service delivery in Tele-health care systems. J. Netw. Comput. Appl. 115, 86–93 (2018)CrossRefGoogle Scholar
  25. 25.
    X. Zhang, L. Yao, Q.Z. Sheng, S.S. Kanhere, T. Gu, D. Zhang, Converting your thoughts to texts: Enabling brain typing via deep feature learning of eeg signals, in 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom), (IEEE, 2018), pp. 1–10Google Scholar
  26. 26.
    P. Mirowski, D. Madhavan, Y. LeCun, R. Kuzniecky, Classification of patterns of EEG synchronization for seizure prediction. Clin. Neurophysiol. 120(11), 1927–1940 (2009)CrossRefGoogle Scholar
  27. 27.
    X. Xie, Z.L. Yu, H. Lu, Z. Gu, Y. Li, Motor imagery classification based on bilinear sub-manifold learning of symmetric positive-definite matrices. IEEE Trans. Neural Syst. Rehabil. Eng. 25(6), 504–516 (2017)CrossRefGoogle Scholar
  28. 28.
    J. Decety, D.H. Ingvar, Brain structures participating in mental simulation of motor behavior: A neuropsychological interpretation. Acta Psychol. 73(1), 13–34 (1990)CrossRefGoogle Scholar
  29. 29.
    K.K. Ang et al., A randomized controlled trial of EEG-based motor imagery brain-computer interface robotic rehabilitation for stroke. Clin. EEG Neurosci. 46(4), 310–320 (2015)CrossRefGoogle Scholar
  30. 30.
    F. Lotte, A tutorial on EEG signal-processing techniques for mental-state recognition in brain–computer interfaces, in Guide to Brain-Computer Music Interfacing, ed. by E. R. Miranda, J. Castet, (Springer, Heidelberg, 2014), pp. 133–161Google Scholar
  31. 31.
    H. Ramoser, J. Muller-Gerking, G. Pfurtscheller, Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans. Rehabil. Eng. 8(4), 441–446 (2000)CrossRefGoogle Scholar
  32. 32.
    F. Lotte, C. Guan, Regularizing common spatial patterns to improve BCI designs: unified theory and new algorithms. IEEE Trans. Biomed. Eng. 58(2), 355–362 (2011)CrossRefGoogle Scholar
  33. 33.
    P. Bashivan, I. Rish, M. Yeasin, and N. Codella, Learning representations from EEG with deep recurrent-convolutional neural networks, in CoRR, vol. abs/1511.06448, 2015Google Scholar
  34. 34.
    Y.R. Tabar, U. Halici, A novel deep learning approach for classification of EEG motor imagery signals. J. Neural Eng. 14(1), 016003 (2016)CrossRefGoogle Scholar
  35. 35.
    H. Cecotti, A. Graser, Convolutional neural networks for P300 detection with application to brain-computer interfaces. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 433–445 (2011)CrossRefGoogle Scholar
  36. 36.
    S. Stober, Learning discriminative features from electroencephalography recordings by encoding similarity constraints, in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (IEEE, 2017), pp. 6175–6179Google Scholar
  37. 37.
    R.T. Schirrmeister et al., Deep learning with convolutional neural networks for EEG decoding and visualization. Hum. Brain Mapp. 38(11), 5391–5420 (2017)CrossRefGoogle Scholar
  38. 38.
    P. Thodoroff, J. Pineau, and A. Lim, Learning robust features using deep learning for automatic seizure detection, in Machine Learning for Healthcare Conference, 2016, pp. 178–190Google Scholar
  39. 39.
    R.T. Canolty et al., High gamma power is phase-locked to theta oscillations in human neocortex. Science 313(5793), 1626–1628 (2006)CrossRefGoogle Scholar
  40. 40.
    K.K. Ang, Z.Y. Chin, H. Zhang, C. Guan, Filter bank common spatial pattern (FBCSP) in brain-computer interface, in 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), (IEEE, 2008), pp. 2390–2397Google Scholar
  41. 41.
    C. Brunner, R. Leeb, G. Müller-Putz, A. Schlögl, G. Pfurtscheller, BCI Competition 2008–Graz data set A, vol 16 (Institute for Knowledge Discovery (Laboratory of Brain-Computer Interfaces), Graz University of Technology, 2008)Google Scholar
  42. 42.
    W. Wang, Y. Huang, Y. Wang, L. Wang, Generalized autoencoder: A neural network framework for dimensionality reduction. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 490–497 (2014)Google Scholar
  43. 43.
    S.U. Amin, M. Alsulaiman, G. Muhammad, M.A. Bencherif, M.S. Hossain, Multilevel weighted feature fusion using convolutional neural networks for EEG motor imagery classification. IEEE Access 7, 18940–18950 (2019). CrossRefGoogle Scholar
  44. 44.
    V.J. Lawhern, A.J. Solon, N.R. Waytowich, S.M. Gordon, C.P. Hung, B.J. Lance, EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. J. Neural Eng. 15(5), 056013 (2018)CrossRefGoogle Scholar
  45. 45.
    S. Sakhavi, C. Guan, S. Yan, Learning temporal information for brain-computer interface using convolutional neural networks. IEEE Transactions on Neural Networks and Learning Systems (99), 1–11 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Syed Umar Amin
    • 1
  • Mansour Alsulaiman
    • 1
  • Ghulam Muhammad
    • 1
  • M. Shamim Hossain
    • 2
    Email author
  • Mohsen Guizani
    • 3
  1. 1.Department of Computer Engineering, College of Computer and Information Sciences (CCIS)King Saud UniversityRiyadhSaudi Arabia
  2. 2.Department of Software Engineering, College of Computer and Information Sciences (CCIS)King Saud UniversityRiyadhSaudi Arabia
  3. 3.Department of Electrical and Computer EngineeringUniversity of IdahoMoscowUSA

Personalised recommendations