Deep Learning for EEG Motor Imagery-Based Cognitive Healthcare
Electroencephalography (EEG) motor imagery signals have recently gained significant attention due to its ability to encode a person’s intent to perform an action. Researchers have used motor imagery signals to help disabled persons control devices, such as wheelchairs and even autonomous vehicles. Hence, the accurate decoding of these signals is important to brain–computer interface (BCI) systems. Such motor imagery-based BCI systems can become an integral part of cognitive modules that are increasingly being used in smart city frameworks. However, the classification and recognition of EEG have consistently been a challenge due to its dynamic time series data and low signal-to-noise ratio. Deep learning methods, such as the convolution neural network (CNN), have achieved remarkable success in computer vision tasks. Considering the limited applications of deep learning for motor imagery EEG classification, this work focuses on developing CNN-based deep learning methods for such purpose. We propose a multiple-CNN feature fusion architecture to extract and fuse features by using subject-specific frequency bands. CNN has been designed with variable filter sizes and split convolutions for the extraction of spatial and temporal information from raw EEG data. A feature fusion technique based on autoencoders is applied. Cross-encoding technique has been proposed and is successfully used to train autoencoders for a novel cross-subject information transfer and augmenting EEG data. This proposed method outperforms the state-of-the-art four-class motor imagery classification methods for subject-specific and cross-subject data. Autoencoder cross-encoding helps to learn subject invariant and generic features for EEG data and achieves more than 10% increase on cross-subject classification results. The fusion approaches show the potential of applying multiple CNN feature fusion techniques for the advancement of EEG-related research.
KeywordsMotor imagery EEG classification Deep learning Convolution neural network Multi-CNNs feature fusion
Unable to display preview. Download preview PDF.
The authors extend their appreciation to the International Scientific Partnership Program ISPP at King Saud University for funding this research work through ISPP-121.
- 7.M.S. Hossain et al., Applying Deep Learning for Epilepsy Seizure Detection and Brain Mapping Visualization. ACM Trans. Multimedia Comput. Commun. Appl. (ACM TOMM) 14(5), 10 (2018). 16 pagesGoogle Scholar
- 8.M. Alhussein et al., Cognitive IoT-cloud integration for smart healthcare: case study for epileptic seizure detection and monitoring. Mobile Netw. Appl., 1–12 (2018)Google Scholar
- 10.L.J. Greenfield, J.D. Geyer, P.R. Carney, Reading EEGs: A practical approach (Lippincott Williams & Wilkins, Philadelphia, PA, 2012)Google Scholar
- 13.L. Tonin, T. Carlson, R. Leeb, J. d. R. Millán, Brain-controlled telepresence robot by motor-disabled people, in 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE, Honolulu, HI, 2011), pp. 4227–4230Google Scholar
- 17.A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Proces. Syst., 1097–1105 (2012)Google Scholar
- 18.Y. LeCun and Y. Bengio, Convolutional networks for images, speech, and time series, in MA Arbib The Handbook of Brain Theory and Neural Networks, MIT PressCambridge, MA 3361, 10, p. 1995, 1995Google Scholar
- 21.Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, Greedy layer-wise training of deep networks. Adv. Neural Inf. Proces. Syst., 153–160 (2007)Google Scholar
- 25.X. Zhang, L. Yao, Q.Z. Sheng, S.S. Kanhere, T. Gu, D. Zhang, Converting your thoughts to texts: Enabling brain typing via deep feature learning of eeg signals, in 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom), (IEEE, 2018), pp. 1–10Google Scholar
- 30.F. Lotte, A tutorial on EEG signal-processing techniques for mental-state recognition in brain–computer interfaces, in Guide to Brain-Computer Music Interfacing, ed. by E. R. Miranda, J. Castet, (Springer, Heidelberg, 2014), pp. 133–161Google Scholar
- 33.P. Bashivan, I. Rish, M. Yeasin, and N. Codella, Learning representations from EEG with deep recurrent-convolutional neural networks, in CoRR, vol. abs/1511.06448, 2015Google Scholar
- 36.S. Stober, Learning discriminative features from electroencephalography recordings by encoding similarity constraints, in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (IEEE, 2017), pp. 6175–6179Google Scholar
- 38.P. Thodoroff, J. Pineau, and A. Lim, Learning robust features using deep learning for automatic seizure detection, in Machine Learning for Healthcare Conference, 2016, pp. 178–190Google Scholar
- 40.K.K. Ang, Z.Y. Chin, H. Zhang, C. Guan, Filter bank common spatial pattern (FBCSP) in brain-computer interface, in 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), (IEEE, 2008), pp. 2390–2397Google Scholar
- 41.C. Brunner, R. Leeb, G. Müller-Putz, A. Schlögl, G. Pfurtscheller, BCI Competition 2008–Graz data set A, vol 16 (Institute for Knowledge Discovery (Laboratory of Brain-Computer Interfaces), Graz University of Technology, 2008)Google Scholar
- 42.W. Wang, Y. Huang, Y. Wang, L. Wang, Generalized autoencoder: A neural network framework for dimensionality reduction. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 490–497 (2014)Google Scholar
- 45.S. Sakhavi, C. Guan, S. Yan, Learning temporal information for brain-computer interface using convolutional neural networks. IEEE Transactions on Neural Networks and Learning Systems (99), 1–11 (2018)Google Scholar