Electrocorticography based motor imagery movements classification using long short-term memory (LSTM) based on deep learning approach
- 161 Downloads
Brain–computer interface (BCI) is an important alternative for disabled people that enables the innovative communication pathway among individual thoughts and different assistive appliances. In order to make an efficient BCI system, different physiological signals from the brain have been utilized for instances, steady-state visual evoked potential, motor imagery, P300, movement-related potential and error-related potential. Among these physiological signals, motor imagery is widely used in almost all BCI applications. In this paper, Electrocorticography (ECoG) based motor imagery signal has been classified using long short-term memory (LSTM). ECoG based motor imagery data has been taken from BCI competition III, dataset I. The proposed LSTM approach has achieved the classification accuracy of 99.64%, which is the utmost accuracy in comparison with other state-of-art methods that have employed the same data set.
KeywordsElectrocorticography (ECoG) Motor imagery (MI) Long short-term memory (LSTM) Brain computer interfaces (BCI) Deep learning
In the present day, the fast growth of machine learning and deep learning approaches enable the investigation of biological signals which happens to be a notable research topic. Professionals have been exploring to figure out and categorize the distinctive biological signals for medical and non-medical application [1, 2]. Invasive and non-invasive strategies are used to capture the biological signals from brain of the human beings. There are two invasive strategies are employed in BCI research, specifically electrocorticography (ECoG) and intracortical neuron recording. In ECoG recording, electrodes are attached on the exterior of the cortex whereas the electrodes are implanted interior the cortex in intracortical neuron recording . The broadly used non-invasive modalities consist of Electroencephalography (EEG), Magnetoencephalography (MEG), Near-Infrared Spectroscopy (NIRS) and Functional Magnetic Resonance Imaging (fMRI).
The trends of mind wave should be decoded in such a manner that persons may able to modulate and interpret their thoughts to deal with a BCI system . These signals are regarded as control signals in BCIs. The widely used physiological control signals are steady state visual evoked potential (SSVEP), slow cortical potentials (SCP), P300 evoked potentials, and motor imagery signal.
In the MI-based BCI framework, the imagination of hand or tongue movement activities is going to be supported by a circumscribed event-related synchronization/desynchronization . In the last few years, a noticeable number of studies have been carried out relating to the MI signal classification. In , ECoG features in terms of band powers (BP) have been extracted and the selected features have been classified using probabilistic neural network (PNN). Erdem Erkan and Ismail Kurnaz proposed a new technique to find optimal channel set namely arc detection algorithm (ADM) in . They used discrete wavelet transform (DWT) for the purpose of feature extraction and to classify the extracted features, support vector machine (SVM), K-nearest neighbors (K-NN) and linear discriminant analysis (LDA) were used. They obtained the highest accuracy of 95% in the classification of ECoG data (BCI Competition III, dataset I). Aswinseshadri. K et al.  employed wavelet packet tree (WPT) on the same dataset to extract features. Authors applied information gain, mutual information and genetic algorithm (GA) to decide on the best-suited features set and then classified these features using K-NN and Naïve Bayes. Feature selection using GA achieved the highest accuracy. A novel feature extraction technique known as Renyi entropy has been implemented in  where they used BLDA to classify the same dataset. Continuous wavelet transform (CWT) based approach has been recommended in . Authors employed PCA to trim down the dimensionality of features set and then implemented LDA, K-NN and SVM to classify the ECoG data. Authors in  extracted features using statistical properties of the bispectrum of ECoG data and then classified the features by K-NN. Zheng et al.  extracted the time–frequency features by the modified S-transform (MST) algorithm, and then the extracted features are classified using SVM. In reference to , authors proposed Hilbert-Huang transform and common spatial subspace decomposition (HCSSD) algorithm to extract time–frequency features and then learning vector quantization neural network (LVQ-NN) was employed to classify the selected features. Chang et al.  employed stockwell transform (ST) and GA for feature extraction and selection respectively and finally Bayesian LDA (BLDA) was used to classify the selected features. Another study in , continuous wavelet transform (CWT) and K-NN are used as features and classifier respectively to classify the ECoG MI data.
Most of the conventional machine learning approaches described in above paragraph have been extracted different features. The performance of the machine learning approaches is highly affected by the feature extraction techniques. The higher dimensionality of the extracted feature set is another computational complexity for the conventional machine learning approaches. In such cases, a variety of feature reduction algorithms should be employed. To avoid the complexity of feature extraction and feature reduction, we have proposed a LSTM based deep learning approach where no need to employ any feature extraction and reduction framework. In this approach, raw ECoG data have been employed to the LSTM model.
In this article, two classes motor imagery ECoG has been classified using LSTM based deep learning approach. The remaining parts of the article has been organized in the following sections i.e. Sections 2 and 3 discusses issues related to methodology, results and discussion respectively; finally, Sect. 4 deals with the conclusion.
2.1 Details about ECoG data
ECoG data has been recorded after 0.5 s from ending of visual cue. This strategy has been followed due to avoid the effect of visually evoked potentials in the ECoG data. The train and test dataset have been captured in two distinct periods when the subjects experienced different mental states. Due to capture data in two distinctive sessions, it is very challenging to classify this dataset. Thus, the classification algorithm should have the ability to classify such dataset which has been captured in distinct sessions.
2.2 LSTM network
The matrices W, R, and b happen to be concatenations of the input weights, the recurrent weights, and the bias of each component, respectively.
2.3 Structural outline of proposed approach
2.4 Performance evaluation metrics
3 Results and discussion
Performance evaluation of proposed approach
Performance evaluation metrics
Performance and methodology comparison of BCI competition III dataset I
K-NN, LDA, SVM
Highest classification accuracy of 95% (SVM)
Highest classification accuracy of 86%
Naïve Bayes, K-NN
Highest classification accuracy 92.45% (Naïve Bayes)
Classification accuracy 91%
K-NN, LDA, SVM
Highest classification accuracy of 92% (SVM)
Classification accuracy 87%
Classification accuracy 95%
Classification accuracy 92%
Classification accuracy 96%
K-NN, LDA, SVM
Highest classification accuracy of 95% (K-NN)
Classification accuracy 99.64%
In this study, two classes of motor imagery ECoG data has been classified. The deep learning-based LSTM approach has been employed in the purpose of classification. The performance of the proposed model has been compared with other previous related studies. The proposed method outperforms state-of-art techniques. Due to the non-stationary nature of ECoG data, it is very challenging to process this data for BCI applications. In conventional machine learning techniques, feature extraction, selection and reduction strategies have been employed to boost the performance. However, these supplementary analyses sometimes create algorithm complexity. Hence, deep learning algorithms get started to prove their possibilities to handle the non-stationary nature of ECoG and EEG data and open new opportunities in BCI research.
This work is funded by the Universiti Malaysia Pahang, Malaysia through the Research Grant, RDU180396.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
- 2.M. Rashid, N. Sulaiman, M. Mustafa, S. Khatun, B.S. Bari, The Classification of EEG Signal Using Different Machine Learning Techniques for BCI Application, in: S.-M.L. Jong-Hwan, KimHyung Myung (Ed.), Robot Intell. Technol. Appl. RiTA 2018. Commun. Comput. Inf. Sci. Vol 1015, Springer, Singapore, 2019: pp. 207–221. https://doi.org/10.1007/978-981-13-7780-8_17 CrossRefGoogle Scholar
- 5.Zhao H, Liu C, Wang H, Li C (2010) Classifying ECoG signals using probabilistic neural network. In: 2010 WASE international conference on information engineering. IEEE, 2010, pp 77–80. https://doi.org/10.1109/ICIE.2010.26
- 6.Aswinseshadri K, Bai VT (2015) Feature selection in brain computer interface using genetics method. In: 2015 IEEE Int Conf Comput Inf Technol Ubiquitous Comput Commun Dependable Auton Secur Comput Pervasive Intell Comput. IEEE, pp 270–275. https://doi.org/10.1109/CIT/IUCC/DASC/PICOM.2015.39
- 8.Islam MR, Fatema U, Bhuiyan MIH, Bashar SK (2016) Classification of electrocorticography based motor imagery movements using continuous wavelet transform. In: 2016 IEEE students’ technology symposium. IEEE, pp 13–17. https://doi.org/10.1109/TechSym.2016.7872647
- 9.Paul S, Zabir I, Sarker T, Fattah SA, Shahnaz C (2017) Higher order statistics of bispectrum and MRP of ECoG signals for motor imagery tasks classification. In: 2017 IEEE region 10 symposium. IEEE, pp 1–4. https://doi.org/10.1109/TENCONSpring.2017.8070109
- 10.Zheng W, Xu F, Shu M, Zhang Y, Yuan Q, Lian J, Zheng Y (2019) Classification of motor imagery electrocorticogram signals for brain–computer interface. In: 2019 9th international IEEE/EMBS conference on neural engineering. IEEE, pp 530–533. https://doi.org/10.1109/NER.2019.8716963
- 13.Aydemir O, Kayikcioglu T (2011) Wavelet transform based classification of invasive brain computer interface data. Radioengineering 20:31–38Google Scholar
- 15.BCI Competition III (2004) http://www.bbci.de/competition/iii/. Accessed 13 Nov 2019
- 19.Precision, recall, sensitivity and specificity | Ubershmekel’s Uberpython Pythonlog (2012) https://uberpython.wordpress.com/2012/01/01/precision-recall-sensitivity-and-specificity/