Advertisement

Single Trial P300 Classification Using Convolutional LSTM and Deep Learning Ensembles Method

  • Raviraj JoshiEmail author
  • Purvi Goel
  • Mriganka Sur
  • Hema A. Murthy
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11278)

Abstract

The odd ball paradigm is a commonly used approach to develop Brain Computer Interfaces (BCIs). EEG signals have shown to elicit a positive deflection known as the P300 event related potential during odd ball experiments. BCIs based on these experiments rely on detection of the P300 potential. EEG signals are noisy, and therefore P300 detection is performed on an average of multiple trials, thus making them inappropriate for BCI applications. We propose a neural network model based on Convolutional Long Short Term Memory (ConvLSTM) for single trial P300 classification. EEG data encodes both spatial and temporal information using multiple EEG sensors. Convolutional neural networks (CNNs) have been known to capture spatial information whereas LSTMs are known to capture temporal information. Our experiments show that the proposed method outperforms previous CNN based approaches on raw EEG signals. The approaches were evaluated on publicly available dataset II of BCI competition III. Another dataset was recorded locally using audio beeps as stimuli to validate these approaches. The ensemble models based on CNNs and ConvLSTM are also proposed. These models perform better than individual architectures.

Keywords

Brain-computer interface Event related potential P300 Convolutional neural networks Convolutional long short term memory 

References

  1. 1.
    Amazon ec2 - p2 instances. https://aws.amazon.com/ec2/instance-types/p2/. Accessed 09 Jan 2018
  2. 2.
    The geodesic sensor net. https://www.egi.com/research-division/geodesic-sensor-net. Accessed 9 Jan 2018
  3. 3.
    Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. OSDI 16, 265–283 (2016)Google Scholar
  4. 4.
    Barsim, K.S., Zheng, W., Yang, B.: Ensemble learning to EEG-based brain computer interfaces with applications on P300-spellersGoogle Scholar
  5. 5.
    Bashivan, P., Rish, I., Yeasin, M., Codella, N.: Learning representations from EEG with deep recurrent-convolutional neural networks (2015). arXiv preprint. arXiv:1511.06448
  6. 6.
    Carabez, E., Sugi, M., Nambu, I., Wada, Y.: Convolutional neural networks with 3D input for P300 identification in auditory brain-computer interfaces. Comput. Intell. Neurosci. 2017 (2017)Google Scholar
  7. 7.
    Cecotti, H., Graser, A.: Convolutional neural networks for P300 detection with application to brain-computer interfaces. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 433–445 (2011)CrossRefGoogle Scholar
  8. 8.
    Chollet, F., et al.: Keras (2015). https://github.com/fchollet/keras
  9. 9.
    Delorme, A., Makeig, S.: Eeglab: an open source toolbox for analysis of single-trial eeg dynamics including independent component analysis. J. Neurosci. Methods 134(1), 9–21 (2004)CrossRefGoogle Scholar
  10. 10.
    Farwell, L.A., Donchin, E.: Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70(6), 510–523 (1988)CrossRefGoogle Scholar
  11. 11.
    Fedjaev, J.: Decoding EEG brain signals using recurrent neural networks (2017)Google Scholar
  12. 12.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014). arXiv preprint arXiv:1412.6980
  13. 13.
    Krusienski, D., Schalk, G.: Wadsworth BCI dataset (P300 evoked potentials), BCI competition III challenge (2004). http://www.bbci.de/competition/iii/
  14. 14.
    Li, Y., Long, J., Yu, T., Yu, Z., Wang, C., Zhang, H., Guan, C.: An EEG-based BCI system for 2-D cursor control by combining mu/beta rhythm and P300 potential. IEEE Trans. Biomed. Eng. 57(10), 2495–2505 (2010)CrossRefGoogle Scholar
  15. 15.
    Liu, M., Wu, W., Gu, Z., Yu, Z., Qi, F., Li, Y.: Deep learning based on batch normalization for P300 signal detection. Neurocomputing 275, 288–297 (2018)CrossRefGoogle Scholar
  16. 16.
    Maddula, R., Stivers, J., Mousavi, M., Ravindran, S., de Sa, V.: Deep recurrent convolutional neural networks for classifying P300 BCI signals. In: Proceedings of the Graz BCI Conference (2017)Google Scholar
  17. 17.
    Mazurowski, M.A., Habas, P.A., Zurada, J.M., Lo, J.Y., Baker, J.A., Tourassi, G.D.: Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance. Neural Netw. 21(2–3), 427–436 (2008)CrossRefGoogle Scholar
  18. 18.
    Naderi, M.A., Mahdavi-Nasab, H.: Analysis and classification of EEG signals using spectral analysis and recurrent neural networks. In: 17th Iranian Conference of Biomedical Engineering (ICBME), pp. 1–4. IEEE (2010)Google Scholar
  19. 19.
    Petrosian, A., Prokhorov, D., Homan, R., Dasheiff, R., Wunsch II, D.: Recurrent neural network based prediction of epileptic seizures in intra-and extracranial EEG. Neurocomputing 30(1–4), 201–218 (2000)CrossRefGoogle Scholar
  20. 20.
    Rakotomamonjy, A., Guigue, V.: BCI competition III: dataset II-ensemble of SVMs for BCI P300 speller. IEEE Trans. Biomed. Eng. 55(3), 1147–1154 (2008)CrossRefGoogle Scholar
  21. 21.
    Rebsamen, B., Guan, C., Zhang, H., Wang, C., Teo, C., Ang, M.H., Burdet, E.: A brain controlled wheelchair to navigate in familiar environments. IEEE Trans. Neural Syst. Rehabil. Eng. 18(6), 590–598 (2010)CrossRefGoogle Scholar
  22. 22.
    Sainath, T.N., Vinyals, O., Senior, A., Sak, H.: Convolutional, long short-term memory, fully connected deep neural networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4580–4584. IEEE (2015)Google Scholar
  23. 23.
    Sun, S., Zhang, C., Zhang, D.: An experimental evaluation of ensemble methods for eeg signal classification. Pattern Recognit. Lett. 28(15), 2157–2163 (2007)CrossRefGoogle Scholar
  24. 24.
    Xingjian, S., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., Woo, W.c.: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. In: Advances in neural information processing systems, pp. 802–810 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Raviraj Joshi
    • 1
    Email author
  • Purvi Goel
    • 1
  • Mriganka Sur
    • 2
  • Hema A. Murthy
    • 1
  1. 1.Indian Institute of Technology MadrasChennaiIndia
  2. 2.Department of Brain and Cognitive SciencesMassachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations