Abstract
From early adoption in rehabilitation, the brain–machine interfaces (BMIs) have dovetailed into applications empowering humans in controlling external devices such as prosthesis and wheelchairs with a high level of autonomy. The success of such brain–machine interfaces depends on the decoding algorithms that translate the brain activity into the human’s intention or cognition state. Taking advantage of this decoding, a machine can have a robust perception of human’s cognitive state and modify its actions accordingly. This decoding process can be viewed as a machine learning problem where features of brain activities are mapped to some labeled events or classes in a controlled environment. This mapping traditionally relies on the subject and task-specific signal processing approaches. Thus, the conventional machine learning methods fail to generalize well and transfer the learned features between different tasks and subjects, especially in out-of-the-lab applications. Recently, deep learning (DL) has shown great success in learning the patterns from very large data and generalizing well on different applications. With respect to brain activity analysis, deep learning and reinforcement learning (RL) techniques can significantly simplify analysis pipelines and facilitate better generalization between subjects, tasks, and also learn intricate coupled dynamics. In this regard, this paper provides information on the state of the art and challenges in implementing deep learning and reinforcement learning algorithms in brain–machine interfaces. In order to demonstrate the use of deep learning techniques in BMIs, we also present a case study of physical human–robot interaction where the brain activity is used to classify the task difficulty while interacting with the robot.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
M. Teplan et al., Fundamentals of EEG measurement. Meas. Sci. Rev. 2(2), 1–11 (2002)
R.P.N. Rao, Brain-Computer Interfacing: An Introduction (Cambridge University Press, Cambridge, 2013)
S. Motamedi-Fakhr, M. Moshrefi-Torbati, M. Hill, C.M. Hill, P.R. White, Signal processing techniques applied to human sleep EEG signals—a review. Biomed. Signal Process. Control 10, 21–33 (2014)
K.A. Guru, E.T. Esfahani, S.J. Raza, R. Bhat, K. Wang, Y. Hammond, G. Wilding, J.O. Peabody, A.J. Chowriappa, Cognitive skills assessment during robot-assisted surgery: separating the wheat from the chaff. BJU Int. 115(1), 166–174 (2015)
A.H. Memar, E.T. Esfahani, Physiological measures for human performance analysis in human-robot teamwork: case of tele-exploration. IEEE Access 6, 3694–3705 (2018)
M. Rahman, W. Karwowski, M. Fafrowicz, P.A. Hancock, Neuroergonomics applications of electroencephalography in physical activities: a systematic review. Front. Hum. Neurosci. 13, 182 (2019)
M.-K. Kim, M. Kim, E. Oh, S.-P. Kim, A review on the computational methods for emotional state estimation from the human EEG. Comput. Math. Methods Med. 2013, 13 pp. (2013)
P. Zarjam, J. Epps, N.H. Lovell, Beyond subjective self-rating: EEG signal classification of cognitive workload. IEEE Trans. Auton. Ment. Dev. 7(4), 301–310 (2015)
E.T. Esfahani, V. Sundararajan, Using brain-computer interfaces to detect human satisfaction in human-robot interaction. Int. J. Humanoid Rob. 08(01), 87–101 (2011)
X. Mao, W. Li, C. Lei, J. Jin, F. Duan, S. Chen, A brain–robot interaction system by fusing human and machine intelligence. IEEE Trans. Neural Syst. Rehabil. Eng. 27(3), 533–542 (2019)
G.K. Karavas, D.T. Larsson, P. Artemiadis, A hybrid BMI for control of robotic swarms: preliminary results, in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2017), pp. 5065–5075
K.K. Ang, K.S.G. Chua, K.S. Phua, C. Wang, Z.Y. Chin, C.W.K. Kuah, W. Low, C. Guan, A randomized controlled trial of EEG-based motor imagery brain-computer interface robotic rehabilitation for stroke. Clin. EEG Neurosci. 46(4), 310–320 (2015)
P. Ofner, A. Schwarz, J. Pereira, G. Müller-Putz, Decoding movements of the upper limb from EEG, in cuttingEEG (2017)
N. Bigdely-Shamlo, T. Mullen, C. Kothe, K.-M. Su, K.A. Robbins, The PREP pipeline: standardized preprocessing for large-scale EEG analysis. Front. Neuroinform. 9, 16 (2015)
W. Zhang, C. Tan, F. Sun, H. Wu, B. Zhang, A review of EEG-based brain-computer interface systems design. Brain Sci. Adv. 4(2), 156–167 (2018)
M.X. Cohen, Analyzing Neural Time Series Data: Theory and Practice (MIT Press, Cambridge, 2014)
M.M. Najafabadi, F. Villanustre, T.M. Khoshgoftaar, N. Seliya, R. Wald, E. Muharemagic, Deep learning applications and challenges in big data analytics. J. Big Data 2(1), 1 (2015)
I. Goodfellow, Y. Bengio, A. Courville, Deep Learning (MIT Press, Cambridge, 2016)
R. Yannick, B. Hubert, A. Isabela, G. Alexandre, F. Jocelyn et al., Deep learning-based electroencephalography analysis: a systematic review (2019). Preprint. arXiv:1901.05498
A. Craik, Y. He, J.L. Contreras-Vidal, Deep learning for electroencephalogram (EEG) classification tasks: a review. J. Neural Eng. 16(3), 031001 (2019)
R.S. Sutton, A.G, Barto, Reinforcement Learning: An Introduction (MIT Press, Cambridge, 2018)
E.A. Pohlmeyer, B. Mahmoudi, S. Geng, N.W. Prins, J.C. Sanchez, Using reinforcement learning to provide stable brain-machine interface control despite neural input reorganization. PloS One 9(1), e87253 (2014)
I. Iturrate, R. Chavarriaga, L. Montesano, J. Minguez, J.d.R. Millán, Teaching brain-machine interfaces as an alternative paradigm to neuroprosthetics control. Sci. Rep. 5, 13893 (2015)
E. Imatz-Ojanguren, E. Irigoyen, T. Keller, Reinforcement learning for hand grasp with surface multi-field neuroprostheses, in International Joint Conference (Springer, Berlin, 2016), pp. 313–322
N.W. Prins, J.C. Sanchez, A. Prasad, Feedback for reinforcement learning based brain–machine interfaces using confidence metrics. J. Neural Eng. 14(3), 036016 (2017)
A.H. Memar, E.T. Esfahani, EEG correlates of motor control difficulty in physical human-robot interaction: a frequency domain analysis, in 2018 IEEE Haptics Symposium (HAPTICS) (IEEE, 2018), pp. 229–234
C. Berka, D.J. Levendowski, M.N. Lumicao, A. Yau, G. Davis, V.T. Zivkovic, R.E. Olmstead, P.D. Tremoulet, P.L. Craven, EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat. Space Environ. Med. 78(5), B231–B244 (2007)
R.W. Homan, J. Herman, P. Purdy, Cerebral location of international 10-20 system electrode placement. Electroencephalogr. Clin. Neurophysiol. 66(4), 376–382 (1987)
A. Gramfort, M. Luessi, E. Larson, D.A. Engemann, D. Strohmeier, C. Brodbeck, R. Goj, M. Jas, T. Brooks, L. Parkkonen et al., MEG and EEG data analysis with MNE-python. Front. Neurosci. 7, 267 (2013)
R.T. Schirrmeister, J.T. Springenberg, L.D.J. Fiederer, M. Glasstetter, K. Eggensperger, M. Tangermann, F. Hutter, W. Burgard, T. Ball, Deep learning with convolutional neural networks for EEG decoding and visualization. Hum. Brain Mapp. 38(11), 5391–5420 (2017)
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
K.K. Ang, Z.Y. Chin, H. Zhang, C. Guan, Filter bank common spatial pattern (FBCSP) in brain-computer interface, in 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence) (IEEE, 2008), pp. 2390–2397
Acknowledgment
We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Manjunatha, H., Esfahani, E.T. (2020). Application of Reinforcement and Deep Learning Techniques in Brain–Machine Interfaces. In: Vinjamuri, R. (eds) Advances in Motor Neuroprostheses. Springer, Cham. https://doi.org/10.1007/978-3-030-38740-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-38740-2_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38739-6
Online ISBN: 978-3-030-38740-2
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)