Abstract
Introduction:Brain-computer interfaces have become an important tool in human computer interactions. The area of applications ranges from simple research to profound stroke therapy. In this paper, a novel approach to motor imagery is proposed. We analyzed left and right hand grasping using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). Material, Methods and Results: We used the wireless g.Nautilus fNIRS (g.tec) with 16 channels of EEG, combined with 8 channels of fNIRS, acquiring optical densities and their corresponding oxygenated (HBO) and deoxygenated (HBR) hemoglobin concentrations. We recorded data from 5 healthy subjects and evaluated the algorithms performance. Real- time positive feedback via functional electrical stimulation and a 3D avatar was provided. Each method: EEG and fNIRS (HBO and HBR) were evaluated separately. The final hybrid prediction was performed by a meta classifier utilizing the scores of the individual linear discriminant analysis classifiers. Discussion and Conclusion: The results indicate that fNIRS is applicable for online MI classification. The classification accuracies using fNIRS appear to be equivalent and in some cases superior to those using EEG. However, taking advantage of all modalities appears to improve the robustness. We therefore consider fNIRS as a simple, powerful and affordable amendment in such applications.
European Commission project RHUMBO – H2020-MSCA-ITN-2018-813234.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bishop, C.M.: Pattern recognition and machine learning. Information science and statistics. Springer, New York, NY (2006). Softcover published in 2016
Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., Muller, K.: Optimizing spatial filters for robust eeg single-trial analysis. IEEE Signal Process. Mag. 25(1), 41–56 (2008). https://doi.org/10.1109/MSP.2008.4408441
Chiarelli, A.M., Croce, P., Merla, A., Zappasodi, F.: Deep learning for hybrid EEG-fNIRS brain-computer interface: application to motor imagery classification. J. Neural Eng. 15(3), 036028 (2018). https://doi.org/10.1088/1741-2552/aaaf82
Delpy, D.T., Cope, M., van der Zee, P., Arridge, S., Wray, S., Wyatt, J.: Estimation of optical pathlength through tissue from direct time of flight measurement. Phys. Med. Biol. 33(12), 1433–1442 (1988). https://doi.org/10.1088/0031-9155/33/12/008
Guger, C., Ramoser, H., Pfurtscheller, G.: Real-time eeg analysis with subject-specific spatial patterns for a brain-computer interface (BCI). IEEE Trans. Rehabil. Eng. 8(4), 447–456 (2000). https://doi.org/10.1109/86.895947
Herculano-Houzel, S.: The human brain in numbers: a linearly scaled-up primate brain. Front. Hum. Neurosci. 3, 31 (2009). https://doi.org/10.3389/neuro.09.031.2009
Hong, K.S., Khan, M.J., Hong, M.J.: Feature extraction and classification methods for hybrid fNIRS-EEG brain-computer interfaces. Front. Hum. Neurosci. 12, 246 (2018). https://doi.org/10.3389/fnhum.2018.00246
Hong, K.S., Naseer, N., Kim, Y.H.: Classification of prefrontal and motor cortex signals for three-class fNIRS-BCI. Neurosci. Lett. 587, 87–92 (2015). https://doi.org/10.1016/j.neulet.2014.12.029
Huppert, T.J., Hoge, R.D., Diamond, S.G., Franceschini, M.A., Boas, D.A.: A temporal comparison of bold, ASL, and NIRS hemodynamic responses to motor stimuli in adult humans. Neuroimage 29(2), 368–382 (2006)
Khalaf, A., Sejdic, E., Akcakaya, M.: A novel motor imagery hybrid brain computer interface using EEG and functional transcranial doppler ultrasound. J. Neurosci. Methods 313, 44–53 (2019). https://doi.org/10.1016/j.jneumeth.2018.11.017
Müller-Putz, G., Scherer, R., Brunner, C., Leeb, R., Pfurtscheller, G.: Better than random? a closer look on BCI results. Int. J. Bioelectromag. 10(1), 52–55 (2008)
Neuper, C., Wörtz, M., Pfurtscheller, G.: Erd/ERS patterns reflecting sensorimotor activation and deactivation. In: Neuper, C., Klimesch, W. (eds.) Event-Related Dynamics of Brain Oscillations, Progress in Brain Research, vol. 159, pp. 211–222. Elsevier (2006). https://doi.org/10.1016/S0079-6123(06)59014-4
Orihuela-Espina, F., Leff, D.R., James, D.R.C., Darzi, A.W., Yang, G.Z.: Quality control and assurance in functional near infrared spectroscopy (fNIRS) experimentation. Phys. Med. Biol. 55(13), 3701–3724 (2010). https://doi.org/10.1088/0031-9155/55/13/009
Pollonini, L., Bortfeld, H., Oghalai, J.S.: Phoebe: a method for real time mapping of optodes-scalp coupling in functional near-infrared spectroscopy. Biomed. Opt. Express 7(12), 5104–5119 (2016)
Ramadan, R.A., Vasilakos, A.V.: Brain computer interface: control signals review. Neurocomputing 223, 26–44 (2017). https://doi.org/10.1016/j.neucom.2016.10.024
Scholkmann, F., et al.: A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology. NeuroImage 85, 6–27 (2014). https://doi.org/10.1016/j.neuroimage.2013.05.004. Celebrating 20 Years of Functional Near Infrared Spectroscopy (fNIRS)
Verma, P., Heilinger, A., Reitner, P., Grünwald, J., Guger, C., Franklin, D.: Performance investigation of brain-computer interfaces that combine EEG and fNIRS for motor imagery tasks. In: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). pp. 259–263, October 2019. https://doi.org/10.1109/SMC.2019.8914083
Zich, C., Debener, S., Thoene, A.K., Chen, L.C., Kranczioch, C.: Simultaneous eeg-fnirs reveals how age and feedback affect motor imagery signatures. Neurobiol. Aging 49, 183–197 (2017). https://doi.org/10.1016/j.neurobiolaging.2016.10.011
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Hirsch, G., Dirodi, M., Xu, R., Reitner, P., Guger, C. (2020). Online Classification of Motor Imagery Using EEG and fNIRS: A Hybrid Approach with Real Time Human-Computer Interaction. In: Stephanidis, C., Antona, M. (eds) HCI International 2020 - Posters. HCII 2020. Communications in Computer and Information Science, vol 1224. Springer, Cham. https://doi.org/10.1007/978-3-030-50726-8_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-50726-8_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-50725-1
Online ISBN: 978-3-030-50726-8
eBook Packages: Computer ScienceComputer Science (R0)