Abstract
This chapter introduces two novel hybrid brain–computer interfaces (BCIs) based on electroencephalography (EEG) and functional transcranial Doppler ultrasound (fTCD). Noninvasive BCIs based on EEG have become popular due to cost-effectiveness, high temporal resolution, and portability of EEG recording devices. However, the performance of the EEG-based BCIs prevents them from a consistent use by the target population, which usually includes individuals with limited speech and physical abilities. Various neuroimaging modalities that measure different brain activities have been used with EEG to improve the speed and accuracy of the noninvasive BCIs. The two hybrid BCIs introduced in this chapter are developed as alternatives to the existing systems, and they were shown to balance between speed and accuracy and outperform the existing hybrid BCIs. Both systems measure electrical brain activity as well as cerebral blood flow velocity using EEG and fTCD, respectively. For these two systems, two different visual presentation paradigms are used to induce simultaneous changes in EEG and fTCD. The visual stimuli in the first system include two objects flickering with different frequencies instructing word generation (WG) and mental rotation (MR) tasks, and the presentation in the second system includes two arrows instructing left and right motor imagery (MI) cognitive tasks. Experimental results show that the flickering MR/WG presentation outperforms the MI presentation by 10% accuracy for task versus task problem. However, the MI presentation outperforms the MR/WG one in terms of transmission rates.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akcakaya, M., et al. (2014). Noninvasive brain–computer interfaces for augmentative and alternative communication. IEEE Reviews in Biomedical Engineering, 7, 31–49. Retrieved February 12, 2019, from http://ieeexplore.ieee.org/document/6684304/.
Alexandrov, A. V., et al. (2007). Practice standards for transcranial doppler ultrasound: Part I-test performance. Journal of Neuroimaging, 17(1), 11–18. https://doi.org/10.1111/j.1552-6569.2006.00088.x.
Allison, B. Z., Wolpaw, E. W., & Wolpaw, J. R. (2007). Brain-computer interface systems: progress and prospects. Expert review of Medical Devices, 4(4): 463–474. Retrieved August 28, 2016, from http://www.ncbi.nlm.nih.gov/pubmed/17605682.
Amiri, S., Fazel-Rezai, R., & Asadpour, V. (2013). A review of hybrid brain-computer interface systems. Advances in Human-Computer Interaction, 2013, 1–8. Retrieved December 10, 2017, from http://www.hindawi.com/journals/ahci/2013/187024/.
Ang, K. K., et al. (2010). Clinical study of neurorehabilitation in stroke using EEG-based motor imagery brain-computer interface with robotic feedback. In 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology (pp. 5549–5552). IEEE. Retrieved October 20, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/21096475.
Ang, K. K., & Guan, C. (2017). EEG-based strategies to detect motor imagery for control and rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(4), 392–401. Retrieved October 20, 2017, from http://ieeexplore.ieee.org/document/7802578/.
Bin, G., et al. (2009). An online multi-channel SSVEP-based brain–computer interface using a canonical correlation analysis method. Journal of Neural Engineering, 6(4), 046002. Retrieved December 9, 2017, from http://stacks.iop.org/1741-2552/6/i=4/a=046002?key=crossref.501c725e9b2f8545feaf7401703ed8c1.
Blair, R. C., & Higgins, J. J. (1980). A comparison of the power of Wilcoxon’s rank-sum statistic to that of student’s T statistic under various non-normal distributions. Journal of Educational Statistics, 5(4), 309. Retrieved August 8, 2016, from http://www.jstor.org/stable/1164905?origin=crossref.
Blank, A. A., French, J. A., Pehlivan, A. U., & O’Malley, M. K. (2014). Current trends in robot-assisted upper-limb stroke rehabilitation: Promoting patient engagement in therapy. Current Physical Medicine and Rehabilitation Reports, 2(3), 184–195. Retrieved October 20, 2017, from http://link.springer.com/10.1007/s40141-014-0056-z.
Blokland, Y., et al. (2014). Combined EEG-fNIRS decoding of motor attempt and imagery for brain switch control: An offline study in patients with tetraplegia. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 22(2), 222–229. Retrieved October 21, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/24608682.
Buccino, A. P., Keles, H. O., & Omurtag, A. (2016). Hybrid EEG-fNIRS asynchronous brain-computer interface for multiple motor tasks. In B. He (ed.), PLOS ONE, 11(1), e0146610. Retrieved October 21, 2017, from http://dx.plos.org/10.1371/journal.pone.0146610.
Chen, X., et al. (2015). High-Speed Spelling with a Noninvasive Brain-Computer Interface. Proceedings of the National Academy of Sciences of the United States of America, 112(44), E6058–E6067. Retrieved January 29, 2018, from http://www.ncbi.nlm.nih.gov/pubmed/26483479.
Coyle, S., Ward, T., Markham, C., & McDarby, G. (2004). On the suitability of near-infrared (NIR) systems for next-generation brain–computer interfaces. Physiological Measurement, 25(4), 815–822. Retrieved September 6, 2016, from http://stacks.iop.org/0967-3334/25/i=4/a=003?key=crossref.2b3bec58c1fb74ea68c7e4e57a6ac10d.
Fager, S., et al. (2011). Access interface strategies. Assistive Technology : The Official Journal of RESNA, 24(1), 25–33. Retrieved November 1, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/22590797.
Farwell, L. A., & Donchin, E. (1988). Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology, 70(6), 510–523. Retrieved November 1, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/2461285.
Fazli, S., et al. (2012). Enhanced performance by a hybrid NIRS–EEG brain computer interface. NeuroImage, 59(1), 519–529. Retrieved October 21, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/21840399.
Hsu, C.-W., & Lin, C.-J. (2002). A comparison of methods for multiclass support vector machines. IEEE Transactions on Neural Networks, 13(2), 415–425. Retrieved August 7, 2016, from http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=991427.
Khalaf, A., Sejdic, E., & Akcakaya, M. (2018a). A novel motor imagery hybrid brain computer interface using EEG and functional transcranial doppler ultrasound. Journal of Neuroscience Methods, Under Rev.
Khalaf, A., Sejdic, E., & Akcakaya, M. (2018b). Towards optimal visual presentation design for hybrid EEG—fTCD brain–computer interfaces. Journal of Neural Engineering, 15(5), 056019. Retrieved September 7, 2018, from http://stacks.iop.org/1741-2552/15/i=5/a=056019?key=crossref.48956e67c2a5bbd451b5d7f08db2ce08.
Khalaf, A., Sybeldon, M., Sejdic, E., & Akcakaya, M. (2017). An EEG and fTCD based BCI for control. In Conference Record—Asilomar Conference on Signals, Systems and Computers.
Khalaf, A., Sybeldon, N., Sejdic, E., & Akcakaya, M. (2018). A brain-computer interface based on functional transcranial doppler ultrasound using wavelet transform and support vector machines. Journal of Neuroscience Methods, 293, 174–182. Retrieved October 20, 2017, from http://www.sciencedirect.com/science/article/pii/S0165027017303515.
Khan, M. J., Hong, M. J., & Hong, K. S. (2014). Decoding of four movement directions using hybrid NIRS-EEG brain-computer interface. Frontiers in Human Neuroscience, 8, 244. Retrieved January 28, 2018, from http://www.ncbi.nlm.nih.gov/pubmed/24808844.
Koo, B., et al. (2015). A hybrid NIRS-EEG system for self-paced brain computer interface with online motor imagery. Journal of Neuroscience Methods, 244, 26–32. Retrieved October 21, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/24797225.
LaFleur, K., et al. (2013). Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface. Journal of Neural Engineering, 10(4), 046003. Retrieved October 20, 2017, from http://stacks.iop.org/1741-2552/10/i=4/a=046003?key=crossref.ac48eeb233dce6b8bef5ccda9dae0dd2.
Lesenfants, D., et al. (2014). An independent SSVEP-based brain–computer interface in locked-in syndrome. Journal of Neural Engineering, 11(3), 035002. Retrieved November 1, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/24838215.
Matteis, M., et al. (2006). Cerebral blood flow velocity changes during meaningful and meaningless gestures—A functional transcranial doppler study. European Journal of Neurology, 13(1), 24–29. Retrieved October 20, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/16420390.
Mellinger, J., et al. (2007). An MEG-based brain–computer interface (BCI). NeuroImage, 36(3), 581–593.
Min, B.-K., Marzelli, M. J., & Yoo, S.-S. (2010). Neuroimaging-based approaches in the brain–computer interface. Trends in Biotechnology, 28(11), 552–560. Retrieved October 21, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/20810180.
Monsein, L. H., et al. (1995). Validation of transcranial doppler ultrasound with a stereotactic neurosurgical technique. Journal of Neurosurgery, 82(6), 972–975. Retrieved August 7, 2016, from http://thejns.org/doi/abs/10.3171/jns.1995.82.6.0972.
Muller-Putz, G., et al. (2015). Towards noninvasive hybrid brain-computer interfaces: Framework, practice, clinical application, and beyond. Proceedings of the IEEE, 103(6), 926–943. Retrieved October 21, 2017 from http://ieeexplore.ieee.org/document/7109824/.
Myrden, A. J. B., et al. (2011). A brain-computer interface based on bilateral transcranial doppler ultrasound. In G. Cymbalyuk (Ed.), PLoS ONE, 6(9), e24170. Retrieved October 21, 2017, from http://dx.plos.org/10.1371/journal.pone.0024170.
Nakanishi, M., et al. (2018). Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis. IEEE Transactions on Biomedical Engineering, 65(1), 104–112. Retrieved January 29, 2018, from http://ieeexplore.ieee.org/document/7904641/.
Naseer, N., & Hong, K.-S. (2015). fNIRS-based brain-computer interfaces: A review. Frontiers in Human Neuroscience, 9, 3. Retrieved January 29, 2018, from http://www.ncbi.nlm.nih.gov/pubmed/25674060.
Nicolas-Alonso, L. F., & Gomez-Gil, J. (2012). Brain computer interfaces, a review. Sensors (Basel, Switzerland), 12(2), 1211–1279. Retrieved October 20, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/22438708.
Obermaier, B., Neuper, C., Guger, C., & Pfurtscheller, G. (2001). Information transfer rate in a five-classes brain-computer interface. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 9(3), 283–288. Retrieved January 24, 2018, from http://ieeexplore.ieee.org/document/948456/.
Peng, H., Long, F., & Ding, C. (2005). Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1226–1238. Retrieved October 21, 2017, from http://ieeexplore.ieee.org/document/1453511/.
Peters, M., & Battista, C. (2008). Applications of mental rotation figures of the Shepard and Metzler type and description of a mental rotation stimulus library. Brain and Cognition, 66(3), 260–264.
Pfurtscheller, G., et al. (2010). The hybrid BCI. Frontiers in Neuroscience, 4, 3. Retrieved January 24, 2018, from http://journal.frontiersin.org/article/10.3389/fnpro.2010.00003/abstract.
Pohjalainen, J., Räsänen, O., & Kadioglu, S. (2015). Feature selection methods and their combinations in high-dimensional classification of speaker likability, intelligibility and personality traits. Computer Speech & Language, 29(1), 145–171. Retrieved December 9, 2017, from http://linkinghub.elsevier.com/retrieve/pii/S0885230813001113.
Putze, F., et al. (2014). Hybrid fNIRS-EEG based classification of auditory and visual perception processes. Frontiers in Neuroscience, 8, 373. Retrieved January 28, 2018, from http://www.ncbi.nlm.nih.gov/pubmed/25477777.
Saeys, Y., Inza, I., & Larranaga, P. (2007). A review of feature selection techniques in bioinformatics. Bioinformatics, 23(19), 2507–2517. Retrieved October 31, 2017, from https://academic.oup.com/bioinformatics/article-lookup/doi/10.1093/bioinformatics/btm344.
Schlögl, A., Lee, F., Bischof, H., & Pfurtscheller, G. (2005). Characterization of four-class motor imagery EEG data for the BCI-competition 2005. Journal of Neural Engineering, 2(4), L14–L22. Retrieved October 20, 2017, from http://stacks.iop.org/1741-2552/2/i=4/a=L02?key=crossref.b48710597fa273469a1d48e6b80b7177.
Shin, J., et al. (2017). Evaluation of a compact hybrid brain-computer interface system. BioMed Research International, 2017, 1–11. Retrieved November 1, 2017, from https://www.hindawi.com/journals/bmri/2017/6820482/.
Stroobant, N., & Vingerhoets, G. (2000). Transcranial doppler ultrasonography monitoring of cerebral hemodynamics during performance of cognitive tasks: A review. Neuropsychology Review, 10(4), 213–231. Retrieved October 21, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/11132101.
van Dokkum, L. E. H., Ward, T., & Laffont, I. (2015). Brain computer interfaces for neurorehabilitation—Its current status as a rehabilitation strategy post-stroke. Annals of Physical and Rehabilitation Medicine, 58(1), 3–8. Retrieved October 21, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/25614021.
Vanegas, M. I., Blangero, A., & Kelly, S. P. (2013). Exploiting individual primary visual cortex geometry to boost steady state visual evoked potentials. Journal of Neural Engineering, 10(3), 036003. Retrieved January 28, 2018, from http://www.ncbi.nlm.nih.gov/pubmed/23548662.
Wang, M., et al. (2015). A new hybrid BCI paradigm based on P300 and SSVEP. Journal of Neuroscience Methods, 244, 16–25. Retrieved October 31, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/24997343.
Wang, Y.-T., et al. (2017). An online brain-computer interface based on SSVEPs measured from non-hair-bearing areas. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(1), 14–21. Retrieved November 1, 2017, from http://ieeexplore.ieee.org/document/7480820/.
Weiskopf, N. et al. (2004). Principles of a brain-computer interface (BCI) based on real-time functional magnetic resonance imaging (fMRI). IEEE Transactions on Biomedical Engineering, 51(6), 966–970. Retrieved September 6, 2016, from http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1300789 ().
Welch, P. (1967). The use of fast fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms. IEEE Transactions on Audio and Electroacoustics, 15(2), 70–73. Retrieved February 27, 2017, from http://ieeexplore.ieee.org/document/1161901/.
Wolpaw, J. R., et al. (2002). Brain-computer interfaces for communication and control. Clinical Neurophysiology : Official Journal of the International Federation of Clinical Neurophysiology, 113(6), 767–791. Retrieved October 31, 2017, from http://www.ncbi.nlm.nih.gov/pubmed/12048038.
Yin, E., et al. (2015a). A dynamically optimized SSVEP brain–computer interface (BCI) speller. IEEE Transactions on Biomedical Engineering, 62(6), 1447–1456. Retrieved May 27, 2018, from http://www.ncbi.nlm.nih.gov/pubmed/24801483.
Yin, X., et al. (2015b). A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching. Journal of Neural Engineering, 12(3), 036004. Retrieved October 21, 2017, from http://stacks.iop.org/1741-2552/12/i=3/a=036004?key=crossref.fe66dac345d35e64effc9b537cbab91c.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Khalaf, A., Sejdic, E., Akcakaya, M. (2020). Hybrid EEG–fTCD Brain–Computer Interfaces. In: Nam, C. (eds) Neuroergonomics. Cognitive Science and Technology. Springer, Cham. https://doi.org/10.1007/978-3-030-34784-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-34784-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34783-3
Online ISBN: 978-3-030-34784-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)