Abstract
Next generation brain computer interfaces (BCI) are expected to provide robust and continuous control mechanism. In this study, we assessed integration of optical brain imaging (fNIR: functional near infrared spectroscopy) to a P300-BCI for improving BCI usability by monitoring cognitive workload and performance. fNIR is a safe and wearable neuroimaging modality that tracks cortical hemodynamics in response to sensory, motor, or cognitive activation. Eight volunteers participated in the study where simultaneous EEG and 16 optode fNIR from anterior prefrontal cortex were recorded while participants engaged with the P300-BCI for spatial navigation. The results showed a significant response in fNIR signals during high, medium and low performance indicating a positive correlation between prefrontal oxygenation changes and BCI performance. This preliminary study provided evidence that the performance of P300-BCI can be monitored by fNIR which in turn can help improve the robustness of the BCI classification.
Chapter PDF
Similar content being viewed by others
References
Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-computer interfaces for communication and control. Clinical Neurophysiology 113, 767–791 (2002)
Farwell, L.A., Donchin, E.: Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and clinical Neurophysiology 70, 510–523 (1988)
Pfurtscheller, G., Neuper, C.: Motor imagery and direct brain-computer communication. Proceedings of the IEEE 89, 1123–1134 (2001)
Mellinger, J., Schalk, G., Braun, C., Preissl, H., Rosenstiel, W., Birbaumer, N., Kübler, A.: An MEG-based brain-computer interface (BCI). Neuroimage 36, 581 (2007)
Coyle, S., Ward, T., Markham, C., McDarby, G.: On the suitability of near-infrared (NIR) systems for next-generation brain–computer interfaces. Physiological Measurement 25, 815 (2004)
Sitaram, R., Zhang, H., Guan, C., Thulasidas, M., Hoshi, Y., Ishikawa, A., Shimizu, K., Birbaumer, N.: Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain–computer interface. NeuroImage 34, 1416–1427 (2007)
Ayaz, H., Izzetoglu, M., Bunce, S., Heiman-Patterson, T., Onaral, B.: Detecting cognitive activity related hemodynamic signal for brain computer interface using functional near infrared spectroscopy, pp. 342–345. IEEE (2007)
Limongi, T., Di Sante, G., Ferrari, M., Quaresima, V.: Detecting mental calculation related frontal cortex oxygenation changes for brain computer interface using multi-channel functional near infrared topography. International Journal of Bioelectromagnetism 11, 86–90 (2009)
Ayaz, H., Shewokis, P., Bunce, S., Schultheis, M., Onaral, B.: Assessment of cognitive neural correlates for a functional near infrared-based brain computer interface system. Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience, 699–708 (2009)
Weiskopf, N., Mathiak, K., Bock, S.W., Scharnowski, F., Veit, R., Grodd, W., Goebel, R., Birbaumer, N.: Principles of a brain-computer interface (BCI) based on real-time functional magnetic resonance imaging (fMRI). IEEE Transactions on Biomedical Engineering 51, 966–970 (2004)
Yoo, S.S., Fairneny, T., Chen, N.K., Choo, S.E., Panych, L.P., Park, H.W., Lee, S.Y., Jolesz, F.A.: Brain-computer interface using fMRI: spatial navigation by thoughts. Neuroreport 15, 1591–1595 (2004)
Ferrez, P.W., Millán, J.R.: Simultaneous real-time detection of motor imagery and error-related potentials for improved BCI accuracy. In: Proceedings of the 4th International Brain-Computer Interface Workshop, pp. 197–202 (2008)
Allison, B., Brunner, C., Kaiser, V., Müller-Putz, G., Neuper, C., Pfurtscheller, G.: Toward a hybrid brain–computer interface based on imagined movement and visual attention. Journal of Neural Engineering 7, 026007 (2010)
Pfurtscheller, G., Allison, B.Z., Bauernfeind, G.N., Brunner, C., Solis Escalante, T., Scherer, R., Zander, T.O., Mueller-Putz, G., Neuper, C., Birbaumer, N.: The hybrid BCI. Frontiers in Neuroscience 4 (2010)
Rebsamen, B., Burdet, E., Zeng, Q., Zhang, H., Ang, M., Teo, C.L., Guan, C., Laugier, C.: Hybrid P300 and mu-beta brain computer interface to operate a brain controlled wheelchair. In: Proceedings of the 2nd International Convention on Rehabilitation Engineering & Assistive Technology, pp. 51-55. Singapore Therapeutic, Assistive & Rehabilitative Technologies (START) Centre, Bangkok, Thailand (2008)
Müller-Putz, G.: Hybrid brain-computer interfaces: current state and future directions. PPT 55, 923–929 (2011)
Mak, J., Arbel, Y., Minett, J., McCane, L., Yuksel, B., Ryan, D., Thompson, D., Bianchi, L., Erdogmus, D.: Optimizing the P300-based brain–computer interface: current status, limitations and future directions. Journal of Neural Engineering 8, 025003 (2011)
Bunce, S.C., Izzetoglu, M., Izzetoglu, K., Onaral, B., Pourrezaei, K.: Functional near-infrared spectroscopy. IEEE Engineering in Medicine and Biology Magazine 25, 54–62 (2006)
Fazli, S., Mehnert, J., Steinbrink, J., Curio, G., Villringer, A., Müller, K.-R., Blankertz, B.: Enhanced performance by a hybrid NIRS–EEG brain computer interface. Neuroimage 59, 519–529 (2012)
Fazli, S., Mehnert, J., Steinbrink, J., Blankertz, B.: Using NIRS as a predictor for EEG-based BCI performance. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4911–4914 (2012)
Mak, J.N., McFarland, D.J., Vaughan, T.M., McCane, L.M., Tsui, P.Z., Zeitlin, D.J., Sellers, E.W., Wolpaw, J.R.: EEG correlates of P300-based brain–computer interface (BCI) performance in people with amyotrophic lateral sclerosis. Journal of Neural Engineering 9, 026014 (2012)
Halder, S., Hammer, E.M., Kleih, S.C., Bogdan, M., Rosenstiel, W., Birbaumer, N., Kübler, A.: Prediction of Auditory and Visual P300 Brain-Computer Interface Aptitude. PLOS One 8, e53513 (2013)
Bledowski, C., Prvulovic, D., Goebel, R., Zanella, F.E., Linden, D.E.J.: Attentional systems in target and distractor processing: a combined ERP and fMRI study. Neuroimage 22, 530–540 (2004)
Mccarthy, G., Luby, M., Gore, J., Goldman-Rakic, P.: Infrequent events transiently activate human prefrontal and parietal cortex as measured by functional MRI. Journal of Neurophysiology 77, 1630–1634 (1997)
Horovitz, S.G., Skudlarski, P., Gore, J.C.: Correlations and dissociations between BOLD signal and P300 amplitude in an auditory oddball task: a parametric approach to combining fMRI and ERP. Magnetic Resonance Imaging 20, 319–325 (2002)
Thomas, M., Sing, H., Belenky, G., Holcomb, H., Mayberg, H., Dannals, R., Wagner, J., Thorne, D., Popp, K., Rowland, L.: Neural basis of alertness and cognitive performance impairments during sleepiness. I. Effects of 24 h of sleep deprivation on waking human regional brain activity. Journal of Sleep Research 9, 335–352 (2008)
Moller, H.J., Rizzo, A.A., Mikulis, D.J.: Prefrontal cortex activation mediates cognitive reserve alertness and attention in the Virtual Classroom: preliminary fMRI findings and clinical implications. In: Virtual Rehabilitation, pp. 146–150 (2007)
Fuster, J.M.: The prefrontal cortex. Academic Press (2008)
Ayaz, H., Bunce, S., Shewokis, P., Izzetoglu, K., Willems, B., Onaral, B.: Using Brain Activity to Predict Task Performance and Operator Efficiency. In: Zhang, H., Hussain, A., Liu, D., Wang, Z. (eds.) BICS 2012. LNCS, vol. 7366, pp. 147–155. Springer, Heidelberg (2012)
Liu, Y., Ayaz, H., Curtin, A., Shewokis, P.A., Onaral, B.: Detection of attention shift for asynchronous P300-based BCI. In: Proc. IEEE Eng. Med. Biol. Soc., pp. 4724–4727 (2012)
Curtin, A., Ayaz, H., Liu, Y., Shewokis, P.A., Onaral, B.: A P300-based EEG-BCI for Spatial Navigation Control. In: Conf. Proc. IEEE Eng.Med. Biol. Soc., pp. 3841–3844 (2012)
Ayaz, H., Shewokis, P.A., Curtin, A., Izzetoglu, M., Izzetoglu, K., Onaral, B.: Using MazeSuite and functional near infrared spectroscopy to study learning in spatial navigation. J. Vis. Exp., e3443 (2011)
Ayaz, H., Allen, S.L., Platek, S.M., Onaral, B.: Maze Suite 1.0: a complete set of tools to prepare, present, and analyze navigational and spatial cognitive neuroscience experiments. Behavior Research Methods 40, 353–359 (2008)
Schalk, G., McFarland, D.J., Hinterberger, T., Birbaumer, N., Wolpaw, J.R.: BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Transactions on Biomedical Engineering 51, 1034–1043 (2004)
Ayaz, H., Shewokis, P.A., Bunce, S., Izzetoglu, K., Willems, B., Onaral, B.: Optical brain monitoring for operator training and mental workload assessment. Neuroimage 59, 36–47 (2012)
Ayaz, H., Izzetoglu, M., Shewokis, P.A., Onaral, B.: Sliding-window motion artifact rejection for functional near-infrared spectroscopy. In: Annual International Conf. on Engineering in Medicine and Biology Society (EMBC), pp. 6567–6570. IEEE (2010)
Halder, S., Agorastos, D., Veit, R., Hammer, E., Lee, S., Varkuti, B., Bogdan, M., Rosenstiel, W., Birbaumer, N., Kübler, A.: Neural mechanisms of brain–computer interface control. NeuroImage 55, 1779–1790 (2011)
Miller, E.K., Cohen, J.D.: An integrative theory of prefrontal cortex function. Annual Review of Neuroscience 24, 167–202 (2001)
Posner, M.I., Rothbart, M.K.: Research on attention networks as a model for the integration of psychological science. Annu. Rev. Psychol. 58, 1–23 (2007)
Ayaz, H., Shewokis, P.A., İzzetoğlu, M., Çakır, M.P., Onaral, B.: Tangram solved? Prefrontal cortex activation analysis during geometric problem solving. In: 34th Annual International IEEE EMBS Conference, pp. 4724–4727. IEEE (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, Y., Ayaz, H., Curtin, A., Onaral, B., Shewokis, P.A. (2013). Towards a Hybrid P300-Based BCI Using Simultaneous fNIR and EEG. In: Schmorrow, D.D., Fidopiastis, C.M. (eds) Foundations of Augmented Cognition. AC 2013. Lecture Notes in Computer Science(), vol 8027. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39454-6_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-39454-6_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39453-9
Online ISBN: 978-3-642-39454-6
eBook Packages: Computer ScienceComputer Science (R0)