Towards Practical Brain-Computer Interfaces pp 355-373 | Cite as
Principles of Hybrid Brain–Computer Interfaces
Abstract
Brain–Computer Interface (BCI) research has developed in the last decade so that BCIs are ready to be used with users outside the research labs. Although a wide range of assistive devices (ADs) exist, the additional usage of a BCI could improve the overall performance or applicability of such a combined system and is called hybrid BCI (hBCI). In this chapter the development of hBCIs starting from specific BCI combinations to very general hBCI based on EEG, biosignals and ADs is presented.
Keywords
Amyotrophic Lateral Sclerosis Spinal Muscular Atrophy Motor Imagery Muscular Fatigue Prosthetic HandNotes
Acknowledgements
This work is partly supported by the European ICT programme projects TOBI: Tools for Brain–Computer Interaction (FP7-224631) and fBNCI: Future Directions for Brain/Neuronal Computer Interaction (FP7-248320). Also, parts were supported by the “Land Steiermark” (project A3-22.N-13/2009-8) and the NeuroCenterStyria. This paper only reflects the authors’ views and funding agencies are not liable for any use that may be made of the information contained herein.
References
- 1.Allison, B.Z., Brunner, C., Kaiser, V., Müller-Putz, G.R., Neuper, C., Pfurtscheller, G.: Toward a hybrid brain–computer interface based on imagined movement and visual attention. J Neural Eng. 7, 026,007 (2010). DOI 10.1088/1741-2560/7/2/026007Google Scholar
- 2.Allison, B.Z., Brunner, C., Altstätter, C., Wagner, I., Grissmann, S., Neuper, C.: A hybrid ERD/SSVEP BCI for continuous simultaneous two dimensional cursor control. J. Neurosci. Methods, in press. DOI 10.1016/j.jneumeth.2012.06.022Google Scholar
- 3.Bauernfeind, G., Leeb, R., Wriessnegger, S., Pfurtscheller, G.: Development, set-up and first results of a one-channel near-infrared spectroscopy system. Biomed. Tech. (Berl.) 53, 36–43 (2008). DOI 10.1515/BMT.2008.005Google Scholar
- 4.Birbaumer, N.: Operant control of slow cortical potentials: a tool in the investigation of the potentials’ meaning and its relation to attentional dysfunction. In: Elbert, T., Rockstroh, B., Lutzenberger, W., Birbaumer, N. (eds.) Self-regulation of the brain and behaviour, pp. 227–239. Springer, New York (1984)Google Scholar
- 5.Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kübler, A., Perelmouter, J., Taub, E., Flor, H.: A spelling device for the paralysed. Nature 398, 297–298 (1999). DOI 10.1038/18581Google Scholar
- 6.Brunner, C., Allison, B.Z., Krusienski, D.J., Kaiser, V., Müller-Putz, G.R., Pfurtscheller, G., Neuper, C.: Improved signal processing approaches in an offline simulation of a hybrid brain–computer interface. J. Neurosci. Methods 188, 165–173 (2010). DOI 10.1016/j.jneumeth.2010.02.002Google Scholar
- 7.Brunner, C., Allison, B.Z., Altstätter, C., Neuper, C.: A comparison of three brain–computer interfaces based on event-related desynchronization, steady state visual evoked potentials, or a hybrid approach using both signals. J. Neural Eng. 8, 025,010 (2011). DOI 10.1088/1741-2560/8/2/025010Google Scholar
- 8.Coyle, S., Ward, T., Markham, C., McDarby, G.: On the suitability of near-infrared (NIR) systems for next-generation brain–computer interfaces. Physiol. Meas. 25, 815–822 (2004)Google Scholar
- 9.Dimitrova, N.A., Dimitrov, G.V.: Interpretation of EMG changes with fatigue: facts, pitfalls, and fallacies. J. Electromyogr. Kinesiol. 13(1), 13–36 (2003). DOI 10.1016/S1050-6411(02)00083-4, http://www.sciencedirect.com/science/article/B6T89-47DPS3S-1/2/b0fc595d474b4418ed7ded7fa4ecc746
- 10.Donchin, E., Spencer, K.M., Wijesinghe, R.: The mental prosthesis: assessing the speed of a P300-based brain–computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 8, 174–179 (2000). DOI 10.1109/86.847808Google Scholar
- 11.Ferrez, P.W., Millán, J.: Error-related EEG potentials generated during simulated brain–computer interaction. IEEE Trans. Biomed. Eng. 55, 923–929 (2008a). DOI 10.1109/TBME.2007.908083Google Scholar
- 12.Ferrez, P.W., Millán, J.: Simultaneous real-time detection of motor imagery and error-related potentials for improved BCI accuracy. In: Proceedings of 4th International Brain–Computer Interface Workshop and Training Course, Graz, Austria (2008b)Google Scholar
- 13.Fazli, S., Mehnert, J., Steinbrink, J., Curio, G., Villringer, A., Müller, K.R., Blankertz, B.: Enhanced performance by a Hybrid NIRS-EEG Brain Computer Interface. Neuroimage 59(1), 519–529 (2011).Google Scholar
- 14.Galán, F., Nuttin, M., Lew, E., Ferrez, P.W., Vanacker, G., Philips, J., Millán, J.D.R.: A brain-actuated wheelchair: Asynchronous and non-invasive brain–computer interfaces for continuous control of robots. Clin. Neurophysiol. 119(9), 2159–2169 (2008). http://dx.doi.org/10.1016/j.clinph.2008.06.001
- 15.Gao, X., Xu, D., Cheng, M., Gao, S.: A BCI-based environmental controller for the motion-disabled. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 137–140 (2003). DOI 10.1109/TNSRE.2003.814449Google Scholar
- 16.Guo, F., Hong, B., Gao, X., Gao, S.: A brain–computer interface using motion-onset visual evoked potential. J. Neural Eng. 5, 477–485 (2008). DOI 10.1088/1741-2560/5/4/011Google Scholar
- 17.Herrmann, M.J., Ehlis, A.C., Wagener, A., Jacob, C.P., Fallgatter, A.J.: Near-infrared optical topography to assess activation of the parietal cortex during a visuo-spatial task. Neuropsychologia 43, 1713–1720 (2005)Google Scholar
- 18.Hong, B., Guo, F., Liu, T., Gao, X., Gao, S.: N200-speller using motion-onset visual response. Clin. Neurophysiol. 120, 1658–1666 (2009). DOI 10.1016/j.clinph.2009.06.026Google Scholar
- 19.Horki, P., Solis-Escalante, T., Neuper, C., Müller-Putz, G.: Combined motor imagery and SSVEP based BCI control of a 2 DoF artificial upper limb. Med. Biol. Eng. Comput. Vol. 49(5), 567–577 (2011). DOI 10.1007/s11517-011-0750-2Google Scholar
- 20.Jin, J., Allison, B.Z., Wang, X., Neuper, C.: A combined brain-computer interface based on P300 potentials and motion-onset visual evoked potentials. J. Neurosci. Methods 205(2), 265–276 (2012)Google Scholar
- 21.Kübler, A., Furdea, A., Halder, S., Hammer, E.M., Nijboer, F., Kotchoubey, B.: A brain–computer interface controlled auditory event-related potential (P300) spelling system for locked-in patients. Ann. N Y Acad. Sci. 1157, 90–100 (2009). DOI 10.1111/j.1749-6632.2008.04122.xGoogle Scholar
- 22.Leeb, R., Sagha, H., Chavarriaga, R., Millán, J.: A hybrid brain–computer interface based on the fusion of electroencephalographic and electromyographic activities. J. Neural Eng. 8(2), 025,011 (2011)Google Scholar
- 23.Li, Y., Long, J., Yu, T., Yu, Z., Wang, C., Zhang, H., Guan, C.: An EEG-based BCI system for 2-D cursor control by combining mu/beta rhythm and P300 potential. IEEE Trans. Biomed. Eng. 57, 2495–2505 (2010). DOI 10.1109/TBME.2010.2055564Google Scholar
- 24.Linortner, P., Ortner, R., Müller-Putz, G.R., Neuper, C., Pfurtscheller, G.: Self-paced control of a hand orthosis using SSVEP-based BCI. In: Proceedings of the13th International Conference on Human–Computer Interaction (2009)Google Scholar
- 25.Liu, T., Goldberg, L., Gao, S., Hong, B.: An online brain–computer interface using non-flashing visual evoked potentials. J. Neural Eng. 7, 036,003 (2010). DOI 10.1088/1741-2560/7/3/036003Google Scholar
- 26.Millán, J., Ferrez, P.W., Galán, F., Lew, E., Chavarriaga, R.: Non-invasive brain-machine interaction. Intern. J. Pattern Recognit. Artif. Intell. 22(5), 959–972 (2008)Google Scholar
- 27.Müller-Putz, G.R., Pfurtscheller, G.: Control of an electrical prosthesis with an SSVEP-based BCI. IEEE Trans. Biomed. Eng. 55, 361–364 (2008). DOI 10.1109/TBME.2007.897815Google Scholar
- 28.Müller-Putz, G.R., Scherer, R., Pfurtscheller, G., Rupp, R.: EEG-based neuroprosthesis control: a step towards clinical practice. Neurosci. Lett. 382, 169–174 (2005)Google Scholar
- 29.Müller-Putz, G.R., Scherer, R., Neuper, C., Pfurtscheller, G.: Steady-state somatosensory evoked potentials: suitable brain signals for brain–computer interfaces? IEEE Trans. Neural Syst. Rehabil. Eng. 14, 30–37 (2006). DOI 10.1109/TNSRE.2005.863842Google Scholar
- 30.Müller-Putz, G.R., Eder, E., Wriessnegger, S.C., Pfurtscheller, G.: Comparison of DFT and lock-in amplifier features and search for optimal electrode positions in SSVEP-based BCI. J. Neurosci. Meth. 168, 174–181 (2008). DOI 10.1016/j.jneumeth.2007.09.024Google Scholar
- 31.Müller-Putz, G.R., Kaiser, V., Solis-Escalante, T., Pfurtscheller, G.: Fast set-up asynchronous brain-switch based on detection of foot motor imagery in 1-channel EEG. Med. Biol. Eng. Comput. 48, 229–233 (2010). DOI 10.1007/s11517-009n-0572-7Google Scholar
- 32.Müller-Putz, G.R., Breitwieser, C., Cincotti, F., Leeb, R., Schreuder, M., Leotta, F., Tavella, M., Bianchi, L., Kreilinger, A., Ramsay, A., Rohm, M., Sagebaum, M., Tonin, L., Neuper, C., del R Millán, J.: Tools for brain–computer interaction: a general concept for a hybrid BCI (hBCI). Frontiers in Neuroinformatics 5, 30 (2011, in revision)Google Scholar
- 33.Neuper, C., Wörtz, M., Pfurtscheller, G.: ERD/ERS patterns reflecting sensorimotor activation and deactivation. In: Neuper, C., Klimesch, W. (eds.) Event-Related Dynamics of Brain Oscillations, Progress in Brain Research, vol. 159, chap. 14, pp. 211–222. Elsevier, Netherlands (2006). DOI 10.1016/S0079-6123(06)59014-4
- 34.Nijboer, F., Furdea, A., Gunst, I., Mellinger, J., McFarland, D.J., Birbaumer, N., Kübler, A.: An auditory brain–computer interface (BCI). Neurosci. Lett. 167, 43–50 (2008)Google Scholar
- 35.Nijholt, A.: BCI for games: a “state of the art” survey. In: Stevens, S., Saldamarco, S. (eds.) Entertainment Computing – ICEC 2008, pp. 225–228. Springer, Berlin/Heidelberg (2009). DOI chap. 29
- 36.Ortner, R., Allison, B., Korisek, G., Gaggl, G., Pfurtscheller, G.: An SSVEP BCI to control a hand orthosis for persons with tetraplegia. IEEE. Trans. Neural Syst. Rehabil. Eng. 19(1), 1–5 (2011)Google Scholar
- 37.Panicker, R.C., Puthusserypady, S., Sun, Y.: An asynchronous P300 BCI with SSVEP-based control state detection. IEEE Trans. Biomed. Eng. 58, 1781–1788 (2011). DOI 10.1109/TBME.2011.2116018Google Scholar
- 38.Pfurtscheller, G., Lopes da Silva F.H.: Event-related desynchronization (ERD) and event-related synchronization (ERS). In: Electroencephalography: basic principles, clinical applications and related fields. Williams & Wilkins, Philadelphia (2005)Google Scholar
- 39.Pfurtscheller, G., Neuper, C.: Motor imagery and direct brain–computer communication. Proc. IEEE 89, 1123–1134 (2001). DOI 10.1109/5.939829Google Scholar
- 40.Pfurtscheller, G., Wörtz, M., Supp, G., da Silva, F.H.L.: Early onset of post-movement beta electroencephalogram synchronization in the supplementary motor area during self-paced finger movement in man. Neurosci. Lett. 339, 111–114 (2003)Google Scholar
- 41.Pfurtscheller, G., Leeb, R., Friedman, D., Slater, M.: Centrally controlled heart rate changes during mental practice in immersive virtual environment: a case study with a tetraplegic. Int. J. Psychophysiol. 68, 1–5 (2008). DOI 10.1016/j.ijpsycho.2007.11.003Google Scholar
- 42.Pfurtscheller, G., Allison, B.Z., Brunner, C., Bauernfeind, G., Solis-Escalante, T., Scherer, R., Zander, T.O., Müller-Putz, G., Neuper, C., Birbaumer, N.: The hybrid BCI. Front. Neurosci. 4, 30 (2010a). DOI 10.3389/fnpro.2010.00003Google Scholar
- 43.Pfurtscheller, G., Bauernfeind, G., Wriessnegger, S.C., Neuper, C.: Focal frontal (de)oxyhemoglobin responses during simple arithmetic. Int. J. Psychophysiol. 76, 186–192 (2010b). DOI 10.1016/j.ijpsycho.2010.03.013Google Scholar
- 44.Rogova, G.L., Nimier, V.: Reliability in information fusion: literature survey. In: Proc. of. the 7th Intl. Conference on Informatin Fusion, Stockholm. pp. 1158–1165 (2004)Google Scholar
- 45.Schalk, G., Wolpaw, J.R., McFarland, D.J., Pfurtscheller, G.: EEG-based communication: Presence of an error potential. Clin. Neurophysiol. 111(12), 2138–2144 (2000)Google Scholar
- 46.Scherer, R., Müller-Putz, G.R., Pfurtscheller, G.: Self-initiation of EEG-based brain–computer communication using the heart rate response. J. Neural Eng. 4, L23–L29 (2007). DOI 10.1088/1741-2560/4/4/L01Google Scholar
- 47.Su, Y., Qi, Y., Luo, J.X., Wu, B., Yang, F., Li, Y., Zhuang, Y.T., Zheng, X.X., Chen, W.D.: A hybrid brain–computer interface control strategy in a virtual environment. J. Zhejiang Univ. Sci. C. 12, 351–361 (2011). DOI 10.1631/jzus.C1000208Google Scholar
- 48.Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791 (2002). DOI 10.1016/S1388-2457n(02)00057-3Google Scholar
- 49.Wriessnegger, S.C., Kurzmann, J., Neuper, C.: Spatio-temporal differences in brain oxygenation between movement execution and imagery: a multichannel near-infrared spectroscopy study. Int. J. Psychophysiol. 67, 54–63 (2008). DOI 10.1016/j.ijpsycho.2007.10.004Google Scholar
- 50.Zander, T.O., Kothe, C.: Towards passive brain–computer interfaces: applying brain–computer interface technology to human-machine systems in general. J. Neural Eng. 8, 025,005 (2011). DOI 10.1088/1741-2560/8/2/025005Google Scholar
- 51.Zander, T.O., Gaertner, M., Kothe, C., Vilimek, R.: Combining eye gaze input with a brain–computer interface for touchless human–computer interaction. Int. J. Hum. Comput. Interaction 27, 38–51 (2011)Google Scholar