Principles of Hybrid Brain–Computer Interfaces

  • Gernot R. Müller-Putz
  • Robert Leeb
  • José d. R. Millán
  • Petar Horki
  • Alex Kreilinger
  • Günther Bauernfeind
  • Brendan Z. Allison
  • Clemens Brunner
  • Reinhold Scherer
Chapter
First Online:
Part of the Biological and Medical Physics, Biomedical Engineering book series (BIOMEDICAL)

Abstract

Brain–Computer Interface (BCI) research has developed in the last decade so that BCIs are ready to be used with users outside the research labs. Although a wide range of assistive devices (ADs) exist, the additional usage of a BCI could improve the overall performance or applicability of such a combined system and is called hybrid BCI (hBCI). In this chapter the development of hBCIs starting from specific BCI combinations to very general hBCI based on EEG, biosignals and ADs is presented.

Keywords

Amyotrophic Lateral Sclerosis Spinal Muscular Atrophy Motor Imagery Muscular Fatigue Prosthetic Hand 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access

Notes

Acknowledgements

This work is partly supported by the European ICT programme projects TOBI: Tools for Brain–Computer Interaction (FP7-224631) and fBNCI: Future Directions for Brain/Neuronal Computer Interaction (FP7-248320). Also, parts were supported by the “Land Steiermark” (project A3-22.N-13/2009-8) and the NeuroCenterStyria. This paper only reflects the authors’ views and funding agencies are not liable for any use that may be made of the information contained herein.

References

  1. 1.
    Allison, B.Z., Brunner, C., Kaiser, V., Müller-Putz, G.R., Neuper, C., Pfurtscheller, G.: Toward a hybrid brain–computer interface based on imagined movement and visual attention. J Neural Eng. 7, 026,007 (2010). DOI 10.1088/1741-2560/7/2/026007Google Scholar
  2. 2.
    Allison, B.Z., Brunner, C., Altstätter, C., Wagner, I., Grissmann, S., Neuper, C.: A hybrid ERD/SSVEP BCI for continuous simultaneous two dimensional cursor control. J. Neurosci. Methods, in press. DOI 10.1016/j.jneumeth.2012.06.022Google Scholar
  3. 3.
    Bauernfeind, G., Leeb, R., Wriessnegger, S., Pfurtscheller, G.: Development, set-up and first results of a one-channel near-infrared spectroscopy system. Biomed. Tech. (Berl.) 53, 36–43 (2008). DOI 10.1515/BMT.2008.005Google Scholar
  4. 4.
    Birbaumer, N.: Operant control of slow cortical potentials: a tool in the investigation of the potentials’ meaning and its relation to attentional dysfunction. In: Elbert, T., Rockstroh, B., Lutzenberger, W., Birbaumer, N. (eds.) Self-regulation of the brain and behaviour, pp. 227–239. Springer, New York (1984)Google Scholar
  5. 5.
    Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kübler, A., Perelmouter, J., Taub, E., Flor, H.: A spelling device for the paralysed. Nature 398, 297–298 (1999). DOI 10.1038/18581Google Scholar
  6. 6.
    Brunner, C., Allison, B.Z., Krusienski, D.J., Kaiser, V., Müller-Putz, G.R., Pfurtscheller, G., Neuper, C.: Improved signal processing approaches in an offline simulation of a hybrid brain–computer interface. J. Neurosci. Methods 188, 165–173 (2010). DOI 10.1016/j.jneumeth.2010.02.002Google Scholar
  7. 7.
    Brunner, C., Allison, B.Z., Altstätter, C., Neuper, C.: A comparison of three brain–computer interfaces based on event-related desynchronization, steady state visual evoked potentials, or a hybrid approach using both signals. J. Neural Eng. 8, 025,010 (2011). DOI 10.1088/1741-2560/8/2/025010Google Scholar
  8. 8.
    Coyle, S., Ward, T., Markham, C., McDarby, G.: On the suitability of near-infrared (NIR) systems for next-generation brain–computer interfaces. Physiol. Meas. 25, 815–822 (2004)Google Scholar
  9. 9.
    Dimitrova, N.A., Dimitrov, G.V.: Interpretation of EMG changes with fatigue: facts, pitfalls, and fallacies. J. Electromyogr. Kinesiol. 13(1), 13–36 (2003). DOI 10.1016/S1050-6411(02)00083-4, http://www.sciencedirect.com/science/article/B6T89-47DPS3S-1/2/b0fc595d474b4418ed7ded7fa4ecc746
  10. 10.
    Donchin, E., Spencer, K.M., Wijesinghe, R.: The mental prosthesis: assessing the speed of a P300-based brain–computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 8, 174–179 (2000). DOI 10.1109/86.847808Google Scholar
  11. 11.
    Ferrez, P.W., Millán, J.: Error-related EEG potentials generated during simulated brain–computer interaction. IEEE Trans. Biomed. Eng. 55, 923–929 (2008a). DOI 10.1109/TBME.2007.908083Google Scholar
  12. 12.
    Ferrez, P.W., Millán, J.: Simultaneous real-time detection of motor imagery and error-related potentials for improved BCI accuracy. In: Proceedings of 4th International Brain–Computer Interface Workshop and Training Course, Graz, Austria (2008b)Google Scholar
  13. 13.
    Fazli, S., Mehnert, J., Steinbrink, J., Curio, G., Villringer, A., Müller, K.R., Blankertz, B.: Enhanced performance by a Hybrid NIRS-EEG Brain Computer Interface. Neuroimage 59(1), 519–529 (2011).Google Scholar
  14. 14.
    Galán, F., Nuttin, M., Lew, E., Ferrez, P.W., Vanacker, G., Philips, J., Millán, J.D.R.: A brain-actuated wheelchair: Asynchronous and non-invasive brain–computer interfaces for continuous control of robots. Clin. Neurophysiol. 119(9), 2159–2169 (2008). http://dx.doi.org/10.1016/j.clinph.2008.06.001
  15. 15.
    Gao, X., Xu, D., Cheng, M., Gao, S.: A BCI-based environmental controller for the motion-disabled. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 137–140 (2003). DOI 10.1109/TNSRE.2003.814449Google Scholar
  16. 16.
    Guo, F., Hong, B., Gao, X., Gao, S.: A brain–computer interface using motion-onset visual evoked potential. J. Neural Eng. 5, 477–485 (2008). DOI 10.1088/1741-2560/5/4/011Google Scholar
  17. 17.
    Herrmann, M.J., Ehlis, A.C., Wagener, A., Jacob, C.P., Fallgatter, A.J.: Near-infrared optical topography to assess activation of the parietal cortex during a visuo-spatial task. Neuropsychologia 43, 1713–1720 (2005)Google Scholar
  18. 18.
    Hong, B., Guo, F., Liu, T., Gao, X., Gao, S.: N200-speller using motion-onset visual response. Clin. Neurophysiol. 120, 1658–1666 (2009). DOI 10.1016/j.clinph.2009.06.026Google Scholar
  19. 19.
    Horki, P., Solis-Escalante, T., Neuper, C., Müller-Putz, G.: Combined motor imagery and SSVEP based BCI control of a 2 DoF artificial upper limb. Med. Biol. Eng. Comput. Vol. 49(5), 567–577 (2011). DOI 10.1007/s11517-011-0750-2Google Scholar
  20. 20.
    Jin, J., Allison, B.Z., Wang, X., Neuper, C.: A combined brain-computer interface based on P300 potentials and motion-onset visual evoked potentials. J. Neurosci. Methods 205(2), 265–276 (2012)Google Scholar
  21. 21.
    Kübler, A., Furdea, A., Halder, S., Hammer, E.M., Nijboer, F., Kotchoubey, B.: A brain–computer interface controlled auditory event-related potential (P300) spelling system for locked-in patients. Ann. N Y Acad. Sci. 1157, 90–100 (2009). DOI 10.1111/j.1749-6632.2008.04122.xGoogle Scholar
  22. 22.
    Leeb, R., Sagha, H., Chavarriaga, R., Millán, J.: A hybrid brain–computer interface based on the fusion of electroencephalographic and electromyographic activities. J. Neural Eng. 8(2), 025,011 (2011)Google Scholar
  23. 23.
    Li, Y., Long, J., Yu, T., Yu, Z., Wang, C., Zhang, H., Guan, C.: An EEG-based BCI system for 2-D cursor control by combining mu/beta rhythm and P300 potential. IEEE Trans. Biomed. Eng. 57, 2495–2505 (2010). DOI 10.1109/TBME.2010.2055564Google Scholar
  24. 24.
    Linortner, P., Ortner, R., Müller-Putz, G.R., Neuper, C., Pfurtscheller, G.: Self-paced control of a hand orthosis using SSVEP-based BCI. In: Proceedings of the13th International Conference on Human–Computer Interaction (2009)Google Scholar
  25. 25.
    Liu, T., Goldberg, L., Gao, S., Hong, B.: An online brain–computer interface using non-flashing visual evoked potentials. J. Neural Eng. 7, 036,003 (2010). DOI 10.1088/1741-2560/7/3/036003Google Scholar
  26. 26.
    Millán, J., Ferrez, P.W., Galán, F., Lew, E., Chavarriaga, R.: Non-invasive brain-machine interaction. Intern. J. Pattern Recognit. Artif. Intell. 22(5), 959–972 (2008)Google Scholar
  27. 27.
    Müller-Putz, G.R., Pfurtscheller, G.: Control of an electrical prosthesis with an SSVEP-based BCI. IEEE Trans. Biomed. Eng. 55, 361–364 (2008). DOI 10.1109/TBME.2007.897815Google Scholar
  28. 28.
    Müller-Putz, G.R., Scherer, R., Pfurtscheller, G., Rupp, R.: EEG-based neuroprosthesis control: a step towards clinical practice. Neurosci. Lett. 382, 169–174 (2005)Google Scholar
  29. 29.
    Müller-Putz, G.R., Scherer, R., Neuper, C., Pfurtscheller, G.: Steady-state somatosensory evoked potentials: suitable brain signals for brain–computer interfaces? IEEE Trans. Neural Syst. Rehabil. Eng. 14, 30–37 (2006). DOI 10.1109/TNSRE.2005.863842Google Scholar
  30. 30.
    Müller-Putz, G.R., Eder, E., Wriessnegger, S.C., Pfurtscheller, G.: Comparison of DFT and lock-in amplifier features and search for optimal electrode positions in SSVEP-based BCI. J. Neurosci. Meth. 168, 174–181 (2008). DOI 10.1016/j.jneumeth.2007.09.024Google Scholar
  31. 31.
    Müller-Putz, G.R., Kaiser, V., Solis-Escalante, T., Pfurtscheller, G.: Fast set-up asynchronous brain-switch based on detection of foot motor imagery in 1-channel EEG. Med. Biol. Eng. Comput. 48, 229–233 (2010). DOI 10.1007/s11517-009n-0572-7Google Scholar
  32. 32.
    Müller-Putz, G.R., Breitwieser, C., Cincotti, F., Leeb, R., Schreuder, M., Leotta, F., Tavella, M., Bianchi, L., Kreilinger, A., Ramsay, A., Rohm, M., Sagebaum, M., Tonin, L., Neuper, C., del R Millán, J.: Tools for brain–computer interaction: a general concept for a hybrid BCI (hBCI). Frontiers in Neuroinformatics 5, 30 (2011, in revision)Google Scholar
  33. 33.
    Neuper, C., Wörtz, M., Pfurtscheller, G.: ERD/ERS patterns reflecting sensorimotor activation and deactivation. In: Neuper, C., Klimesch, W. (eds.) Event-Related Dynamics of Brain Oscillations, Progress in Brain Research, vol. 159,  chap. 14, pp. 211–222. Elsevier, Netherlands (2006). DOI 10.1016/S0079-6123(06)59014-4
  34. 34.
    Nijboer, F., Furdea, A., Gunst, I., Mellinger, J., McFarland, D.J., Birbaumer, N., Kübler, A.: An auditory brain–computer interface (BCI). Neurosci. Lett. 167, 43–50 (2008)Google Scholar
  35. 35.
    Nijholt, A.: BCI for games: a “state of the art” survey. In: Stevens, S., Saldamarco, S. (eds.) Entertainment Computing – ICEC 2008, pp. 225–228. Springer, Berlin/Heidelberg (2009). DOI  chap. 29
  36. 36.
    Ortner, R., Allison, B., Korisek, G., Gaggl, G., Pfurtscheller, G.: An SSVEP BCI to control a hand orthosis for persons with tetraplegia. IEEE. Trans. Neural Syst. Rehabil. Eng. 19(1), 1–5 (2011)Google Scholar
  37. 37.
    Panicker, R.C., Puthusserypady, S., Sun, Y.: An asynchronous P300 BCI with SSVEP-based control state detection. IEEE Trans. Biomed. Eng. 58, 1781–1788 (2011). DOI 10.1109/TBME.2011.2116018Google Scholar
  38. 38.
    Pfurtscheller, G., Lopes da Silva F.H.: Event-related desynchronization (ERD) and event-related synchronization (ERS). In: Electroencephalography: basic principles, clinical applications and related fields. Williams & Wilkins, Philadelphia (2005)Google Scholar
  39. 39.
    Pfurtscheller, G., Neuper, C.: Motor imagery and direct brain–computer communication. Proc. IEEE 89, 1123–1134 (2001). DOI 10.1109/5.939829Google Scholar
  40. 40.
    Pfurtscheller, G., Wörtz, M., Supp, G., da Silva, F.H.L.: Early onset of post-movement beta electroencephalogram synchronization in the supplementary motor area during self-paced finger movement in man. Neurosci. Lett. 339, 111–114 (2003)Google Scholar
  41. 41.
    Pfurtscheller, G., Leeb, R., Friedman, D., Slater, M.: Centrally controlled heart rate changes during mental practice in immersive virtual environment: a case study with a tetraplegic. Int. J. Psychophysiol. 68, 1–5 (2008). DOI 10.1016/j.ijpsycho.2007.11.003Google Scholar
  42. 42.
    Pfurtscheller, G., Allison, B.Z., Brunner, C., Bauernfeind, G., Solis-Escalante, T., Scherer, R., Zander, T.O., Müller-Putz, G., Neuper, C., Birbaumer, N.: The hybrid BCI. Front. Neurosci. 4, 30 (2010a). DOI 10.3389/fnpro.2010.00003Google Scholar
  43. 43.
    Pfurtscheller, G., Bauernfeind, G., Wriessnegger, S.C., Neuper, C.: Focal frontal (de)oxyhemoglobin responses during simple arithmetic. Int. J. Psychophysiol. 76, 186–192 (2010b). DOI 10.1016/j.ijpsycho.2010.03.013Google Scholar
  44. 44.
    Rogova, G.L., Nimier, V.: Reliability in information fusion: literature survey. In: Proc. of. the 7th Intl. Conference on Informatin Fusion, Stockholm. pp. 1158–1165 (2004)Google Scholar
  45. 45.
    Schalk, G., Wolpaw, J.R., McFarland, D.J., Pfurtscheller, G.: EEG-based communication: Presence of an error potential. Clin. Neurophysiol. 111(12), 2138–2144 (2000)Google Scholar
  46. 46.
    Scherer, R., Müller-Putz, G.R., Pfurtscheller, G.: Self-initiation of EEG-based brain–computer communication using the heart rate response. J. Neural Eng. 4, L23–L29 (2007). DOI 10.1088/1741-2560/4/4/L01Google Scholar
  47. 47.
    Su, Y., Qi, Y., Luo, J.X., Wu, B., Yang, F., Li, Y., Zhuang, Y.T., Zheng, X.X., Chen, W.D.: A hybrid brain–computer interface control strategy in a virtual environment. J. Zhejiang Univ. Sci. C. 12, 351–361 (2011). DOI 10.1631/jzus.C1000208Google Scholar
  48. 48.
    Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791 (2002). DOI 10.1016/S1388-2457n(02)00057-3Google Scholar
  49. 49.
    Wriessnegger, S.C., Kurzmann, J., Neuper, C.: Spatio-temporal differences in brain oxygenation between movement execution and imagery: a multichannel near-infrared spectroscopy study. Int. J. Psychophysiol. 67, 54–63 (2008). DOI 10.1016/j.ijpsycho.2007.10.004Google Scholar
  50. 50.
    Zander, T.O., Kothe, C.: Towards passive brain–computer interfaces: applying brain–computer interface technology to human-machine systems in general. J. Neural Eng. 8, 025,005 (2011). DOI 10.1088/1741-2560/8/2/025005Google Scholar
  51. 51.
    Zander, T.O., Gaertner, M., Kothe, C., Vilimek, R.: Combining eye gaze input with a brain–computer interface for touchless human–computer interaction. Int. J. Hum. Comput. Interaction 27, 38–51 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Gernot R. Müller-Putz
    • 1
  • Robert Leeb
    • 2
  • José d. R. Millán
    • 2
  • Petar Horki
    • 1
  • Alex Kreilinger
    • 1
  • Günther Bauernfeind
    • 1
  • Brendan Z. Allison
    • 1
  • Clemens Brunner
    • 1
  • Reinhold Scherer
    • 1
  1. 1.Institute for Knowledge Discovery, BCI-LabGraz University of TechnologyGrazAustria
  2. 2.Chair in Non-Invasive Brain-Machine InterfaceÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland

Personalised recommendations