Predicting Motor Intentions with Closed-Loop Brain-Computer Interfaces

  • Matthias Schultze-Kraft
  • Mario Neumann
  • Martin Lundfall
  • Patrick Wagner
  • Daniel Birman
  • John-Dylan Haynes
  • Benjamin Blankertz
Chapter
First Online:
Part of the SpringerBriefs in Electrical and Computer Engineering book series (BRIEFSELECTRIC)

Abstract

We present two studies in which brain-computer interfaces (BCIs) used two related EEG signals, the readiness potential (RP) and the lateralized readiness potential (LRP), in order to predict and feed back motor intentions in real-time. In each of the studies, the experimental task was designed as a game that the subjects played against the BCI. In one of the experiments, subjects played a “duel” game against the BCI. They were challenged to perform spontaneous button presses but to withhold any movement when interrupted by a stop signal. This stop signal was controlled by the BCI that had been trained to predict movements by detecting the occurrence of RPs in the ongoing EEG. In the other experiment, participants played a “matching pennies” game. They won a point if they raised a different hand than the opponent at the end of a countdown and lost a point otherwise. The opponent was played by the BCI, who had been trained to predict from the LRP in the ongoing EEG which hand the subject would move at the end of the countdown. Hence, in both experiments a key feature of the BCI was its closed-loop nature, that is the ability to predict the motor intention in real-time and provide an immediate feedback of the prediction to the subject. In both experiments, prediction accuracies of the BCI were substantially higher compared to random predictions, thereby demonstrating the success of this approach. This allows researchers to use BCIs as research tools to address questions from cognitive neuroscience and provide new insights into the coupling of motor preparatory signals and the corresponding actions.

This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This work was supported by the Bernstein Focus: Neurotechnology from the German Federal Ministry of Education and Research (BMBF grant 01GQ0850), by the Bernstein Computational Neuroscience Program (BMBF grant 01GQ1001C), the Research Training Group “Sensory Computation in Neural Systems” (GRK 1589/1-2), the Collaborative Research Center “Volition and Cognitive Control: Mechanisms, Modulations, Dysfunctions” (SFB 940/1) and the German Research Foundation (DFG grants EXC 257 and KFO 247).

References

  1. 1.
    Allison BZ, Dunne S, Leeb R, Millán JDR, Nijholt A (2012) Towards practical brain-computer interfaces: bridging the gap from research to realworld applications. Springer Science & Business Media, HeidelbergGoogle Scholar
  2. 2.
    Bai O, Rathi V, Lin P, Huang D, Battapady H, Fei D-YY, Schneider L, Houdayer E, Chen X, Hallett M (2011) Prediction of human voluntary movement before it occurs. Clin Neurophysiol 122(2):364–372CrossRefGoogle Scholar
  3. 3.
    Bai O, Vorbach S, Hallett M, Floeter MK (2006) Movement-related cortical potentials in primary lateral sclerosis. Ann Neurol 59(4):682–690CrossRefGoogle Scholar
  4. 4.
    Birbaumer N, Ghanayim N, Hinterberger T, Iversen I, Kotchoubey B, Kübler A, Perelmouter J, Taub E, Flor H (1999) A spelling device for the paralysed. Nature 398(6725):297–298CrossRefGoogle Scholar
  5. 5.
    Blankertz B, Acqualagna L, Dähne S, Haufe S, Schultze-Kraft M, Sturm I, U2¢umlic M, Wenzel MA, Curio G, Müller K-R (2016) The Berlin brain-computer interface: progress beyond communication and control. Front Neurosci 10:530Google Scholar
  6. 6.
    Blankertz B, Dornhege G, Krauledat M, Müller K-R, Curio G (2007) The non-invasive Berlin brain-computer interface: fast acquisition of effective performance in untrained subjects. NeuroImage 37(2):539–550CrossRefGoogle Scholar
  7. 7.
    Blankertz B, Dornhege G, Lemm S, Krauledat M, Curio G, Müller K-R (2006) The Berlin brain-computer interface: machine learning based detection of user specific brain states. J UCS 12(6):581–607Google Scholar
  8. 8.
    Blankertz B, Lemm S, Treder M, Haufe S, Müller K-R (2011) Singletrial analysis and classiffcation of ERP components - a tutorial. NeuroImage 56(2):814–825CrossRefGoogle Scholar
  9. 9.
    Blankertz B, Tangermann M, Vidaurre C, Fazli S, Sannelli C, Haufe S, Maeder C, Ramsey LE, Sturm I, Curio G, Müller KR (2010) The Berlin brain-computer interface: non-medical uses of BCI technology. Front Neurosci 4:198CrossRefGoogle Scholar
  10. 10.
    Brass M, Haggard P (2008) The what, when, whether model of intentional action. Neuroscientist 14(4):319–325CrossRefGoogle Scholar
  11. 11.
    Brunia CH, Voorn FJ, Berger MP (1985). Movement related slow potentials. II. A contrast between finger and foot movements in left-handed subjects. Electroencephalogr Clin Neurophysiol 60:135–145Google Scholar
  12. 12.
    Brunner C, Birbaumer N, Blankertz B, Guger C, Kübler A, Mattia D, del R. Millán J, Miralles F, Nijholt A, Opisso E, Ramsey N, Salomon P, Müller-Putz GR (2015). BNCI Horizon 2020: towards a roadmap for the BCI community. Brain Comput Interfaces 2(1):1–10Google Scholar
  13. 13.
    Coles MG, Gratton G, Donchin E (1988) Detecting early communication: using measures of movement-related potentials to illuminate human information processing. Biol Psychol 26(1):69–89CrossRefGoogle Scholar
  14. 14.
    Cui RQ, Huter D, Lang W, Deecke L (1999) Neuroimage of voluntary movement: topography of the Bereitschaftspotential, a 64-channel DC current source density study. NeuroImage 9(1):124–134CrossRefGoogle Scholar
  15. 15.
    Eimer M (1998) The lateralized readiness potential as an on-line measure of central response activation processes. Behav Res Methods Instrum Comput 30(1):146–156CrossRefGoogle Scholar
  16. 16.
    Haufe S, Kim J-W, Kim I-H, Sonnleitner A, Schrauf M, Curio G, Blankertz B (2014) Electrophysiology-based detection of emergency braking intention in real-world driving. J Neural Eng 11(5):056011CrossRefGoogle Scholar
  17. 17.
    Haufe S, Treder MS, Gugler MF, Sagebaum M, Curio G, Blankertz B (2011) EEG potentials predict upcoming emergency brakings during simulated driving. J Neural Eng 8(5):056001Google Scholar
  18. 18.
    Haynes J-D (2011) Decoding and predicting intentions. Ann N Y Acad Sci 1224(1):9–21MathSciNetCrossRefGoogle Scholar
  19. 19.
    Kohlmorgen J, Dornhege G, Braun M, Blankertz B, Müller K-R, Curio G, Hagemann K, Bruns A, Schrauf M, Kincses W (2007) Improving human performance in a real operating environment through real-time mental workload detection. In: Dornhege G, del R. Millán J, Hinterberger T, McFarland D, Müller K-R (eds) Toward brain-computer interfacing. MIT press, Cambridge, MA, pp 409–422Google Scholar
  20. 20.
    Kornhuber HH, Deecke L (1965) Hirnpotentialänderungen bei Willkürbewegungen und passiven Bewegungen des Menschen: Bereitschaftspotential und reafferente Potentiale. Pflügers Arch 284:1–17CrossRefGoogle Scholar
  21. 21.
    Libet B, Gleason CA, Wright EW, Pearl DK (1983) Time of conscious intention to act in relation to onset of cerebral activity (readiness-potential). The unconscious initiation of a freely voluntary act. Brain J Neurol 106(3):623–642Google Scholar
  22. 22.
    Maoz U, Ye S, Ross IB, Mamelak AN, Koch C (2012) Predicting action content on-line and in real time before action onset - an intracranial human study. In: Bartlett PL, Pereira FCN, Burges CJC, Bottou L, Weinberger KQ (eds) NIPS, pp 881–889Google Scholar
  23. 23.
    Müller K-R, Tangermann M, Dornhege G, Krauledat M, Curio G, Blankertz B (2008) Machine learning for real-time single-trial EEG-analysis: from brain-computer interfacing to mental state monitoring. J Neurosci Methods 167(1):82–90CrossRefGoogle Scholar
  24. 24.
    Naumann L, Schultze-Kraft M, Dähne S, Blankertz B (2017) Prediction of difficulty levels in video games from ongoing EEG. In: Gamberini L, Spagnolli A, Jacucci G, Blankertz B, Freeman J (eds) Symbiotic interaction, vol 9961. Springer International Publishing, Cham, pp 125–136Google Scholar
  25. 25.
    Salvaris M, Haggard P (2014) Decoding intention at sensorimotor timescales. PLoS ONE 9(2):e85100CrossRefGoogle Scholar
  26. 26.
    Schultze-Kraft M, Birman D, Rusconi M, Allefeld C, Görgen K, Dähne S, Blankertz B, Haynes J-D (2016a) The point of no return in vetoing self-initiated movements. Proc Nat Acad Sci 113(4):1080–1085Google Scholar
  27. 27.
    Schultze-Kraft M, Dähne S, Gugler M, Curio G, Blankertz B (2016) Unsupervised classification of operator workload from brain signals. J Neural Eng 13(3):036008CrossRefGoogle Scholar
  28. 28.
    Wolpaw J, Wolpaw EW (2012) Brain-computer interfaces: principles and practice. OUP, USAGoogle Scholar

Copyright information

© The Author(s) 2017

Authors and Affiliations

  • Matthias Schultze-Kraft
    • 1
    • 2
    • 3
  • Mario Neumann
    • 1
  • Martin Lundfall
    • 1
  • Patrick Wagner
    • 1
  • Daniel Birman
    • 3
  • John-Dylan Haynes
    • 2
    • 3
    • 4
  • Benjamin Blankertz
    • 1
    • 2
    • 3
  1. 1.Neurotechnology GroupTechnische Universität BerlinBerlinGermany
  2. 2.Bernstein Focus: NeurotechnologyBerlinGermany
  3. 3.Bernstein Center for Computational Neuroscience BerlinBerlinGermany
  4. 4.Berlin Center for Advanced NeuroimagingBerlinGermany

Personalised recommendations