Experimental Brain Research

, Volume 154, Issue 4, pp 521–526 | Cite as

A multimodal brain-based feedback and communication system

  • Thilo Hinterberger
  • Nicola Neumann
  • Mirko Pham
  • Andrea Kübler
  • Anke Grether
  • Nadine Hofmayer
  • Barbara Wilhelm
  • Herta Flor
  • Niels Birbaumer
Research Article

Abstract

The Thought Translation Device (TTD) is a brain-computer interface based on the self-regulation of slow cortical potentials (SCPs) and enables completely paralyzed patients to communicate using their brain potentials. Here, an extended version of the TTD is presented that has an auditory and a combined visual and auditory feedback modality added to the standard visual feedback. This feature is necessary for locked-in patients who are no longer able to focus their gaze. In order to test performance of physiological regulation with auditory feedback 54 healthy participants were randomly assigned to visual, auditory or combined visual-auditory feedback of slow cortical potentials. The training consisted of three sessions with 500 trials per session with random assignment of required cortical positivity or negativity in half of the trials. The data show that physiological regulation of SCPs can be learned with auditory and combined auditory and visual feedback although the performance of auditory feedback alone was significantly worse than with visual feedback alone.

Keywords

Slow cortical potentials (SCPs) Auditory feedback Visual feedback Thought translation device (TTD) 

References

  1. Birbaumer N, Elbert T, Canavan AGM, Rockstroh B (1990) Slow potentials of the cerebral cortex and behaviour. Physiol Rev 70:1–41PubMedGoogle Scholar
  2. Birbaumer N, Flor H, Ghanayim N, Hinterberger T, Iverson I, Taub E, Kotchoubey B, Kübler A, Perelmouter J (1999) A brain-controlled spelling device for the completely paralyzed. Nature 398:297–298CrossRefPubMedGoogle Scholar
  3. Birbaumer N, Kübler A, Ghanayim N, Hinterberger T, Perelmouter J, Kaiser J, Iversen I, Kotchoubey B, Neumann N, Flor H (2000a) The thought translation device (TTD) for completely paralyzed patients. IEEE Trans Rehabil Eng 8:190–193CrossRefPubMedGoogle Scholar
  4. Birbaumer N, Kübler A, Hinterberger T (2000b) Ein Hirn-Computer-Interface zur Kommunikation nach vollständigem oder teilweisem Ausfall der Motorik (“thought-translation-device” TTD). Management of neuromuscular diseases. Sonderheft September 2000, 2–3Google Scholar
  5. Brener JM (1982) Psychobiological mechanisms in biofeedback. In: White L, Tursky B (eds) Clinical biofeedback: efficacy and mechanisms. Guilford, New York, pp 24–47Google Scholar
  6. Hinterberger T (1999) Entwicklung und Optimierung eines Gehirn-Computer-Interfaces mit langsamen Hirnpotentialen. Dissertation in der Fakultät für Physik an der Eberhard-Karls-Universität Tübingen, Schwäbische VerlagsgesellschaftGoogle Scholar
  7. Hinterberger T, Kotchoubey B, Kaiser J, Kübler A, Neumann N, Perelmouter J, Strehl U, Birbaumer N (2000) Anwendungen der Selbstkontrolle langsamer kortikaler Potentiale. Verhaltenstherapie 10:219–227CrossRefGoogle Scholar
  8. Hinterberger T, Kübler A, Kaiser J, Neumann N, Birbaumer N (2003a) A brain-computer-interface (BCI) for the locked-in: comparison of different EEG classifications for the Thought Translation Device (TTD). Clin Neurophysiol 114:416–425CrossRefPubMedGoogle Scholar
  9. Hinterberger T, Mellinger J, Birbaumer N (2003b) The Thought Translation Device: structure of a multimodal brain-computer communication system. Proceedings of the 1st International IEEE EMBS Conference on Neural Engineering, 2003, pp 603–606Google Scholar
  10. Hinterberger T, Veit R, Strehl U, Erb M, Kotchoubey B, Flor H, Birbaumer N (in press) The relationship between BOLD-signal of fMRI with slow cortical potentials. Exp Brain ResGoogle Scholar
  11. Kaiser J, Perelmouter J, Iversen I, Neumann N, Ghanayim N, Hinterberger T, Kübler A, Kotchoubey B, Birbaumer N (2001) Self-initiation of EEG-based communication in paralyzed patients. Clin Neurophysiol 112:551–554CrossRefPubMedGoogle Scholar
  12. Kaiser J, Kübler A, Hinterberger T, Neumann N, Birbaumer N (2002) A non-invasive communication device for the paralyzed: a case report. Minimal Inv Neurosurg 45:19–23CrossRefGoogle Scholar
  13. Kotchoubey B, Schleichert H, Lutzenberger W, Birbaumer N (1997) A new method for self-regulation of slow cortical potentials in a timed paradigm. Appl Psychophysiol Biofeedback 22:77–93CrossRefPubMedGoogle Scholar
  14. Kotchoubey B, Strehl U, Holzapfel S, Blankenhorn V, Fröscher W, Birbaumer N (1999) Negative potential shifts in the prediction of the outcome of neurofeedback therapy in epilepsy. Clin Neurophysiol 110:683–686CrossRefPubMedGoogle Scholar
  15. Kübler A, Kotchoubey B, Ghanayim N, Hinterberger T, Perelmouter J, Schauer M, Fritsch C, Birbaumer N (1998) A thought translation device for brain computer communication. Studia Psychol 40:17–31Google Scholar
  16. Kübler A, Kotchoubey B, Hinterberger T, Ghanayim N, Perelmouter J, Schauer M, Fritsch C, Taub E, Birbaumer N (1999) The thought-translation-device: a neurophysiological approach to communication in total motor paralysis. Exp Brain Res 124:223–232CrossRefPubMedGoogle Scholar
  17. Kübler A, Neumann N, Kaiser J, Kotchoubey B, Hinterberger T, Birbaumer N (2001a) Brain-computer communication: self-regulation of slow cortical potentials for verbal communication. Arch Phys Med Rehabil 82:1533–1539CrossRefPubMedGoogle Scholar
  18. Kübler A, Kotchoubey B, Kaiser J, Wolpaw JR, Birbaumer N (2001b) Brain-computer communication: unlocking the locked-in. Psychol Bull 127:358–375CrossRefPubMedGoogle Scholar
  19. Lal SKL, Henderson RJ, Carter N, Bath A, Hart MG, Langeluddecke P, Hunyor SN (1998) Effect of feedback signal and psychological characteristics on blood pressure self-manipulation capability. Psychophysiology 35:405–412CrossRefPubMedGoogle Scholar
  20. Llinas RR (2001) I of the vortex: from neurons to self. MIT Press, Cambridge, MAGoogle Scholar
  21. Mishkin M, Malamut B, Bachevalier J (1984) Memories and habits’ two neural systems. In: Squire L, Batters N (eds) Neurobiology of human learning and memory. Guilford, New York, pp 65–77Google Scholar
  22. Neumann N, Birbaumer N (in press) Predictors of successful self-control during brain-computer communication. J Neurol Neurosurg PsychiatryGoogle Scholar
  23. Neumann N, Kübler A, Kaiser J, Hinterberger N, Birbaumer N (2003) Conscious perception of brain states: mental strategies for brain-computer communication. Neuropsychologia 41:1028–1036CrossRefPubMedGoogle Scholar
  24. Neuper C, Müller GR, Kübler A, Birbaumer N, Pfurtscheller G (2003) Clinical application of an EEG-based brain-computer interface: a case study in a patient with severe motor impairment. Clin Neurophysiol 114:399–409CrossRefPubMedGoogle Scholar
  25. Newell A, Rosenbloom PS (1981) Mechanisms of skill acquisition and the law of practice. In: Anderson JR (ed) Cognitive skills and their acquisition. Erlbaum, Hillsdale, NJ, pp 1–55Google Scholar
  26. Nicolelis MAL (2001) Actions from thoughts. Nature 409:403–407CrossRefPubMedGoogle Scholar
  27. Perelmouter J, Kotchoubey B, Kübler A, Birbaumer N (1999) Language support program for thought-translation devices. Automedica 18:67–84Google Scholar
  28. Rockstroh B, Elbert T, Canavan A, Lutzenberger W, Birbaumer N (1989) Slow cortical potentials and behavior. Urban & Schwarzenberg, BaltimoreGoogle Scholar
  29. Rolls ET (1999) The brain and emotion. Oxford University Press, OxfordGoogle Scholar
  30. Teitelbaum P, Schallert T, Whishaw IQ (1983) Sources of spontaneity in “motivated” behavior. In: Satinoff E, Teitelbaum P (eds) Handbook of behavioral neurobiology, vol 6. Plenum, New York, pp 23–65Google Scholar
  31. Wenger MA, Cullen TD (1972) Studies of autonomic balance in children and adults. In: Greenfield N, Sternbach RA (eds) Handbook of psychophysiology. Holt, Rinehart and Winston, New YorkGoogle Scholar

Copyright information

© Springer-Verlag 2004

Authors and Affiliations

  • Thilo Hinterberger
    • 1
  • Nicola Neumann
    • 1
  • Mirko Pham
    • 1
  • Andrea Kübler
    • 1
  • Anke Grether
    • 1
  • Nadine Hofmayer
    • 1
  • Barbara Wilhelm
    • 1
  • Herta Flor
    • 3
  • Niels Birbaumer
    • 1
    • 2
  1. 1.Institute of Medical Psychology and Behavioral NeurobiologyUniversity of TübingenTübingenGermany
  2. 2.Center for Cognitive NeuroscienceUniversity of TrentoTrentoItaly
  3. 3.Department of Neuropsychology at the University of HeidelbergCentral Institute of Mental HealthMannheimGermany

Personalised recommendations