Brain-Computer Interfaces pp 113-135

Part of the The Frontiers Collection book series (FRONTCOLL) | Cite as

Detecting Mental States by Machine Learning Techniques: The Berlin Brain–Computer Interface

  • Benjamin Blankertz
  • Michael Tangermann
  • Carmen Vidaurre
  • Thorsten Dickhaus
  • Claudia Sannelli
  • Florin Popescu
  • Siamac Fazli
  • Márton Danóczy
  • Gabriel Curio
  • Klaus-Robert Müller
Chapter

Abstract

The Berlin Brain-Computer Interface (BBCI) uses a machine learning approach to extract user-specific patterns from high-dimensional EEG-features optimized for revealing the user’s mental state. Classical BCI applications are brain actuated tools for patients such as prostheses (see Section 4.1) or mental text entry systems ([1] and see [2–5] for an overview on BCI). In these applications, the BBCI uses natural motor skills of the users and specifically tailored pattern recognition algorithms for detecting the user’s intent. But beyond rehabilitation, there is a wide range of possible applications in which BCI technology is used to monitor other mental states, often even covert ones (see also [6] in the fMRI realm). While this field is still largely unexplored, two examples from our studies are exemplified in Sections 4.3 and 4.4.

References

  1. 1.
    B. Blankertz, M. Krauledat, G. Dornhege, J. Williamson, R. Murray-Smith, and K.-R. Müller, A note on brain actuated spelling with the Berlin Brain-Computer Interface, In C. Stephanidis, (Ed.), Universal access in HCI, Part II, HCII 2007, ser. LNCS, vol. 4555. Springer, Berlin Heidelberg, pp. 759–768 (2007).Google Scholar
  2. 2.
    G. Dornhege, J. del R. Millán, T. Hinterberger, D. McFarland, and K.-R. Müller (Eds.), Toward brain-computer interfacing. MIT Press, Cambridge, MA, (2007).Google Scholar
  3. 3.
    J.R. Wolpaw, N. Birbaumer, D.J. McFarland, G. Pfurtscheller, and T.M. Vaughan, Brain-computer interfaces for communication and control, Clin Neurophysiol, 113(6), 767–791, (2002).CrossRefPubMedGoogle Scholar
  4. 4.
    B. Allison, E. Wolpaw, and J. Wolpaw, Brain-computer interface systems: progress and prospects. Expert Rev Med Devices, 4(4), 463–474, (2007).CrossRefPubMedGoogle Scholar
  5. 5.
    G. Pfurtscheller, C. Neuper, and N. Birbaumer, Human Brain-Computer Interface, In A. Riehle and E. Vaadia (Eds.), Motor cortex in voluntary movements, CRC Press, New York NY: ch. 14, pp. 367–401, (2005).Google Scholar
  6. 6.
    J. Haynes, K. Sakai, G. Rees, S. Gilbert, and C. Frith, Reading hidden intentions in the human brain, Curr Biol, 17, 323–328, (2007).CrossRefPubMedGoogle Scholar
  7. 7.
    B. Blankertz, G. Dornhege, S. Lemm, M. Krauledat, G. Curio, and K.-R. Müller, The Berlin Brain-Computer Interface: machine learning based detection of user specific brain states. J Universal Computer Sci, 12(6), 581–607, (2006).Google Scholar
  8. 8.
    T. Elbert, B. Rockstroh, W. Lutzenberger, and N. Birbaumer, Biofeedback of slow cortical potentials. I. Electroencephalogr Clin Neurophysiol, 48, 293–301, (1980).CrossRefGoogle Scholar
  9. 9.
    N. Birbaumer, A. Kübler, N. Ghanayim, T. Hinterberger, J. Perelmouter, J. Kaiser, I. Iversen, B. Kotchoubey, N. Neumann, and H. Flor, The thought translation device (TTD) for completly paralyzed patients. IEEE Trans Rehabil Eng, 8(2), 190–193, (June 2000).CrossRefPubMedGoogle Scholar
  10. 10.
    M. Schreuder, B. Blankertz, and M. Tangermann, A new auditory multi-class brain-computer interface paradigm: Spatial hearing as an informative cue. PLoS ONE, 5(4),p. e9813, (2010).Google Scholar
  11. 11.
    H. H. Kornhuber and L. Deecke, Hirnpotentialänderungen bei Willkürbewegungen und passiven Bewegungen des Menschen: Bereitschaftspotential und reafferente Potentiale. Pflugers Arch, 284, 1–17, (1965).CrossRefGoogle Scholar
  12. 12.
    W. Lang, M. Lang, F. Uhl, C. Koska, A. Kornhuber, and L. Deecke, Negative cortical DC shifts preceding and accompanying simultaneous and sequential movements. Exp Brain Res, 74(1), 99–104, (1988).Google Scholar
  13. 13.
    R.Q. Cui, D. Huter, W. Lang, and L. Deecke, Neuroimage of voluntary movement: topography of the Bereitschaftspotential, a 64-channel DC current source density study. Neuroimage, 9(1), 124–134, (1999).CrossRefPubMedGoogle Scholar
  14. 14.
    B. Blankertz, G. Dornhege, M. Krauledat, K.-R. Müller, V. Kunzmann, F. Losch, and G. Curio, The Berlin Brain-Computer Interface: EEG-based communication without subject training. IEEE Trans Neural Syst Rehabil Eng, 14(2), 147–152, (2006). [Online]. Available: http://dx.doi.org/10.1109/TNSRE.2006.875557. Accessed 14 Sept 2010.
  15. 15.
    B. Blankertz, G. Dornhege, M. Krauledat, V. Kunzmann, F. Losch, G. Curio, and K.-R. Müller, The berlin brain-computer interface: machine-learning based detection of user specific brain states, In G. Dornhege, J. del R. Millán, T. Hinterberger, D. McFarland, and K.-R. Müller (Eds.), Toward Brain-Computer Interfacing, MIT, Cambridge, MA, pp. 85–101, (2007).Google Scholar
  16. 16.
    B. Blankertz, G. Dornhege, C. Schäfer, R. Krepki, J. Kohlmorgen, K.-R. Müller, V. Kunzmann, F. Losch, and G. Curio, Boosting bit rates and error detection for the classification of fast-paced motor commands based on single-trial EEG analysis. IEEE Trans Neural Syst Rehabil Eng, 11(2), 127–131, (2003). [Online]. Available: http://dx.doi.org/10.1109/TNSRE.2003.814456. Accessed on 14 Sept 2010.Google Scholar
  17. 17.
    D. Krusienski, G. Schalk, D.J. McFarland, and J. Wolpaw, A mu-rhythm matched filter for continuous control of a brain-computer interface. IEEE Trans Biomed Eng, 54(2), 273–280, (2007).CrossRefPubMedGoogle Scholar
  18. 18.
    V.V. Nikulin and T. Brismar, Phase synchronization between alpha and beta oscillations in the human electroencephalogram. Neuroscience, 137, 647–657, (2006).CrossRefPubMedGoogle Scholar
  19. 19.
    G. Pfurtscheller and F.H.L. da Silva, Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol, 110(11), 1842–1857, (Nov 1999).CrossRefPubMedGoogle Scholar
  20. 20.
    P.L. Nunez, R. Srinivasan, A.F. Westdorp, R.S. Wijesinghe, D.M. Tucker, R.B. Silberstein, and P.J. Cadusch, EEG coherency I: statistics, reference electrode, volume conduction, Laplacians, cortical imaging, and interpretation at multiple scales. Electroencephalogr Clin Neurophysiol, 103(5), 499–515, (1997).CrossRefPubMedGoogle Scholar
  21. 21.
    B. Blankertz, R. Tomioka, S. Lemm, M. Kawanabe, and K.-R. Müller, Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Process Mag, 25(1), 41–56, (Jan. 2008). [Online]. Available: http://dx.doi.org/10.1109/MSP.2008.4408441. Accessed on 14 Sept 2010.
  22. 22.
    D.J. McFarland, L.M. McCane, S.V. David, and J.R. Wolpaw, Spatial filter selection for EEG-based communication. Electroencephalogr Clin Neurophysiol, 103, 386–394, (1997).CrossRefPubMedGoogle Scholar
  23. 23.
    N. Hill, T.N. Lal, M. Tangermann, T. Hinterberger, G. Widman, C.E. Elger, B. Schölkopf, and N. Birbaumer, Classifying event-related desynchronization in EEG, ECoG and MEG signals, In G. Dornhege, J. del R. Millán, T. Hinterberger, D. McFarland, and K.-R. Müller (Eds.), Toward brain-computer interfacing, MIT, Cambridge, MA, pp. 235–260, (2007).Google Scholar
  24. 24.
    M. Grosse-Wentrup, K. Gramann, and M. Buss, Adaptive spatial filters with predefined region of interest for EEG based brain-computer-interfaces, In B. Schölkopf, J. Platt, and T. Hoffman, (Eds.), Advances in neural information processing systems 19, pp. 537–544, (2007).Google Scholar
  25. 25.
    G. Dornhege, M. Krauledat, K.-R. Müller, and B. Blankertz, General signal processing and machine learning tools for BCI, In G. Dornhege, J. del R. Millán, T. Hinterberger, D. McFarland, and K.-R. Müller, (Eds.), Toward brain-computer interfacing, MIT, Cambridge, MA, pp. 207–233, (2007).Google Scholar
  26. 26.
    L.C. Parra, C.D. Spence, A.D. Gerson, and P. Sajda, Recipes for the linear analysis of EEG, Neuroimage, 28(2), 326–341, (2005).CrossRefPubMedGoogle Scholar
  27. 27.
    K. Fukunaga, Introduction to statistical pattern recognition, 2nd ed. Academic, San Diego CA, (1990).Google Scholar
  28. 28.
    C. Guger, H. Ramoser, and G. Pfurtscheller, Real-time EEG analysis with subject-specific spatial patterns for a Brain Computer Interface (BCI). IEEE Trans Neural Syst Rehabil Eng, 8(4), 447–456, (2000).Google Scholar
  29. 29.
    H. Ramoser, J. Müller-Gerking, and G. Pfurtscheller, Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans Rehabil Eng, 8(4), 441–446, (2000).CrossRefPubMedGoogle Scholar
  30. 30.
    S. Lemm, B. Blankertz, G. Curio, and K.-R. Müller, Spatio-spectral filters for improving classification of single trial EEG. IEEE Trans Biomed Eng, 52(9), 1541–1548, (2005). [Online]. Available: http://dx.doi.org/10.1109/TBME.2005.851521. Accessed on 14 Sept 2010.Google Scholar
  31. 31.
    G. Dornhege, B. Blankertz, M. Krauledat, F. Losch, G. Curio, and K.-R. Müller, Optimizing spatio-temporal filters for improving brain-computer interfacing, In Y. Weiss, B. Schölkopf, and J. Platt, (Eds.), Advances in neural Information Processing Systems (NIPS 05), vol. 18. MIT, Cambridge, MA, pp. 315–322, (2006).Google Scholar
  32. 32.
    R. Tomioka, K. Aihara, and K.-R. Müller, Logistic regression for single trial EEG classification, In B. Schölkopf, J. Platt, and T. Hoffman (Eds.), Advances in neural information processing systems 19, MIT, Cambridge, MA, pp. 1377–1384, (2007).Google Scholar
  33. 33.
    K.-R. Müller, C.W. Anderson, and G.E. Birch, Linear and non-linear methods for brain-computer interfaces. IEEE Trans Neural Syst Rehabil Eng, 11(2), 165–169, (2003).CrossRefPubMedGoogle Scholar
  34. 34.
    R.O. Duda, P.E. Hart, and D.G. Stork, Pattern classification, 2nd ed. Wiley, New York, (2001).Google Scholar
  35. 35.
    C. Stein, Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proceeding of 3rd Berkeley Symposium Mathematical Statistics. Probability 1, 197–206, (1956).Google Scholar
  36. 36.
    J.H. Friedman, Regularized discriminant analysis. J Am Stat Assoc, 84(405), 165–175, (1989).CrossRefGoogle Scholar
  37. 37.
    O. Ledoit and M. Wolf, A well-conditioned estimator for large-dimensional covariance matrices. J Multivar Anal, 88, 365–411, (2004).CrossRefGoogle Scholar
  38. 38.
    C. Vidaurre, N. Krämer, B. Blankertz, and A. Schlögl, Time domain parameters as a feature for EEG-based Brain Computer Interfaces. Neural Netw, 22, 1313–1319, (2009).Google Scholar
  39. 39.
    J. Schäfer and K. Strimmer, A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Stat Appl Genet Mol Biol, 4, Article32, (2005) [Online]. Available: http://www.bepress.com/sagmb/vol4/iss1/art32/.
  40. 40.
    C. Neuper, R. Scherer, M. Reiner, and G. Pfurtscheller, Imagery of motor actions: Differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. Brain Res Cogn Brain Res, 25(3), 668–677, (2005).CrossRefPubMedGoogle Scholar
  41. 41.
    V.V. Nikulin, F.U. Hohlefeld, A.M. Jacobs, and G. Curio, Quasi-movements: a novel motor-cognitive phenomenon, Neuropsychologia, 46(2), 727–742, (2008). [Online]. Available: http://dx.doi.org/10.1016/j.neuropsychologia.2007.10.008. Accessed on 14 Sept 2010.
  42. 42.
    B. Blankertz, G. Dornhege, M. Krauledat, K.-R. Müller, and G. Curio, The non-invasive Berlin Brain-Computer Interface: fast acquisition of effective performance in untrained subjects. Neuroimage, 37(2), 539–550, (2007). [Online]. Available: http://dx.doi.org/10.1016/j.neuroimage.2007.01.051. Accessed on 14 Sept 2010.
  43. 43.
    J.R. Wolpaw, D.J. McFarland, and T.M. Vaughan, Brain-computer interface research at the Wadsworth Center. IEEE Trans Rehabil Eng, 8(2), 222–226, (2000).Google Scholar
  44. 44.
    A. Schlögl, J. Kronegg, J. Huggins, and S.G. Mason, Evaluation Criteria for BCI Research, In G. Dornhege, J. del R. Millán, T. Hinterberger, D. McFarland, and K.-R. Müller, (Eds.), Towards Brain-Computer Interfacing, MIT, Cambridge, MA, pp. 297–312, 2007.Google Scholar
  45. 45.
    G. Dornhege, Increasing information transfer rates for brain-computer interfacing, Ph.D. dissertation, University of Potsdam, (2006).Google Scholar
  46. 46.
    K.-R. Müller and B. Blankertz, Toward noninvasive brain-computer interfaces. IEEE Signal Process Mag, 23(5), 125–128, (Sept 2006).CrossRefGoogle Scholar
  47. 47.
    J. Williamson, R. Murray-Smith, B. Blankertz, M. Krauledat, and K.-R. Müller, Designing for uncertain, asymmetric control: Interaction design for brain-computer interfaces. Int J Hum-Comput Stud, 67(10), 827–841, (2009).Google Scholar
  48. 48.
    B. Blankertz, F. Losch, M. Krauledat, G. Dornhege, G. Curio, and K.-R. Müller, The Berlin Brain-Computer Interface: Accurate performance from first-session in BCI-naive subjects. IEEE Trans Biomed Eng, 55(10), 2452–2462, 2008. [Online]. Available: http://dx.doi.org/10.1109/TBME.2008.923152. Accessed on 14 Sept 2010.
  49. 49.
    A. Kübler and K.-R. Müller, An introduction to brain computer interfacing, In G. Dornhege, J. del R. Millán, T. Hinterberger, D. McFarland, and K.-R. Müller, (Eds.), Toward Brain-Computer Interfacing, MIT, Cambridge, MA, pp. 1–25, (2007).Google Scholar
  50. 50.
    B. Blankertz, C. Sannelli, S. Halder, E. M. Hammer, A. Kübler, K. R. Müller, G. Curio, and T. Dickhaus, Neurophysiological predictor of SMR-based BCI performance. NeuroImage, 51, 1303–1309, (2010).Google Scholar
  51. 51.
    T. Dickhaus, C. Sannelli, K.-R. Müller, G. Curio, and B. Blankertz, Predicting BCI performance to study BCI illiteracy. BMC Neurosci 2009, 10(Suppl 1), P84, (2009).Google Scholar
  52. 52.
    B. Blankertz and C. Vidaurre, Towards a cure for BCI illiteracy: Machine-learning based co-adaptive learning. BMC Neuroscience 2009, 10, (Suppl 1), P85, (2009).CrossRefGoogle Scholar
  53. 53.
    C. Vidaurre and B. Blankertz, Towards a cure for BCI illiteracy, Open Access Brain Topogr, 23, 1303–1309, (2010).Google Scholar
  54. 54.
    C. Vidaurre, A. Schlögl, B. Blankertz, M. Kawanabe, and K.-R. Müller, Unsupervised adaptation of the LDA classifier for Brain-Computer Interfaces, in Proceedings of the 4th International Brain-Computer Interface Workshop and Training Course 2008. Verlag der Technischen Universität Graz, (2008), pp. 122–127.Google Scholar
  55. 55.
    J. Kohlmorgen, G. Dornhege, M. Braun, B. Blankertz, K.-R. Müller, G. Curio, K. Hagemann, A. Bruns, M. Schrauf, and W. Kincses, Improving human performance in a real operating environment through real-time mental workload detection, In G. Dornhege, J. del R. Millán, T. Hinterberger, D. McFarland, and K.-R. Müller, (Eds.), Toward Brain-Computer Interfacing, MIT, Cambridge, MA, pp. 409–422, (2007).Google Scholar
  56. 56.
    K.-R. Müller, M. Tangermann, G. Dornhege, M. Krauledat, G. Curio, and B. Blankertz, Machine learning for real-time single-trial EEG-analysis: from brain-computer interfacing to mental state monitoring. J Neurosci Methods, 167(1), 82–90, (2008). [Online]. Available: http://dx.doi.org/10.1016/j.jneumeth.2007.09.022. Accessed on 14 Sept 2010.
  57. 57.
    F. Popescu, S. Fazli, Y. Badower, B. Blankertz, and K.-R. Müller, Single trial classification of motor imagination using 6 dry EEG electrodes, PLoS ONE, 2(7), (2007). [Online]. Available: http://dx.doi.org/10.1371/journal.pone.0000637
  58. 58.
    S. Fazli, M. Danóczy, M. Kawanabe, and F. Popescu, Asynchronous, adaptive BCI using movement imagination training and rest-state inference. IASTED’s Proceedings on Artificial Intelligence and Applications 2008. Innsbruck, Austria, ACTA Press Anaheim, CA, USA, (2008), pp. 85–90. [Online]. Available: http://portal.acm.org/citation.cfm?id=1712759.1712777
  59. 59.
    L. Ramsey, M. Tangermann, S. Haufe, and B. Blankertz, Practicing fast-decision BCI using a “goalkeeper” paradigm. BMC Neurosci 2009, 10(Suppl 1), P69, (2009).CrossRefGoogle Scholar
  60. 60.
    M. Krauledat, G. Dornhege, B. Blankertz, G. Curio, and K.-R. Müller, The Berlin brain-computer interface for rapid response. Biomed Tech, 49(1), 61–62, (2004).Google Scholar
  61. 61.
    A. Kübler, B. Kotchoubey, J. Kaiser, J. Wolpaw, and N. Birbaumer, Brain-computer communication: Unlocking the locked in. Psychol Bull, 127(3), 358–375, (2001).CrossRefPubMedGoogle Scholar
  62. 62.
    A. Kübler, F. Nijboer, J. Mellinger, T. M. Vaughan, H. Pawelzik, G. Schalk, D. J. McFarland, N. Birbaumer, and J. R. Wolpaw, Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface. Neurology, 64(10), 1775–1777, (2005).CrossRefPubMedGoogle Scholar
  63. 63.
    N. Birbaumer and L. Cohen, Brain-computer interfaces: communication and restoration of movement in paralysis. J Physiol, 579, 621–636, (2007).CrossRefPubMedGoogle Scholar
  64. 64.
    N. Birbaumer, C. Weber, C. Neuper, E. Buch, K. Haapen, and L. Cohen, Physiological regulation of thinking: brain-computer interface (BCI) research. Prog Brain Res, 159, 369–391, (2006).Google Scholar
  65. 65.
    L. Hochberg, M. Serruya, G. Friehs, J. Mukand, M. Saleh, A. Caplan, A. Branner, D. Chen, R. Penn, and J. Donoghue, Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature, 442(7099), 164–171, (Jul 2006).CrossRefPubMedGoogle Scholar
  66. 66.
    J. Conradi, B. Blankertz, M. Tangermann, V. Kunzmann, and G. Curio, Brain-computer interfacing in tetraplegic patients with high spinal cord injury. Int J Bioelectromagnetism, 11, 65–68, (2009).Google Scholar
  67. 67.
    R. Krepki, B. Blankertz, G. Curio, and K.-R. Müller, The Berlin Brain-Computer Interface (BBCI): towards a new communication channel for online control in gaming applications. J Multimedia Tools, 33(1), 73–90, (2007). [Online]. Available: http://dx.doi.org/10.1007/s11042-006-0094-3. Accessed on 14 Sept 2010.
  68. 68.
    R. Krepki, G. Curio, B. Blankertz, and K.-R. Müller, Berlin brain-computer interface - the hci communication channel for discovery. Int J Hum Comp Studies, 65, 460–477, (2007), special Issue on Ambient Intelligence.CrossRefGoogle Scholar
  69. 69.
    R. Leeb, F. Lee, C. Keinrath, R. Scherer, H. Bischof, and G. Pfurtscheller, Brain-computer communication: motivation, aim, and impact of exploring a virtual apartment. IEEE Trans Neural Syst Rehabil Eng, 15(4), 473–482, (2007).CrossRefPubMedGoogle Scholar
  70. 70.
    A. Gerson, L. Parra, and P. Sajda, Cortically coupled computer vision for rapid image search. IEEE Trans Neural Syst Rehabil Eng, 14(2), 174–179, (2006).CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Benjamin Blankertz
    • 1
    • 2
  • Michael Tangermann
    • 1
  • Carmen Vidaurre
    • 1
  • Thorsten Dickhaus
    • 1
  • Claudia Sannelli
    • 1
  • Florin Popescu
    • 2
  • Siamac Fazli
    • 1
  • Márton Danóczy
    • 1
  • Gabriel Curio
    • 3
  • Klaus-Robert Müller
    • 1
  1. 1.Machine Learning LaboratoryBerlin Institute of TechnologyBerlinGermany
  2. 2.Fraunhofer FIRST (IDA)BerlinGermany
  3. 3.Campus Benjamin FranklinCharité University Medicine BerlinBerlinGermany

Personalised recommendations