Advertisement

Towards BCI-Based Implicit Control in Human–Computer Interaction

  • Thorsten O. Zander
  • Jonas Brönstrup
  • Romy Lorenz
  • Laurens R. Krol
Chapter
Part of the Human–Computer Interaction Series book series (HCIS)

Abstract

In this chapter a specific aspect of Physiological Computing, that of implicit Human–Computer Interaction, is defined and discussed. Implicit Interaction aims at controlling a computer system by behavioural or psychophysiological aspects of user state, independently of any intentionally communicated command. This introduces a new type of Human–Computer Interaction, which in contrast to most forms of interaction implemented nowadays, does not require the user to explicitly communicate with the machine. Users can focus on understanding the current state of the system and developing strategies for optimally reaching the goal of the given interaction. For example, the system can assess the user state by means of passive Brain-Computer Interfaces, which the user needs not even be aware of. Based on this information and the given context the system can adapt automatically to the current strategies of the user. In a first study, a proof of principle is given, by implementing an Implicit Interaction to guide simple cursor movements in a 2D grid to a target. The results of this study clearly indicate the high potential of Implicit Interaction and introduce a new bandwidth of applications for passive Brain-Computer Interfaces.

Keywords

Amyotrophic Lateral Sclerosis User State Motor Imagery Technical System Implicit Information 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Acqualagna L, Blankertz B (2011) A gaze independent spelling based on rapid serial visual presentation. In: Proceedings IEEE engineering medicine biology society conference, pp 4560–4563Google Scholar
  2. Bai O, Lin P, Vorbach S, Floeter MK, Hattori N, Hallett M (2008) A high performance sensorimotor beta rhythm-based brain-computer interface associated with human natural motor behavior. J Neural Eng 5(1):24–35Google Scholar
  3. Balderas D, Zander TO, Bachl F, Neuper C, Scherer R (2011) Restricted boltzmann machines as useful tool for detecting oscillatory EEG components. In: Proceedings of the 5th international brain-computer interface conference, Graz, pp 68–71Google Scholar
  4. Beverina F, Palmas G, Silvoni S, Piccione F, Giove S (2003) User adaptive BCIs: SSVEP and P300 based interfaces. J Psych Nology 1:331–354Google Scholar
  5. Bin G, Gao X, Yan Z, Hong B, Gao S (2009) An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. J Neural Eng 6(4):046,002Google Scholar
  6. Birbaumer N (2006) Breaking the silence: brain-computer interfaces (BCI) for communication and motor control. Psychophysiology 43(6):517–532CrossRefGoogle Scholar
  7. Birbaumer N, Cohen LG (2007) Brain-computer interfaces: communication and restoration of movement in paralysis. J Physiol 579(3):621–636CrossRefGoogle Scholar
  8. Birbaumer N, Elbert T, Canavan AG, Rockstroh B (1990) Slow potentials of the cerebral cortex and behavior. Physiol Rev 70(1):1–41Google Scholar
  9. Birbaumer N, Ghanayim N, Hinterberger T, Iversen I, Kotchoubey B, Kübler A, Perelmouter J, Taub E, Flor H (1999) A spelling device for the paralysed. Nature 398(6725):297–298CrossRefGoogle Scholar
  10. Birbaumer N, Hinterberger T, Kübler A, Neumann N (2003) The thought-translation device (TTD): neurobehavioral mechanisms and clinical outcome. IEEE Trans Neural Syst Rehabil Eng 11(2):120–123CrossRefGoogle Scholar
  11. Blankertz B, Schäfer C, Dornhege G, Curio G (2002a) Single trial detection of EEG error potentials: a tool for increasing BCI transmission rates. In: Dorronsoro JR (ed) Artificial neural networks—ICANN 2002, vol 2415., Lecture notes in computer science Springer, Berlin, pp 1137–1143CrossRefGoogle Scholar
  12. Blankertz B, Schäfer C, Dornhege G, Curio G (2002b) Single trial detection of EEG error potentials: a tool for increasing BCI transmission rates. In: Artificial Neural Networks—ICANN 2002, Springer, pp 1137–1143Google Scholar
  13. Blankertz B, Krauledat M, Dornhege G, Williamson J, Murray-Smith R, Müller KR (2007) A note on brain actuated spelling with the Berlin brain-computer interface. In: Stephanidis C (ed) Universal access in human-computer interaction, Lecture notes in computer science, vol 4557. Springer, pp 759–768Google Scholar
  14. Blankertz B, Sannelli C, Halder S, Hammer E, Kübler A, Müller KR, Curio G, Dickhaus T (2010) Neurophysiological predictor of SMR-based BCI performance. NeuroImage 51(4):1303–1309CrossRefGoogle Scholar
  15. Blankertz B, Lemm S, Treder M, Haufe S, Müller KR (2011) Single-trial analysis and classification of ERP components—a tutorial. NeuroImage 56(2):814–825CrossRefGoogle Scholar
  16. Brouwer AM, van Erp J (2010) A tactile P300 brain-computer interface. Frontiers in Neuroscience 4(19):1–11Google Scholar
  17. Canento F, Fred A, Silva H, Gamboa H, Lourenço A (2011) Multimodal biosignal sensor data handling for emotion recognition. In: Sensors, 2011 IEEE, pp 647–650Google Scholar
  18. Coyle S, Ward T, Markham C, McDarby G (2004) On the suitability of near-infrared (NIR) systems for next-generation brain-computer interfaces. Physiol Meas 25(4):815–822CrossRefGoogle Scholar
  19. Daly JJ, Wolpaw JR (2008) Brain-computer interfaces in neurological rehabilitation. Lancet Neurol 7(11):1032–1043CrossRefGoogle Scholar
  20. Delorme A, Kothe C, Vankov A, Bigdely-Shamlo N, Oostenveld R, Zander TO, Makeig S (2010) MATLAB-based tools for BCI research. In: Tan DS, Nijholt A (eds) Brain-computer interfaces, human-computer interaction series, Springer, London, pp 241–259Google Scholar
  21. Duda RO, Hart PE et al (1973) Pattern classification and scene analysis. Wiley, New YorkzbMATHGoogle Scholar
  22. Duda RO, Hart PE, Stork DG (2001) Pattern classification, 2nd edn. Wiley, New YorkzbMATHGoogle Scholar
  23. Fairclough SH (2008) BCI and physiological computing for computer games: differences, similarities and intuitive control. In: Proceedings of CHIGÇÖ08Google Scholar
  24. Fairclough SH (2009) Fundamentals of physiological computing. Interact Comput 21(1):133–145CrossRefGoogle Scholar
  25. Fang F, Liu Y, Shen Z (2003) Lie detection with contingent negative variation. Int J Psychophysiol 50(3):247–255CrossRefGoogle Scholar
  26. Farwell LA, Donchin E (1988) Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 70(6):510–523CrossRefGoogle Scholar
  27. Ferrez PW, Millán JdR (2008) Simultaneous real-time detection of motor imagery and error-related potentials for improved BCI accuracy. In: Proceedings of the 4th international brain-computer interface workshop and training course, pp 197–202Google Scholar
  28. Furdea A, Halder S, Krusienski D, Bross D, Nijboer F, Birbaumer N, Kübler A (2009) An auditory oddball (P300) spelling system for brain-computer interfaces. Psychophysiology 46(3):617–625CrossRefGoogle Scholar
  29. Gangadhar G, Chavarriaga R, Millán JdR (2009) Fast recognition of anticipation-related potentials. IEEE Trans Biomed Eng 56(4):1257–1260Google Scholar
  30. Gao X, Xu D, Cheng M, Gao M (2003) A BCI-based environmental controller for the motion-disabled. IEEE Trans Neural Syst Rehabil Eng 11(2):137–140CrossRefGoogle Scholar
  31. Grimes D, Tan DS, Hudson SE, Pradeep S, Rao RP (2008) Feasibility and pragmatics of classifying working memory load with an electroencephalograph. In: Czerwinski M (ed) Proceedings of the twenty-sixth annual SIGCHI conference on Human factors in computing systems. ACM, New York, pp 835–844CrossRefGoogle Scholar
  32. Guger C, Edlinger G, Harkam W, Niedermayer I, Pfurtscheller G (2003) How many people are able to operate an EEG-based brain-computer interface (BCI)? IEEE Trans Neural Syst Rehabil Eng 11(2):145–147 (a publication of the IEEE Eng Med Biol Soc)Google Scholar
  33. Guger C, Daban S, Sellers E, Holzner C, Krausz G, Carabalona R, Gramatica F, Edlinger G (2009) How many people are able to control a P300-based brain-computer interface (BCI)? Neurosci Lett 462(1):94–98CrossRefGoogle Scholar
  34. Hajcak G, McDonald N, Simons RF (2003) To err is autonomic: error-related brain potentials, ANS activity, and post-error compensatory behavior. Psychophysiology 40(6):895–903CrossRefGoogle Scholar
  35. Hankins TC, Wilson GF (1998) A comparison of heart rate, eye activity, EEG and subjective measures of pilot mental workload during flight. Aviat Space Environ Med 69(4):360–367Google Scholar
  36. Hill NJ, Schölkopf B (2012) An online brain-computer interface based on shifting attention to concurrent streams of auditory stimuli. J Neural Eng 9(2):026,011Google Scholar
  37. Hinterberger T, Schmidt S, Neumann N, Mellinger J, Blankertz B, Curio G, Birbaumer N (2004) Brain-computer communication and slow cortical potentials. IEEE Trans Biomed Eng 51(6):1011–1018CrossRefGoogle Scholar
  38. Holroyd CB, Coles MGH (2002) The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol Rev 109(4):679–708CrossRefGoogle Scholar
  39. Jacob RJ, Leggett JJ, Myers BA, Pausch R (1993) Interaction styles and input/output devices. Behav Inf Technol 12(2):69–79CrossRefGoogle Scholar
  40. Kohlmorgen J, Dornhege G, Braun M, Blankertz B, Müller KR, Curio G, Hagemann K, Bruns A, Schrauf M, Kincses W (2007) Improving human performance in a real operating environment through real-time mental workload detection. In: Dornhege G, Millán JdR, Hinterberger T, McFarland DJ, Müller KR (eds) Toward brain-computer interfacing, Neural information processing series, MIT Press, Cambridge, pp 409–422Google Scholar
  41. Kothe CA, Makeig S (2013) BCILAB: a platform for brain-computer interface development. J Neural Eng 10(5):056,014Google Scholar
  42. Krauledat M, Tangermann M, Blankertz B, Müller KR (2008) Towards zero training for brain-computer interfacing. PLoS one 3(8):e2967CrossRefGoogle Scholar
  43. Kübler A, Birbaumer N (2008) Brain-computer interfaces and communication in paralysis: extinction of goal directed thinking in completely paralysed patients? Clin Neurophysiol 119(11):2658–2666CrossRefGoogle Scholar
  44. Kübler A, Nijboer F, Mellinger J, Vaughan TM, Pawelzik H, Schalk G, McFarland DJ, Birbaumer N, Wolpaw JR (2005) Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface. Neurology 64(10):1775–1777CrossRefGoogle Scholar
  45. Lal TN, Schröder M, Hill NJ, Preissl H, Hinterberger T, Mellinger J, Bogdan M, Rosenstiel W, Hofmann T, Birbaumer N, Schölkopf B (2005) A brain computer interface with online feedback based on magnetoencephalography. In: ICML ‘05 Proceedings of the 22nd international conference on machine learning, pp 465–472Google Scholar
  46. Lazarus RS, Speisman JC, Mordkoff AM (1963) The relationship between autonomic indicators of psychological stress: heart rate and skin conductance. Psychosom Med 25(1):19–30Google Scholar
  47. LeBlanc J, Blais B, Barabe B, Cote J (1976) Effects of temperature and wind on facial temperature, heart rate, and sensation. J Appl Physiol 40(2):127–131Google Scholar
  48. Lee JH, Ryu J, Jolesz FA, Cho ZH, Yoo SS (2009) Brain-machine interface via real-time fMRI: preliminary study on thought-controlled robotic arm. Neurosci Lett 450(1):1–6CrossRefGoogle Scholar
  49. Lorenz R, Pascual J, Blankertz B, Vidaurre C (2013) Towards a holistic assessment of the user experience with hybrid BCIs. J Neural Eng (submitted)Google Scholar
  50. Mak JN, Wolpaw J (2009) Clinical applications of brain-computer interfaces: current state and future prospects. IEEE Rev Biomed Eng 2:187–199CrossRefGoogle Scholar
  51. McFarland DJ, Wolpaw JR (2008) Brain-computer interface operation of robotic and prosthetic devices. Computer 41(10):52–56CrossRefGoogle Scholar
  52. McFarland DJ, Sarnacki WA, Wolpaw JR (2010) Electroencephalographic (EEG) control of three-dimensional movement. J Neural Eng 7(3):036,007Google Scholar
  53. Mellinger J, Schalk G, Braun C, Preissl H, Rosenstiel W, Birbaumer N, Kübler A (2007) An MEG-based brain-computer interface (BCI). NeuroImage 36(3):581–593CrossRefGoogle Scholar
  54. Müller KR, Krauledat M, Dornhege G, Curio G, Blankertz B (2004) Machine learning techniques for brain-computer interfaces. Biomed Eng 49(1):11–22Google Scholar
  55. Müller KR, Tangermann M, Dornhege G, Krauledat M, Curio G, Blankertz B (2008) Machine learning for real-time single-trial EEG-analysis: from brain-computer interfacing to mental state monitoring. J Neurosci Methods 167(1):82–90CrossRefGoogle Scholar
  56. Müller-Putz G, Scherer R, Neuper C, Pfurtscheller G (2006a) Steady-state somatosensory evoked potentials: suitable brain signals for brain-computer interfaces? IEEE Trans Neural Syst Rehabil Eng 14(1):30–37CrossRefGoogle Scholar
  57. Müller-Putz GR, Scherer R, Pfurtscheller G, Rupp R (2005) EEG-based neuroprosthesis control: a step towards clinical practice. Neurosci Lett 382(1–2):169–174CrossRefGoogle Scholar
  58. Müller-Putz GR, Scherer R, Pfurtscheller G, Rupp R (2006b) Brain-computer interfaces for control of neuroprostheses: from synchronous to asynchronous mode of operation. Biomed Tech 51(2):57–63CrossRefGoogle Scholar
  59. Nicolas-Alonso LF, Gomez-Gil J (2012) Brain computer interfaces, a review. Sensors 12(2):1211–1279CrossRefGoogle Scholar
  60. Nijboer F, Sellers EW, Mellinger J, Jordan MA, Matuz T, Furdea A, Halder S, Mochty U, Krusienski DJ, Vaughan TM, Wolpaw JR, Birbaumer N, Kübler A (2008) A P300-based brain-computer interface for people with amyotrophic lateral sclerosis. Clin Neurophysiol 119(8):1909–1916CrossRefGoogle Scholar
  61. Papadelis C, Chen Z, Kourtidou-Papadeli C, Bamidis PD, Chouvarda I, Bekiaris E, Maglaveras N (2007) Monitoring sleepiness with on-board electrophysiological recordings for preventing sleep-deprived traffic accidents. Clin Neurophysiol 118(9):1906–1922CrossRefGoogle Scholar
  62. Patel SH, Azzam PN (2005) Characterization of N200 and P300: selected studies of the event-related potential. Int J Med Sci p 147 2(4):147–154Google Scholar
  63. Pfurtscheller G (1977) Graphical display and statistical evaluation of event-related desynchronization (ERD). Electroencephalogr Clin Neurophysiol 43(5):757–760CrossRefGoogle Scholar
  64. Pfurtscheller G (1992) Event-related synchronization (ERS): an electrophysiological correlate of cortical areas at rest. Electroencephalogr Clin Neurophysiol 83(1):62–69CrossRefGoogle Scholar
  65. Pfurtscheller G, Lopes da Silva FH (1999) Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol: Off J Int Fed Clin Neurophysiol 110(11):1842–1857CrossRefGoogle Scholar
  66. Pfurtscheller G, Guger C, Müller G, Krausz G, Neuper C (2000) Brain oscillations control hand orthosis in a tetraplegic. Neurosci Lett 292(3):211–214CrossRefGoogle Scholar
  67. Pfurtscheller G, Allison BZ, Brunner C, Bauernfeind G, Solis-Escalante T, Scherer R, Zander T, Müller-Putz G, Neuper C, Bierbaumer N (2010) The hybrid BCI. Front Neurosci 4(30):1–11Google Scholar
  68. Picard RW (1999) Affective computing for HCI. In: Human computer interaction, vol 1. pp 829–833Google Scholar
  69. Protzak J, Ihme K, Zander TO (2013) A passive brain-computer interface for supporting gaze-based human-machine interaction. In: Universal access in human-computer interaction. Design methods, tools, and interaction techniques for e inclusion, Springer, pp 662–671Google Scholar
  70. Renard Y, Lotte F, Gibert G, Congedo M, Maby E, Delannoy V, Bertrand O, Lécuyer A (2010) Openvibe: an open-source software platform to design, test, and use brain-computer interfaces in real and virtual environments. Presence: Teleoper Virtual Environ 19(1):35–53CrossRefGoogle Scholar
  71. Reuderink B, Farquhar J, Poel M, Nijholt A (2011) A subject-independent brain-computer interface based on smoothed, second-order base lining. In: Engineering in medicine and biology society, EMBC, 2011 annual international conference of the IEEE, pp 4600–4604Google Scholar
  72. Riccio A, Mattia D, Simione L, Olivetti M, Cincotti F (2012) Eye-gaze independent EEG-based brain-computer interfaces for communication. J Neural Eng 9(4):045,001Google Scholar
  73. Rivera K, Cooke NJ, Bauhs JA (1996) The effects of emotional icons on remote communication. In: Conference companion on human factors in computing systems, ACM, pp 99–100Google Scholar
  74. Rötting M, Zander T, Trösterer S, Dzaack J (2009) Implicit interaction in multimodal human-machine systems. In: Industrial engineering and ergonomics, Springer, pp 523–536Google Scholar
  75. Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR (2004) BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans Biomed Eng 51(6):1034–1043CrossRefGoogle Scholar
  76. Schmidt A (2000) Implicit human computer interaction through context. Pers Technol 4(2–3):191–199CrossRefGoogle Scholar
  77. Schmidt NM, Blankertz B, Treder MS (2012) Online detection of error-related potentials boosts the performance of mental typewriters. BMC Neurosci 13(1):19CrossRefGoogle Scholar
  78. Schöllkopf B, Smola AJ (2002) Learning with kernels: support vector machines, regularization, optimization, and beyond. The MIT Press, CambridgeGoogle Scholar
  79. Schreuder M, Rost T, Tangermann M (2011) Listen, you are writing! Speeding up online spelling with a dynamic auditory BCI. Front Neurosci 5Google Scholar
  80. Sellers EW, Donchin E (2006) A P300-based brain-computer interface: initial tests by ALS patients. Clin Neurophysiol 117(3):538–548CrossRefGoogle Scholar
  81. Shi Y, Ruiz N, Taib R, Choi E, Chen F (2007) Galvanic skin response (GSR) as an index of cognitive load. In: CHI’07 extended abstracts on human factors in computing systems, ACM, pp 2651–2656Google Scholar
  82. Squire P, Parasuraman R (2010) Effects of automation and task load on task switching during human supervision of multiple semi-autonomous robots in a dynamic environment. Ergonomics 53(8):951–961CrossRefGoogle Scholar
  83. Tavella M, Leeb R, Rupp R, Millán JdR (2010) Towards natural non-invasive hand neuroprostheses for daily living. In: Proceedings IEEE engineering in medicine biology society conference, pp 126–129Google Scholar
  84. Treder MS, Blankertz B (2010) (C)overt attention and visual speller design in an ERP-based brain-computer interface. Behav Brain Funct 6(1):28CrossRefGoogle Scholar
  85. Treder MS, Schmidt NM, Blankertz B (2011) Gaze-independent brain-computer interfaces based on covert attention and feature attention. J Neural Eng 8(6):066,003Google Scholar
  86. Van Schie HT, Mars RB, Coles MG, Bekkering H (2004) Modulation of activity in medial frontal and motor cortices during error observation. Nat Neurosci 7(5):549–554CrossRefGoogle Scholar
  87. Vapnik VN, Chervonenkis AY (1971) On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab Appl 16(2):264–280zbMATHMathSciNetCrossRefGoogle Scholar
  88. Vidal JJ (1973) Toward direct brain-computer communication. Annu Rev Biophys Bioeng 2(1):157–180CrossRefGoogle Scholar
  89. Vidal JJ (1977) Real-time detection of brain events in EEG. Proc IEEE 65(5):633–641MathSciNetCrossRefGoogle Scholar
  90. Vilimek R, Zander TO (2009) BC (eye): combining eye-gaze input with brain-computer interaction. Universal access in human-computer interaction. Springer, Intelligent and Ubiquitous Interaction Environments, pp 593–602Google Scholar
  91. Weiskopf N, Veit R, Erb M, Mathiak K, Grodd W, Goebel R, Birbaumer N (2003) Physiological self-regulation of regional brain activity using real-time functional magnetic resonance imaging (fMRI): methodology and exemplary data. NeuroImage 19(3):577–586CrossRefGoogle Scholar
  92. Whitworth B (2005) Polite computing. Behav Inf Technol 24(5):353–363CrossRefGoogle Scholar
  93. Wolpaw JR (2004) Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. In: Proceedings of the national academy of sciences 101(51):17,849-17,854Google Scholar
  94. Wolpaw JR, McFarland D, Vaughan T (2000) Brain-computer interface research at the Wadsworth Center. IEEE Trans Rehabil Eng 8(2):222–226CrossRefGoogle Scholar
  95. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM (2002) Brain-computer interfaces for communication and control. Clin Neurophysiol 113(6):767–791CrossRefGoogle Scholar
  96. Zander TO (2011) Utilizing brain-computer interfaces for human-machine systems. PhD thesis, Universitätsbibliothek TU, BerlinGoogle Scholar
  97. Zander TO, Jatzev S (2012) Context-aware brain-computer interfaces: exploring the information space of user, technical system and environment. J Neural Eng 9(1):016,003Google Scholar
  98. Zander TO, Kothe C (2011) Towards passive brain-computer interfaces: applying brain-computer interface technology to human-machine systems in general. J Neural Eng 8(2):025,005Google Scholar
  99. Zander TO, Gaertner M, Kothe C, Vilimek R (2010a) Combining eye gaze input with a brain-computer interface for touchless human-computer interaction. Int J Human Comput Inter 27(1):38–51CrossRefGoogle Scholar
  100. Zander TO, Kothe C, Jatzev S, Gaertner M (2010b) Enhancing human-computer interaction with input from active and passive brain-computer interfaces. In: Tan DS, Nijholt A (eds) Brain-computer interfaces: applying our minds to human-computer interaction. Human-Computer Interaction Series, Springer, pp 181–199CrossRefGoogle Scholar
  101. Zander TO, Ihme K, Gärtner M, Rötting M (2011) A public data hub for benchmarking common brain-computer interface algorithms. J Neural Eng 8(2):025,021Google Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  • Thorsten O. Zander
    • 1
  • Jonas Brönstrup
    • 1
  • Romy Lorenz
    • 1
  • Laurens R. Krol
    • 1
  1. 1.Team PhyPABerlin Institute of TechnologyBerlinGermany

Personalised recommendations