Enhancing Human-Computer Interaction with Input from Active and Passive Brain-Computer Interfaces

  • Thorsten O. Zander
  • Christian Kothe
  • Sabine Jatzev
  • Matti Gaertner
Part of the Human-Computer Interaction Series book series (HCIS)


This chapter introduces a formal categorization of BCIs, according to their key characteristics within HCI scenarios. This comprises classical approaches, which we group into active and reactive BCIs, and the new group of passive BCIs. Passive BCIs provide easily applicable and yet efficient interaction channels carrying information on covert aspects of user state, while adding little further usage cost. All of these systems can also be set up as hybrid BCIs, by incorporating information from outside the brain to make predictions, allowing for enhanced robustness over conventional approaches. With these properties, passive and hybrid BCIs are particularly useful in HCI. When any BCI is transferred from the laboratory to real-world situations, one faces new types of problems resulting from uncontrolled environmental factors—mostly leading to artifacts contaminating data and results. The handling of these situations is treated in a brief review of training and calibration strategies. The presented theory is then underpinned by two concrete examples. First, a combination of Event Related Desynchronization (ERD)-based active BCI with gaze control, defining a hybrid BCI as solution for the midas touch problem. And second, a passive BCI based on human error processing, leading to new forms of automated adaptation in HCI. This is in line with the results from other recent studies of passive BCI technology and shows the broad potential of this approach.


User State Machine Error Mental Workload User Training Short Dwell Time 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We gratefully thank Matthias Roetting for continuous financial support, and Roman Vilimek, Siemens AG, as well as Jessika Reissland, TU Berlin, for their professional co-working, their beneficial comments, and their support.


  1. Asteriadis S, Tzouveli P, Karpouzis K, Kollias S (2009) Estimation of behavioral user state based on eye gaze and head pose—application in an e-learning environment. Multimed Tools Appl 41:469–493 CrossRefGoogle Scholar
  2. Becker H, Meek C, Chickering DM (2007) Modeling contextual factors of click rates. In: Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, Vancouver Google Scholar
  3. Blankertz B, Schäfer C, Dornhege G, Curio G (2002) Single trial detection of EEG error potentials: A tool for increasing BCI transmission rates. In: Artificial Neural Networks—ICANN 2002, pp 1137–1143 Google Scholar
  4. Blankertz B, Dornhege G, Krauledat M, Müller K, Curio G (2007) The non-invasive Berlin Brain–Computer Interface: Fast acquisition of effective performance in untrained subjects. NeuroImage 37(2):539–550 CrossRefGoogle Scholar
  5. Bolt RA (1982) Eyes at the Interface. In: Proceedings of the 1982 Conference on Human Factors in Computing Systems. ACM Press, New York, pp 360–362 CrossRefGoogle Scholar
  6. Cajochen C, Kräuchi K, von Arx M, Möri D, Graw P, Wirz-Justice A (1996) Daytime melatonin administration enhances sleepiness and theta/alpha activity in the waking EEG. Neurosci Lett 207(3):209–213 CrossRefGoogle Scholar
  7. Chanel G, et al (2006) Emotion assessment: Arousal evaluation using EEG’s and peripheral physiological signals. Multimedia Content Representation, Classification and Security, pp 530–537 Google Scholar
  8. Chen D, Vertegaal R (2004) Using mental load for managing interruptions in physiologically attentive user interfaces. Extended Abstracts of SIGCHI 2004 Conference on Human Factors in Computing Systems, pp 1513–1516 Google Scholar
  9. Engell-Nielsen T, Glenstrup AJ, Hansen JP (2003) Eye gaze interaction: A new media—not just a fast mouse. In: Itoh K, Komatsubara A, Kuwano S (eds) Handbook of Human Factors/Ergonomics. AsakuraPublishing, Tokyo, Japan, pp 445–455 Google Scholar
  10. Fang F, Liu Y, Shen Z (2003) Lie detection with contingent negative variation. Int J Psychophysiol 50(3):247–255 CrossRefGoogle Scholar
  11. Farwell LA, Donchin E (1988) Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 70(6):510–523 CrossRefGoogle Scholar
  12. Ferrez PW, del Millán R (2008) Error-related EEG potentials generated during simulated brain-computer interaction. IEEE Trans Biomed Eng 55(3):923–929 CrossRefGoogle Scholar
  13. Grimes D, Tan DS, Hudson SE, Shenoy P, Rao RP (2008) Feasibility and pragmatics of classifying working memory load with an electroencephalograph. In: Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy, 2008). ACM, New York, pp 835–844 CrossRefGoogle Scholar
  14. Heekeren HR, Marrett S, Ungerleider LG (2008) The neural systems that mediate human perceptual decision making. Nat Rev Neurosci 9:467–479 CrossRefGoogle Scholar
  15. Holroyd CB (2004) A note on the oddball N200 and the feedback ERN. In: Ullsperger M, Falkenstein M (eds) Errors, Conflicts, and the Brain. Current Opinions on Performance Monitoring. Max-Planck Institute for Human Cognitive and Brain Sciences, Leipzig, pp 211–218 Google Scholar
  16. Hutchinson TF (1993) Eye gaze computer interfaces: Computers that sense eye positions on the display. Computer 26:65–67 CrossRefGoogle Scholar
  17. Jacob RJK (1993) What you look at is what you get. IEEE Comput 26:65–66 CrossRefGoogle Scholar
  18. Jacob RJK, Legett JJ, Myers BA, Pausch R (1993) Interaction styles and input/output devices. Behav Inf Technol 12:69–79 CrossRefGoogle Scholar
  19. Jatzev S, Zander TO, DeFilippis M, Kothe C, Welke S, Rötting M (2008) Examining causes for non-stationarities: The loss of controllability is a factor which induces non-stationarities. In: Proceedings of the 4th Int BCI Workshop & Training Course, Graz University of Technology. Publishing House, Graz, Austria Google Scholar
  20. Kelly S, Lalor E, Reilly R, Foxe J (2005) Visual spatial attention tracking using high-density SSVEP data for independent brain–computer communication. IEEE Trans Rehabil Eng 13(2):172–178 CrossRefGoogle Scholar
  21. Kohlmorgen J, Dornhege G, Braun M, Blankertz B, Müller K, Curio G, Hagemann K, Bruns A, Schrauf M, Kincses W (2007) Improving human performance in a real operating environment through real-time mental workload detection. In: Toward Brain-Computer Interfacing. MIT Press, incollection 24:409–422 Google Scholar
  22. Krauledat M (2008) Analysis of nonstationarities in EEG signals for improving brain-computer interface performance. PhD thesis, Technische Universität Berlin, Fakultät IV—Elektrotechnik und Informatik Google Scholar
  23. Miller GA (1994) The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychol Rev 101(2):343–352 CrossRefGoogle Scholar
  24. Moldt D, von Scheve C (2002) Attribution and adaptation: The case of social norms and emotion in human-agent interaction. In: Proceedings of “The Philosophy and Design of Socially Adept Technologies”, Workshop Held in Conjunction with CHI’02, April 20th, Minneapolis/Minnesota, USA. National Research Council Canada, Ottawa Google Scholar
  25. Müller K, Tangermann M, Dornhege G, Krauledat M, Curio G, Blankertz B (2008) Machine learning for real-time single-trial EEG-analysis: From brain-computer interfacing to mental state monitoring. J Neurosci Methods 167:82–90 CrossRefGoogle Scholar
  26. Murata A (2006) Eye gaze input versus mouse: Cursor control as a function of age. Int J Hum Comput Interact 21:1–14 Google Scholar
  27. Nieuwenhuis S, Holroyd CB, Mol N, Coles MG (2004) Reinforcement-related brain potentials from medial frontal cortex: Origins and functional significance. Neurosci Biobehav Rev 28:441–448 CrossRefGoogle Scholar
  28. Nilsson S, Gustafsson T, Carleberg P (2007) Hands free interaction with virtual information in a real environment. In: Proceedings of COGAIN 2007, Leicester, UK, pp 53–57 Google Scholar
  29. Park N, Zhu W, Jung Y, McLaughlin M, Jin S (2005) Utility of haptic data in recognition of user state. In: Proceedings of HCI International 11. Lawrence Erlbaum Associates, Mahwah Google Scholar
  30. Pfurtscheller G, Neuper C, Flotzinger D, Pregenzer M (1997) EEG-based discrimination between imagination of right and left hand movement. Electroencephalogr Clin Neurophysiol 103(6):642–651 CrossRefGoogle Scholar
  31. Pfurtscheller G, Leeb R, Keinrath C, Friedman D, Neuper C, Guger C, Slater M (2006) Walking from thought. Brain Res 1071(1):145–152 CrossRefGoogle Scholar
  32. Pierce J, Stearns B, Pausch R (1999) Voodoo dolls: Seamless interaction at multiple scales in virtual environments. In: Tagungsband Symposium on Interactive 3D graphics, Atlanta, GA, USA. ACM Press, New York, pp 141–145 Google Scholar
  33. Posner MI, Cohen Y (1984) Components of visual orienting. In: Bouma H, Bouwhuis DG (eds) Attention and Performance, vol 10. Erlbaum, Hillsdale, NJ, pp 531–556 Google Scholar
  34. Popescu F, Fazli S, Badower Y, Blankertz B, Müller K-R (2007) Single trial classification of motor imagination using 6 dry EEG electrodes. PLoS ONE 2(7). DOI  10.1371/journal.pone.0000637
  35. Premack D, Woodruff G (1978) Does the chimpanzee have a “theory of mind”? Behav Brain Sci 4:515–526 CrossRefGoogle Scholar
  36. Reissland J, Zander TO (2009) Automated detection of bluffing in a game—Revealing a complex covert user state with a passive BCI. In: Proceedings of the Human Factors and Ergonomics Society Europe Chapter Annual Meeting, Linkoeping, Sweden Google Scholar
  37. Rötting M, Zander T, Trösterer S, Dzaack J (2009) Implicit interaction in multimodal human-machine systems. In: Schlick C (ed) Methods and Tools of Industrial Engineering and Ergonomics. Springer, Berlin Google Scholar
  38. Schmidt EA, Kincses WE, Schrauf M, Haufe S, Schubert R, Curio G, Ag D (2009) Assessing driver’s vigilance state during monotonous driving, June 2009 Google Scholar
  39. Shenoy P, Krauledat M, Blankertz B, Rao RPN, Müller K-R (2006) Towards adaptive classification for BCI. J Neural Eng 3(1):R13–R23 CrossRefGoogle Scholar
  40. Shinkareva SV, Mason RA, Malave VL, Wang W, Mitchell TM, Just MA (2008) Using fMRI brain activation to identify cognitive states associated with perception of tools and dwellings. PLoS ONE 3(1):e1394 CrossRefGoogle Scholar
  41. Sibert LE, Jacob RJK (2000) Evaluation of eye gaze interaction. CHI’00. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press, New York, pp 281–288 CrossRefGoogle Scholar
  42. Toga AW, Mazziotta JC (2000) Brain Mapping. Gulf Professional Publishing, Amsterdam Google Scholar
  43. Tomioka R, Dornhege G, Nolte G, Blankertz B, Aihara K, Müller K-R (2006) Spectrally weighted common spatial pattern algorithm for single trial EEG classification.Technical Report, 21. Dept. of Mathematical Engineering. Tokyo, Japan: University of Tokyo Google Scholar
  44. Vilimek R, Zander TO (2009) BC(eye): Combining eye-gaze input with brain-computer interaction. In: Proceedings of the HCII 2009. Springer, Heidelberg Google Scholar
  45. Wiener EL (1989) Human factors of advanced technology (“glass cockpit”) transport aircraft. NASA Contractor Report 177528, NASA Ames Research Center Google Scholar
  46. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM (2002) Brain-computer interfaces for communication and control. Clin Neurophysiol Off J Int Fed Clin Neurophysiol 113(6):767–791. PMID: 12048038 CrossRefGoogle Scholar
  47. Wriessnegger S, Scherer R, Mörth K, Maier C, Pfurtscheller G (2006) CN. Heart rate-controlled EEG-Based BCI: The Graz hybrid BCI. In: Proc of the 3rd Int BCI Workshop & Training Course, Graz, Austria, 2006. Graz University of Technology Publishing House, Graz Google Scholar
  48. Zander TO, Kothe C, Welke S, Roetting M (2008) Enhancing human-machine systems with secondary input from passive brain-computer interfaces. In: Proc of the 4th Int BCI Workshop & Training Course, Graz, Austria, 2008. Graz University of Technology Publishing House, Graz Google Scholar
  49. Zander TO, Jatzev S (2009) Detecting affective covert user states with passive brain-computer interfaces. In: Proceedings of the ACII 2009. IEEE Computer Society Press, Los Alamitos, CA Google Scholar

Copyright information

© Springer-Verlag London Limited 2010

Authors and Affiliations

  • Thorsten O. Zander
    • 1
    • 2
  • Christian Kothe
    • 1
    • 2
  • Sabine Jatzev
    • 1
    • 2
  • Matti Gaertner
    • 1
    • 2
  1. 1.Team PhyPATU BerlinBerlinGermany
  2. 2.Department of Psychology and Ergonomics, Chair for Human-Machine SystemsBerlin Institute of TechnologyBerlinGermany

Personalised recommendations