Abstract
This chapter provides an overview of the functionality and the underlying principles of the brain-computer interfaces (BCI) developed by the Chair in Non-Invasive Brain-Machine Interface (CNBI) of the Swiss Federal Institute of Technology (EPFL), as well as exemplary applications where those have been successfully evaluated. Our laboratory mainly develops non-invasive BCI systems based on electroencephalographic (EEG) signals and, thus, devoid of medical hazards, real-time, portable, relatively low-cost and minimally obtrusive. Our research is pushing forward asynchronous paradigms offering a spontaneous, user-driven and largely ecological interaction. Furthermore, we stand on the machine-learning way to BCI with emphasis on personalization, configurability and adaptability, coupled with mutual learning training protocols, so that elaborate signal processing and pattern recognition methods are optimally combined with the user’s learnable modulation of brain signals towards high and robust performances and universal usability. Additionally, cognitive mental state monitoring is employed to shape or refine the interaction. Shared-control approaches allow smart, context-aware robotics to complement the BCI channel for more fine-grained control and reduction of the user’s mental workload. Last but not least, hybrid BCI designs exploit additional physiological signals to augment the BCI modality and enrich the control paradigm, thus also exploiting potential residual capabilities of disabled end-users.
The applicability and effectiveness of the aforementioned principles is hereby demonstrated in four exemplary applications evaluated with both able-bodied and motor-disabled end-users. These applications include a hybrid, motor imagery (MI)-based speller, a telepresence robot equipped with shared-control, cognitive mental state monitoring paradigms able to recognize and correct errors, and, finally, a car driving application where a passive BCI enabled on a smart car assists towards increased safety and improved driving experience. Remarkably, our results show that the performance of end-users with disabilities was similar to that of a group of healthy users, who were more familiar with the experiment and the environment. This demonstrates that end-users are able to successfully use BCI technology.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Allison BZ, Brunner C, Kaiser V, Müller-Putz GR, Neuper C, Pfurtscheller G (2010) Toward a hybrid brain–computer interface based on imagined movement and visual attention. J Neural Eng 7:026007
Aloise F, Schettini F, Aricó P, Leotta F, Salinari S, Mattia D, Babiloni F, Cincotti F (2011) P300-based brain-computer interface for environmental control: an asynchronous approach. J Neural Eng 8:025025
Ang KK, Guan C, Chua KS, Ang BT, Kuah C, Wang C, Phua KS, Chin ZY, Zhang H (2011) A large clinical study on the ability of stroke patients to use an EEG-based motor imagery brain-computer interfaces. Clin EEG Neurosci 42(4):253–258
Biasiucci A, Leeb R, Al-Khodairy A, Buhlmann V, Millán JdR (2013) Motor recovery after stroke by means of BCI-guided functional electrical stimulation. In: Proceedings of 5th international BCI meeting, Asilomar, California, US
Billinger M, Brunner C, Müller-Putz GR (2013) Single-trial connectivity estimation for classification of motor imagery data. J Neural Eng 10(4):046006
Birbaumer N, Ghanayim N, Hinterberger T, Iversen I, Kotchoubey B, Kübler A, Perelmouter J, Taub E, Flor H (1999) A spelling device for the paralysed. Nature 398(6725):297–298
Blankertz B, Dornhege G, Schäfer C, Krepki R, Kohlmorgen J, Müller K-R, Kunzmann V, Losch F, Curio G (2003) Boosting bit rates and error detection for the classification of fast-paced motor commands based on single-trial EEG analysis. IEEE Trans Neural Syst Rehabil Eng 11:127–131
Blankertz B, Tangermann M, Vidaurre C, Fazli S, Sannelli C, Haufe S, Maeder C, Ramsey L, Sturm I, Curio G et al (2010) The Berlin brain–computer interface: non-medical uses of BCI technology. Front Neurosci 4:198
Carlson T, Demiris Y (2008) Human-wheelchair collaboration through prediction of intention and adaptive assistance. In: Proceedings of IEEE international conference on robotics and automation, Pasadena, pp 3926–3931
Carlson T, Millán JdR (2013) Brain–controlled wheelchairs: a robotic architecture. IEEE Robot Autom Mag 20(1):65–73
Carlson T, Leeb R, Chavarriaga R, Millán JdR (2012) Online modulation of the level of assistance in shared control systems. In: Proceedings of IEEE international conference on systems man and cybernetics, Seoul, pp 3321–3326
Carmena JM (2013) Advances in neuroprosthetic learning and control. PLoS Biol 11(5):e1001561
Chavarriaga R, Millán JdR (2010) Learning from EEG error-related potentials in noninvasive brain-computer interfaces. IEEE Trans Neural Syst Rehabil Eng 18(4):381–388
Chavarriaga R, Perrin X, Siegwart R, Millán JdR (2012) Anticipation- and error-related EEG signals during realistic human-machine interaction: a study on visual and tactile feedback. In: Proceedings of 34th annual international conference of the IEEE engineering in medicine and biology society, San Diego, pp 6723–6726
Chavarriaga R, Gheorghe L, Zhang H, Khaliliardali Z, Millán JdR (2013) Detecting cognitive states for enhancing driving experience. In: Proceedings of 5th international brain-computer interface meeting 2013, Pacific Grove, Asilomar, California, US
Chavarriaga R, Sobolewski A, Millán JdR (2014) Errare machinale est: the use of error-related potentials in brain-machine interfaces. Front Neurosci 8:208
Collinger JL, Wodlinger B, Downey JE, Wang W, Tyler-Kabara EC, Weber DJ, McMorland AJC, Velliste M, Boninger ML, Schwartz AB (2013) High-performance neuroprosthetic control by an individual with tetraplegia. Lancet 381(9866):557–564
Decety J (1996) The neurophysiological basis of motor imagery. Behav Brain Res 77(1–2):45–52
de Negueruela C, Broschart M, Menon C, Millán JdR (2010) Brain-computer interfaces for space applications. J Pers Ubiquitous Comput 15(5):527–537
Ehrsson HH, Geyer S, Naito E (2003) Imagery of voluntary movement of fingers, toes, and tongue activates corresponding body-part-specific motor representations. J Neurophysiol 90:3304–3316
Fazli S, Mehnert J, Steinbrink J, Curio G, Villringer A, Müller K-R, Blankertz B (2012) Enhanced performance by a hybrid NIRS–EEG brain computer interface. Neuroimage 59:519–529
Ferrez PW, Millán JdR (2008) Error-related EEG potentials generated during simulated brain-computer interaction. IEEE Trans Biomed Eng 55:923–929
Ferrez PW, Millán JdR (2008) Simultaneous real-time detection of motor imagery and error-related potentials for improved BCI accuracy. In: Proceedings of 4th international brain-computer interface workshop and training course, Graz, pp 197–202
Galán F, Ferrez PW, Oliva F, Guàrdia J, Millán JdR (2007) Feature extraction for multi-class BCI using canonical variates analysis. In: Proceedings of IEEE international symposium on intelligent signal processing, Alcalá de Henares
Galán F, Nuttin M, Lew E, Ferrez PW, Vanacker G, Philips J, Millán JdR (2008) A brain-actuated wheelchair: asynchronous and non-invasive brain-computer interfaces for continuous control of robots. Clin Neurophysiol 119(9):2159–2169
Gao X, Xu D, Cheng M, Gao S (2003) A BCI-based environmental controller for the motion-disabled. IEEE Trans Neural Syst Rehabil Eng 11:137–140
Gheorghe LA, Chavarriaga R, Millán JdR (2013) Steering timing prediction in a driving simulator task. In: Proceedings of 35th annual international conference of the IEEE engineering in medicine and biology society, Osaka, Japan, pp 6913–6916
Guger C, Edlinger G, Harkam W, Niedermayer I, Pfurtscheller G (2003) How many people are able to operate an EEG-based brain-computer interface (BCI)? IEEE Trans Neural Syst Rehabil Eng 11:145–147
Haufe S, Treder MS, Gugler MF, Sagebaum M, Curio G, Blankertz B (2011) EEG potentials predict upcoming emergency brakings during simulated driving. J Neural Eng 8(5):056001
Haufe S, Kim J-W, Kim I-H, Sonnleitner A, Schrauf M, Curio G, Blankertz B (2014) Electrophysiology-based detection of emergency braking intention in real-world driving. J Neural Eng 11:056011
Hochberg LR, Bacher D, Jarosiewicz B, Masse NY, Simeral JD, Vogel J, Haddadin S, Liu J, Cash SS, van der Smagt P, Donoghue JP (2012) Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485(7398):372–375
Höhne J, Holz E, Staiger-Sälzer P, Müller K-R, Kübler A, Tangermann M (2014) Motor imagery for severely motor-impaired patients: evidence for brain-computer interfacing as superior control solution. PLoS ONE 9:e104854
Hu TC, Tucker AC (1971) Optimal computer search trees and variable-length alphabetical codes. SIAM J Appl Math 21(4):514–532
Iturrate I, Chavarriaga R, Montesano L, Minguez J, Millán JdR (2014) Latency correction of error potentials between different experiments reduces calibration time for single-trial classification. J Neural Eng 11:036005
Jeannerod M, Frak V (1999) Mental imaging of motor activity in humans. Curr Opin Neurobiol 9(6):735–739
Kaufmann T, Holz EM, Kübler A (2013) Comparison of tactile, auditory and visual modality for brain-computer interface use: a case study with a patient in the locked-in state. Front Neurosci 7:129
Khaliliardali Z, Chavarriaga R, Gheorghe LA, Millán JdR (2012) Detection of anticipatory brain potentials during car driving. In: Proceedings of 34th annual international conference of the IEEE engineering in medicine and biology society, San Diego, vol 2012, pp 3829–3832
Kim I-H, Kim J-W, Haufe S, Lee S-W (2015) Detection of braking intention in diverse situations during simulated driving based on EEG feature combination. J Neural Eng 12:016001
Kohlmorgen J, Dornhege G, Braun ML, Blankertz B, Müller KR, Curio G, Hagemann K, Bruns A, Schrauf M, Kincses WE (2007) Improving human performance in a real operating environment through real-time mental workload detection. In: Dornhege G, Millán JdR, Hinterberger T, McFarland D, Müller KR (eds) Toward brain-computer interfacing. MIT Press, Cambridge, pp 409–422
Krepki R, Blankertz B, Curio G, Müller K-R (2007) The Berlin brain-computer interface (BBCI): towards a new communication channel for online control in gaming applications. J Multimed Tools Appl 33:73–90
Kübler A, Birbaumer N (2008) Brain-computer interfaces and communication in paralysis: extinction of goal directed thinking in completely paralysed patients? Clin Neurophysiol 119(11):2658–2666
Leeb R, Sagha H, Chavarriaga R, Millán JdR (2011) A hybrid brain-computer interface based on the fusion of electroencephalographic and electromyographic activities. J Neural Eng 8(2):025011
Leeb R, Gwak K, Kim D-S, Millán JdR (2013) Freeing the visual channel by exploiting vibrotactile BCI feedback. In: Proceedings of 35th annual international conference of the IEEE engineering in medicine and biology society, Osaka, Japan
Leeb R, Perdikis S, Tonin L, Biasiucci A, Tavella M, Molina A, Al-Khodairy A, Carlson T, Millán JdR (2013) Transferring brain-computer interfaces beyond the laboratory: successful application control for motor-disabled users. Artif Intell Med 59(2):121–132
Leeb R, Tonin L, Carlson T, Millán JdR (2015) Towards independence: a BCI telepresence robot for people with severe motor disabilities. Proc IEEE 103:969–982
Lew E, Chavarriaga R, Silvoni S, Millán JdR (2012) Detection of self-paced reaching movement intention from EEG signal. Front Neuroeng 5:13
Lew E, Chavarriaga R, Silvoni S, Millán JdR (2014) Single trial prediction of self-paced reaching directions from EEG signals. Front Neurosci 8:222
Lin C-T, Ko L-W, Sheng T-K (2009) Computational intelligent brain computer interaction and its applications on driving cognition. IEEE Comput Intell Mag 4(4):32–46
Lotte F, Larrue F, Mühl C (2013) Flaws in current human training protocols for spontaneous brain-computer interfaces: lessons learned from instructional design. Front Hum Neurosci 7:568
Mak JN, Wolpaw EW (2009) Clinical applications of brain-computer interfaces: current state and future prospects. IEEE Rev Biomed Eng 2:187–199
Marshall D, Coyle D, Wilson S, Callaghan M (2013) Games, gameplay, and BCI: the state of the art. IEEE Trans Comput Intell AI Games 5(2):82–99
Mason SG, Bashashati A, Fatourechi M, Navarro KF, Birch GE (2007) A comprehensive survey of brain interface technology designs. Ann Biomed Eng 35(2):137–169
Millán JdR (2003) Adaptive brain interfaces. Commun ACM 46:74–80
Millán JdR (2004) On the need for on-line learning in brain-computer interfaces. In: Proceedings of international joint conference on neural networks, Budapest
Millán JdR, Franzé M, Mouriño J, Cincotti F, Babiloni F (2002) Relevant EEG features for the classification of spontaneous motor-related tasks. Biol Cybern 86:89–95
Millán JdR, Renkens F, Mouriño J, Gerstner W (2004) Noninvasive brain-actuated control of a mobile robot by human EEG. IEEE Trans Biomed Eng 51(6):1026–1033
Millán JdR, Buttfield A, Vidaurre C, Krauledat M, Schögl A, Shenoy P, Blankertz B, Rao RPN, Cabeza R, Pfurtscheller G, Müller K-R (2007) Adaptation in brain-computer interfaces. In: Dornhege G, Millán JdR, Hinterberger T, McFarland D, Müller KR (eds) Toward brain-computer interfacing. MIT Press, Cambridge, pp 303–325
Millán JdR, Ferrez PW, Galán F, Lew E, Chavarriaga R (2008) Non-invasive brain-machine interaction. Int J Pattern Recognit Artif Intell 22(5):959–972
Millán JdR, Rupp R, Müller-Putz GR, Murray-Smith R, Giugliemma C, Tangermann M, Vidaurre C, Cincotti F, Kübler A, Leeb R, Neuper C, Müller KR, Mattia D (2010) Combining brain-computer interfaces and assistive technologies: state-of-the-art and challenges. Front Neurosci 4:161
Miltner WHR, Braun CH, Coles MGH (1997) Event-related brain potentials following incorrect feedback in a time-estimation task: evidence for a “generic” neural system for error detection. J Cogn Neurosci 9(6):788–798
Müller K-R, Tangermann M, Dornhege G, Krauledat M, Curio G, Blankertz B (2008) Machine learning for real-time single-trial EEG-analysis: from brain-computer interfacing to mental state monitoring. J Neurosci Methods 167:82–90
Müller-Putz GR, Scherer R, Pfurtscheller G, Rupp R (2005) EEG-based neuroprosthesis control: a step towards clinical practice. Neurosci Lett 382:169–174
Navarro X, Krueger TB, Lago N, Micera S, Stieglitz T, Dario P (2005) A critical review of interfaces with the peripheral nervous system for the control of neuroprostheses and hybrid bionic systems. J Peripher Nerv Syst 10(3):229–258
O’Doherty JE, Lebedev MA, Ifft PJ, Zhuang KZ, Shokur S, Bleuler H, Nicolelis MAL (2011) Active tactile exploration using a brain-machine-brain interface. Nature 479:228–231
Orsborn AL, Moorman HG, Overduin SA, Shanechi MM, Dimitrov DF, Carmena JM (2014) Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron 82(6):1380–1393
Perdikis S, Leeb R, Williamson J, Ramsay A, Tavella M, Desideri L, Hoogerwerf E-J, Al-Khodairy A, Murray-Smith R, Millán JdR (2014) Clinical evaluation of BrainTree, a motor imagery hybrid BCI speller. J Neural Eng 11(3):036003
Perrin X, Chavarriaga R, Colas F, Siegwart R, Millán JdR (2010) Brain-coupled interaction for semi-autonomous navigation of an assistive robot. Robot Auton Syst 58(12):1246–1255
Pfurtscheller G, Lopes da Silva FH (1999) Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol 110:1842–1857
Pfurtscheller G, Neuper C (2001) Motor imagery and direct brain-computer communication. Proc IEEE 89:1123–1134
Pfurtscheller G, Allison BZ, Bauernfeind G, Brunner C, Solis Escalante T, Scherer R, Zander TO, Müller-Putz G, Neuper C, Birbaumer N (2010) The hybrid BCI. Front Neurosci 4:42
Pfurtscheller G, Solis-Escalante T, Ortner R, Linortner P, Muller-Putz GR (2010) Self-paced operation of an SSVEP-based orthosis with and without an imagery-based “brain switch”: a feasibility study towards a hybrid BCI. IEEE Trans Neural Syst Rehabil Eng 18:409–414
Ramos-Murguialday A, Broetz D, Rea M, Läer L, Yilmaz Ö, Brasil FL, Liberati G, Curado MR, Garcia-Cossio E, Vyziotis A, Cho W, Agostini M, Soares E, Soekadar S, Caria A, Cohen LG, Birbaumer N (2013) Brain-machine interface in chronic stroke rehabilitation: a controlled study. Ann Neurol 74(1):100–108
Rohm M, Schneiders M, Müller C, Kreilinger A, Kaiser V, Müller-Putz GR, Rupp R (2013) Hybrid brain-computer interfaces and hybrid neuroprostheses for restoration of upper limb functions in individuals with high-level spinal cord injury. Artif Intell Med 59(2):133–142
Saeedi S, Carlson T, Chavarriaga R, Millán JdR (2013) Making the most of context-awareness in brain-computer interfaces. In: Proceedings of IEEE international conference on cybernetics, Lausanne, Switzerland
Sajda P, Pohlmeyer E, Wang J, Parra LC, Christoforou C, Dmochowski J, Hanna B, Bahlmann C, Singh MK, Chang S-F (2010) In a blink of an eye and a switch of a transistor: cortically coupled computer vision. Proc IEEE 98(3):462–478
Schalk G, Wolpaw JR, McFarland DJ, Pfurtscheller G (2000) EEG-based communication: presence of an error potential. Clin Neurophysiol 111:2138–2144
Scherer R, Müller-Putz GR, Pfurtscheller G (2007) Self-initiation of EEG-based brain-computer communication using the heart rate response. J Neural Eng 4:L23–L29
Schreuder M, Blankertz B, Tangermann M (2010) A new auditory multi-class brain-computer interface paradigm: spatial hearing as an informative cue. PLoS ONE 5(4):e9813
Silvoni S, Ramos-Murguialday A, Cavinato M, Volpato C, Cisotto G, Turolla A, Piccione F, Birbaumer N (2011) Brain-computer interface in stroke: a review of progress. Clin EEG Neurosci 42(4):245–252
Tangermann MW, Krauledat M, Grzeska K, Sagebaum M, Vidaurre C, Blankertz B, Müller K-R (2008) Playing pinball with non-invasive BCI. In: Advances in neural information processing systems 21, Vancouver, pp 1641–1648
Tonin L, Leeb R, Tavella M, Perdikis S, Millán JdR (2010) The role of shared-control in BCI-based telepresence. In: Proceedings of 2010 IEEE international conference on systems, man and cybernetics, Istanbul, pp 1462–1466
Tonin L, Leeb R, Millán JdR (2012) Time-dependent approach for single trial classification of covert visuospatial attention. J Neural Eng 9(4):045011
Tonin L, Leeb R, Sobolewski A, Millán JdR (2013) An online EEG BCI based on covert visuospatial attention in absence of exogenous stimulation. J Neural Eng 10(5):056007
Vanhooydonck D, Demeester E, Nuttin M, Van Brussel H (2003) Shared control for intelligent wheelchairs: an implicit estimation of the user intention. In: Proceedings of 1st international workshop advances in service robot, Bardolino, pp 176–182
Vidaurre C, Kawanabe M, von Bünau P, Blankertz B, Müller K-R (2011) Toward unsupervised adaptation of LDA for brain-computer interfaces. IEEE Trans Biomed Eng 58(3):587–597
Vidaurre C, Sannelli C, Müller K-R, Blankertz B (2011) Co-adaptive calibration to improve bci efficiency. J Neural Eng 8:025009
Wolpaw J, Winter-Wolpaw E (2012) Brain-computer interfaces: principles and practice. Oxford University Press, Oxford/New York
Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM (2002) Brain-computer interfaces for communication and control. Clin Neurophysiol 113(6):767–791
Yin E, Zhou A, Jiang J, Chen F, Liu Y, Hu D (2013) A novel hybrid BCI speller based on the incorporation of SSVEP into the P300 paradigm. J Neural Eng 10:026012
Zhang H, Chavarriaga R, Gheorghe LA, Millán JdR (2013) Inferring driver’s turning direction through detection of error related brain activity. In: Proceedings of 35th annual international conference of the IEEE engineering in medicine and biology society, Osaka, vol 2013, pp 2196–2199
Zhang H, Chavarriaga R, Millán JdR (2014) Towards implementation of motor imagery using brain connectivity features. In: Proceedings of 6th international brain-computer interface conference, Graz, Austria
Zickler C, Di Donna V, Kaiser V, Al-Khodairy A, Kleih S, Kuebler A, Malavasi M, Mattia D, Mongardi S, Neuper C, Rohm M, Rupp R (2009) Brain computer interaction applications for people with disabilities: defining user needs and user requirements. In: Proceedings of AAATE, Florence, p 5
Acknowledgements
The authors acknowledge the use of text from their prior publications [27, 34, 37, 45, 66] in Sect. 6.3.
This work was supported by the European ICT programme projects TOBI: Tools for Brain-Computer Interaction (FP7-224631) and Opportunity: Activity and Context Recognition with Opportunistic Sensor Configuration (ICT-225938), the Swiss National Science Foundation NCCR Robotics, Nissan Motor Co. Ltd., the Hasler Foundation, and the dissemination by the European ICT coordination and support action BNCI Horizon 2020 (FP7-ICT-609593). The writing up of this work was also supported by the Swiss canton of Valais-Wallis and by the City of Sion. This paper only reflects the authors’ views and funding agencies are not liable for any use that may be made of the information contained herein.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Leeb, R., Chavarriaga, R., Perdikis, S., Iturrate, I., Millán, J.d.R. (2015). Moving Brain-Controlled Devices Outside the Lab: Principles and Applications. In: Lee, SW., Bülthoff, H., Müller, KR. (eds) Recent Progress in Brain and Cognitive Engineering. Trends in Augmentation of Human Performance, vol 5. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-7239-6_6
Download citation
DOI: https://doi.org/10.1007/978-94-017-7239-6_6
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-017-7238-9
Online ISBN: 978-94-017-7239-6
eBook Packages: Biomedical and Life SciencesBiomedical and Life Sciences (R0)