Skip to main content
SpringerLink
Log in
Book cover

Towards Practical Brain-Computer Interfaces pp 239–250Cite as

Framework for BCIs in Multimodal Interaction and Multitask Environments

  • Chapter
  • First Online:

  • 2800 Accesses

Part of the book series: Biological and Medical Physics, Biomedical Engineering ((BIOMEDICAL))

Abstract

The development of Brain Computer Interfaces (BCIs) enters a phase in which these devices are no longer restricted to applications in controlled, single-task environments. For instance, BCIs for gaming or high-end operator stations will function as part of a multimodal user interface in a multitask environment. This phase introduces new issues that were not relevant for the development of the initial special-use applications and we should address these issues systematically. In this paper, we will present the potential conflicts and how models of information processing can help to cope with these. We will conclude with providing guidelines.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Anderson, E.J., Mannan, S.K., Rees, G., Sumner, P., Kennard, C.: Overlapping functional anatomy for working memory and visual search. Exp. Brain Res. 200(1), 91–107 (2010)

    Google Scholar 

  2. Arnell, K.M.: Visual, auditory, and cross-modality dual-task costs: Electrophysiological evidence for an amodal bottleneck on working memory consolidation. Percept. Psychophys. 68(3), 447–457 (2006)

    Google Scholar 

  3. Bashashati, A., Fatourechi, M., Ward, R.K., Birch, G.E.: A survey of signal processing algorithms in brain–computer interfaces based on electrical brain signals. J. Neural Eng. 4(2), R32–R57 (2007)

    Google Scholar 

  4. Boles, D.B., Bursk, J.H., Phillips, J.B., Perdelwitz, J.R.: Predicting dual-task performance with the multiple resource questionnaire. Hum. Factors 49, 32–45 (2007)

    Google Scholar 

  5. Brouwer, A.-M., Van Erp, J.B.F.: A tactile P300 brain–computer interface. Front. Neurosci. 4(19), 1–12 (2010)

    Google Scholar 

  6. Brouwer, A.-M., Van Erp, J.B.F., Aloise, F., Cincotti, F.: Tactile, visual and bimodal P300s: Could bimodal P300s boost BCI performance? SRX Neurosci. 2010, 1–9 (2010)

    Google Scholar 

  7. Charron, S., Koechlin, E.: Divided representation of concurrent goals in the human frontal lobes. Science 328(5976), 360–363 (2010)

    Google Scholar 

  8. Curran, E.A., Stokes, M.J.: Learning to control brain activity: A review of the production and control of EEG components for driving brain–computer interface (BCI) systems. Brain Cogn. 51(3), 326–336 (2003)

    Google Scholar 

  9. Gazes, Y., Rakitin, B.C., Steffener, J., Habeck, C., Butterfield, B., Ghez, C., Stern, Y.: Performance degradation and altered cerebral activation during dual performance: Evidence for a bottom-up attentional system. Behav. Brain Res. 210(2), 229–239 (2010)

    Google Scholar 

  10. Hiraga, C.Y., Garry, M.I., Carson, R.G., Summers, J.J.: Dual-task interference: Attentional and neurophysiological influences. Behav. Brain Res. 205(1), 10–18 (2009)

    Google Scholar 

  11. ISO ISO 2941-11.: Ergonomic requirements for office work with visual display terminals (VDTs) Guidance on usability. International Organization for Standardization (1998)

    Google Scholar 

  12. Just, M.A., Keller, T.A., Cynkar, J.: A decrease in brain activation associated with driving when listening to someone speak. Brain Res. 1205(C), 70–80 (2008)

    Google Scholar 

  13. Kamienkowski, J.E., Sigman, M.: Delays without mistakes: Response time and error distributions in dual-task. PLoS ONE 3(9), (2008). art. no. e3196

    Google Scholar 

  14. Low, K.A., Leaver, E.E., Kramer, A.F., Fabiani, M., Gratton, G.: Share or compete? Load-dependent recruitment of prefrontal cortex during dual-task performance. Psychophysiology 46(5), 1069–1079 (2009)

    Google Scholar 

  15. Marois, R., Larson, J.M., Chun, M.M., Shima, D.: Response-specific sources of dual-task interference in human pre-motor cortex. Psychol Res. 70(6), 436–447 (2006)

    Google Scholar 

  16. Mochizuki, H., Tashiro, M., Gyoba, J., Suzuki, M., Okamura, N., Itoh, M., Yanai, K.: Brain activity associated with dual-task management differs depending on the combinations of response modalities. Brain Res. 1172(1), 82–92 (2007)

    Google Scholar 

  17. Mühl, C., Gürkök, H., Plass-Oude Bos, D., Thurlings, M.E., Scherffig, L., Duvinage, M., Elbakyan, A.A., Kang, S., Poel, M., Heylen, D.: Bacteria Hunt: Evaluating multi-paradigm BCI interaction. J. Multimodal User Interfaces 4(1), 11–25 (2010)

    Google Scholar 

  18. Pfurtscheller, G., Allison, B.Z., Bauernfeind, G., Brunner, C., Solis-Escalante, T., Scherer, R., Zander, T.O., Müller Putz, G., Neuper, C., Birbaumer, N.: The hybrid BCI. Front. Neurosci. 4, 30, 1–12 (2010)

    Google Scholar 

  19. Rémy, F., Wenderoth, N., Lipkens, K., Swinnen, S.P.: Dual-task interference during initial learning of a new motor task results from competition for the same brain areas. Neuropsychologia 48(9), 2517–2527 (2010)

    Google Scholar 

  20. Ruthruff, E., Johnston, J.C., Remington, R.W.: How strategic is the central bottleneck: Can it be overcome by trying harder? J. Exp. Psychol. Hum. Percept. Perform. 35(5), 1368–1384 (2009)

    Google Scholar 

  21. Sangals, J., Sommer, W.: The impact of intervening tasks on response preparation. J. Exp. Psychol. Hum. Percept. Perform. 36(2), 415–429 (2010)

    Google Scholar 

  22. Sigman, M., Dehaene, S.: Dynamics of the central bottleneck: Dual-task and task uncertainty. PLoS Biol. 4(7), 1227–1238 (2006)

    Google Scholar 

  23. Sigman, M., Dehaene, S.: Brain mechanisms of serial and parallel processing during dual-task performance. J. Neurosci. 28(30), 7585–7598 (2008)

    Google Scholar 

  24. Spence, C., Driver, J.: Cross-modal links in attention between audition, vision, and touch: implications for interface design. Int. J. Cogn. Ergon. 1(4), 351–373 (1997)

    Google Scholar 

  25. Stelzel, C., Brandt, S.A., Schubert, T.: Neural mechanisms of concurrent stimulus processing in dual tasks. NeuroImage 48(1), 237–248 (2009)

    Google Scholar 

  26. Stelzel, C., Kraft, A., Brandt, S.A., Schubert, T.: Dissociable neural effects of task order control and task set maintenance during dual-task processing. J. Cogn. Neurosci. 20(4), 613–628 (2008)

    Google Scholar 

  27. Stelzel, C., Schumacher, E.H., Schubert, T., D’Esposito, M.: The neural effect of stimulus-response modality compatibility on dual-task performance: An fMRI study. Psychol. Res. 70(6), 514–525 (2006)

    Google Scholar 

  28. Thurlings, M.E., Van Erp, J.B.F., Brouwer, A.-M., Blankertz, B., Werkhoven, P.J.: Control-display mapping in brain–computer interfaces. Ergonomics, 55(5), (2012).

    Google Scholar 

  29. Thurlings, M.E., Van Erp, J.B.F., Brouwer, A.-M., Werkhoven, P.J.: EEG-Based navigation from a Human Factors perspective. In: Tan, D.S., Nijholt, A. (eds.) Brain–Computer Interfaces, Human–Computer Interaction series, pp. 117–132. Spinger, London (2010)

    Google Scholar 

  30. Van Erp, J.B.F., Van Veen, H.A.H.C.: Vibrotactile in-vehicle navigation system. Transp. Res. Part F Traffic Psychol. Behav. 7(4–5), 247–256 (2004)

    Google Scholar 

  31. Van Erp, J.B.F., Werkhoven, P.J.: Vibro-tactile and visual asynchronies: Sensitivity and consistency. Perception 33, 103–111 (2004)

    Google Scholar 

  32. Van Erp, J.B.F., Werkhoven, P.J.: Validation of Principles for Tactile Navigation Displays. HFES 50th annual meeting, 1687–1691. Santa Monica: Human Factors and Ergonomics Society. (2006)

    Google Scholar 

  33. Van Erp, J.B.F., Tangermann, M., Lotte, F.: Brain-Computer Interfaces: Beyond Medical Applications. IEEE computer, 45(4), 26–34 (2012).

    Google Scholar 

  34. Van Erp, J.B.F., Veltman, J.A., Grootjen, M.: Brain-Based Indices for User System Symbiosis. In: Tan, D.S., Nijholt, A. (eds.) Brain–Computer Interfaces, Human–Computer Interaction series, pp. 201–219. Spinger, London (2010)

    Google Scholar 

  35. Wickens, C.D.: Multiple resources and mental workload. Hum. Factors 50(3), 449–455 (2008)

    Google Scholar 

  36. Wolpaw, J., Birbaumer, N.: Brain–computer interfaces for communication and control. In: Selzer, M., Clarke, S., Cohen, L., Duncan, P., Gage, F. (eds.) Textbook of neural repair and rehabilitation; neural repair and plasticity, pp. 602–614. Cambridge University Press (2006)

    Google Scholar 

  37. Zander, T.O., Kothe, C., Jatzev, S., Gaertner, M.: Enhancing Human–Computer Interaction with Input from Active and Passive Brain–Computer Interfaces. In: Tan, D.S., Nijholt, A., (eds.) Brain–Computer Interfaces, Human–Computer Interaction series, pp. 181–200. Spinger, London (2010)

    Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the support of the BrainGain Smart Mix Programme of the Netherlands Ministry of Economic Affairs and the Netherlands Ministry of Education, Culture and Science. This research has been supported by the GATE project, funded by the Netherlands Organization for Scientific Research (NWO) and the Netherlands ICT Research and Innovation Authority (ICT Regie).

Author information

Authors and Affiliations

  1. TNO, Kampweg 5, 3769DE, Soesterberg, The Netherlands

    Jan B. F. van Erp, Anne-Marie Brouwer, Marieke E. Thurlings & Peter J. Werkhoven

Authors
  1. Jan B. F. van Erp
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anne-Marie Brouwer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marieke E. Thurlings
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter J. Werkhoven
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jan B. F. van Erp .

Editor information

Editors and Affiliations

  1. Baltimore Drive 5700Apt 167, La Mesa, 91942, California, USA

    Brendan Z. Allison

  2. Starlab Neuroscience Research, Teodor Roviralta 45, Barcelona, 08022, Spain

    Stephen Dunne

  3. , School of Engineering, Swiss Federal Institute of Technology, EPFL STI-CNBI, ELB 141, Station 11, Lausanne, 1015, Switzerland

    Robert Leeb

  4. , School of Engineering, Swiss Federal Institute of Technology, EPFL STI-CNBI, ELB 138. Station 11, Lausanne, 1015, Switzerland

    José Del R. Millán

  5. , Faculty EEMCS, University of Twente, Enschede, 7500, Netherlands

    Anton Nijholt

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

van Erp, J.B.F., Brouwer, AM., Thurlings, M.E., Werkhoven, P.J. (2012). Framework for BCIs in Multimodal Interaction and Multitask Environments. In: Allison, B., Dunne, S., Leeb, R., Del R. Millán, J., Nijholt, A. (eds) Towards Practical Brain-Computer Interfaces. Biological and Medical Physics, Biomedical Engineering. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29746-5_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-29746-5_12

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-29745-8

  • Online ISBN: 978-3-642-29746-5

  • eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)

Publish with us

Policies and ethics

Search

Navigation