Skip to main content

Combining Brain-Computer Interfaces and Haptics: Detecting Mental Workload to Adapt Haptic Assistance

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNISA,volume 7282)

Abstract

In this paper we introduce the combined use of Brain-Computer Interfaces (BCI) and Haptic interfaces. We propose to adapt haptic guides based on the mental activity measured by a BCI system. This novel approach is illustrated within a proof-of-concept system: haptic guides are toggled during a path-following task thanks to a mental workload index provided by a BCI. The aim of this system is to provide haptic assistance only when the user’s brain activity reflects a high mental workload. A user study conducted with 8 participants shows that our proof-of-concept is operational and exploitable. Results show that activation of haptic guides occurs in the most difficult part of the path-following task. Moreover it allows to increase task performance by 53% by activating assistance only 59% of the time. Taken together, these results suggest that BCI could be used to determine when the user needs assistance during haptic interaction and to enable haptic guides accordingly.

Keywords

  • Brain-Computer Interface
  • EEG
  • Force-Feedback
  • Adaptation
  • Mental Workload
  • Guidance

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain–computer interfaces for communication and control. Clinical Neurophysiology 113(6), 767–791 (2002)

    CrossRef  Google Scholar 

  2. Farwell, L.A., Donchin, E.: Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology 70(6), 510–523 (1988)

    CrossRef  Google Scholar 

  3. Guger, C., Harkam, W., Hertnaes, C., Pfurtscheller, G.: Prosthetic control by an EEG-based brain-computer interface (BCI). In: Proc. European Conference for the Advancement of Assistive Technology (1999)

    Google Scholar 

  4. Allison, B., Graimann, B., Gräser, A.: Why use a BCI if you are healthy? In: ACE Workshop - Brain-Computer Interfaces and Games, pp. 7–11 (2007)

    Google Scholar 

  5. George, L., Lécuyer, A.: An overview of research on ”passive” brain-computer interfaces for implicit human-computer interaction. In: International Conference on Applied Bionics and Biomechanics, Venise, Italy (2010)

    Google Scholar 

  6. Zander, T.O., Kothe, C.: Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general. Journal of Neural Engineering 8(2) (2011)

    Google Scholar 

  7. Nijholt, A., Bos, D.P.O., Reuderink, B.: Turning shortcomings into challenges: Brain–computer interfaces for games. Entertainment Computing 1(2), 85–94 (2009)

    CrossRef  Google Scholar 

  8. Müller-Putz, G.R., Scherer, R., Neuper, C., Pfurtscheller, G.: Steady-state somatosensory evoked potentials: suitable brain signals for brain-computer interfaces? IEEE Transactions on Neural Systems and Rehabilitation Engineering 14(1), 30–37 (2006)

    CrossRef  Google Scholar 

  9. Cincotti, F., Kauhanen, L., Aloise, F., Palomäki, T., Caporusso, N., Jylänki, P., Mattia, D., Babiloni, F., Vanacker, G., Nuttin, M., Marciani, M.G., Millán, J.d.R.: Vibrotactile feedback for brain-computer interface operation. In: Computational Intelligence and Neuroscience (2007)

    Google Scholar 

  10. Chatterjee, A., Aggarwal, V., Ramos, A., Acharya, S., Thakor, N.: A brain-computer interface with vibrotactile biofeedback for haptic information. Journal of NeuroEngineering and Rehabilitation 4(1) (2007)

    Google Scholar 

  11. Feygin, D., Keehner, M., Tendick, R.: Haptic guidance: experimental evaluation of a haptic training method for a perceptual motor skill. In: Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 40–47 (2002)

    Google Scholar 

  12. Bluteau, J., Sabine, C., Yohan, P., Edouard, G.: Haptic Guidance Improves the Visuo-Manual Tracking of Trajectories. PLoS ONE 3(3) (2008)

    Google Scholar 

  13. Asseldonk, E.H.F., Wessels, M., Stienen, A.H., van der Helm, F.C., van der Kooij, H.: Influence of haptic guidance in learning a novel visuomotor task. Journal of Physiology 103(3–5), 276–285 (2009)

    Google Scholar 

  14. Lécuyer, A., Lotte, F., Reilly, R., Leeb, R., Hirose, M., Slater, M.: Brain-computer interfaces, virtual reality, and videogames. Computer 41(10), 66–72 (2008)

    CrossRef  Google Scholar 

  15. Mühl, C., Gürkök, H., Plass-Oude Bos, D., Thurlings, M., Scherffig, L., Duvinage, M., Elbakyan, A., Kang, S., Poel, M., Heylen, D.: Bacteria Hunt: A multimodal, multiparadigm BCI game. In: International Summer Workshop on Multimodal Interfaces, University of Genua (2010)

    Google Scholar 

  16. Kohlmorgen, J., Dornhege, G., Braun, M., Blankertz, B., Müller, K.-R., Curio, G., Hagemann, K., Bruns, A., Schrauf, M., Kincses, W.: Improving human performance in a real operating environment through real-time mental workload detection. In: Dornhege, G., Millán, J.d.R., Hinterberger, T., McFarland, D., Müller, K.-R. (eds.) Toward Brain-Computer Interfacing, pp. 409–422. MIT press, Cambridge (2007)

    Google Scholar 

  17. Heger, D., Putze, F., Schultz, T.: Online workload recognition from EEG data during cognitive tests and human-machine interaction. In: Advances in Artificial Intelligence, pp. 410–417 (2010)

    Google Scholar 

  18. Pope, A.T., Bogart, E.H., Bartolome, D.S.: Biocybernetic system evaluates indices of operator engagement in automated task. Biological Psychology 40(1), 187–195 (1995)

    CrossRef  Google Scholar 

  19. Renard, Y., Lotte, F., Gibert, G., Congedo, M., Maby, E., Delannoy, V., Bertrand, O., Lécuyer, A.: OpenViBE: An Open-Source Software Platform to Design, Test and Use Brain-Computer Interfaces in Real and Virtual Environments. Presence 19, 35–53 (2010)

    CrossRef  Google Scholar 

  20. Hamadicharef, B., Zhang, H., Guan, C., Wang, C., Phua, K.S., Tee, K.P., Ang, K.K.: Learning EEG-based Spectral-Spatial Patterns for Attention Level Measurement. In: IEEE International Symposium on Circuits and Systems, Taipei, Taiwan, Province of China, pp. 1465–1468 (2009)

    Google Scholar 

  21. Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., Müller, K.R.: Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Processing Magazine 25(1), 41–56 (2008)

    CrossRef  Google Scholar 

  22. Peng, H., Long, F., Ding, C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(8), 1226–1238 (2005)

    CrossRef  Google Scholar 

  23. Williams, J., Michelitsch, G.: Designing effective haptic interaction: inverted damping. In: Extended Abstracts on Human Factors in Computing Systems, pp. 856–857. ACM (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

George, L., Marchal, M., Glondu, L., Lécuyer, A. (2012). Combining Brain-Computer Interfaces and Haptics: Detecting Mental Workload to Adapt Haptic Assistance. In: Isokoski, P., Springare, J. (eds) Haptics: Perception, Devices, Mobility, and Communication. EuroHaptics 2012. Lecture Notes in Computer Science, vol 7282. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31401-8_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-31401-8_12

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-31400-1

  • Online ISBN: 978-3-642-31401-8

  • eBook Packages: Computer ScienceComputer Science (R0)