Combining Brain-Computer Interfaces and Haptics: Detecting Mental Workload to Adapt Haptic Assistance

  • Laurent George
  • Maud Marchal
  • Loeiz Glondu
  • Anatole Lécuyer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7282)


In this paper we introduce the combined use of Brain-Computer Interfaces (BCI) and Haptic interfaces. We propose to adapt haptic guides based on the mental activity measured by a BCI system. This novel approach is illustrated within a proof-of-concept system: haptic guides are toggled during a path-following task thanks to a mental workload index provided by a BCI. The aim of this system is to provide haptic assistance only when the user’s brain activity reflects a high mental workload. A user study conducted with 8 participants shows that our proof-of-concept is operational and exploitable. Results show that activation of haptic guides occurs in the most difficult part of the path-following task. Moreover it allows to increase task performance by 53% by activating assistance only 59% of the time. Taken together, these results suggest that BCI could be used to determine when the user needs assistance during haptic interaction and to enable haptic guides accordingly.


Brain-Computer Interface EEG Force-Feedback Adaptation Mental Workload Guidance 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain–computer interfaces for communication and control. Clinical Neurophysiology 113(6), 767–791 (2002)CrossRefGoogle Scholar
  2. 2.
    Farwell, L.A., Donchin, E.: Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology 70(6), 510–523 (1988)CrossRefGoogle Scholar
  3. 3.
    Guger, C., Harkam, W., Hertnaes, C., Pfurtscheller, G.: Prosthetic control by an EEG-based brain-computer interface (BCI). In: Proc. European Conference for the Advancement of Assistive Technology (1999)Google Scholar
  4. 4.
    Allison, B., Graimann, B., Gräser, A.: Why use a BCI if you are healthy? In: ACE Workshop - Brain-Computer Interfaces and Games, pp. 7–11 (2007)Google Scholar
  5. 5.
    George, L., Lécuyer, A.: An overview of research on ”passive” brain-computer interfaces for implicit human-computer interaction. In: International Conference on Applied Bionics and Biomechanics, Venise, Italy (2010)Google Scholar
  6. 6.
    Zander, T.O., Kothe, C.: Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general. Journal of Neural Engineering 8(2) (2011)Google Scholar
  7. 7.
    Nijholt, A., Bos, D.P.O., Reuderink, B.: Turning shortcomings into challenges: Brain–computer interfaces for games. Entertainment Computing 1(2), 85–94 (2009)CrossRefGoogle Scholar
  8. 8.
    Müller-Putz, G.R., Scherer, R., Neuper, C., Pfurtscheller, G.: Steady-state somatosensory evoked potentials: suitable brain signals for brain-computer interfaces? IEEE Transactions on Neural Systems and Rehabilitation Engineering 14(1), 30–37 (2006)CrossRefGoogle Scholar
  9. 9.
    Cincotti, F., Kauhanen, L., Aloise, F., Palomäki, T., Caporusso, N., Jylänki, P., Mattia, D., Babiloni, F., Vanacker, G., Nuttin, M., Marciani, M.G., Millán, J.d.R.: Vibrotactile feedback for brain-computer interface operation. In: Computational Intelligence and Neuroscience (2007)Google Scholar
  10. 10.
    Chatterjee, A., Aggarwal, V., Ramos, A., Acharya, S., Thakor, N.: A brain-computer interface with vibrotactile biofeedback for haptic information. Journal of NeuroEngineering and Rehabilitation 4(1) (2007)Google Scholar
  11. 11.
    Feygin, D., Keehner, M., Tendick, R.: Haptic guidance: experimental evaluation of a haptic training method for a perceptual motor skill. In: Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 40–47 (2002)Google Scholar
  12. 12.
    Bluteau, J., Sabine, C., Yohan, P., Edouard, G.: Haptic Guidance Improves the Visuo-Manual Tracking of Trajectories. PLoS ONE 3(3) (2008)Google Scholar
  13. 13.
    Asseldonk, E.H.F., Wessels, M., Stienen, A.H., van der Helm, F.C., van der Kooij, H.: Influence of haptic guidance in learning a novel visuomotor task. Journal of Physiology 103(3–5), 276–285 (2009)Google Scholar
  14. 14.
    Lécuyer, A., Lotte, F., Reilly, R., Leeb, R., Hirose, M., Slater, M.: Brain-computer interfaces, virtual reality, and videogames. Computer 41(10), 66–72 (2008)CrossRefGoogle Scholar
  15. 15.
    Mühl, C., Gürkök, H., Plass-Oude Bos, D., Thurlings, M., Scherffig, L., Duvinage, M., Elbakyan, A., Kang, S., Poel, M., Heylen, D.: Bacteria Hunt: A multimodal, multiparadigm BCI game. In: International Summer Workshop on Multimodal Interfaces, University of Genua (2010)Google Scholar
  16. 16.
    Kohlmorgen, J., Dornhege, G., Braun, M., Blankertz, B., Müller, K.-R., Curio, G., Hagemann, K., Bruns, A., Schrauf, M., Kincses, W.: Improving human performance in a real operating environment through real-time mental workload detection. In: Dornhege, G., Millán, J.d.R., Hinterberger, T., McFarland, D., Müller, K.-R. (eds.) Toward Brain-Computer Interfacing, pp. 409–422. MIT press, Cambridge (2007)Google Scholar
  17. 17.
    Heger, D., Putze, F., Schultz, T.: Online workload recognition from EEG data during cognitive tests and human-machine interaction. In: Advances in Artificial Intelligence, pp. 410–417 (2010)Google Scholar
  18. 18.
    Pope, A.T., Bogart, E.H., Bartolome, D.S.: Biocybernetic system evaluates indices of operator engagement in automated task. Biological Psychology 40(1), 187–195 (1995)CrossRefGoogle Scholar
  19. 19.
    Renard, Y., Lotte, F., Gibert, G., Congedo, M., Maby, E., Delannoy, V., Bertrand, O., Lécuyer, A.: OpenViBE: An Open-Source Software Platform to Design, Test and Use Brain-Computer Interfaces in Real and Virtual Environments. Presence 19, 35–53 (2010)CrossRefGoogle Scholar
  20. 20.
    Hamadicharef, B., Zhang, H., Guan, C., Wang, C., Phua, K.S., Tee, K.P., Ang, K.K.: Learning EEG-based Spectral-Spatial Patterns for Attention Level Measurement. In: IEEE International Symposium on Circuits and Systems, Taipei, Taiwan, Province of China, pp. 1465–1468 (2009)Google Scholar
  21. 21.
    Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., Müller, K.R.: Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Processing Magazine 25(1), 41–56 (2008)CrossRefGoogle Scholar
  22. 22.
    Peng, H., Long, F., Ding, C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(8), 1226–1238 (2005)CrossRefGoogle Scholar
  23. 23.
    Williams, J., Michelitsch, G.: Designing effective haptic interaction: inverted damping. In: Extended Abstracts on Human Factors in Computing Systems, pp. 856–857. ACM (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Laurent George
    • 1
    • 2
    • 4
  • Maud Marchal
    • 1
    • 2
    • 4
  • Loeiz Glondu
    • 3
    • 4
  • Anatole Lécuyer
    • 1
    • 4
  1. 1.INRIARennesFrance
  2. 2.INSARennesFrance
  3. 3.ENS CachanBruzFrance
  4. 4.IRISARennesFrance

Personalised recommendations