An Improved Visual-Tactile P300 Brain Computer Interface

  • Hongyan Sun
  • Jing Jin
  • Yu Zhang
  • Bei Wang
  • Xingyu Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10635)


Recently, the bimodal BCI has attracted more and more attention. Previous studies have reported that the classification performance of bimodal system was better than that of unimodal system. Based on the fundamental visual-tactile P300 BCI, this paper made a change on the flash pattern of visual stimuli expecting to improve its performance by enhancing the link between visual and tactile modalities. Two patterns were tested in this paper, which respectively were picture-vibrate pattern (producing the visual effect of vibration) and color-change pattern (changing blue to green). The results showed that the picture-vibrate pattern achieved higher classification accuracy and information transfer rate than color-change pattern. The average online bit rate of picture-vibrate pattern including the breaking time between selections, reached 12.49 bits/min, while the color-change pattern’s online bit rate reached 8.87 bits/min on average.


Brain computer interface P300 Visual-tactile Picture-vibrate pattern Color-change pattern 



This work was supported by the Grant National Natural Science Foundation of China, under Grant Nos. 91420302, 61573142. This work was also supported by the Fundamental Research Funds for the Central Universities (WH1516018, 222201717006) and Shanghai Chenguang Program under Grant 14CG31.


  1. 1.
    Mak, J.N., Wolpaw, J.R.: Clinical applications of brain-computer interfaces: current state and future prospects. IEEE Rev. Biomed. Eng. 2, 187–199 (2009)CrossRefGoogle Scholar
  2. 2.
    Wolpaw, J.R., Birbaumer, N., Mcfarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113(6), 767–791 (2002)CrossRefGoogle Scholar
  3. 3.
    Farwell, L.A., Donchin, E.: Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70(6), 510–523 (1988)CrossRefGoogle Scholar
  4. 4.
    Sellers, E.W., Mcfarland, D.J.: Toward enhanced P300 speller performance. J. Neurosci. Meth. 167(1), 15–21 (2008)CrossRefGoogle Scholar
  5. 5.
    Jin, J., Sellers, E.W., Zhou, S., Zhang, Y., Wang, X., Cichocki, A.: A p300 brain-computer interface based on a modification of the mismatch negativity paradigm. Int. J. Neural Syst. 25(3), 595–599 (2015)CrossRefGoogle Scholar
  6. 6.
    Jin, J., Allison, B.Z., Zhang, Y., Wang, X., Cichocki, A.: An ERP-based BCI using an oddball paradigm with different faces and reduced errors in critical functions. Int. J. Neural Syst. 24(8), 1450027 (2014)CrossRefGoogle Scholar
  7. 7.
    Hill, N.J., Lal, T.N., Bierig, K., Birbaumer, N., Schölkopf, B.: An auditory paradigm for brain–computer interfaces. In: Saul, L.K., Weiss, Y., Botton, L. (eds.) Advances in Neural Information Processing Systems, vol. 17, pp. 569–576. MIT Press, Cambridge (2005)Google Scholar
  8. 8.
    Brouwer, A.M., van Erp, J.B.F.: A tactile P300 brain-computer interface. Front. Neurosci. 4, 19 (2010)Google Scholar
  9. 9.
    Aloise, F., Lasorsa, I., Schettini, F., Brouwer, A., Mattia, D., Babiloni, F., Salinari, S., Marciani, M.G., Cincotti, F.: Multimodal stimulation for a P300-based BCI. Int. J. Bioelectromagn. 9(3), 128–130 (2007)Google Scholar
  10. 10.
    Kaufmann, T., Holz, E.M., Kübler, A.: Comparison of tactile, auditory, and visual modality for brain-computer interface use: a case study with a patient in the locked-in state. Front. Neurosci. 7, 129 (2012)Google Scholar
  11. 11.
    Ortner, R., Lugo, Z., Prückl, R., Hintermüller, C., Noirhomme, Q., Guger, C.: Performance of a tactile P300 speller for healthy people and severely disabled patients. In: Conference Proceedings of IEEE Engineering in Medicine and Biology Society, pp. 2259–2262. IEEE Press, Osaka (2013)Google Scholar
  12. 12.
    van der Waal, M., Severens, M., Geuze, J., Desain, P.: Introducing the tactile speller: an ERP-based brain-computer interface for communication. J. Neural Eng. 9(4), 045002 (2012)CrossRefGoogle Scholar
  13. 13.
    Mori, H., Makino, S., Rutkowski, Tomasz M.: Multi–command chest tactile brain computer interface for small vehicle robot navigation. In: Imamura, K., Usui, S., Shirao, T., Kasamatsu, T., Schwabe, L., Zhong, N. (eds.) BHI 2013. LNCS, vol. 8211, pp. 469–478. Springer, Cham (2013). doi: 10.1007/978-3-319-02753-1_47 CrossRefGoogle Scholar
  14. 14.
    Rutkowski, T.M., Mori, H.: Tactile and bone-conduction auditory brain computer interface for vision and hearing impaired users. J. Neurosci. Methods 244, 45–51 (2015)CrossRefGoogle Scholar
  15. 15.
    Kaufmann, T., Herweg, A., Kübler, A.: Toward brain-computer interface based wheelchair control utilizing tactually-evoked event-related potentials. J. Neuroeng. Rehabil. 11(1), 7 (2014)CrossRefGoogle Scholar
  16. 16.
    Herweg, A., Gutzeit, J., Kleih, S., Kübler, A.: Wheelchair control by elderly participants in a virtual environment with a brain-computer interface (BCI) and tactile stimulation. Biol. Psychol. 121, 117–124 (2016)CrossRefGoogle Scholar
  17. 17.
    Brouwer, A.M., Erp, J.B.F.V., Aloise, F., Cincotti, F.: Tactile, visual, and bimodal P300s: could bimodal P300s boost BCI performance. SRX Neurosci. 2010, 1–9 (2010)CrossRefGoogle Scholar
  18. 18.
    Thurlings, M.E., Brouwer, A.M., Erp, J.B.F.V., Werkhoven, P.: Gaze-independent ERP-BCIs: augmenting performance through location-congruent bimodal stimuli. Front. Neurosci. 8, 143 (2014)Google Scholar
  19. 19.
    Yin, E., Zeyl, T., Saab, R., Hu, D., Zhou, Z., Chau, T.: An auditory-tactile visual saccade-independent P300 brain–computer interface. Int. J. Neural Syst. 26(1), 1650001 (2016)CrossRefGoogle Scholar
  20. 20.
    Driver, J., Noesselt, T.: Multisensory interplay reveals crossmodal influences on ‘Sensory-specific’ brain regions, neural responses, and judgments. Neuron 57(1), 11–23 (2008)CrossRefGoogle Scholar
  21. 21.
    Hoffmann, U., Vesin, J.M., Ebrahimi, T., Diserens, K.: An efficient P300-based brain-computer interface for disabled subjects. J. Neurosci. Meth. 167(1), 115–125 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Hongyan Sun
    • 1
  • Jing Jin
    • 1
  • Yu Zhang
    • 1
  • Bei Wang
    • 1
  • Xingyu Wang
    • 1
  1. 1.Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of EducationEast China University of Science and TechnologyShanghaiChina

Personalised recommendations