An Online P300 Brain–Computer Interface Based on Tactile Selective Attention of Somatosensory Electrical Stimulation
- 502 Downloads
P300 component of event related potentials in response to visual and auditory stimulation has been widely used in brain–computer interfaces (BCI). In clinical applications, tactile stimulus based on somatosensory electrical stimulation is an alternative for patients with impaired vision or hearing. This study presents an online P300 BCI based on somatosensory electrical stimulation paradigm. P300 signals were elicited by tactile selective attention of electrical stimuli on four fingers.
Fifteen healthy subjects participated in this study. Participants’ task was to focus their attention on the target finger and count the number. The classification of P300 signals was performed by step-wise linear discriminate analysis.
The average classification accuracy of the somatosensory BCI was 79.81 ± 7.91%, with the information transfer rate at 4.9 ± 1.3 bits/min. The BCI performance on different time windows was also evaluated in the present study.
Our results demonstrate the feasibility of employing somatosensory electrical stimuli to build a practical online P300 BCI without taxing the visual and auditory channel, providing a wider application prospect in clinical applications and daily life. We anticipate our diagram to be a starting point for more explorations on utilizing electrical somatosensory stimuli in conjunction with portable BCI for neural rehabilitation.
KeywordsBrain–computer interface (BCI) P300 Tactile selective attention Electrical stimulation Somatosensory evoked potentials (SEP)
Brain computer interfaces (BCIs) enable users to set up bidirectional connections between the world and their minds by translating the brain signals into computer control commands and sending back feedback signals [1, 2, 3]. BCIs have attracted growing attention from researchers for studying, mapping, enhancing, and repairing human cognitive, sensory and motor functions . Due to its non-invasiveness and relatively high communication speed, BCIs based on electroencephalogram (EEG) have become one of the popular choices. Among all kinds of EEG potentials suitable for BCIs, the P300 wave is a positive EEG deflection in the human event-related potentials (ERPs) which occurs around 300 ms after the target stimulus has been presented. Over the past several decades, P300 s are still popular to drive BCI systems since they are relatively robust and reliable .
The vast majority of P300 based BCI systems have used visual and sometimes auditory stimuli to elicit P300 potentials [6, 7, 8, 9]. Though the fundamental research on ERPs mainly focuses on visual and auditory elicited P300 s, somatosensory stimuli can also be an interesting alternative option for general BCI purposes. Mueller–Putz’s group applied tactile stimuli to left/right index fingers, and asked the subject to pay attention to one of the fingers, different evoked potentials could be detected to discriminate target and non-target fingers which confirms that somatosensory stimulation is suitable for BCIs . Somatosensory modality based BCI has the advantages that it only frees the eyes and the ears of the user and not require them to literally stare at and pay attention to a screen/speaker, but also gets complete involvement of sensory nervous pathways which can be a strength in clinical rehabilitation applications. Vision and/or hearing impaired users and patients who are experiencing spinal cord/brain injuries (e.g. cerebral palsy) can utilize the power of somatosensory based P300 BCIs, and for those people, the robust and reliability of BCI systems may come for the first priority . Hence, somatosensory P300 s can be an appropriate candidate for practical BCIs in spite of their relatively slow communication speed.
Somatosensory stimuli can be delivered by electrical stimulators and tactile vibrators. Several previous reports present the potential use of vibro-tactile stimulus for P300 BCIs [12, 13, 14]. Either graspable oscillating objects or a set of grouped vibrators on an adjustable vest or belt was used in their studies. In alternative, electrical somatosensory stimuli can be used for BCI purpose due to its advantages in flexible stimulation protocols design and simplified hardware design. Electrical somatosensory stimulation has been commonly used in clinical rehabilitations [15, 16]. It has higher power efficiency than vibrators and can be easily generated by small chips under the precise timing control of various BCI systems. By changing stimulation frequency, duration, and intensity, we can easily provide personal tactile stimulation protocol for individual users. Recently studies showed that application of BCI can be used to greatly improve the efficiency of neural rehabilitation [11, 17, 18, 19, 20]. The rationale behind the use of the electric somatosensory BCI is that it involves somatosensory neural circuits during the rehabilitation, which can improve the neural plasticity of nerve system, while visual or auditory stimuli do not have such function.
This paper is to propose a P300 based online BCI system by using electrical somatosensory stimulation. In this study, electrical stimuli were sent separately to different fingers of the participants who would selectively pay attention to the chosen ‘target’ finger. Our hypothesis is that electrical somatosensory stimuli on different target/non-target body locations could evoke detectable and consistent P300 s which were differentiable. In addition, different time windows for online data analysis were evaluated to optimize the classification performance.
In total, fifteen healthy subjects from university aged 23–29 years (8 male, 7 female) were participated in this study. All subjects had normal or corrected to normal vision, normal hearing and normal somatosensory functions. None of them reported a clinical history of psychiatric conditions or brain diseases. All subjects were instructed about the experiment before the start of the recordings. This study was carried out in accordance with the recommendations of The Research Ethics Committee of academy with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by The Research Ethics Committee of academy.
2.2 Recording Technologies
EEG activity was acquired at Fz, C3, Cz, and C4 electrode sites of the 10–20 system. A ground electrode was attached to the forehead. The EEG electrodes were referenced to linked right mastoid electrodes and the impedances of all electrodes were below 5 kΩ. EEG data were sampled with a frequency of 1000 Hz and filtered before storage by a 0.1 Hz high pass, a 100 Hz low pass and a 50 Hz notch filter. The experiment was controlled by Psychophsics Toolbox Version 3 running on Matlab platform (The Math Works, Inc.).
The tactile selective attention task was conducted using electrical stimuli. The electrical stimuli were generated by electrical muscle stimulation device (Shenhe medical instrument corporation, Zhuhai, China), delivered through flexible patch electrodes placed on a subject’s fingers. A trigger signal was collected to synchronize the stimulation and record the EEG in responding to event types. The time duration of electrical stimulation was set to 1 ms, and the stimuli interval was 800 ms.
2.4 Experimental Protocol
For each subject, there were two blocks (one training block and one test block). The training block was performed before the experiment to train the classifier, and the training model was applied to the test block to classify the subject’s EEG data. Each block consisted of four sequences of electrical stimuli. Each of the four sequences consisted of electrical pulses on each finger 18 times in a random order. Participants took a 3–5 min breaks between blocks.
2.5 Classification and Analysis
To generate features for the classification analysis, continuous EEG data from each participant was first down sampled with a factor of four and then low-pass filtered at 30 Hz. Individual channel data was segmented into epochs, starting at the stimuli onset and ending at 800 ms after the stimulus onset. The activity from -200 ms to 0 was used as a baseline. To evaluate and find the optimized time window for classification, we extracted EEG data from two different windows: (1) 0–800 ms (whole length), (2) 250–550 ms (typical P300 time range from pre-experiments).
3.1 BCI Classification
3.2 Event-Related Potentials
The present study is aimed at building an online P300 BCI system based on tactile selective attention paradigm by using somatosensory electrical stimulations. The classification results of our proposed BCI paradigm were very encouraging. Our results show that the electrical stimulation can produce remarkable P300 component amplitudes, moreover, the differences between targets and non-targets in P300 wave are clear. Therefore, the paradigm using somatosensory electrical stimulation can be a practical candidate for P300 based BCI.
Over the last several decades, EEG based BCI have been widely used in the fields of neural engineering, clinical rehabilitation, and the fundamental research of neuroscience . In practical BCI systems, many modalities can be chosen as the BCI inputs, for example, pictures and shapes as visual input, sounds and music as auditory input [24, 25, 26]. Among these modalities, visual and auditory modalities become popular for their high speeds, high accuracies, and low amount of training time. However, somatosensory modality can also be an interesting option as it not only frees the eyes and ears of the user and is less noticeable in daily life usage scenarios but also shows great potentials in clinical rehabilitation training. Few literatures have thoroughly depicted and verified the usefulness of somatosensory modality as an input of BCI system. Here, we propose a simple but practical electrical stimulation paradigm in somatosensory modality.
In the current study, the ERP waveform morphology in response to target stimuli showed typical P300 waveform, while similar and smaller waveform during 200–300 ms could be seen in ERP recording in response to non-target stimuli. Such presentation of non-target ERP can be seen in several previous reports in auditory , visual  and tactile paradigm . By contrast, we found that another visual  and auditory  did not produce any P300 s for non-target cases. It would be interesting to further investigate the waveform morphology in response to non-target stimuli under different paradigms. However, this study focuses on the ability and performance of BCI based on somatosensory electrical stimulation paradigm. Although somatosensory ERP shows similar patterns in target and non-target cases, our results suggest that the difference in amplitude might be practical enough for distinguishing. It also demonstrates that somatosensory P300 elicited by electrical stimulation on different body locations could serve as the classification features in BCI applications.
Previously, a somatosensory paradigm based on vibro-tactile stimuli around the waist was proposed in . Our method explores the possibility of using electrical stimuli to elicit ERP components. Besides, electrical stimulation has several advantages: (1) the general implementation of electrical stimulator is simpler, easier, and smaller than vibrators; (2) the energy consumption of electrical stimulation generator is optimal. Therefore, for practical BCI systems, our paradigm is supposed to be better in portability and wearability. For its involvement of sensory pathways, electrical somatosensory stimulation is preferably used to improve sensory function after spinal cord and brain injuries, for example, the spastic hands. Hence, the electrical somatosensory stimuli paradigm proposed here shows great potential for clinical usage: Firstly, the electrical stimulation protocols can be set up based on extensive clinical experience to optimize the outcome. Secondly, most patients suffered from central nervous system injuries may experience difficulties when they were asked to do typical visual/auditory tasks in classical BCI systems. For example, the patients may lose the ability to see/listen clearly, to move their heads voluntarily after stroke. In this situation, our method not only provides an effective way to bridge the communication between the human brain and machines, but also useful for rehabilitation trainings. Besides, for other users, this paradigm will allow the user to operate BCI tasks less noticeably than traditional visual/auditory paradigms, and it is not taxing normal routine visual and auditory systems.
We showed that somatosensory electrical stimulation based on tactile selective attention on different body locations could produce remarkable P300 waves. The results have confirmed that electrical somatosensory stimuli can be an interesting option for BCI inputs alternatively to commonly used visual stimuli. Currently, our paradigm is less accurate and slower than visual paradigms. Though, in clinical rehabilitation scenarios, the accuracy is less important than reliability, our method still achieves a relatively acceptable performance. In the experimental design, we only select channel Cz, C3, C4 and Fz from the cap, for the consideration that our primary goal is to demonstrate the feasibility of electrical somatosensory stimulation in BCI application, especially for neural rehabilitation purpose. Therefore, we strictly select few channel and commonly-used simple classification algorithms. However, the relatively low performance may be improved by developing new stimulation scheme and new classification algorithm.
In the present study, we used a simple and classical SWLDA classifier. One of the key problems in classification is to find proper features of epochs to discriminate target and non-targets. We compared two types of time windows in our experiment. First we expected that using more data could increase classification performance. On the contrary, the time window of 250–550 ms was better in classification accuracy and ITR. It was apparent that P300 elicited by tactile selective attention of electrical stimuli was clear enough to distinction target and non-targets. An explanation could be that amplitude of P300 was relatively more significant and stable in 250–550 ms window and the classification model might use these features to steadily classify target and non-targets. As in the ERP results (Fig. 4), the majority of typical P300 features were clustered around 300 ms which felt in the range of 250–550 ms. Nevertheless, the time window of 0–800 ms might contain abundant EEG information which could disturb the classifier. In our study, the P300 feature was significant in 250 ms-550 ms after stimuli onset for all subjects. Possibly, factors like gender and age could have varied individual difference in response to electrical stimulation. To further improve the performance of our BCI paradigm, adaptive parameter selection for time window should be considered.
The current study also shows some deficiencies in several aspects. Firstly, our paradigm still has a long way toward performance improvements. Better stimulation scheme and adaptive classifier can be introduced. Single-trial extraction method may contribute as well. Secondly, here we used computer screen to send instructions to participants which is still occupying the visual channel of sensory system. Another future research direction will be to adapt the paradigm so that the subject can actively appoint targets instead of passively receive targets, which can be more suitable for real-world scenarios.
In this paper, an online BCI system based on electrical somatosensory stimulation was proposed. The somatosensory P300 elicited by attention to specific fingers was demonstrated to be an effective input for BCI, suggesting that a larger command set could be achieved by extending the stimulation locations on the body. The feasibility of a BCI in somatosensory modality was demonstrated by the bitrate at a medium speed of 4.9 ± 1.3 bits/min. BCI based on somatosensory modality brings the involvement of sensory nervous pathways during usage. It will not affect visual and audial function when using somatosensory BCI. Future study may further improves the BCI performance by optimizing the stimulation sequence and classification algorithm.
This work is supported by the National Nature Science Foundation of China (61501524, 61431007), Key Technologies R & D Program of Tianjin (15ZCZDSY00600), CAMS Innovation Fund for Medical Sciences (2016-I2 M-3-023, 2016-I2 M-2-006).
- 1.Hughes, M. A. (2014). Engineering brain–computer interfaces: Past, present and future. Journal of Neurosurgical Sciences, 58(2), 117–123.Google Scholar
- 6.Simon, N., Käthner, I., Ruf, C. A., Pasqualotto, E., Kübler, A., & Halder, S. (2015). An auditory multiclass brain–computer interface with natural stimuli: Usability evaluation with healthy participants and a motor impaired end user. Frontiers in Human Neuroscience, 8(1039), 1039.Google Scholar
- 7.Mccane, L. M., Heckman, S. M., Mcfarland, D. J., Townsend, G., Mak, J. N., Sellers, E. W., et al. (2015). P300-based brain–computer interface (BCI) event-related potentials (ERPs): People with amyotrophic lateral sclerosis (ALS) vs. age-matched controls. Clinical Neurophysiology, 126(11), 2124–2131.CrossRefGoogle Scholar
- 12.Yajima, H., Makino, S., & Rutkowski, T. M. (2014). P300 responses classification improvement in tactile BCI with touch-sense glove. In Signal and information processing association annual summit and conference (APSIPA). https://doi.org/10.1109/APSIPA.2014.7041770.
- 14.Brouwer, A. M., & van Erp, J. B. (2010). A tactile P300 brain–computer interface. Frontiers in Neuroscience, 4, 19.Google Scholar
- 19.King, C. E., Wang, P. T., McCrimmon, C. M., Chou, C. C., Do, A. H., & Nenadic, Z. (2015). The feasibility of a brain–computer interface functional electrical stimulation system for the restoration of overground walking after paraplegia. Journal of Neuroengineering and Rehabilitation, 12, 80.CrossRefGoogle Scholar
- 25.Simon, N., Käthner, I., Ruf, C. A., Pasqualotto, E., Kübler, A., & Halder, S. (2014). An auditory multiclass brain–computer interface with natural stimuli: Usability evaluation with healthy participants and a motor impaired end user. Frontiers in Human Neuroscience, 8(1039), 1039.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.