BCI–VR-Based Hand Soft Rehabilitation System with Its Applications in Hand Rehabilitation After Stroke

Abstract

The recovery of hand motor function can effectively improve the living standard of stroke patients and relieve their psychological anxiety. Traditional physical rehabilitation training is unable to target the cause of motor function loss; therefore, the rehabilitation effect is not ideal. The objective of this study is to propose a hand rehabilitation system combining brain–computer interface (BCI), soft hand rehabilitation glove and virtual reality (VR), and explore its effectiveness on hand movement disorders in stroke patients. The corresponding comparison experiments conducted on 11 stroke patients demonstrated that the proposed BCI-based hand rehabilitation system can not only mobilize more cerebral cortex to participate in the process of hand motor rehabilitation, but also enhance the muscle strength, muscle tension, and improve the hand motor dysfunction of stroke patients.

1 Introduction

The number of elderly people suffering from vascular disorders has increased rapidly in developed countries. It has become a social problem, with consequences of paralysis and worsening living conditions. Many diseases can cause paralysis, for example, stroke, spinal cord injury (SCI), amyotrophic lateral sclerosis, and multiple sclerosis, in which stroke accounts for the largest proportion. According to the report of the World Health Organization (WHO) in 2016, stroke is now the second leading cause of death in the world. Most poststroke patients experience partial paralysis, which occurs mainly in the upper limbs, especially the hands [1, 2]. Ghassemi et al. [3] research has shown that six months after the onset of stroke, about 65% of stroke patients still have hand dysfunction, only 15% of stroke patients can recover about half of their hand function, and only 3% of patients are able to recover more than 70% of their original functions [4].

Stroke treatment for hand rehabilitation may require different equipment and methods, but physiotherapy (PT) is a central component of the rehabilitation process. Improving hand function requires repetitive task practice rehabilitation, which includes breaking tasks down into individual movements and practicing them to improve hand strength, accuracy, and range of motion. Constraint-induced motor therapy (CIMT), neuromuscular stimulation (NMS) and mental practice with motor imagery are some of the most common treatments for the rehabilitation of paraplegic hands after stroke, and their efficacy has been well established [5, 6]. However, these techniques have some important limitations, especially for patients in chronic stages. For example, nearly 50 percent of chronic patients with severe functional affectation do not experience improvement with CIMT, and residual motor activity is necessary for CIMT; therefore, CIMT is not suitable for stroke patients with severe limb weakness [7, 8].

Stroke is a disease of brain tissue damage caused by sudden rupture of brain blood vessels or blockage of blood vessels that prevent blood from flowing into the brain. The postoperative rehabilitation of stroke patients not only requires passive rehabilitation activities to help reduce muscle and bone atrophy, but also requires active exercises to improve central nervous tension, activate physiological functions of various systems and organs, and improve plasticity of damaged brain cells, and ultimately improve the motor function of patients. Traditional rehabilitation methods and rehabilitation equipment can not solve the above problems well, and the emergence of brain computer interface (BCI) and virtual reality (VR) technology provides a new way for the active rehabilitation of stroke patients and the improvement of brain plasticity. BCI can effectively obtain patients' active rehabilitation intention, and VR technology can effectively improve central nervous tension and activate the physiological functions of various system organs through immersive experience. How to combine the two methods with traditional rehabilitation methods and equipments, effectively extract patients’ rehabilitation intentions, improve patients’ rehabilitation enthusiasm and initiative, effectively improve the brain motor plasticity of stroke patients, so as to achieve scientific and effective rehabilitation effect, is a problem that needs to be considered in modern rehabilitation medicine.

Hence, there is a need for improved approaches to support motor rehabilitation therapy for stroke patients, especially for patients in the chronic stages. Some new approaches to support therapy have been developed and have been gaining attention, such as robot devices [9], functional electrical stimulation [10, 11], or virtual reality [12]. While these and other common approaches often consider neuroscientific principles and have fostered understanding of how the brain improves during stroke therapy, they typically do not utilize direct measures of brain activity.

BCI uses neural activity to directly control external devices with real-time feedback. Some BCI systems combine neural activity with feedback devices to create closed-loop multi-modal feedback designed to enhance Hebbian plasticity and thereby help restore lost motor function [13,14,15,16]. Numerous studies have shown that BCI therapy can induce long-lasting neurological changes and improve upper limb motor function in patients with subacute and chronic stroke [17, 18].

BCI systems can be combined with different types of external devices to assist the execution and learning of movements. In the approach for movement restoration, stroke patients wear EEG caps to perform motor imagery (MI) exercises. Decoded brain oscillations can be used to trigger feedback mechanism to reproduce imagined movement with a paralyzed limb, such as functional electrical stimulation (FES) or a robot assistant device. Hence, reward feedback only occurs when the patient imagines the desired movement. This feedback loop is most effective with “closed-loop” feedback, meaning that feedback is presented in real-time, ideally through informative, clear feedback that supports effective co-adaptation between the end-user and the system [19, 20].

The external devices combined with BCIs could be assistant robotic devices, or VR avatars. The robot-assisted devices that work in coordination with BCI systems can effectively help patients achieve movement functions that their affected limbs cannot achieve. Initially, robot hand rehabilitation systems that combined with BCI were mostly multi-degree-of-freedom exoskeletons [21,22,23]. Most of the multi-degree-of-freedom exoskeletons require the biological joints to be aligned with those of the exoskeleton, and a few have passive degrees of freedom or self-alignment features. Their rigid mechanical design makes the device robust and reliable, capable of delivering high levels of forces for more challenging rehabilitation programs. However, most of these robotic devices require experienced supervision to ensure patient safety, due to the high strength and force of the actuators prone to injury of the patient joints.

To address these existing limitations, soft materials have been explored in the manufacture of rehabilitation exoskeleton and a type of exoskeleton named “soft wearable robot” has been designed [24,25,26,27]. This type of exoskeleton is designed to be more in line with human limbs and a series of soft robots for assisting and rehabilitation training have been developed, including soft robot gloves [28], elbow sleeves [29], and the exoskeleton of the entire arm [30, 31].

For the application of hand rehabilitation devices, some researchers have tested the BCI-hand robot system for patients with chronic and subacute stroke, while some researchers have verified the system for healthy participants [32, 33]. Frolovet et al. [34] conducted a randomized controlled trial of 55 stroke patients in the BCI group. The experimenter underwent an MI–BCI intervention to assist the hand exoskeleton drive to open and clench the affected limb. In contrast, the 19 stroke patients in the control group only performed an auxiliary hand exoskeleton drive to open and clench the affected limbs and hands without MI–BCI intervention (only passive rehabilitation). The results of the study showed that the Action Research Arm Test (ARAT) and Fugl–Meyer Assessment (FMA) scores of stroke patients in the experimental group increased by 21.8% and 36.4%, respectively, while in the comparison group, the two scores increased by only 5.1% and 15.8%. In another study, Wang et al. conducted a randomized controlled trial [35]. Thirteen chronic stroke patients received robotic hand training based on electroencephalogram combined with movement observation (BCI group), and 11 chronic stroke patients received robotic hand training without movement observation and electroencephalogram (control group). Their results showed that the recovery of upper limb motor function of stroke patients in the BCI group was significantly improved over the long-term, which was not achieved by stroke patients in the control group.

In 2019, Bin he et al. proposed and validated a noninvasive framework using EEG to achieve the neural control of a robotic device for continuous random target tracking [36]. The proposed framework enhanced BCI learning by nearly 60% for traditional center-out tasks, and by more than 500% in the more realistic continuous pursuit task. Such combined advances in the quality of neural decoding and the practical utility of noninvasive robotic arm control will have major implications for the eventual development and implementation of neurorobotics by means of noninvasive BCI.

VR is a relatively recent approach that may enable simulated practice of functional tasks at a higher dosage than traditional therapies [12]. VR has been defined as the “use of interactive simulations created with computer hardware and software to present users with opportunities to engage in environments that appear and feel similar to real-world objects and events” [37]. VR has been used in a neurological rehabilitation population to improve upper [38] and lower extremity function and gait [39] as well as cognition, perception, and functional tasks such as crossing a street, driving, preparing food, and shopping. VR can help the user relearn lost movements due to disease with immersive avatars that can demonstrate and perform these movements only when the patients imagine or attempt them correctly. It is important that the subjects feel a sense of “body ownership” over the virtual limbs; that is, the virtual limbs feel like each subject’s real limbs [40, 41].

The objective of this study is to propose a BCI based hand rehabilitation system that combines soft hand rehabilitation glove and VR, and explore the effectiveness of the proposed system on hand movement disorders in stroke patients. The proposed rehabilitation system combines MI therapy with a VR avatar and soft rehabilitation glove, provides real-time feedback based on subject’s EEG signals. Comparison rehabilitation experiments were conducted on 11 stroke patients and surface electromyography (sEMG) evaluation system, Fugl–Meyer assessment and modified Barthel index (MBI) were used to evaluate the experimental results.

The rest of this article is organized as follows: Sect. 2 introduces the proposed BCI–VR-based hand rehabilitation system, including the system design and the construction of each part. Section 3 explains the experiment design and results. The fourth section discusses and analyzes the experiment results, and Sect. 5 draws the conclusion.

2 Materials and Methods

2.1 System Design

Figure 1 shows the system design of the proposed hand soft rehabilitation system based on BCI and VR. Four modules construct the proposed system: EEG signal collection and analysis module, control center, VR module, and hand soft rehabilitation module. The EEG signal collection and analysis module is responsible for the collection, amplification, preprocessing, and pattern recognition of the user's scalp EEG signal. The control center is the information exchange center of the system. On the one hand, it receives the EEG signal processing results and sends them to the virtual reality module and hand soft rehabilitation exoskeleton. The two modules then adopt corresponding rehabilitation exercises according to the EEG signal processing results. On the other hand, the rehabilitation therapist can also set the rehabilitation parameters through the human–machine interface of the control center, so as to control patients to carry out specific rehabilitation exercises. The VR module provides patients with different backgrounds and modes of the virtual reality environment, so as to enhance the immersion and commitment of patients in the rehabilitation process and improve the rehabilitation effect. At the same time, by synchronizing the rehabilitation movement information of patients' hands, virtual reality can also give patients timely and accurate visual and auditory feedback. The hand soft rehabilitation module includes two parts: soft rehabilitation exoskeleton and hand position information detection. The hand soft rehabilitation exoskeleton receives the rehabilitation exercise information sent by the control center and controls the patient's hand to move accordingly. The hand position information detection is responsible for detecting the real-time motion state of hand and communicating with the VR module. The VR module will show the patient's hand motion state in the form of dynamic visual and voice prompts, and build visual and auditory feedback for the patient.

Fig. 1

The proposed hand soft rehabilitation system based on BCI and VR: a system diagram; b system operation diagram

Since the normal communication pathway (brain–nerve–muscle) in stroke patients is destroyed (shown by the blue arrow in Fig. 1), additional communication pathways must be established. Therefore, the system proposed in this paper constructs two rehabilitation modes for users: passive rehabilitation mode and active rehabilitation mode. In passive rehabilitation mode, rehabilitation therapists specify rehabilitation training plans for patients and set rehabilitation training parameters, including rehabilitation movements, training duration, and frequency, through the human–machine interface of the control center. The control center controls the hand soft rehabilitation exoskeleton driving the patient's hand to carry out rehabilitation training in accordance with the rehabilitation parameters that are set. At the same time, the VR module receives the rehabilitation parameter information and the motion state information of the soft rehabilitation exoskeleton, so as to provide patients with real-time station information and motion video of hands, and give patients synchronous visual and auditory feedback. In the passive rehabilitation mode, patients passively receive rehabilitation training information and execute the rehabilitation plans formulated by rehabilitation therapist, they are the information receivers and action executors. The direction of information flow and the rehabilitation process are shown in Fig. 1 with the red arrows. In the active rehabilitation mode, the patient sends rehabilitation wishes through the brain–computer interface, and the external system assists the patient to complete rehabilitation actions. The patient is the sender of information and the active executor of rehabilitation actions. The rehabilitation intention is sent out from the patient, through EEG signal acquisition and analysis, control center, virtual reality, hand soft rehabilitation exoskeleton, and finally, is implemented. The system gives the patient triple feedback of movement, sound, and vision, thus constructs a closed-loop path of external information transmission for the patient. In the active rehabilitation mode, the direction of information flow and the rehabilitation process are shown as the green arrows in Fig. 1. Each part of the system will be described in detail below.

2.2 EEG Signal Acquisition and Analysis

The EEG signal acquisition and analysis module mainly completes the EEG signal acquisition, amplification, preprocessing, and pattern recognition. The results of EEG signal analysis directly affect the performance of the BCI system, which is the core part of the BCI system. It consists of EEG signal acquisition and EEG signal analysis.

2.2.1 EEG Signal Acquisition

In this paper, a 32-channel wireless EEG acquisition system provided by Neuracle is used to collect the EEG signals of users. Figure 2a illustrates the main construction of the Neuracle EEG acquisition system, including a 32-electrode cap, a wireless power amplifier, and a wireless router. The system sampling frequency can be set to 100–250 Hz, and in this paper, the sampling frequency is set to be 250 Hz. Figure 2b presents the electrode position of the Neuracle EEG cap. The EEG acquisition procedure is as follows: the subject generates EEG signals, the electrode cap collects EEG signals and transmits them to the amplifier terminal through wireless Wi-Fi, and finally, the amplified EEG signals are transmitted to the EEG analysis part for signal analysis.

Fig. 2

a Main construction of the Neuracle EEG acquisition system; b electrode positions of the Neuracle EEG cap

2.2.2 EEG Signal Analysis

Due to the complexity of brain structure, the weakness of brain signals and the overlap of brain signal sources in the transmission process, there are many difficulties in the feature extraction and analysis of EEG signals, such as high misjudgment rate, low information transmission rate, and poor robustness. Providing an efficient, accurate, and stable EEG signal analysis method for BCI system, and creating a real-time, accurate information extraction and analysis method that meets the information interaction demand of the BCI system is an urgent problem in current BCI research.

Traditional EEG signal analysis is mostly based on the framework of features, which usually uses the features of time domain, frequency domain, space domain or the fusion of multiple features, and judges different brain activity states according to the dissimilarity among the features.

The commonly used feature extraction methods include signal bandpass energy (amplitude) [42], autoregressive model [43], phase characteristics [44], frequency domain statistics, wavelet (packet) transformation [45, 46], HHT transformation [47], and information entropy [48]. Although these methods have achieved good results, the commonality between the same brain activity features and the differences between different brain activity features are not obvious in traditional Euclidian space. Meanwhile, the existence of jump points and discrete points in feature data often leads to the misjudgment of the BCI system. Therefore, the traditional EEG signal analysis methods in the Euclidian space still cannot meet the application requirements of the BCI system.

In recent years, the Riemannian geometric method for symmetric positive definite (SPD) matrices has attracted much attention in signal processing [49, 50]. As the Riemannian geometric method can operate the covariance matrix and its subspace directly on the basis of its geometric structure, it has better performance of information extraction and analysis than classical signal processing method based on feature vectors. Therefore, increasingly more attention has been paid to the analysis of EEG signals in the Riemannian space.

Barachant et al. [49] proposed a Riemannian space analysis method for EEG signals, minimum distance to Riemannian mean (MDRM) method, which completed the feature extraction and classification of EEG signals in Riemannian space. The basic idea of MDRM method is: calculating the Riemannian mean of all the SPD matrices of EEG signals in a given label training set, when an EEG signal to be classified is given, the Riemannian distance between its SPD matrix and the mean of each class is calculated, the category label is defined as the label of the closest class. This method makes full use of the geometric framework of Riemannian manifold, and its classification performance is comparable to that of CSP + LDA(Common Spatial Pattern + Lenear Discriminant Analysis). However, the weaknesses of this method are obvious: the calculation of Riemannian mean is complex and time-consuming. The SPD matrix, which is the Riemannian feature of EEG, cannot be used as the input of classical classifiers, such as LDA and SVM (Support Vector Machine). Therefore, their classification effects cannot be further improved. Due to the above problems, many subsequent improved methods have been proposed, such as the tangent space linear discriminate analysis (TSLDA) method [49], projection classification of Riemann tangent space [51], and Fisher's metric discriminate criterion method [52]. All of them have achieved good analysis results. Nowadays, increasingly more EEG analysis methods based on the Riemann space are proposed.

In view of the great advantages and potential of Riemannian space analysis method in EEG signal analysis, the Riemann space PSD-kNN algorithm is used in this paper. The PSD-kNN algorithm is proposed by Gao et al. [53] in 2020 and achieved much better analysis results compared with traditional Euclidean methods. Three parts construct the PSD-kNN algorithm: the construction of the PSD matrix, the calculation of the similarity/dissimilarity, and the classification using kNN. The flow chart of the PSD-kNN algorithm is shown in Fig. 3.

Fig. 3

The flow chart of the PSD-kNN algorithm in Riemannian space

2.2.2.1 Construction of PSD Matrix

Suppose the n^th epoch for the i^th subject of the multi-channel raw EEG data at time $t$ can be expressed as a vector:

$${{\varvec{S}}}_{n}^{({\varvec{i}})}(t)=[{{\varvec{s}}}_{n1}^{({\varvec{i}})}(t),...,{{\varvec{s}}}_{nM}^{({\varvec{i}})}(t)],t=1,...T,$$

(1)

Thus, the n^th epoch measure data matrix (representing M channels of measured data for a duration of T seconds) for the i^th subject is given by

$${{\varvec{S}}}_{n}^{({\varvec{i}})}=[{{\varvec{s}}}_{n}^{({\varvec{i}})}(1),...,{{\varvec{s}}}_{n}^{({\varvec{i}})}(T)],n=1,...,N$$

(2)

After being labeled carefully, for the i.^th subject, the EEG signals of known categories can be expressed with Formula (3)

$$\boldsymbol{\Phi }=\left\{\left[\begin{array}{c}{{\varvec{S}}}_{1}\\ {{\varvec{\iota}}}_{1}\end{array}\right],...,\left[\begin{array}{c}{{\varvec{S}}}_{n}\\ {{\varvec{\iota}}}_{n}\end{array}\right],...,\left[\begin{array}{c}{{\varvec{S}}}_{k}\\ {{\varvec{\iota}}}_{k}\end{array}\right]\right\}$$

(3)

where ${\iota }_{n}\in \{\mathrm{1,2},...,k\}$ represents the class tag belonging to the n^th cycle of the motor imagery EEG signals.

For the n^th epoch of the multi-channel EEG data matrix ${{\varvec{S}}}_{n}$, the column vector ${{\varvec{s}}}_{n}(t)={[{{\varvec{s}}}_{n1}(t),...,{{\varvec{s}}}_{nM}(t)]}^{T},t=1,...,T$, is the measurement from the M channels and can be considered as a wide-sense stationary vector. Therefore, its ensemble mean and the covariance can be evaluated approximately by obtaining the corresponding time averages.

That is:

$${\mu }_{n}=E[{{\varvec{s}}}_{n}]\cong \frac{1}{T}\sum_{t=1}^{T}{{\varvec{s}}}_{n}(t)$$

(4)

$$\begin{aligned}{{\varvec{R}}}_{n}\left(\tau \right)&=E\left[\left\{{{\varvec{s}}}_{n}\left(t+\tau \right)-{\mu }_{n}\right\}{\left\{{{\varvec{s}}}_{n}\left(t\right)-{\mu }_{n}\right\}}^{T}\right]\\ &\cong \frac{1}{T}\sum_{t=1}^{T}\left\{{{\varvec{s}}}_{n}\left(t+\tau \right)-{\mu }_{n}\right\}{\left\{{{\varvec{s}}}_{n}\left(t\right)-{\mu }_{n}\right\}}^{T}\end{aligned}$$

(5)

The covariance matrix ${{\varvec{R}}}_{n}(\tau )$ is positive semi-definite. If ${{\varvec{R}}}_{n}(\tau )$ is also finite in its ${\mathcal{l}}_{1}$-norm sum, taking the discrete Fourier transform, the ‘Hermitian positive definite’ PSD matrix of the signal at frequency $\upomega$ can be obtained:

$${{\varvec{P}}}_{n}(\omega )=\frac{1}{2\pi }\sum_{\tau }{{\varvec{R}}}_{n}(\tau ){\mathbf{e}}^{-\mathrm{j}\omega \tau }$$

(6)

Ideally, $\tau$, the time shift range in Eq. (6) should be $(-\mathbf{\infty },+\mathbf{\infty })$. In practice, owing to the finite number of samples, Eq. (6) is evaluated for $\tau \in [-(T-1),T-1]$. In Eq. (6), any discrete frequency value $\omega ={\omega }_{i}$ within the range of $[{\omega }_{min},{\omega }_{max}]$ can be chosen to evaluate the corresponding ${{\varvec{P}}}_{n}(\omega )$. Hence, the nth epoch multichannel EEG signal matrix ${{\varvec{S}}}_{n}$ can be characterized by its PSD matrix sequence ${{\varvec{P}}}_{n}(\omega )$ in the frequency range. In order to compute the PSD matrix using the finite measured EEG data, the Nuttall–Strand algorithm is employed in the PSD-kNN algorithm. The Nuttall–Strand algorithm uses forward–backward linear prediction to iteratively estimate the residual covariance matrices arriving at an accurate positive semi-definite estimate of the PSD matrix with high-frequency resolution.

2.2.2.2 The Calculation of the Similarity/Dissimilarity of PSD Matrix

PSD matrices are Hermitian and positive definite; they are no longer free points in the signal space, rather they form a manifold $\mathcal{M}$. Each PSD matrix P (at a particular frequency $\omega$) can be regarded as a point on the manifold $\mathcal{M}$ and at different frequencies $\omega \in [{\omega }_{\mathrm{min}},{\omega }_{\mathrm{max}}]$, and the positive definite PSD matrix of a signal epoch can be represented as a series of points forming a curve on $\mathcal{M}$. For the same frequency range $[{\omega }_{\mathrm{min}},{\omega }_{\mathrm{max}}]$, two PSD matrices of the m^th and n^th epochs describe two sequences of points (two separate curves) on $\mathcal{M}$ denoted by ${{\varvec{P}}}_{m}(\omega )$ and ${{\varvec{P}}}_{n}(\omega )$, respectively.

In order to distinguish the categories of the EEG signals to the maximum extent, it is necessary to establish a similarity/dissimilarity measurement to maximize the similarity between the same categories of EEG signals and minimize the similarity between the different categories of EEG signals. For the two curves on the manifold described by two PSD matrices ${{\varvec{P}}}_{m}(\omega )$ and ${{\varvec{P}}}_{n}(\omega )$, a Riemannian distance ${{\varvec{d}}}_{R}({{\varvec{P}}}_{m}(\omega ),{{\varvec{P}}}_{n}(\omega ))$ is a non-negative real valued function of $\omega$, measuring the distance between the two curves on the manifold $\mathcal{M}$ at frequency $\omega$. As $\omega$ varies, the distance between the curves ${{\varvec{P}}}_{m}(\omega )$ and ${{\varvec{P}}}_{n}(\omega )$ in the range of $[{\upomega }_{\mathrm{min}},{\omega }_{\mathrm{max}}]$ can be defined as:

$$\begin{aligned}\mathcal{d}({{\varvec{P}}}_{m}(\omega ),{{\varvec{P}}}_{n}(\omega ))&={\int }_{{\omega }_{min}}^{{\omega }_{max}}{\mathcal{d}}_{R}(\omega )d\omega \cong \sum_{i}{\mathcal{d}}_{R}({{\varvec{\omega}}}_{{\varvec{i}}})\Delta {\omega }_{i}\\ &=\sum_{i}{\mathcal{d}}_{R}({{\varvec{P}}}_{m}({\omega }_{i}),{{\varvec{P}}}_{n}({\omega }_{i}))\Delta {\omega }_{i}\end{aligned}$$

(7)

It is easy to show that this Riemann integral satisfies the axioms of distance function. If equal frequency increment is used, that is, $\Delta {\omega }_{i}$ is a constant, then we can define the dissimilarity between the two given PSD curves as:

$$\begin{aligned}{\varvec{d}}{\varvec{i}}{\varvec{s}}{\varvec{s}}{\varvec{i}}{\varvec{m}}{\varvec{i}}{\varvec{l}}{\varvec{a}}{\varvec{r}}{\varvec{i}}{\varvec{t}}{\varvec{y}}&=\mathcal{d}({{\varvec{P}}}_{m}(\omega ),{{\varvec{P}}}_{n}(\omega ))\\ &=\sum_{i}{\mathcal{d}}_{R}({{\varvec{P}}}_{m}({\omega }_{i}),{{\varvec{P}}}_{n}({\omega }_{i}))\end{aligned}$$

(8)

Equation (8) shows that when the two EEG PSD matrices are of the same categories, the curve distance described by the two PSD matrices is short and the dissimilarity is small, which can be judged to be relatively similar. Therefore, Eq. (8) can be used as the basis for subsequent classification of EEG signals.

2.2.2.3 The Classification Using kNN

As a classic classification algorithm, k-nearest neighbor (kNN) algorithm is mainly used in data mining technology. The kNN algorithm is used to classify the EEG signal characteristics, the distance between the training data set and the classification object is calculated, and the k-nearest neighbors are selected in ascending order. Finally, according to the classification labels of k neighborhoods, the obtained labels are assigned to the classification attributes of the classification objects.

In general, there is no rule to choose the best value of k in the kNN algorithm. If the sample size is infinite, the larger is k, the better is the performance of the kNN classifier. In actual experiments, the sample size is finite and if the sample size is not very large, it is necessary to select the appropriate k to make the algorithm achieve a better classification result.

2.3 Control Center Module

As the system brain, the control center can make logical judgment on the received signal instructions and issue corresponding control instructions to other modules. At the same time, functional parameters can be set on the human–computer interaction interface. Various parameters in the rehabilitation training process (such as port setting, control instruction sending window, continuous instruction sending button, virtual reality environment selection button, rehabilitation training gear, detection function, training duration) can be set through this interface. The human–computer interaction interface is shown in Fig. 4.

Fig. 4

The human–machine interface of the control center

The port baud rate, data bit, stop bit, check bit and other parameters of the control center interface and the soft rehabilitation glove exoskeleton system can be set according to the configuration parameters of the Bluetooth module, so that the two can communicate normally. After the above port configuration is completed, you can click the open serial port button. If you normally open the serial port and communicate, the red indicator will display green. You can enter the rehabilitation action command in the text indicator box. The clear sending area button can clear all the rehabilitation action commands and stop the rehabilitation action immediately. The timing sending option controls the sending interval of rehabilitation action commands. Rehabilitation action instructions are shown in Table 1.

Table 1 Rehabilitation action control instructions and corresponding rehabilitation actions

Because the VR module develops several rehabilitation strategies with different difficulties, the Browse button can select the rehabilitation strategy in the local file. The detection function area is the evaluation mode of the lack of hand motor ability described above. The maximum extension angle and the minimum bending angle of each finger of the user can be displayed in the text prompt box. Refresh the detection function area button to re evaluate the lack of hand motor ability. The control center presets four different speed gears, that is, select the appropriate speed gear for patients with different rehabilitation periods and disease degrees for rehabilitation training. Regular training options can be selected according to the recommendations of rehabilitation doctors, in minutes.

2.4 Virtual Reality Module

The virtual reality (VR) module is established based on Unity3D engine. The virtual environment uses the first-person display of the patients' hand state, so that patients can get a strong sense of immersion and mutual feeling. The virtual reality (VR) module and EEG signal processing module use TCP/IP protocol to communicate, and the programming language is C#. The flow chart of data transmission between virtual reality (VR) module and sensor is shown in Fig. 5.

Fig. 5

Flow chart of data transmission between sensor and upper computer

The VR module is suitable for use with rehabilitation equipment because the rehabilitation equipment can not only assist the affected hand to stretch, grasp, and other movements, but also provides a certain degree of visual feedback. The use of virtual reality is designed to promote task-oriented and repetitive motor training of new motor skills while using a variety of stimulating environments.

In this paper, four rehabilitation task strategies were developed in the VR module: virtual study environment, billiards game, tree-chopping game, and exploration game. The difficulty levels increased gradually, different difficulty levels could gradually help improve the motor ability rehabilitation of patients with affected limbs.

Virtual study environment: the virtual study environment with high degree of reduction and high sense of immersion will bring patients into reality for initial passive rehabilitation, while virtual reality will give patients immersive visual feedback. The virtual study environment is shown in Fig. 6a.

Fig. 6

Rehabilitation task strategy in VR module: a the virtual environment of the study; b the virtual environment of billiards game; c the virtual environment of tree-chopping game; d the virtual environment of exploration game; e the ten gestures used in the exploration game

Virtual environment of billiards game: In this VR game, the virtual arm can grasp the billiard cue through the collision body detection, the MPU6050 sensor can obtain the position information, adjust the direction to reach the billiard ball to hit, and complete the different strength of the punch through clenching the fist for energy storage. The aim of this game is to allow patients to imagine their own hand movements and stimulate the recovery of damaged neural pathways. The VR environment of the billiards game is shown in Fig. 6b.

Virtual environment of tree-chopping game: the patient grasps the force of cutting bamboo by clenching his fist. If the force is not enough, the patient needs to keep the force several times. Task-based virtual games guide patients to achieve rehabilitation training goals. The VR environment of tree-chopping game is shown in Fig. 6c.

Virtual environment of exploration game: patients explore in the virtual environment and control the virtual characters to end the game, turn left, turn right, jump, squat, forward, backward, accelerate running, jog, and return to the initial state with 10 gestures from 0 to 9. The 10 gestures representing 0–9 are shown in Fig. 6e, and the exploring game's VR environment is shown in Fig. 6d.

2.5 Hand Soft Rehabilitation Module

The hand soft rehabilitation module is composed of two parts: hand soft rehabilitation gloves and hand soft rehabilitation control box, which is a multi-degree-of-freedom hand rehabilitation exoskeleton based on line drive. One end of the driving line is arranged on the hand soft rehabilitation glove, the other end is on the side wall of the control box. Considering that the driving lines will interfere with each other during operation, the five driving lines are all in independent driving line channels, which avoids the coupling problem. In addition, the line driving channel will reduce the loss of tension. The hand soft rehabilitation exoskeleton module is shown in Fig. 7.

Fig. 7

Hand soft rehabilitation module: hand soft rehabilitation gloves and hand soft rehabilitation control box

In order to make patients feel comfortable when wearing rehabilitation gloves, hand soft rehabilitation gloves are made of microfiber and polyester fabric, and cowhide patches are added to the palms and joints to improve wear resistance. The thermosetting pressure plates on the back of the fingers and the joints under the gloves are in a walking band to ensure that the rotation center of the connecting rod structure coincides with the rotation axis of the human joints. Otherwise, the force transmission may not be on the safety track, and causing damage to the user's fingers. The driving force of the driving system is transmitted to the finger joints through the flexible steel wire rope, and the coupling action between the fingers during the rehabilitation training process is avoided through the driving wire conduit. Combining with the recommendations of rehabilitation doctors, in order to transmit the driving force in the direction of fingers, the block guide rail is placed at the distal interphalangeal joint (DIP), proximal interphalangeal joint (PIP) and metacarpophalangeal joint (MCP) of the fingers on the back of the hand, and the block guide rail at the palm is placed at the distal phalanx, middle phalanx and metacarpal bone near metacarpophalangeal joint (MCP). The physical drawing of soft rehabilitation gloves and drive line drive catheter is shown in Fig. 8.

Fig. 8

Multi angle display of soft rehabilitation gloves and drive line conduit

Through the analysis of the existing drive system, servo motor is chosen as the power output source to provide power for patients' rehabilitation training. Servo motor is usually called steering engine, which has the advantages of high control accuracy, low power, large output torque, simple control and easy operation. The selected steering engine model is SG90 produced by Dongguan Desheng Intelligent Technology Co., Ltd. The steering engine is a servo driver controlled by pulse width modulation (PWM). The rotation angle of the steering wheel of the steering engine SG90 is 0°–180°, which is controlled by signals with a cycle of 20 ms and a width of 0.5–2.5 ms.

Through the analysis of finger bones and kinematics, and combined with the recommendations of rehabilitation doctors, after the field investigation and Research on the force on the fingers of patients during finger rehabilitation training, the system finally uses five steering engines with a limit output torque of 20 kg cm to assist the rehabilitation training of five fingers of patients. The specific physical drawing of the steering engine is shown in Fig. 9.

Fig. 9

Steering engine SG90

Five bending sensors are connected to the circuit board, which adopt the STM32F40LCBUb as the main control chip. The main control circuit board is integrated with voltage and resistance conversion circuit, data format selection switch, MicOR-USB charging interface, Bluetooth module and MPU6050 sensor. The bending sensor and the main control circuit board are shown in Fig. 10.

Fig. 10

Bending sensor of hand soft rehabilitation glove: a the bending sensor is a flex bending sensor produced by Shanghai Fengyou Information Technology Co., Ltd. with high sensitivity; b five bending sensors and the main control board. A TDK mpu6050 attitude sensor is integrated on the main control board to collect spatial attitude data during hand movement

Detection function of hand soft rehabilitation module: the bending sensor is fundamental to realize the detection function of the hand soft rehabilitation module. It is composed of an electronic element with variable resistance (9 K ohms when not bent, 22 K ohms when bent at 180°). The bending angle of the sensor can be obtained by quantifying the resistance value. The sensor is integrated into the hand soft rehabilitation glove to obtain the maximum range and acceleration of the user's finger movement. When the user wears the hand soft rehabilitation gloves, bending fingers will drive the bending sensor to deform. At this time, the sensor resistance will change, and the main control circuit board will send real-time data packets to the control center and virtual reality module. The control center records relevant data, and the virtual reality module refreshes the virtual reality environment according to the received data to achieve synchronization with the actual finger movement of the patient.

The hand soft rehabilitation control box is provided with five servo motors (steering engine), each steering engine’s driving line is connected to the corresponding finger to control its movement. The reason for choosing the steering engine as the driving unit is that the steering engine has high control accuracy and can accurately control the driving stroke, which can adapt to the needs of different users. When the steering engine is initialized, the operating angle is 0°, corresponding to the unbending state of the finger. The maximum operating angle is 180°, and the user's finger is completely bent at this time. As the Arduino controller controls the five steering engines separately, users can be assisted to complete the actions of a single finger, full fingers, and opposite fingers by controlling the operation of a different steering engine. The internal structure of the hand soft rehabilitation control box is shown in Fig. 11.

Fig. 11

The internal structure of the hand soft rehabilitation control box

3 Experiment Setup and Results

In collaboration with the Rehabilitation Department of Shandong Provincial Hospital of Traditional Chinese Medicine, our team conducted a series of rehabilitation experiments from November 2020 to March 2021. Participants in the rehabilitation experiment were all stroke patients of the Rehabilitation Department of Shandong Provincial Hospital of Traditional Chinese Medicine.

3.1 Experiment Setup

3.1.1 Subjects

During the subjects’ selection procession, the following criteria were followed:

Inclusion criteria: 1. Stroke or cerebral infarction occurred for the first time (first illness). 2. Unilateral limb loss. 3. Meets the diagnostic criteria established by the 4th National Cerebrovascular Academic Conference: 4.35 ≤ age ≤ 65 years old. 5. No cognitive impairment, MMSE (mini-mental state examination) score ≥ 27 points. 6. Fewer or no serious complications. 7. Cerebral infarction confirmed by CT or MRI. 8. Have not received formal motor imaging treatment before clinical trials. 9. Clear consciousness, good compliance, can clearly express the feelings of rehabilitation training. 10. Agrees to participate in this clinical trial and have signed an informed consent form.

Exclusion criteria: 1. Deterioration of the disease, new infarction, or large-scale cerebral infarction. 2. Those who have a history of epilepsy, cerebral hemorrhage, and severe heart, lung, liver, kidney, and other important organ failures. 3. Severe cognition (MMSE score < 27 points) and communication barriers that cannot be trained. 4. Patients receiving other clinical central nervous system interventions at the time of enrollment. 5. Those who cannot complete the basic course of treatment and have poor compliance. 6. Allergic to metals, polymer materials or other related foreign bodies used in this equipment. 7. There is a local skin infection or damage below the elbow of the upper limb. 8. The affected limb is congenital or due to other reasons before the onset of the disease, resulting in limb deformity, abnormal planing, skeletal differences, and joint insufficiency. 9. Participated in other clinical trials 1 month before the trial. 10. The accuracy of sports imagination has not reached 60%.

Interruption criteria: 1. If any adverse reactions or other complications occur during the experiment, it is not advisable to continue the experiment. 2. The compliance becomes worse and does not meet the test standards. 3. Withdrawal by oneself or unable to continue clinical trials for other reasons.

Based on the above criteria, a total of 11 stroke patients (9 males and 2 females, aged between 28 and 65) participated in this study. The stroke patients’ information is shown in Table 2. It is worth noting that the rehabilitation experiment conducted in this paper did not affect patients’ other rehabilitation training in the hospital, and patients’ normal rehabilitation training in the hospital was carried out as usual.

Table 2 Rehabilitation experiment subjects’ information

During the experiments, all participants were randomly divided into two groups: a BCI–VR rehabilitation training group and a soft rehabilitation glove training group. The BCI–VR rehabilitation group performed the active rehabilitation training with BCI, VR and rehabilitation glove, and the soft rehabilitation glove rehabilitation group performed the passive rehabilitation training only with the rehabilitation glove. The training time, frequency, and experiment duration of the two groups were the same, but the training content was different. The information and grouping of participants are listed in Table 2.

3.1.2 Experiment Schedule

The rehabilitation experiment is divided into six stages, from stage 0 to stage 5, of which stage 0 is the preparation stage, stage 5 is the rehabilitation evaluation stage, and stages 1–4 is the rehabilitation training stage. Stage 0 lasts for one week (week 0). The main purpose of stage 0 is to let participants understand the content of rehabilitation training, master the operation of rehabilitation equipment, and be familiar with the rehabilitation process. Stages 1–4 are rehabilitation training stages (from the first week to the 16th week), and each stage lasts for four weeks. These four stages gradually carry out rehabilitation training strategies with different degrees of difficulty. The difficulty, complexity and intensity of rehabilitation training are gradually increasing. In the rehabilitation training stage, participants receive rehabilitation training at least 5 times a week, and the length of each training is not less than 1 h and not more than 2 h, so as to avoid fatigue and boredom of participants. Stage 5 is the rehabilitation evaluation stage, from the 17th week to the 20th week. In stage 5, the participants did not receive rehabilitation training, only received rehabilitation evaluation tests once a week. The main purpose was to observe the sustainability and effectiveness of the rehabilitation effect after the first to fourth stages of rehabilitation training, and to conduct a comparative observation experiment before and after rehabilitation. The specific rehabilitation schedule is shown in Table 3.

Table 3 Experiment schedule

3.1.3 EEG Signal Analysis Experiment

In the EEG analysis experiment, 60 trials of EEG data were collected for every participant per day, and the experiment lasted for 5 days, that is, 300 trials of training data were collected for every participant. In every trial, participants sat in a comfortable chair in a quiet environment and wore an electrode cap to collect EEG signals, with a computer monitor 60–70 cm away from the patient's face. Every trial lasted 10 s. After two seconds’ blank screen time, a beep prompted the subject that the trial was beginning. Then the subject opened/clenched his/her affected hand according to the virtual hand’s action on the computer screen. The opening/clenching action lasted for five seconds, followed by a two-second rest before moving on to the next trial. The paradigm of one trial is depicted in Fig. 12. In each trial, the system randomly determined whether patients performed opening or clenching actions.

Fig. 12

The paradigm of the EEG analysis experiment trial

3.1.4 Evaluation Method

Before and after the rehabilitation training, all the participants were evaluated by a therapist who did not know the grouping situation. The evaluation methods included the surface electromyography (sEMG) evaluation system, FMA [54] and MBI [55]. The sEMG rehabilitation evaluation system adopts the rehabilitation evaluation system based on analytic hierarchy process and fuzzy evaluation method developed by our research group [56]. FMA includes two items of fingers and wrist, and MBI includes two items closely related to hand functions: eating and dressing.

3.1.4.1 Surface electromyography (sEMG) evaluation system

In the sEMG evaluation system used in this paper, the sEMG signals of the affected and healthy sides of patients with specific movements are collected through a three-electrode module. Then the features of sEMG are extracted and the difference between them is calculated. Based on the basic knowledge of clinical rehabilitation evaluation, the analytic hierarchy process (AHP) and Fuzzy Comprehensive Evaluation (FCE) are organically combined to construct the rehabilitation evaluation model, and finally the rehabilitation evaluation of patients based on sEMG signals is realized.

The three electrodes used in the sEMG system contain one reference electrode and two measuring electrodes. During the measurement, the reference electrode should be placed away from the active muscle, and the two measuring electrodes should be placed on the surface of the active muscle. For example, when measuring the sEMG signal of the abductor brevis muscle, the reference electrode can be placed at the wrist, and two measuring electrodes can be placed on the abductor brevis muscle surface.

After collecting the sEMG signals, three features are extracted: integrated Electromyogram (iEMG) value, root mean square (RMS) and median frequency (MF), which can respectively represent muscle strength, muscle tension and fatigue degree of muscle. Finally, the AHP_FCE evaluation model is established to evaluate the degree of rehabilitation of patients based on above features. For the specific details of signal acquisition, model building and rehabilitation assessment of the evaluation system, please refer to literature [56].

3.1.4.2 FMA (Fugel–Meyer assessment scale)

FMA was further quantified and accurately developed by Fugel–Meyer et al., based on the Brunnstrom Scale 6-level functional classification. It is a motor function assessment method specially designed for stroke patients. It covers five areas of motion, sensation, balance, range of motion and pain, and contains 113 assessment items. Among them, the assessment of motor function is the most widely used and recognized assessment method in the clinical and scientific evaluation of stroke efficacy.

3.1.4.3 MBI (modified Barthel index rating scale)

At present, the Barthel Index (BI) is the most widely used basic activity of daily living (BADL) evaluation method for stroke patients in clinic, which reflects the self-care ability of patients in self-care activities such as dressing, eating, personal hygiene, as well as metastatic activities such as sitting, standing and walking. BI is simple and clear, but its sensitivity is low, and the scale is slow to reflect the functional changes of patients. In 1989, Canadian scholars Shah and vanchay [55] subdivided the BI level into five levels and made the modified Barthel Index (MBI).

Meanwhile, the EEG topographic map, EEG signal analysis, and the statistical analysis of the evaluation data were all adopted to evaluate the rehabilitation results between the two comparison groups.

In the statistical analysis, SPSS19.0 software was used. ${\chi }^{2}$ test was used for counting data, independent sample $t$ test or independent sample nonparametric rank sum test was used for inter-group comparison of measurement data, and paired sample $t$ test or paired sample nonparametric rank sum test was used for intra-group comparison. $P\le 0.05$ indicates significant difference.

3.2 Experiment Results

3.2.1 EEG Analysis Results

In this paper, the PSD-kNN algorithm is used to analyze the EEG signals. The key points of the algorithm need to be clarified in the process of EEG signal analysis: This algorithm uses the kNN algorithm for classification. The question in this experiment is how much K can be selected to achieve a more satisfactory effect?

3.2.1.1 The Selection of k

For small data sets, the selection of k value has great influence on the classification accuracy of kNN algorithm. In this paper, the average classification accuracy of the five participants in the BCI–VR group under different k values was calculated and the curve of classification accuracy was drawn, as shown in Fig. 13.

Fig. 13

Classification accuracy of different k values

Figure 13 shows, as k value changes, that the accuracy rate also changes. In the process of k increasing gradually, the accuracy curve rises gradually. When k equals 23, the upward trend of the curve slows down and the subsequent classification accuracy rates are all above 80%, and the classification accuracy rate reaches the maximum value, 86.7%, at k equals 29. As k continues to increase, the accuracy drops slightly at k equals 31. The above analysis shows that, in the case of a certain training sample set, a relatively ideal classification effect can be obtained by selecting a reasonable k value, an excessive k value is unnecessary. For a small sample training data set, too large a k value may cause deviation, leading to the decline of accuracy. Therefore, in the following experiments, k is set to be 29.

3.2.2 EEG Analysis Results

3.2.2.1 EEG Topographic Map

For the participants in the BCI–VR group, two states (relaxed state and clenching hand state) and EEG topology maps were obtained at two stages (stage 0 and stage 5), aiming at comparing and analyzing the influence of the 16-week BCI–VR rehabilitation training on the brain activation degree of the participants.

Figure 14 is the EEG topology map of participant s7 in two states at two stages, in which (a) and (b) were collected in stage 0, (a) was in relaxed state, (b) was in clenching hand state, and (c) and (d) were collected in stage 5, (c) was in relaxed state, and (d) was in clenching hand state. Figure 14 shows that in relaxed state, the EEG topology map of stage 0 and stage 5 had little difference, that is, there was no obvious difference in voltage distribution between different brain regions in relaxed state. While in clenching hand stage, the difference of EEG topology map in stage 0 and stage 5 was obvious. As the affected hand of s7 was the right hand, in (b) and (d), the EEG topology map had obvious voltage variation in the left region, and both the voltage amplitude and the voltage changes area, figure (d) were bigger than figure (b), which shows that 16 weeks BCI–VR rehabilitation training had obvious activation in the cerebral cortex, the degree and scope of activation cerebral cortex were significantly expanded.

Fig. 14

The EEG topographic map of s7 in two states at two stages: a EEG topographic map of s7 in the relaxed state at stage 0; b EEG topographic map of s7 in clenching hand state at stage 0; c EEG topographic map of s7 in relaxed state at stage 5; d EEG topographic map of s7 in the clenching hand state at stage 5

3.2.2.2 sEMG (Surface Electromyography, sEMG) Evaluation

The sEMG rehabilitation evaluation experiment was conducted for all participants at stage 0 (before rehabilitation training) and stage 5 (after rehabilitation training), and the rehabilitation effects of the two groups were compared, respectively. Figure 15 is the strength muscle comparison diagram of two participants (s3—soft rehabilitation glove group, s1—BCI–VR group) at stage 0 and stage 5, and the detecting part is abductor pollicis brevis muscle. In Fig. 15a and b are the muscle strength comparison diagram of s3, (a) is in stage 0, (b) is in stage 5, (c) and (d) are the muscle strength comparison diagram of s1, (c) is in stage 0, and (d) is in stage 5. In the muscle strength comparison diagrams, the red line represents the muscle strength curve of the healthy hand, and the blue line represents the muscle strength curve of the affected hand.

Fig. 15

Muscle strength comparison diagram of two participants (s3—soft rehabilitation glove group, s1—BCI–VR group), before and after rehabilitation training, the detecting part is abductor pollicis brevis muscle, the action is clenching hand. In figure, the red line is the muscle strength of health hand, and the blue line is the muscle strength of the affected hand: a Muscle strength comparison of participant s3 before rehabilitation training; b Muscle strength comparison of participant s3 after rehabilitation training; c Muscle strength comparison of participant s1 before rehabilitation training; d Muscle strength comparison of participant s1 after rehabilitation training

Figure 15 shows that, no matter for participant s3 or participant s1, for the same participant, the muscle strength of the affected hand was much weaker than that of the healthy hand at stage 0, as shown in (a) and (c). After 16 weeks of rehabilitation training, the muscle strength of the affected hand was obviously improved, even though there was still a certain gap between the muscle strength of the affected hand and the healthy hand. In terms of muscle strength improvement degree of affected hand, participant s1 had a slight advantage over s3, indicating that BCI–VR rehabilitation training had an advantage over traditional passive rehabilitation training in muscle strength improvement, but the advantage was not obvious.

3.2.2.3 Statistical Analysis

Table 4 shows the statistical analysis of sEMG evaluation data of patients in the two groups (BCI–VR group and soft rehabilitation glove group) before and after rehabilitation training. Two features were tested: muscle strength and muscle tension. For each feature, four muscle parts were selected for detection: flexor carpi muscle, extensor carpi muscle, extensor digitorum muscle, and abductor pollicis brevis muscle. Before rehabilitation training, there was no significant difference in muscle tension and muscle strength of the four muscle parts between two groups (P > 0.05). After rehabilitation training, the features between two groups were statistically significantly different (P < 0.05), which indicates the rehabilitation effects of the proposed BCI–VR rehabilitation system. For the same group, the differences between the two stages (stage 0, before rehabilitation training and stage 5, after rehabilitation training) were large, and the difference for the BCI–VR group (P < 0.001) is larger than that of soft rehabilitation glove group (maximum P is 0.044). This phenomenon shows that the rehabilitation effect of the BCI–VR rehabilitation training system proposed in this paper is more obvious and superior than the traditional passive rehabilitation glove system.

Table 4 Muscle tension and muscle strength comparison of the two groups at two stages

In the rehabilitation evaluation process, in addition to the sEMG-based rehabilitation evaluation system, FMA (Fugl–Meyer assessment) and MBI (modified Barthel index), which are commonly used in stroke patients’ rehabilitation evaluation, were also adopted in this paper. In FMA evaluation, only the patient's hand and wrist functions were scored for rehabilitation; in MBI, only the eating and dressing functions, which are closely related to hand functions, were scored. Statistical analysis of FMA and MBI scores of patients in the two groups before and after rehabilitation training is shown in Table 5.

Table 5 FMA and MBI score comparison of the two groups at two stages

In terms of the FMA score of patients, there was no significant difference in the hand motor function between the two groups before rehabilitation training (P = 0.809, P = 0.241), but after rehabilitation training, the FMA score of patients in the two groups was significantly improved, while the rehabilitation score of patients in the BCI–VR group was more significantly improved (P = 0.026, P = 0.001). Compared to the functional rehabilitation of wrist and finger, it could be seen that the motor function of fingers was more obviously improved (P = 0.001).

The MBI score was similar to the FMA score. Before the rehabilitation training, there was no significant difference in eating and dressing in the two groups, but after the 16-week rehabilitation training, the scores of eating and dressing were significantly improved (P < 0.05), and the improvement of dressing ability was better than that of eating (P = 0.001).

4 Discussion

In this paper, the PSD-kNN algorithm is used as the EEG signal analysis algorithm. The relationship between the value of K and the classification accuracy for a small data set in a brain–computer interface system is verified through a series of experiments, as depicted in Fig. 13. Figure 13 shows that for small data sets and the choosing of k, bigger is better is not always true. If the value of k is too large, it may cause overfitting of the system, leading to the decline of classification accuracy. Therefore, when using kNN algorithm for feature classification, it is necessary to conduct system test experiments and select the k value with the best performance for a specific number of data sets.

In order to verify and analyze the activation of cerebral lesion regions and the improvement of brain plasticity of stroke patients by the proposed system, EEG signals were collected in the BCI–VR rehabilitation group at two states before and after rehabilitation. Figure 14 shows the EEG distribution of S7 in the BCI–VR rehabilitation group before and after rehabilitation training. As can be seen in Fig. 14, after 16 weeks of BCI–VR rehabilitation training, more areas of the contralateral cerebral cortex of the affected limb were activated in ‘clenching hand’ state, and the extent and range of activation were significantly expanded and improved upon compared with before rehabilitation training. This indicates that the BCI–VR rehabilitation system proposed by this paper can mobilize more cerebral cortex to participate in the process of hand movement rehabilitation while patients carry out hand movement rehabilitation, thus improving brain plasticity and improving hand movement dysfunction caused by lesions.

In the process of measuring the rehabilitation degree of patients' hand movement function, this paper adopts the sEMG-based rehabilitation evaluation system developed by our team. Aiming at the rehabilitation of patients' hand function, four muscles of patients were evaluated in terms of muscle strength and muscle tension. The four muscles selected for evaluation were the flexor carpi, extensor carpi, extensor digitorum, and abductor pollicis brevis. Statistical analysis was also performed for all patients in the two rehabilitation groups. Figure 15 shows the muscle strength comparison between the affected and healthy hands of patient S3 (soft rehabilitation glove group) and S1 (BCI–VR group). The action of hand is ‘clenching hand’ and the detection muscle is abductor brevis pollicis. As can be seen from Fig. 15, the 16-week rehabilitation training, whether BCI–VR rehabilitation training or soft rehabilitation glove rehabilitation training, significantly improved the muscle strength of the affected hand. Before rehabilitation training, compared with the healthy hand, the muscle strength of the affected hand was weak for both patients. After 16 weeks of rehabilitation training, both patients’ muscle strength of the affected hands increased significantly. Even though there was still a gap compared with the healthy hand, the ascension of muscle strength was obvious. This depicted that continuous rehabilitation training, either active or passive training, can obviously improve the muscle strength of the training limb. The phenomenon that there was no significant difference in the degree of improvement between the two groups indicates that there was little difference between passive rehabilitation and active rehabilitation in terms of short-term recovery of limb muscle strength. The evaluation of the sustained effect of the active and passive rehabilitation training requires long-term follow-up observation and motor function evaluation of patients. In this paper, due to the influence of patients discharged from hospital, follow-up evaluations of patients were not carried out; this is a future area to be improved upon by this research team.

The statistical analysis of muscle strength and muscle tension of patients in the two groups showed that regardless of BCI–VR rehabilitation group or soft rehabilitation glove group, the improvement of muscle strength and muscle tension in the four parts of patients was obvious. By contrasting the rehabilitation effect of the two groups, it can be seen that the rehabilitation effect of the BCI–VR rehabilitation group is better than that of the soft rehabilitation glove group. The reason is that the loss of motor function in stroke patients is caused by brain injury. In passive rehabilitation training, the direction of information transmission is from muscle to brain, which is opposite to the real information transmission direction of the human body. In this kind of rehabilitation training mode, the brain can receive stimulation from muscles’ training, but the recovery effect is limited.

While the BCI technology in the BCI–VR rehabilitation system makes the brain directly involved in the patients’ hand movement control, the stimulation to the lesions in active rehabilitation is more direct than that in the passive rehabilitation. At the same time, because of the addition of VR technology, the visual stimulation in the virtual environment and the hands’ synchronous movement feedback makes the rehabilitation more immersive; therefore, the BCI–VR rehabilitation training can achieve better rehabilitation effects than traditional passive rehabilitation training.

Comparison of the recovery degree of muscle strength and muscle tension between the two groups led to the finding that the recovery effect of muscle strength and muscle tension in the BCI–VR rehabilitation group was obvious, while the recovery effect of muscle tension in the soft rehabilitation glove group was better than that of muscle strength. Muscle tension refers to the tension of muscles in the relaxed state and the resistance encountered during passive exercise; muscle strength mainly reflects the strength of muscle contraction. For stroke patients, muscle tension represents the state of muscle spasm. Therefore, the above analysis indicates that the BCI–VR rehabilitation system has a significant effect on improving the muscle spasm and muscle contraction ability of stroke patients, while passive rehabilitation, such as soft rehabilitation gloves, has a better effect on improving the spasm than improving the active muscle contraction ability of patients.

Table 5 is the statistical analysis of FMA and MBI scores of all participants. This paper only focuses on the improvement of patients' hand functions, so FMA scores only focus on the hand and wrist functions related to hand functions, while MBI scores only focus on eating and dressing. As can be seen from Table 5, both the BCI–VR group and the soft rehabilitation glove group showed significant improvement in FMA and MBI scores before and after rehabilitation training, and the improvement degree of FMA was better than that of MBI. The reason is that FMA only scores patients' wrist and finger motor ability, and MBI requires patients to complete eating and dressing, which are closely related to hands. Both eating and dressing are not only related to hand function, but are also closely related to patients' somatosensory, positional, and spatial senses, which require close cooperation between patients' hands, brain, and eyes, thus eating and dressing are more difficult than simple hand actions. Therefore, the improvement of these two items needs longer rehabilitation training and targeted training to achieve better results.

In this paper, the BCI–VR-based hand soft rehabilitation system combines BCI technology with the hand soft rehabilitation technology and constructs a brain–nerve –muscle information transmission pathway in vitro for stroke patients. At the same time, the addition of VR technology gives patients the real-time feedback with visual and motion, and feedback to the patients’ neural pathways’ results to the brain. Therefore, the closed-loop path of brain information transmission and control is constructed. This rehabilitation system transforms the traditional passive physical rehabilitation training into active rehabilitation training, which not only helps users to complete the rehabilitation training of the hand function, but also enables patients' brains to actively participate in and control hand movements, improve patients' brain plasticity, and patients' motor function.

The feasibility and effectiveness of the proposed system were verified by a series of rehabilitation experiments for stroke patients in the Rehabilitation Department of Shandong University of Traditional Chinese Medicine. However, the rehabilitation training of stroke patients is a complex process, and the hand muscles are also extremely rich. Therefore, there are still many problems to be solved for the rehabilitation of patients' hand motor function. Thoughts, upon completion of this paper, are as follows:

In this paper, the PSD-kNN algorithm is used as the EEG signal analysis algorithm. The PSD-kNN algorithm transforms the EEG signals from the traditional Euclidian space into the Riemannian space and analyzes the EEG signals from the angle of Riemannian geometry so as to achieve a satisfactory classification effect.

Although the algorithm proposed in this paper completes the conversion of EEG signals from European space to Riemannian space and the analysis of the geometric distribution of EEG signals in Riemannian space. However, the proposed algorithm defines the sum of the distances between the curves described by two PSD matrices as the definition of dissimilarity between them. Since the distances are scalar, the algorithm does not consider the geometric distribution and shape of the PSD matrix in the Riemannian space in the calculation process. Although this paper has experimentally verified the feasibility and effectiveness of this definition of dissimilarity, can the dissimilarity of two PSD matrices in Riemannian space be constructed by fully considering the spatial geometric distribution characteristics of the PSD matrices?

The BCI–VR-based soft hand rehabilitation system proposed in this paper has a better activation degree and range of cerebral cortex in stroke patients than traditional passive hand rehabilitation. The reason is that the BCI–VR-based rehabilitation system not only constitutes a closed control loop of the brain–nerve–muscle, but also involves the patients more thoroughly immersed in rehabilitation training, which gives the patient a greater effect of neural remodeling. However, it is worth noting that neurological remodeling after injury can be positive or negative. In order to restore central nervous system function, intervention measures that can cause remodeling are required to be targeted and have no negative effects [2]. If BCI produces negative effects, abnormal synapses and negative remodeling can be formed through repeated wrong signal adjustment, which may cause patients to have unintentional actions, even compulsion, hallucination, and sensory distortion, etc. This will pose a severe ethical problem for the future application of BCI [10]. Therefore, when using BCI technology for rehabilitation of limb motor function in stroke patients, it is necessary to investigate the positive and negative aspects of neural remodeling brought by this system to patients, which is also a problem that needs to be considered emphatically in the future using BCI technology for rehabilitation of motor function. That is, how to determine the positive and negative of patients' neural remodeling? How to use the BCI technique to produce positive neural remodeling?

The rehabilitation system proposed in this paper adopts the design of soft rehabilitation glove, avoiding the hidden danger caused by the traditional mechanical rehabilitation system of the hand. For rehabilitation action design, the rehabilitation gloves can help patients complete most of the major hand movements, such as grasping, clenching, and finger bending, which can help patients to complete daily life tasks, such as dressing, eating, and so on. However, the hand muscles are rich, and the muscles related to finger motor ability also include the interosseous metacarpal muscle, interosseous dorsal muscle, pollicis radalis, adductor pollicis, and vermiform muscle. These muscles belong to small muscle groups, and the rehabilitation of small muscle groups can enable patients to achieve the execution of fine and complicated movements. Due to the integrated design of the rehabilitation gloves in this system, it is impossible to carry out targeted training and rehabilitation for the muscles in these small muscle groups. How to construct a more flexible and meticulous hand function rehabilitation system that can carry out rehabilitation training for small muscle groups of patients' hands and assist patients to complete fine movements is a problem that needs to be considered in the direction of subsequent hand function rehabilitation.

Due to the limitations of objective conditions, the duration of the experiment in this paper was 5 months for stroke patients, without long-term follow-up observation and regular rehabilitation evaluation after discharge. Half a year or even more than a year of follow-up observation and regular rehabilitation evaluation for the effectiveness and sustainability of rehabilitation methods will provide better guiding significance.

5 Conclusions

This study constructed a soft rehabilitation system based on BCI–VR technology for stroke patients’ hand rehabilitation training. The proposed system combines BCI technology with rehabilitation soft gloves to realize an in vitro channel of brain–nerve–muscle information transmission, transforming the traditional passive rehabilitation into active rehabilitation in which the brain actively participates. At the same time, with the addition of VR technology, patients will have a higher degree of brain participation and investment in rehabilitation training, so as to achieve better rehabilitation effects. In order to verify the role and effect of the proposed system in hand function rehabilitation of stroke patients, comparative rehabilitation experiments were conducted on 11 stroke patients in the Rehabilitation Department of The Affiliated Hospital of Shandong University of Traditional Chinese Medicine. The rehabilitation comparison experiments lasted for 16 weeks; 11 stroke patients were randomly divided into a BCI–VR active rehabilitation group and a soft rehabilitation glove passive rehabilitation group. The two groups had the same rehabilitation time and frequency, and the rehabilitation equipment and content were different. This study systematically analyzed the EEG topographic map, muscle strength, muscle tensor recovery of hand muscles, and FMA and MBI scores of the two groups of patients. The results of the comparative rehabilitation experiment showed that the system proposed in this paper could effectively improve the activation degree and activation range of cerebral lesions in stroke patients, enhance the muscle strength of patients' hands, improve the hand spasm, and effectively increase the FMA and MBI scores of patients. The proposed BCI–VR hand soft rehabilitation system could provide stroke patients with a scientific, effective, and highly targeted rehabilitation training method to effectively improve the rehabilitation training effect and enhance the functional exercise level.