Keywords

1 Introduction

Brain is the senior commander of human body, which controls all kinds of information communication between human body and external environment through peripheral nerve and muscle channels. However, with the emergence of global aging problem, a variety of brain diseases are also increasing, such as stroke, epilepsy, depression and so on, which seriously endanger the life safety of patients; In addition, the rapid development of science and technology has greatly changed people's way of travel. While people get convenient transportation, there are also many traffic accidents, such as brain and nervous system damage of drivers, amputation and other problems caused by traffic accidents, which lead to the loss of the ability of human body to control its own muscles [3]. Although these diseases or accidents cut off the channel of information communication between the human brain and the external environment, the brain of the victims can produce consciousness or thinking. Therefore, researchers at home and abroad are trying to help the victims recover and improve their quality of life by using external auxiliary equipment.

In recent years, with the continuous development of computer technology, more and more scientists are committed to the field of brain science. They study the interactive method of combining computer and human brain, and reflect the real intention of patients by recording their EEG signals, so as to carry out rehabilitation treatment, which effectively promotes the brain computer interface, BCI) [5] technology development. Brain computer interface technology refers to a control system that does not rely on human muscle tissue and neural pathways to create channels between the human brain and external devices, so as to realize the communication between the brain and the external environment. As shown in Fig. 1, BCI technology is used to build an external pathway between the brain and the legs, so as to realize the control of the brain over the legs. Brain computer interface technology is not only widely used in biomedicine and neural rehabilitation, but also has significant advantages in education, military, entertainment and so on. BCI was first formed in the 1970s and grew rapidly in the late 1990s. Until now, researchers at home and abroad have never stopped exploring BCI. In recent years, with the in-depth development of artificial intelligence technology, it has opened up a new way for the research of BCI technology. For example, Li [9] proposed the algorithm of using multi-core learning mode to optimize support vector machine, which can quickly classify and recognize EEG with cognitive ability; Hajinoroozi et al. [10] used the method of convolutional neural networks (CNN) to study the EEG of drivers, so as to predict and regress their cognitive ability; Qiao [11] et al. Established a spatiotemporal convolution model to classify and recognize motor imagery EEG signals.

Fig. 1.
figure 1

BCI channel

Full size image

Motor imaging (MI) refers to the rehearsal of a behavior that is about to be triggered by the brain after receiving external stimulation [12]. At this time, the brain only has the intention to imagine the action, but not the real behavior. When the brain imagines a specific behavior, the related motor areas become active due to stimulation, which enhances the discharge process of neurons and leads to the change of their potential, resulting in event-related changes, and ultimately achieve the purpose of motor control. By collecting the motor imagery EEG signal at the time of brain discharge, and analyzing and processing the signal, different classification algorithms are used to identify the data to obtain the motor imagery intention. Finally, the external device completes the execution of related actions by judging the imported signal [13], and successfully analyzes people's action intention. Motor imagery is widely used in BCI system, sports training, rehabilitation training of lower limb patients and other fields [14]. It is an important tool to study the brain activation, neural network function and psychological process of human body under external stimulation. It is of great significance to the research of medicine and biological brain science.

Based on the theory of EEG, this paper designs EEG experiments of lower limb motor imagery to collect EEG data from 20 subjects. Aiming at the problems of nonstationarity, difficulty in feature extraction and low classification accuracy of motor imagery EEG signal, a multi domain fusion method of feature extraction of EEG signal from time domain, frequency domain, time-frequency domain and spatial domain is proposed, At the same time, the ensemble learning algorithm is used to classify and recognize the fused features, and two kinds of EEG signal classifiers, bagging and gradient boosting, are constructed for experiments. The final classification accuracy reaches 87.8% and 93%, which is better than the traditional SVM EEG signal classification method.

2 Experiment

In this paper, through the construction of the experimental platform of motor imagination, we use the real person leg raising video to stimulate the subjects’ motor imagination, which can efficiently and accurately obtain the EEG characteristics of the subjects, and the EEG signal extraction of the subjects uses the safe and convenient non-invasive method, During the experiment, the subjects need to wear a 64 lead quick cap EEG acquisition cap that meets the international 10–20 electrode positioning standard. The EEG signal collected is transmitted to the signal processor through Weaver EEG paste, and then the EEG signal is amplified by a certain proportion through the brain amp amplifier. The experimental paradigm is designed by using E-Prime software to realize synchronous communication.

In this study, a total of 20 college students, male and female, aged 18–26 years old and healthy, without other diseases, were invited. The design of this experiment is based on the motor imagination experiment of resting state and task state under visual stimulation. The human leg raising video is used to induce and stimulate the subjects, and the five electrode channels (FC1, FC2, C1, C2, CZ) of the subjects are explored, as shown in Fig. 2. Before the experiment, each subject is required to carry out a week of motor imagination training to improve the motor imagination ability. At the same time, the whole experimental process and precautions are introduced to the subjects in detail to ensure that the subjects have a clear understanding of the experimental content. In order to ensure that the subjects have a good mental state, they are required to fall asleep before 22 o'clock one day before the experiment; One hour before the experiment, the hair was washed and dried with a hair dryer to ensure a smaller impedance; During the experiment, the subjects are required to blink as little as possible, reduce the number of eye movements and swallowing saliva and other behaviors that affect the effect of the experiment.

Fig. 2.
figure 2

Video capture of human body in resting state and task state

Full size image

During the experiment, each person collected 5 groups of experiments, 40 times in each group, 20 times in the resting state and 20 times in the task state, 10 s each time. Before the beginning of each experiment, the screen will display the experiment instructions. After the subjects are ready, they press the keyboard “Q” key to start the experiment. A red “+” will appear in the center of the screen in 0–1 s to remind the subjects to prepare for the experiment; 1–3 s, the screen does not show any content, so that the subjects can relax physically and mentally; In 3–7 s, sit in or leg up videos were randomly displayed on the screen. When the leg up videos appeared, the subjects imagined the movement. When the sit in videos appeared, the subjects only needed to keep their mind blank and did not do any imaginary actions; The rest time is 7–10 s, and the subjects will not be disturbed by the EMG signal generated by fatigue. The experimental process is shown in Fig. 3.

Fig. 3.
figure 3

Flow chart of single experiment

Full size image

3 Methods

3.1 Data Preprocessing

The original EEG signal collected through the experiment contains a lot of interference noise, such as eye movement, head movement, ECG and 50 Hz power frequency interference. Therefore, before feature extraction of EEG signal, it is often necessary to carry out data preprocessing to effectively filter the noise, as shown in Fig. 4.

Fig. 4.
figure 4

Original EEG map

Full size image

The data preprocessing of EEG signal mainly includes: electrode location, removal of useless electrode, re reference, filtering, segmentation, replacement of bad segment, blind source separation and removal of artifacts, among which filtering and blind source separation are particularly important. Because most of the EEG signals of motor imagery of lower limbs are of the same waveform α Wave and β Therefore, the 0.1–40 Hz EEG signal is selected as the band of interest, and the band-pass (low-pass, high pass and sag filter) filter is used for filtering. After filtering, the EEG signal is analyzed by independent component analysis, and different EEG components are separated. The artifact identification and elimination operation are carried out on the separated EEG signal by using the adjust artifact elimination method. As shown in Fig. 5, the EEG signal after preprocessing is shown, and the noise component is significantly reduced, and the signal-to-noise ratio is also greatly improved.

Fig. 5.
figure 5

EEG signal after pretreatment

Full size image

3.2 Feature Extraction

After preprocessing the collected EEG signals, some electrodes need to be selected to extract their features. Feature extraction is to represent the imagination intention of the brain by using as few feature vectors as possible. It is the basis and basis of classification and recognition in the later stage, and is a necessary part of EEG signal processing. This paper explores the ERD/ERS phenomenon in the brain of the subjects during the experiment, determines the most obvious frequency band and time period of the right leg motor imagination, and represents the two information in the time domain, frequency domain, time-frequency domain and spatial domain respectively. Finally, it is fused into the form of multi domain feature vector, which effectively overcomes the limitations of single feature.

Because of the complexity and non stationarity of EEG signal, the time-domain feature is often abandoned by researchers. It is the characterization of the amplitude of EEG signal at different times, mainly including the maximum, minimum and average of the amplitude of EEG signal. These three common time-domain feature information include all the time information data of EEG signal, which has a strong intuitive feature selection of EEG signal. Frequency domain feature is the change of EEG signal amplitude with frequency. It can identify the correlation of different EEG signals by depicting the spectral feature information of EEG signals in different frequency bands. Power spectral density (PSD) is a common method to study the frequency domain characteristics of EEG signal, which takes frequency as an independent variable to reflect the power value of a specific frequency component. In this paper, the kurtosis, skewness, standard deviation and average power of EEG signal are selected as frequency domain characteristic information by increasing the characteristic number of power spectral density. The feature of time-frequency domain is the dimension reflecting the change of EEG signal frequency with time. By using the method of short-time Fourier transform and introducing the time window function, the non-stationary EEG signal can be effectively extracted, but the time window function cannot meet the local change of time and frequency. Therefore, the processed EEG signal is decomposed and reconstructed by using the method of discrete wavelet transform, Simple and stable time-frequency characteristic information can be obtained. Spatial domain feature extraction is mainly to construct spatial filter for task state and resting state data, and to maximize the covariance difference between the two types of data by using matrix diagonalization and variance scaling method, so as to show the feature vector with high discrimination, as shown in Fig. 6, which is the spatial domain feature map of electrode channel C1 and CZ. The multi domain fusion matrix is obtained by fusing the feature information of time domain, frequency domain, time-frequency domain and spatial domain of the above EEG signal features, which solves the problem of difficult feature extraction caused by the high non-stationary of EEG signal, and brings convenience for the subsequent classification and recognition.

Fig. 6.
figure 6

C1 and CZ airspace characteristic map

Full size image

3.3 Classification and Identification

Different classification algorithms are used to classify and identify the extracted feature information, which can help patients to control the external equipment. Compared with the traditional SVM method, this paper proposes an integrated learning algorithm of bagging and gradient boosting to analyze EEG information, and verifies the advantages and disadvantages of the classification method by comparing its classification accuracy.

Bagging algorithm is one of the integrated learning algorithms, which is characterized by independent sub learners, and its dependence is not strong, and can be generated synchronously [15]. It selects the classification tree in decision tree as weak classifier. After integrating m weak classifiers, bagging can reduce the variance of training set and increase deviation, so that bagging will not show the fitting phenomenon on the training set. Therefore, when using bagging algorithm to classify EEG signals after feature extraction, it can randomly sample and obtain the subset and generate the base classifier after training, The accuracy of EEG signal classification is greatly improved, up to 87.8%. As shown in Fig. 7, the accuracy of multi domain fusion feature classification is shown when using bagging algorithm to iterate for 50 times.

Fig. 7.
figure 7

Bagging classifier

Full size image

Boosting algorithm is an ensemble learning algorithm that combines multiple weak classifiers into strong classifiers according to the weight. Its principle is to randomly extract samples, add the same initial weight to each sample, observe the performance of weak classifiers after each training round, and increase the proportion of wrong samples, so that such samples can get more attention in the next round, Until m weak classifiers are trained and combined into strong classifiers according to weight, the accuracy of weak classification algorithm can be effectively improved [16]. The gradient boosting algorithm is the optimization of boosting algorithm. It constructs a weak classifier which can reduce the classification error rate along the steepest direction of the gradient by gradient lifting [17]. It can solve the problem of second classification of EEG signal and effectively improve the anti noise ability of the model, with the highest accuracy of 93%, Fig. 8 shows the classification accuracy of multi domain fusion features when the gradient boosting algorithm is used for 50 iterations.

Fig. 8.
figure 8

Gradient boosting classifier

Full size image

4 Conclusion

In this paper, the EEG data of 20 subjects are collected and explored by building a lower limb motor imagery EEG experimental platform. The multi domain (time domain, frequency domain, time-frequency domain and spatial domain) feature fusion method is used to effectively extract the feature information of complex and high-dimensional EEG signals. At the same time, the ensemble learning algorithm bagging and gradient boosting are used as classifiers, The classification accuracy of EEG signal is greatly improved, but the EEG signal data collected in this experiment is still small data samples, and the experimental objects are normal people. The generalization ability of the classifier model to the EEG signal data of real patients is poor. In the later stage, the EEG signal data of real patients will be collected and the sample size will be expanded to improve the universality of the classifier model.