Incorporating hand-crafted features into deep learning models for motor imagery EEG-based classification

Abstract

Motor imagery (MI) is a mental process that produces two types of event-related potentials called event-related desynchronization (ERD) and event-related synchronization (ERS). We can record ERD and ERS in an electroencephalogram (EEG) and use them to identify a MI execution. However, the classification of MI is a challenging task because ERD and ERS exhibit inter- and intra-subject variability. Recently, researchers have proposed deep learning models to solve this problem. Although they achieve cutting-edge results, the amount of data available for training constrains their learning ability. To address this issue, we propose to incorporate hand-crafted features, which have a strong inductive bias, into deep learning models at different levels of depth, which have a soft inductive bias, without making them lose their ability to discover new features from data. Our approach enables the design of models that benefit from deep learning and traditional machine learning models for MI EEG-based classification. In this manner, it is possible to build compact machine learning models that perform better than pure deep learning models in a small data setting. Results of experiments on the public datasets 2a and 2b of the BCI Competition IV demonstrate that a model built following our proposed strategy achieves state-of-the-art accuracy on MI EEG-based classification.

Appendix A common spatial patterns

The CSP mathematical procedure to find out spatial filters for a two-class problem is described below.

First, calculate the covariance matrices of training epochs of each class $k \in \{a, b\}$:

$$\begin{aligned} R_k = \frac{1}{n_k} \sum _{i=1}^{n_k} X_{k,i} X_{k,i}^T \end{aligned}$$

(A1)

where:

• $X_{k,i} \in \mathbb {R}^{c \times t}$ is the i-th epoch of class k, composed of t samples from c channels.

• $n_k$ is the number of epochs of class k.

• $R_k \in \mathbb {R}^{c \times c}$ is the covariance matrix of training epochs of class k.

Then, with the covariance matrices $R_a$ and $R_b$, solve the generalized eigenvalue problem in the matrix form:

$$\begin{aligned} R_a W = (R_a + R_b) W \Lambda \end{aligned}$$

(A2)

where:

• $W \in \mathbb {R}^{c \times c}$ (with $c =$ number of channels) contains the generalized eigenvectors $w_{j= \{1,...,c\}}$ of $R_a$ and $R_a + R_b$.

• $\Lambda \in \mathbb {R}^{c \times c}$ is a diagonal matrix which contains the generalized eigenvalues $\lambda _{j= \{1, ...,c\}}$, corresponding to the eigenvectors $w_j$, of $R_a$ and $R_a + R_b$.

Finally, each eigenvector $w_j$ is a spatial filter. Grosse-Wentrup and Buss [31] proposed an extension of the CSP mathematical procedure for multiclass problems.