T3SFNet: A Tuned Topological Temporal-Spatial Fusion Network for Motor Imagery with Rehabilitation Exoskeleton

Abstract

In recent years, motor imagery-based brain computer interfaces (MI-BCI) combined with exoskeleton robot has proved to be a promising method for spinal cord injury (SCI) rehabilitation training. The core of BCI is to achieve a high accurate movement prediction based on patient’s MI. The inconsistent response frequency of MI in different trials and subjects leads to the limited performance accuracy of MI movement prediction method for the single subject. The individual differences in the activation patterns of MI brain regions bring a greater challenge to the generalization ability of the method. According to the MI mechanism, this paper proposes a graph-based tuned topological temporal-spatial fusion network (T3SFNet) for MI electroencephalography (MI-EEG) limb movement prediction. The proposed method designs a learnable EEG tuning mechanism to fuse and enhance the subject’s MI response band data, and then uses a channel node-based graph convolutional network and a temporal-spatial fusion convolutional network to extract the topological features and spatiotemporal coupling features of the fused band data respectively. We evaluate the proposed approach on two MI datasets and show that our method outperforms state-of-the-art methods in both within-subject and cross-subject situations. Furthermore, our method shows surprising results on the small-sample migration test, reaching the prediction baseline with only $5\%$ of the data sample size. Ablation experiments of the model demonstrate the effectiveness and necessity of the proposed framework.