Active Feedback Framework with Scan-Path Clustering for Deep Affective Models
The attention of subjects to EEG-based emotion recognition experiments could seriously affect their emotion induction level and annotation quality of EEG data. Therefore, it is important to evaluate the raw EEG data before training the classification model. In this paper, we propose a framework to filter out low quality EEG data from participants with low attention using eye tracking data and boost the performance of deep affective models with CNN and LSTM. We introduce a novel attention-deprived experiment with dual tasks, in which the dominant task is auditory continuous performance test, identical pairs version (CPT-IP) and the subtask is emotion eliciting experiment. Motivated by the idea that subjects with attention share similar scan-path patterns under the same clips, we adopt the cosine distance based spatial-temporal scan-path analysis with eye tracking data to cluster these similar scan-paths. The average accuracy of emotion recognition using the selected EEG data with attention is about 3% higher than that of original training dataset without filtering. We also found that with the increasing distance of scan-paths between outliers and cluster center, the performance of corresponding EEG data tends to decrease.
KeywordsEye tracking Scan-path Attention evaluation EEG data filtering
This work was supported in part by the grants from the National Key Research and Development Program of China (Grant No. 2017YFB1002501), the National Natural Science Foundation of China (Grant No. 61673266), and the Fundamental Research Funds for the Central Universities.
- 3.Duan, R.N., Zhu, J.Y., Lu, B.L.: Differential entropy feature for EEG-based emotion classification. In: 6th International IEEE/EMBS Conference on Neural Engineering, pp. 81–84. IEEE (2013)Google Scholar
- 5.Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD, vol. 96, pp. 226–231 (1996)Google Scholar
- 6.Huang, A.: Similarity measures for text document clustering. In: Proceedings of the Sixth New Zealand Computer Science Research Student Conference, Christchurch, New Zealand, pp. 49–56 (2008)Google Scholar
- 8.Lu, Y., Zheng, W.L., Li, B., Lu, B.L.: Combining eye movements and EEG to enhance emotion recognition. In: IJCAI, vol. 15, pp. 1170–1176 (2015)Google Scholar
- 11.Qian, G., Sural, S., Gu, Y., Pramanik, S.: Similarity between Euclidean and cosine angle distance for nearest neighbor queries. In: Proceedings of the 2004 ACM Symposium on Applied Computing, pp. 1232–1237. ACM (2004)Google Scholar
- 13.Shi, Z.F., Zhou, C., Zheng, W.L., Lu, B.L.: Attention evaluation with eye tracking glasses for EEG-based emotion recognition. In: 8th International IEEE/EMBS Conference on Neural Engineering, pp. 86–89. IEEE (2017)Google Scholar
- 15.Yan, X., Zheng, W.L., Liu, W., Lu, B.L.: Investigating gender differences of brain areas in emotion recognition using LSTM neural network. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, E.S. (eds.) ICONIP 2017. LNCS, vol. 10637, pp. 820–829. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70093-9_87CrossRefGoogle Scholar