Emotion Annotation Using Hierarchical Aligned Cluster Analysis

  • Wei-Ye Zhao
  • Sheng Fang
  • Ting Ji
  • Qian Ji
  • Wei-Long Zheng
  • Bao-Liang Lu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10637)


The correctness of annotation is quite important in supervised learning, especially in electroencephalography(EEG)-based emotion recognition. The conventional EEG annotations for emotion recognition are based on the feedback like questionnaires about emotion elicitation from subjects. However, these methods are subjective and divorced from experiment data, which lead to inaccurate annotations. In this paper, we pose the problem of annotation optimization as temporal clustering one. We mainly explore two types of clustering algorithms: aligned clustering analysis (ACA) and hierarchical aligned clustering analysis (HACA). We compare the performance of questionnaire-based, ACA-based, HACA-based annotation on a public EEG dataset called SEED. The experimental results demonstrate that our proposed ACA-based and HACA-based annotation achieve an accuracy improvement of \(2.59\%\) and \(4.53\%\) in average, respectively, which shows their effectiveness for emotion recognition.


Neural data analysis Time series analysis EEG annotations Emotion recognition 



This work was supported in part by grants from the National Key Research and Development Program of China (Grant No. 2017YFB1002501), the National Natural Science Foundation of China (Grant No. 61673266), the Major Basic Research Program of Shanghai Science and Technology Committee (Grant No. 15JC1400103), ZBYY-MOE Joint Funding (Grant No. 6141A02022604), and the Technology Research and Development Program of China Railway Corporation (Grant No. 2016Z003-B).


  1. 1.
    Brodley, C.E., Friedl, M.A.: Identifying mislabeled training data. J. Artif. Intell. Res. 11, 131–167 (1999)MATHGoogle Scholar
  2. 2.
    Frénay, B., Verleysen, M.: Classification in the presence of label noise: a survey. IEEE Trans. Neural Netw. Learn. Syst. 25(5), 845–869 (2014)CrossRefGoogle Scholar
  3. 3.
    Guan, D., Yuan, W., Lee, Y.K., Lee, S.: Nearest neighbor editing aided by unlabeled data. Inf. Sci. 179(13), 2273–2282 (2009)CrossRefGoogle Scholar
  4. 4.
    Guan, D., Yuan, W., Ma, T., Khattak, A.M., Chow, F.: Cost-sensitive elimination of mislabeled training data. Inf. Sci. 402, 170–181 (2017)CrossRefGoogle Scholar
  5. 5.
    Kim, K.H., Bang, S.W., Kim, S.R.: Emotion recognition system using short-term monitoring of physiological signals. Med. Biol. Eng. Comput. 42(3), 419–427 (2004)CrossRefGoogle Scholar
  6. 6.
    Lam, C.P., Stork, D.G.: Evaluating classifiers by means of test data with noisy labels. In: IJCAI, pp. 513–518 (2003)Google Scholar
  7. 7.
    Lu, Y., Zheng, W.L., Li, B., Lu, B.L.: Combining eye movements and EEG to enhance emotion recognition. In: IJCAI, pp. 1170–1176 (2015)Google Scholar
  8. 8.
    Sáez, J.A., Galar, M., Luengo, J., Herrera, F.: A first study on decomposition strategies with data with class noise using decision trees. In: Corchado, E., Snášel, V., Abraham, A., Woźniak, M., Graña, M., Cho, S.-B. (eds.) HAIS 2012. LNCS, vol. 7209, pp. 25–35. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-28931-6_3 CrossRefGoogle Scholar
  9. 9.
    Schuller, B., Rigoll, G., Lang, M.: Hidden Markov model-based speech emotion recognition. In: Proceedings of the 2003 International Conference on Multimedia and Expo, vol. 1, p. I–401. IEEE (2003)Google Scholar
  10. 10.
    Shi, L.C., Jiao, Y.Y., Lu, B.L.: Differential entropy feature for EEG-based vigilance estimation. In: 35th Annual International Conference of the IEEE, Engineering in Medicine and Biology Society, pp. 6627–6630. IEEE (2013)Google Scholar
  11. 11.
    Shimodaira, H., Noma, K.I., Nakai, M., Sagayama, S., et al.: Dynamic time-alignment kernel in support vector machine. In: NIPS, vol. 2, pp. 921–928 (2001)Google Scholar
  12. 12.
    Wang, X.W., Nie, D., Lu, B.L.: Emotional state classification from EEG data using machine learning approach. Neurocomputing 129, 94–106 (2014)CrossRefGoogle Scholar
  13. 13.
    Zheng, W.L., Lu, B.L.: Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans. Auton. Ment. Dev. 7(3), 162–175 (2015)CrossRefGoogle Scholar
  14. 14.
    Zheng, W.L., Zhu, J.Y., Lu, B.L.: Identifying stable patterns over time for emotion recognition from EEG. IEEE Trans. Affect. Comput. (2017). doi: 10.1109/TAFFC.2017.2712143
  15. 15.
    Zheng, W.L., Zhu, J.Y., Peng, Y., Lu, B.L.: EEG-based emotion classification using deep belief networks. In: IEEE International Conference on Multimedia and Expo, pp. 1–6. IEEE (2014)Google Scholar
  16. 16.
    Zhou, F., De la Torre, F., Hodgins, J.K.: Aligned cluster analysis for temporal segmentation of human motion. In: 8th IEEE International Conference on Automatic Face & Gesture Recognition, pp. 1–7. IEEE (2008)Google Scholar
  17. 17.
    Zhou, F., De la Torre, F., Hodgins, J.K.: Hierarchical aligned cluster analysis for temporal clustering of human motion. IEEE Trans. Pattern Anal. Mach. Intell. 35(3), 582–596 (2013)CrossRefGoogle Scholar
  18. 18.
    Zhu, X., Wu, X.: Class noise vs. attribute noise: a quantitative study. Artif. Intell. Rev. 22(3), 177–210 (2004)CrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Wei-Ye Zhao
    • 1
  • Sheng Fang
    • 1
  • Ting Ji
    • 1
  • Qian Ji
    • 1
  • Wei-Long Zheng
    • 1
  • Bao-Liang Lu
    • 1
    • 2
    • 3
  1. 1.Department of Computer Science and EngineeringCenter for Brain-like Computing and Machine IntelligenceShanghaiChina
  2. 2.Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive EngineeringShanghaiChina
  3. 3.Brain Science and Technology Research CenterShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations