Emotion Recognition Based on Electroencephalogram Using a Multiple Instance Learning Framework

  • Xiaowei Zhang
  • Yue Wang
  • Shengjie Zhao
  • Jinyong Liu
  • Jing Pan
  • Jian Shen
  • Tingzhen Ding
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10955)


Electroencephalogram (EEG)-based emotion recognition has been widely researched in the field of affective computing. Nevertheless, EEG signals which reflect brain activity are always unstable, it is inappropriate for traditional analysis methods to treat each sliding time window of signals as independent sample during classification. In this study, we employ a multi-instance learning (MIL) framework for EEG-based emotion recognition and regard sliding time windows from the same EEG signal as a whole by learning two MIL models based on Citation-kNN and mi-SVM algorithms. Experiment results show that our methods can achieve higher classification accuracy of 74.21% and 77.50% on two affective dimensions (valence and arousal) respectively when comparing with traditional single-instance classification algorithms. We believe that MIL framework can improve the generalization performance of EEG-based emotion recognition further, and provide new inspiration for affective computing.


Affective computing Emotion recognition Electroencephalogram Multi-instance Learning 



This work was supported by the National Basic Research Program of China (973 Program) (No.2014CB744600), the state key development program of China (No.2017YFE0111900), the National Natural Science Foundation of China (grant No.61402211, No.61210010) and the Fundamental Research Funds for the Central Universities (lzujbky-2017-196, lzujbky-2017-kb08). The authors acknowledge European Community’s Seventh Framework Program (FP7/2007-2011) for their DEAP database.


  1. 1.
    Picard, R.W.: Affective Computing, vol. 1, 1st edn, pp. 71–73. IGI Global, Hershey (1997)Google Scholar
  2. 2.
    Calvo, R.A., D’Mello, S.: Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans. Affect. Comput. 1(1), 18–37 (2010)CrossRefGoogle Scholar
  3. 3.
    Tkalčič, M., Burnik, U., Košir, A.: Using affective parameters in a content-based recommender system for images. User Model. User-Adapt. Interact. 20(4), 279–311 (2010)CrossRefGoogle Scholar
  4. 4.
    Anderson, K., Mcowan, P.W.: A real-time automated system for the recognition of human facial expressions. IEEE Trans. Syst. Man Cybern. Part B Cybern. Publ. IEEE Syst. Man Cybern. Soc. 36(1), 96–105 (2006)CrossRefGoogle Scholar
  5. 5.
    van der Wal, C.N., Kowalczyk, W.: Detecting changing emotions in human speech by machine and humans. Appl. Intell. 39(4), 675–691 (2013)CrossRefGoogle Scholar
  6. 6.
    Wagner, J., Kim, N.J., Andre, E.: From physiological signals to emotions: implementing and comparing selected methods for feature extraction and classification. In: IEEE International Conference on Multimedia and Expo, pp. 940–943. IEEE (2005)Google Scholar
  7. 7.
    Mao, C., et al.: EEG-based biometric identification using local probability centers. In: International Joint Conference on Neural Networks, pp. 1–8. IEEE (2015)Google Scholar
  8. 8.
    Chen, J., et al.: Feature-level fusion of multimodal physiological signals for emotion recognition. In: IEEE International Conference on Bioinformatics and Biomedicine, pp. 395–399. IEEE (2015)Google Scholar
  9. 9.
    Paus, T., Sipila, P.K., Strafella, A.P.: Synchronization of neuronal activity in the human primary motor cortex by transcranial magnetic stimulation: an EEG study. J. Neurophysiol. 86(4), 1983–1990 (2001)CrossRefGoogle Scholar
  10. 10.
    Chanel, G., et al.: Short-term emotion assessment in a recall paradigm. Int. J. Hum Comput Stud. 67(8), 607–627 (2009)CrossRefGoogle Scholar
  11. 11.
    Kaplan, A.Y., et al.: Nonstationary nature of the brain activity as revealed by EEG/MEG: methodological, practical and conceptual challenges. Signal Process. 85(11), 2190–2212 (2005)CrossRefGoogle Scholar
  12. 12.
    Sanei, S., Chambers, J.A.: EEG signal processing. In: The Fernow Watershed Acidification Study, pp. 207–236. Springer, Netherlands (2013)Google Scholar
  13. 13.
    Sadatnejad, K., et al.: EEG Representation Using Multi-instance Framework on The Manifold of Symmetric Positive Definite Matrices for EEG-based Computer Aided Diagnosis (2017)Google Scholar
  14. 14.
    Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89(1–2), 31–71 (1997)CrossRefGoogle Scholar
  15. 15.
    Kandemir, M., Hamprecht, F.A.: Computer-aided diagnosis from weak supervision: a benchmarking study. Comput. Med. Imaging Graph. 42, 44–50 (2015)CrossRefGoogle Scholar
  16. 16.
    Huo, J., Gao, Y., Yang, W., Yin, H.: Abnormal event detection via multi-instance dictionary learning. In: Yin, H., Costa, J.A.F., Barreto, G. (eds.) IDEAL 2012. LNCS, vol. 7435, pp. 76–83. Springer, Heidelberg (2012). Scholar
  17. 17.
    Fang, Y., Chang, L.: Multi-instance feature learning based on sparse representation for facial expression recognition. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015. LNCS, vol. 8935, pp. 224–233. Springer, Cham (2015). Scholar
  18. 18.
    Lee, C.-C., et al.: Affective state recognition in married couples’ interactions using PCA-based vocal entrainment measures with multiple instance learning. In: D’Mello, S., Graesser, A., Schuller, B., Martin, J.-C. (eds.) ACII 2011. LNCS, vol. 6975, pp. 31–41. Springer, Heidelberg (2011). Scholar
  19. 19.
    Wu, B., et al.: Music emotion recognition by multi-label multi-layer multi-instance multi-view learning. In: ACM International Conference on Multimedia, pp. 117–126. ACM (2014)Google Scholar
  20. 20.
    Jafari, A., et al.: An EEG artifact identification embedded system using ICA and multi-instance learning. In: IEEE International Symposium on Circuits and Systems, pp. 1–4. IEEE (2017)Google Scholar
  21. 21.
    Maron, O., Lozano-Pérez, T.: A framework for multiple-instance learning. In: Advances in Neural Information Processing Systems, vol. 200, no. 2, pp. 570–576 (1998)Google Scholar
  22. 22.
    Weidmann, N., Frank, E., Pfahringer, B.: A two-level learning method for generalized multi-instance problems. In: Lavrač, N., Gamberger, D., Blockeel, H., Todorovski, L. (eds.) ECML 2003. LNCS (LNAI), vol. 2837, pp. 468–479. Springer, Heidelberg (2003). Scholar
  23. 23.
    Wang, J., Zucker, J.D.: Solving the multiple-instance problem: a lazy learning approach. In: Seventeenth International Conference on Machine Learning, pp. 1119–1126. Morgan Kaufmann Publishers Inc. (2000)Google Scholar
  24. 24.
    Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: Advances in Neural Information Processing Systems, vol. 15, no. 2, pp. 561–568 (2002)Google Scholar
  25. 25.
    Koelstra, S., et al.: DEAP: a database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2012)CrossRefGoogle Scholar
  26. 26.
    Bos, D.O.: EEG-based Emotion Recognition (2008)Google Scholar
  27. 27.
    Lang, P.J.: The emotion probe. Studies of motivation and attention. Am. Psychol. 50(5), 372 (1995)CrossRefGoogle Scholar
  28. 28.
    Zhao, Q., et al.: Automatic identification and removal of ocular artifacts in EEG–improved adaptive predictor filtering for portable applications. IEEE Trans. Nanobiosci. 13(2), 109–117 (2014)CrossRefGoogle Scholar
  29. 29.
    Seitsonen, E.R., et al.: EEG spectral entropy, heart rate, photoplethysmography and motor responses to skin incision during sevoflurane anaesthesia. Acta Anaesthesiol. Scand. 49(3), 284–292 (2005)CrossRefGoogle Scholar
  30. 30.
    Inuso, G., et al.: Brain activity investigation by EEG processing: wavelet analysis, kurtosis and Renyi’s entropy for artifact detection. In: International Conference on Information Acquisition, pp. 195–200. IEEE (2007)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Xiaowei Zhang
    • 1
  • Yue Wang
    • 1
  • Shengjie Zhao
    • 1
  • Jinyong Liu
    • 1
  • Jing Pan
    • 1
  • Jian Shen
    • 1
  • Tingzhen Ding
    • 1
  1. 1.School of Information Science and EngineeringLanZhou UniversityLanzhouChina

Personalised recommendations