Mutual Information-Based Emotion Recognition
Emotions that arise in viewers in response to videos play an essential role in content-based indexing and retrieval. However, the emotional gap between low-level features and high-level semantic meanings is not well understood. This paper proposes a general scheme for video emotion identification using mutual information-based feature selection followed by regression. Continuous arousal and valence values are used to measure video affective content in dimensional arousal-valence space. Firstly, rich audio-visual features are extracted from video clips. The minimum redundancy and maximum relevance feature selection is then used to select most representative feature subsets for arousal and valence modelling. Finally support vector regression is employed to model arousal and valence estimation functions. As evaluated via tenfold cross-validation, the estimation results achieved by our scheme for arousal and valence are: mean absolute error, 0.1358 and 0.1479, variance of absolute error, 0.1074 and 0.1175, respectively. Encouraging results demonstrate the effectiveness of our proposed method.
Key wordsAffective content analysis Mutual information-based feature selection Support vector regression
This work was supported by a CSC-Newcastle scholarship.
- 2.Kang, H.B.: Affective content detection using HMMs. Proceedings of the 11th ACM International Conference on Multimedia (MM), Berkeley, California, USA, pp. 259–262 (2003)Google Scholar
- 3.Xu, M., Jin, J.S., Luo, S., Duan, L.: Hierarchical movie affective content analysis based on arousal and valence features. Proceeding of the 16th ACM International Conference on Multimedia (MM), Vancouver, Canada, pp. 677–680 (2008)Google Scholar
- 4.Xu, M., Chia, L.T., Jin, J.: Affective content analysis in comedy and horror videos by audio emotional event detection. In: IEEE International Conference on Multimedia and Expo (ICME), pp. 621–625. IEEE Press, New York (2005)Google Scholar
- 6.Zhang, S.L., Tian, Q., Huang, Q.M., Gao, W., Li, S.P.: Utilizing Affective Analysis for Efficient Movie Browsing. 16th IEEE International Conference on Image Processing, Vols 1-6, pp. 1833–1836 (2009)Google Scholar
- 8.Zhang, S., Xu, Y.J., Jia, J., Cai, L.H.: Analysis and Modeling of Affective Audio Visual Speech Based on PAD Emotion Space. 6th International Symposium on Chinese Spoken Language Processing, pp. 281–284 (2008)Google Scholar
- 17.Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:21–27:27 (2011)Google Scholar
- 18.Hsu, C.-W., Chang, C.-C., Lin, C.-J.: A Practical Guide to Support Vector Classification. Department of Computer Science and Information Engineering, National Taiwan University, Taipei (2003)Google Scholar
- 19.Cui, Y., Lin, J.S., Zhang, S.L., Luo, S.H., Tian, Q.: Correlation-Based Feature Selection and Regression. Advances in Multimedia Information Processing-PCM, Pt I 6297, pp. 25–35 (2010)Google Scholar
- 21.Hu, X., Deng, F., Li, K., Zhang, T., Chen, H., Jiang, X., Lv, J., Zhu, D., Faraco, C., Zhang, D., Mesbah, A., Han, J., Hua, X., Xie, L., Miller, S., Lei, G., Liu, T.: Bridging low-level features and high-level semantics via fMRI brain imaging for video classification. Proceedings of the International Conference on Multimedia (MM), Firenze, Italy, pp. 451–460 (2010)Google Scholar