Abstract
In recent years, the diffusion of social networks has made available large amounts of user-generated data containing people’s opinions and feelings. Such data are mostly unstructured and hence need to be enriched with a large set of metadata to allow for efficient data indexing and querying. In this work we focus on videos and we extend traditional metadata extraction techniques by taking into account emotional metadata, in order to enable data analysis from an affective perspective. To this purpose, we present a 3-phase methodology for the automatic extraction of emotional metadata from videos through facial expression recognition algorithms. We also propose a simple but versatile model for metadata that takes into account variations in emotions among video chunks. Experiments on a real-world video dataset show that our non-linear classifier reaches a remarkable 72% classification accuracy in facial expression recognition.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Benitez-Garcia, G., Nakamura, T., Kaneko, M.: Multicultural facial expression recognition based on differences of western-caucasian and east-asian facial expressions of emotions. IEEE Trans. Inf. Syst. 5, 1317–1324 (2018)
Cohen, I., Sebe, N., Garg, A., Chen, L.S., Huang, T.S.: Facial expression recognition from video sequences: temporal and static modeling. Comput. Vis. Image Underst. 91, 160–187 (2003)
Diamantini, C., Mircoli, A., Potena, D., Storti, E.: Semantic disambiguation in a social information discovery system. In: Proceedings of the 2015 International Conference on Collaboration Technologies and Systems (CTS), pp. 326–333 (2015)
Ekman, P.: An argument for basic emotions. Cogn. Emotion 6, 169–200 (1992)
Felbo, B., Mislove, A., Sgaard, A., Rahwan, I., Lehmann, S.: Using millions of Emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1616–1626 (2017)
Fourati, M., Jedidi, A., Gargouri, F.: Generic descriptions for movie document: an experimental study. In: Proceedings of IEEE/ACS International Conference on Computer Systems and Applications, AICCSA 2017, pp. 766–773, October 2018
Huang, J., Yuan, C.: Weighted-PCANet for face recognition. In: Arik, S., Huang, T., Lai, W.K., Liu, Q. (eds.) ICONIP 2015. LNCS, vol. 9492, pp. 246–254. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-26561-2_30
Mircoli, A., Cucchiarelli, A., Diamantini, C., Potena, D.: Automatic emotional text annotation using facial expression analysis. In: Proceedings of CEUR Workshop 1848, pp. 188–196 (2017)
Mo, S., Niu, J., Su, Y., Das, S.K.: A novel feature set for video emotion recognition. Neurocomputing 291, 11–20 (2018)
Poria, S., Peng, H., Hussain, A., Howard, N., Cambria, E.: Ensemble application of convolutional neural networks and multiple kernel learning for multimodal sentiment analysis. Neurocomputing 261, 217–230 (2017)
Pramerdofer, C., Kampel, M.: Facial expression recognition using convolutional neural networks: state of the art. arXiv preprint arXiv:1612.02903 (2016)
Sailunaz, K., Dhaliwal, M., Rokne, J., Alhajj, R.: Emotion detection from text and speech: a survey. Soc. Netw. Anal. Min. 8(1) (2018)
Sariyanidi, E., Gunes, H., Cavallaro, A.: Automatic analysis of facial affect: a survey of registration, representation and recognition. IEEE Trans. Patt. Anal. Mach. Intell. 37, 1113–1133 (2015)
Sikos, L.F., Powers, D.M.W.: Knowledge-driven video information retrieval with LOD: Lrom semi-structured to structured video metadata. In: Proceedings of the 2015 Workshop on Exploiting Semantic Annotations in Information Retrieval (ESAIR), pp. 35–37 (2015)
Soltani, M., Zarzour, H., Babahenini, M.C.: Facial emotion detection in massive open online courses. In: Rocha, Á., Adeli, H., Reis, L.P., Costanzo, S. (eds.) WorldCIST’18 2018. AISC, vol. 745, pp. 277–286. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-77703-0_28
Sun, W., Zhao, H., Jin, Z.: A complementary facial representation extracting method based on deep learning. Neurocomputing 306, 246–259 (2018)
Sun, Y., Sebe, N., Lew, M.S., Gevers, T.: Authentic emotion detection in real-time video. In: Sebe, N., Lew, M., Huang, T.S. (eds.) CVHCI 2004. LNCS, vol. 3058, pp. 94–104. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24837-8_10
Viola, P., Jones, M.: Robust real-time object detection. Int. J. Comput. Vis., 137–154 (2001)
Yu, Z., Zhang, C.: Image based static facial expression recognition with multiple deep network learning. In: Proceedings of the 2015 International Conference on Multimodal Interaction (ICMI), pp. 435–442 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Mircoli, A., Cimini, G. (2018). Automatic Extraction of Affective Metadata from Videos Through Emotion Recognition Algorithms. In: Benczúr, A., et al. New Trends in Databases and Information Systems. ADBIS 2018. Communications in Computer and Information Science, vol 909. Springer, Cham. https://doi.org/10.1007/978-3-030-00063-9_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-00063-9_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00062-2
Online ISBN: 978-3-030-00063-9
eBook Packages: Computer ScienceComputer Science (R0)