Abstract
Affective labeling of multimedia content can be useful in recommender systems. In this paper we compare the effect of implicit and explicit affective labeling in an image recommender system. The implicit affective labeling method is based on an emotion detection technique that takes as input the video sequences of the users’ facial expressions. It extracts Gabor low level features from the video frames and employs a kNN machine learning technique to generate affective labels in the valence-arousal-dominance space. We performed a comparative study of the performance of a content-based recommender (CBR) system for images that uses three types of metadata to model the users and the items: (i) generic metadata, (ii) explicitly acquired affective labels and (iii) implicitly acquired affective labels with the proposed methodology. The results showed that the CBR performs best when explicit labels are used. However, implicitly acquired labels yield a significantly better performance of the CBR than generic metadata while being an unobtrusive feedback tool.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Adomavicius, G., Sankaranarayanan, R., Sen, S., Tuzhilin, A.: Incorporating contextual information in recommender systems using a multidimensional approach. ACM Transactions on Information Systems (TOIS) 23(1), 103–145 (2005)
Adomavicius, G., Tuzhilin, A.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering 17(6), 734–749 (2005)
Arapakis, I., Moshfeghi, Y., Joho, H., Ren, R.: Integrating facial expressions into user profiling for the improvement of a multimodal recommender system. In: IEEE International Conference on Multimedia and Expo., ICME 2009, pp. 1440–1443 (2009)
Bartlett, M.S., Littlewort, G.C., Frank, M.G., Lainscsek, C., Fasel, I.R., Movellan, J.R.: Automatic Recognition of Facial Actions in Spontaneous Expressions. Journal of Multimedia 1(6), 22–35 (2006)
Chen, L., Chen, G., Xu, C., March, J., Benford, S.: EmoPlayer: A media player for video clips with affective annotations. Interacting with Computers 20(1), 17–28 (2008)
Eckhardt, M., Picard, R.: A more effective way to label affective expressions. In: 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, pp. 1–2 (September 2009)
Hanjalic, A.: Adaptive extraction of highlights from a sport video based on excitement modeling. IEEE Transactions on Multimedia 7(6), 1114–1122 (2005)
Herlocker, J.L., Konstan, J.A., Terveen, L., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. 22(1), 5–53 (2004)
Joho, H., Jose, J.M., Valenti, R., Sebe, N.: Exploiting facial expressions for affective video summarisation. In: Proceeding of the ACM International Conference on Image and Video Retrieval - CIVR 2009, page 1 (2009)
Kierkels, J.J.M., Pun, T.: Simultaneous exploitation of explicit and implicit tags in affect-based multimedia retrieval. IEEE (September 2009)
Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical Report A-8. Technical report, University of Florida (2005)
Lehman, E.L., Romano, J.P.: Testing Statistical Hypotheses (2005)
Odić, A., Kunaver, M., Tasič, J., Košir, A.: Open issues with contextual information in existing recommender system databases. In: ERK 2010 Proceedings (2010)
Pantic, M., Vinciarelli, A.: Implicit human-centered tagging Social Sciences. IEEE Signal Processing Magazine 26(6), 173–180 (2009)
Picard, R.W., Daily, S.B.: Evaluating affective interactions: Alternatives to asking what users feel. In: CHI Workshop on Evaluating Affective Interfaces: Innovative Approaches (2005)
Pogacnik, M., Tasic, J., Meza, M., Kosir, A.: Personal Content Recommender Based on a Hierarchical User Model for the Selection of TV Programmes. User Modeling and User-Adapted Interaction: The Journal of Personalization Research 15(5), 425–457 (2005)
Shan, M.-K., Kuo, F.-F., Chiang, M.-F., Lee, S.-Y.: Emotion-based music recommendation by affinity discovery from film music. Expert Syst. Appl. 36(4), 7666–7674 (2009)
Soleymani, M., Davis, J., Pun, T.: A collaborative personalized affective video retrieval system. In: 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, pp. 1–2 (September 2009)
Tkalčič, M., Burnik, U., Košir, A.: Using affective parameters in a content-based recommender system for images. User Modeling and User-Adapted Interaction: The Journal of Personalization Research 20(4), 279–311 (2005)
Tkalčič, M., Odić, A., Košir, A., Tasič, J.: Comparison of an Emotion Detection Technique on Posed and Spontaneous Datasets. In: Proceedings of the 19th ERK Conference, Portorož (2010)
Tkalčič, M., Tasič, J., Košir, A.: The LDOS-PerAff-1 Corpus of Face Video Clips with Affective and Personality Metadata. In: Proceedings of Multimodal Corpora: Advances in Capturing, Coding and Analyzing Multimodality (Malta, 2010), LREC, p. 111 (2009)
Valenti, R., Yucel, Z., Gevers, T.: Robustifying eye center localization by head pose cues. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 612–618 (June 2009)
Wang, Y., Guan, L.: Recognizing Human Emotional State From Audiovisual Signals. IEEE Transactions on Multimedia 10(4), 659–668 (2008)
Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions. IEEE Trans. Pattern Analysis & Machine Intelligence 31(1), 39–58 (2009)
Zhi, R., Ruan, Q.: Facial expression recognition based on two-dimensional discriminant locality preserving projections. Neurocomputing 71(7-9), 1730–1734 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tkalčič, M., Odić, A., Košir, A., Tasič, J.F. (2012). Impact of Implicit and Explicit Affective Labeling on a Recommender System’s Performance. In: Ardissono, L., Kuflik, T. (eds) Advances in User Modeling. UMAP 2011. Lecture Notes in Computer Science, vol 7138. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28509-7_32
Download citation
DOI: https://doi.org/10.1007/978-3-642-28509-7_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28508-0
Online ISBN: 978-3-642-28509-7
eBook Packages: Computer ScienceComputer Science (R0)