Impression-Aware Video Stream Retrieval System with Temporal Color-Sentiment Analysis and Visualization
To retrieve Web video intuitively, the concept of “impression” is of great importance, because many users consider feelings and moods to be one of the most significant factors motivating them to watch videos. In this paper, we propose an impression-aware video stream retrieval system for querying the visual impression of video streams by analyzing the temporal change in sentiments. As a metric of visual impression, we construct a 180-dimensional vector space called as color-impression space; each dimension corresponds to a specific adjective representing humans’ color perception. The main feature of this system is a context-dependent query processing mechanism to generate a ranking by considering the temporal transition of each video’s visual impressions on viewers’ emotion. We design an impression-aware noise reduction mechanism that dynamically reduces the number on non-zero features for each item mapped in the high-dimensional color-impression space by extracting the dominant salient impression features from a video stream. This system allows users to retrieve videos by submitting emotional queries such as “Find videos whose overall impression is happy and which have several sad and cool scenes”. Through this query processing mechanism, users can effectively retrieve videos without requiring detailed information about them.
Keywordsvideo-search impression visualization sentiment analysis
Unable to display preview. Download preview PDF.
- 1.Cisco: Cisco Visual Networking Index: Forecast and Methodology, 2009-2014 (2010), http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper_c11-481360.pdf
- 2.Cunningham, S.J., Nichols, D.M.: How people find videos. In: Proceedings of the 8th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 201–210 (2008)Google Scholar
- 4.Kiyoki, Y., Kitagawa, T., Hitomi, Y.: A fundamental framework for realizing semantic interoperability in a multidatabase environment. Journal of Integrated Computer-Aided Engineering 2(1), 3–20 (1995)Google Scholar
- 8.Hou, X., Zhang, L.: Color conceptualization. In: Proceedings of the 15th International Conference on Multimedia, pp. 265–268. ACM (2007)Google Scholar
- 12.Kobayashi, S.: Color Image Scale. Oxford University Press (1992)Google Scholar
- 14.Lehane, B., O’Connor, N.E., Lee, H., Smeaton, A.F.: Indexing of Fictional Video Content for Event Detection and Summarisation. EURASIP Journal on Image and Video Processing, Article ID 14615, 15 pages (2007)Google Scholar
- 16.Arifin, S., Cheung, P.Y.K.: A computation method for video segmentation utilizing the pleasure-arousal-dominance emotional information. In: Proceedings of the 15th ACM International Conference on Multimedia, pp. 68–77 (2007)Google Scholar
- 17.Kurabayashi, S., Ueno, T., Kiyoki, Y.: A Context-Based Whole Video Retrieval System with Dynamic Video Stream Analysis Mechanisms. In: Proceedings of the 11th IEEE International Symposium on Multimedia (ISM 2009), pp. 505–510 (2009)Google Scholar