Abstract
This paper presents a study on classifying music by affective visual information extracted frommusic videos. The proposed audio-visual approach analyzes genre specific utilization of color. A comprehensive set of color specific image processing features used for affect and emotion recognition derived from psychological experiments or art-theory is evaluated in the visual and multi-modal domain against contemporary audio content descriptors. The evaluation of the presented color features is based on comparative classification experiments on the newly introduced ‘Music Video Dataset’. Results show that a combination of the modalities can improve non-timbral and rhythmic features but show insignificant effects on high performing audio features.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Crete, F., et al.: The blur effect: Perception and estimation with a new no-reference perceptual blur metric. In: Electronic Imaging 2007, p. 64920 (2007)
Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part III. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006)
Fu, Z., Lu, G., Ting, K.M., Zhang, D.: A survey of audio-based music classification and annotation. IEEE Transactions on Multimedia 13(2), 303–319 (2011)
Gillet, O., Essid, S., Richard, G.: On the correlation of automatic audio and visual segmentations of music videos. IEEE Trans. on Circuits and Sys. for Video Tech. (2007)
Hanbury, A.: Circular statistics applied to colour images. In: 8th Computer Vision Winter Workshop, vol. 91, pp. 53–71. Citeseer (2003)
Itten, J., Van Haagen, E.: The art of color: The subjective experience and objective rationale of color. Van Nostrand Reinhold, New York (1973)
Lidy, T., Rauber, A.: Evaluation of feature extractors and psycho-acoustic transformations for music genre classification. In: ISMIR (2005)
Machajdik, J., Hanbury, A.: Affective image classification using features inspired by psychology and art theory. In: Proc. Int. Conf. on Multimedia, pp. 83–92 (2010)
Manjunath, B.S., Ohm, J.-R., Vasudevan, V.V., Yamada, A.: Color and texture descriptors. IEEE Trans. on Circuits and Sys. for Video Tech. 11(6), 703–715 (2001)
Plataniotis, K.N., Venetsanopoulos, A.N.: Color image proc. and applications (2000)
Rubner, Y., Tomasi, C., Guibas, L.J.: The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision 40(2), 99–121 (2000)
Schettini, R., Ciocca, G., Zuffi, S., et al.: A survey of methods for colour image indexing and retrieval in image databases. In: Color Imaging Science: Exploiting Digital Media (2001)
Schindler, A., Rauber, A.: A music video information retrieval approach to artist identification. In: 10th Symp. on Computer Music Multidisciplinary Research (2013)
Tzanetakis, G., Cook, P.: Marsyas: A framework for audio analysis. Organised Sound (2000)
Valdez, P., Mehrabian, A.: Effects of color on emotions. Journal of Experimental Psychology: General 123(4), 394 (1994)
Vedaldi, A., Soatto, S.: Quick shift and kernel methods for mode seeking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 705–718. Springer, Heidelberg (2008)
Wei-ning, W., Ying-lin, Y., Sheng-ming, J.: Image retrieval by emotional semantics: A study of emotional space and feature extraction. In: IEEE International Conference on Systems, Man and Cybernetics (2006)
Wildenauer, H., Blauensteiner, P., Hanbury, A., Kampel, M.: Motion detection using an improved colour model. In: Bebis, G., et al. (eds.) ISVC 2006. LNCS, vol. 4292, pp. 607–616. Springer, Heidelberg (2006)
Yazdani, A., Kappeler, K., Ebrahimi, T.: Affective content analysis of music video clips. In: Music Information Retrieval with User-Centered and Multimodal Strategies (2011)
Zhang, S., Huang, Q., Jiang, S., Gao, W., Tian, Q.: Affective visualization and retrieval for music video. IEEE Transactions on Multimedia 12(6), 510–522 (2010)
Zuiderveld, K.: Contrast limited adaptive histogram equalization. In: Graphics Gems IV, pp. 474–485. Academic Press Professional, Inc. (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Schindler, A., Rauber, A. (2015). An Audio-Visual Approach to Music Genre Classification through Affective Color Features. In: Hanbury, A., Kazai, G., Rauber, A., Fuhr, N. (eds) Advances in Information Retrieval. ECIR 2015. Lecture Notes in Computer Science, vol 9022. Springer, Cham. https://doi.org/10.1007/978-3-319-16354-3_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-16354-3_8
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16353-6
Online ISBN: 978-3-319-16354-3
eBook Packages: Computer ScienceComputer Science (R0)