Semantic Based Background Music Recommendation for Home Videos

  • Yin-Tzu Lin
  • Tsung-Hung Tsai
  • Min-Chun Hu
  • Wen-Huang Cheng
  • Ja-Ling Wu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8326)


In this paper, we propose a new background music recommendation scheme for home videos and two new features describing the short-term motion/tempo distribution in visual/aural content. Unlike previous researches that merely matched the visual and aural contents through a perceptual way, we incorporate the textual semantics and content semantics while determining the matching degree of a video and a song. The key idea is that the recommended music should contain semantics that relate to the ones in the input video and that the rhythm of the music and the visual motion of the video should be harmonious enough. As a result, a few user-given tags and automatically annotated tags are used to compute their relation to the lyrics of the songs for selecting candidate musics. Then, we use the proposed motion-direction histogram (MDH) and pitch tempo pattern (PTP) to do the second-run selection. The user preference to the music genre is also taken into account as a filtering mechanism at the beginning. The primitive user evaluation shows that the proposed scheme is promising.


Visual Motion Input Video Background Music Music Genre Recommendation Scheme 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bertin-Mahieux, T., Ellis, D.P., Whitman, B., Lamere, P.: The Million Song Dataset. In: ISMIR (2011)Google Scholar
  2. 2.
    Chen, C.-H., Weng, M.-F., Jeng, S.-K., Chuang, Y.-Y.: Emotion-based music visualization using photos. In: Satoh, S., Nack, F., Etoh, M. (eds.) MMM 2008. LNCS, vol. 4903, pp. 358–368. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    Chu, W.-T., Tsai, S.-Y.: Rhythm of Motion Extraction and Rhythm-Based Cross-Media Alignment for Dance Videos. IEEE Trans. MM 14(1), 129–141 (2012)Google Scholar
  4. 4.
    Dunker, P., Dittmar, C., Begau, A., Nowak, S., Gruhne, M.: Semantic High-Level Features for Automated Cross-Modal Slideshow Generation. In: Int. Workshop on Content-Based Multimedia Indexing, pp. 144–149 (June 2009)Google Scholar
  5. 5.
    Dunker, P., Popp, P., Cook, R.: Content-aware Auto-soundtracks for Personal Photo Music Slideshows. In: IEEE ICME (2011)Google Scholar
  6. 6.
    Feng, J., Ni, B.: Auto-Generation of Professional Background Music for Home-made Videos. In: Int. Conf. on Internet Multi. Comput. and Serv. (2010)Google Scholar
  7. 7.
    Goto, M.: A chorus section detection method for musical audio signals and its application to a music listening station. IEEE Trans. Audio, Speech, Lang. Process. 14(5), 1783–1794 (2006)CrossRefGoogle Scholar
  8. 8.
    Li, L.-J., Su, H., Xing, E.P., Li, F.-F.: Object Bank: A High-Level Image Representation for Scene Classification & Semantic Feature Sparsification. In: NIPS, pp. 1378–1386 (2010)Google Scholar
  9. 9.
    Miller, G.A.: WordNet: A Lexical Database for English. Commun. ACM 38(11), 39–41 (1995)CrossRefGoogle Scholar
  10. 10.
    Pampalk, E.: Computational Models of Music Similarity and Their Application in Music Information Retrieval. Ph.d., Vienna University of Technology (2006)Google Scholar
  11. 11.
    Wang, J., Chng, E., Xu, C.: Fully and Semi-automatic Music Sports Video Composition. In: IEEE ICME, pp. 1897–1900 (2006)Google Scholar
  12. 12.
    Yeh, M.-C., Cheng, K.-T.: A String Matching Approach for Visual Retrieval and Classification. In: ACM MIR, p. 52. ACM Press, New York (2008)Google Scholar
  13. 13.
    Zettl, H.: In: Dorai, C., Venkatesh, S. (eds.) Media Computing: Computational Media Aesthetics, pp. 11–38. Springer, USGoogle Scholar
  14. 14.
    Zhang, W., Xing, L., Huang, Q., Gao, W.: A System for Automatic Generation of Music Sports-Video. In: IEEE ICME, pp. 1286–1289 (2005)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Yin-Tzu Lin
    • 1
  • Tsung-Hung Tsai
    • 2
  • Min-Chun Hu
    • 2
    • 3
  • Wen-Huang Cheng
    • 2
  • Ja-Ling Wu
    • 1
  1. 1.National Taiwan UniversityTaipeiTaiwan, R.O.C.
  2. 2.Academia SinicaTaipeiTaiwan, R.O.C.
  3. 3.National Cheng Kung UniversityTainanTaiwan, R.O.C.

Personalised recommendations