Advertisement

Query-by-Dancing: A Dance Music Retrieval System Based on Body-Motion Similarity

  • Shuhei TsuchidaEmail author
  • Satoru FukayamaEmail author
  • Masataka GotoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11295)

Abstract

This paper presents Query-by-Dancing, a dance music retrieval system that enables a user to retrieve music using dance motions. When dancers search for music to play when dancing, they sometimes find it by referring to online dance videos in which the dancers use motions similar to their own dance. However, previous music retrieval systems could not support retrieval specialized for dancing because they do not accept dance motions as a query. Therefore, we developed our Query-by-Dancing system, which uses a video of a dancer (user) as the input query to search a database of dance videos. The query video is recorded using an ordinary RGB camera that does not obtain depth information, like a smartphone camera. The poses and motions in the query are then analyzed and used to retrieve dance videos with similar poses and motions. The system then enables the user to browse the music attached to the videos it retrieves so that the user can find a piece that is appropriate for their dancing. An interesting problem here is that a simple search for the most similar videos based on dance motions sometimes includes results that do not match the intended dance genre. We solved this by using a novel measure similar to tf-idf to weight the importance of dance motions when retrieving videos. We conducted comparative experiments with 4 dance genres and confirmed that the system gained an average of 3 or more evaluation points for 3 dance genres (waack, pop, break) and that our proposed method was able to deal with different dance genres.

Keywords

Dance Music Video Retrieval system Body-motion 

Notes

Acknowledgments

This work was supported in part by JST ACCEL Grant Number JPMJAC1602, Japan.

References

  1. 1.
    Cao, Z., Simon, T., Wei, S., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: The 2017 IEEE Conference on Computer Vision and Pattern Recognition (2017)Google Scholar
  2. 2.
    Casey, M.A., Veltkamp, R.C., Goto, M., Leman, M., Rhodes, C., Slaney, M.: Content-based music information retrieval: current directions and future challenges. Proc. IEEE 96(4), 668–696 (2008)CrossRefGoogle Scholar
  3. 3.
    Chen, J., Chen, A.: Query by rhythm: an approach for song retrieval in music databases. In: Proceedings of the 8th International Workshop on Research Issues in Data Engineering: Continuous-Media Databases and Applications, pp. 139–146 (1998)Google Scholar
  4. 4.
    Ghias, A., Logan, J., Chamberlin, D., Smith, B.C.: Query by humming - musical information retrieval in an audio database. In: Proceedings of ACM Multimedia 1995, pp. 231–236 (1995)Google Scholar
  5. 5.
    Jang, J.-S.R., Lee, H.-R., Yeh, C.-H.: Query by tapping: a new paradigm for content-based music retrieval from acoustic input. In: Shum, H.-Y., Liao, M., Chang, S.-F. (eds.) PCM 2001. LNCS, vol. 2195, pp. 590–597. Springer, Heidelberg (2001).  https://doi.org/10.1007/3-540-45453-5_76CrossRefGoogle Scholar
  6. 6.
    Maezawa, A., Goto, M., Okuno, H.G.: Query-by-conducting: an interface to retrieve classical-music interpretations by real-time tempo input. In: The 11th International Society of Music Information Retrieval, pp. 477–482 (2010)Google Scholar
  7. 7.
    Müller, M.: Fundamentals of Music Processing - Audio, Analysis, Algorithms, Applications. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-21945-5CrossRefGoogle Scholar
  8. 8.
    Salvador, S., Chan, P.: Toward accurate dynamic time warping in linear time and space. J. Intell. Data Anal. 11(5), 561–580 (2007)CrossRefGoogle Scholar
  9. 9.
    Schedl, M., Gómez, E., Urbano, J.: Music information retrieval: recent developments and applications. Found. Trends Inf. Retr. 8(2–3), 127–261 (2014)CrossRefGoogle Scholar
  10. 10.
    Smiraglia, R.P.: Musical works as information retrieval entities: epistemological perspectives. In: The 2nd International Society of Music Information Retrieval, pp. 85–91 (2001)Google Scholar
  11. 11.
    Turnbull, D., Barrington, L., Torres, D., Lanckriet, G.: Semantic annotation and retrieval of music and sound effects. IEEE Trans. Audio, Speech, Lang. Process. 16(2), 467–476 (2008)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.National Institute of Advanced Industrial Science and Technology (AIST)TsukubaJapan

Personalised recommendations