Skip to main content
SpringerLink
Log in

Knowledge-Based Multimodal Music Similarity

  • Conference paper
  • First Online:
The Semantic Web: ESWC 2023 Satellite Events (ESWC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13998))

Included in the following conference series:

  • 514 Accesses

Abstract

Music similarity is an essential aspect of music retrieval, recommendation systems, and music analysis. Moreover, similarity is of vital interest for music experts, as it allows studying analogies and influences among composers and historical periods.

Current approaches to musical similarity rely mainly on symbolic content, which can be expensive to produce and is not always readily available. Conversely, approaches using audio signals typically fail to provide any insight about the reasons behind the observed similarity.

This research addresses the limitations of current approaches by focusing on the study of musical similarity using both symbolic and audio content. The aim of this research is to develop a fully explainable and interpretable system that can provide end-users with more control and understanding of music similarity and classification systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    ChoCo SPARQL Endpoint: https://polifonia.disi.unibo.it/choco/sparql.

  2. 2.

    MusicBrainz: https://musicbrainz.org/.

  3. 3.

    Discogs: https://www.discogs.com/.

References

  1. Adegbija, T.: JazzNet: a dataset of fundamental piano patterns for music audio machine learning research. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE (2023)

    Google Scholar 

  2. Balke, S., Arifi-Müller, V., Lamprecht, L., Müller, M.: Retrieving audio recordings using musical themes. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 281–285 (2016)

    Google Scholar 

  3. Baltrušaitis, T., Ahuja, C., Morency, L.P.: Multimodal machine learning: a survey and taxonomy (2017)

    Google Scholar 

  4. de Berardinis, J., Meroño-Peñuela, A., Poltronieri, A., Presutti, V.: Choco: a chord corpus and a data transformation workflow for musical harmony knowledge graphs. In: Manuscript Under Review (2022)

    Google Scholar 

  5. de Berardinis, J., Meroño-Peñuela, A., Poltronieri, A., Presutti, V.: The harmonic memory: a knowledge graph of harmonic patterns as a trustworthy framework for computational creativity. In: The Web Conference (2023)

    Google Scholar 

  6. de Berardinis, J., Vamvakaris, M., Cangelosi, A., Coutinho, E.: Unveiling the hierarchical structure of music by multi-resolution community detection. Trans. Int. Soc. Music Inf. Retrieval 3(1), 82–97 (2020)

    Google Scholar 

  7. Bizer, C., Heath, T., Berners-Lee, T.: Linked data - the story so far. Int. J. Semant. Web Inf. Syst. 5(3), 1–22 (2009)

    Article  Google Scholar 

  8. Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)

    Article  Google Scholar 

  9. Chefer, H., Gur, S., Wolf, L.: Transformer interpretability beyond attention visualization (2020)

    Google Scholar 

  10. Corrêa, D.C., Rodrigues, F.A.: A survey on symbolic data-based music genre classification. Expert Syst. Appl. 60, 190–210 (2016)

    Article  Google Scholar 

  11. Downie, J.S.: The scientific evaluation of music information retrieval systems: foundations and future. Comput. Music. J. 28(2), 12–23 (2004)

    Article  Google Scholar 

  12. Du, X., Chen, K., Wang, Z., Zhu, B., Ma, Z.: Bytecover2: towards dimensionality reduction of latent embedding for efficient cover song identification. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 616–620 (2022)

    Google Scholar 

  13. Dörfler, M., Bammer, R., Grill, T.: Inside the spectrogram: convolutional neural networks in audio processing. In: 2017 International Conference on Sampling Theory and Applications (SampTA), pp. 152–155 (2017)

    Google Scholar 

  14. Giraud, M., Groult, R., Leguy, E., Levé, F.: Computational fugue analysis. Comput. Music. J. 39(2), 77–96 (2015)

    Article  Google Scholar 

  15. de Haas, W.B., Wiering, F., Veltkamp, R.C.: A geometrical distance measure for determining the similarity of musical harmony. Int. J. Multimed. Inf. Retrieval 2(3), 189–202 (2013)

    Article  Google Scholar 

  16. Hanna, P., Robine, M., Rocher, T.: An alignment based system for chord sequence retrieval. In: Proceedings of the 9th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 101–104 (2009)

    Google Scholar 

  17. Harte, C., Sandler, M.B., Abdallah, S.A., Gómez, E.: Symbolic representation of musical chords: a proposed syntax for text annotations. In: ISMIR, vol. 5, pp. 66–71 (2005)

    Google Scholar 

  18. Humphrey, E.J., Salamon, J., Nieto, O., Forsyth, J., Bittner, R.M., Bello, J.P.: JAMS: a JSON annotated music specification for reproducible MIR research. In: ISMIR, pp. 591–596 (2014)

    Google Scholar 

  19. Karydis, I., Lida Kermanidis, K., Sioutas, S., Iliadis, L.: Comparing content and context based similarity for musical data. Neurocomputing 107, 69–76 (2013). Timely Neural Networks Applications in Engineering

    Article  Google Scholar 

  20. Knees, P., Schedl, M.: A survey of music similarity and recommendation from music context data. ACM Trans. Multimedia Comput. Commun. Appl. 10(1), 1–21 (2013)

    Article  Google Scholar 

  21. Kowald, D., Schedl, M., Lex, E.: The unfairness of popularity bias in music recommendation: a reproducibility study. In: Jose, J.M., et al. (eds.) ECIR 2020. LNCS, vol. 12036, pp. 35–42. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-45442-5_5

    Chapter  Google Scholar 

  22. Laurier, C., Grivolla, J., Herrera, P.: Multimodal music mood classification using audio and lyrics. In: 2008 Seventh International Conference on Machine Learning and Applications, pp. 688–693 (2008)

    Google Scholar 

  23. Lerdahl, F.: Tonal pitch space. Music Percept.: Interdisc. J. 5(3), 315–349 (1988)

    Article  Google Scholar 

  24. Li, P.C., Su, L., Yang, Y.H., Su, A.W.Y.: Analysis of expressive musical terms in violin using score-informed and expression-based audio features. In: International Society for Music Information Retrieval Conference (2015)

    Google Scholar 

  25. Madjiheurem, S., Qu, L., Walder, C.: Chord2vec: learning musical chord embeddings. In: Proceedings of the Constructive Machine Learning Workshop at 30th Conference on Neural Information Processing Systems (NIPS2016), Barcelona, Spain (2016)

    Google Scholar 

  26. McFee, B., Barrington, L., Lanckriet, G.: Learning content similarity for music recommendation. IEEE Trans. Audio Speech Lang. Process. 20(8), 2207–2218 (2012)

    Article  Google Scholar 

  27. Medina, R., Smith, L., Wagner, D.: Content-based indexing of musical scores. In: Proceedings of the 2003 Joint Conference on Digital Libraries, pp. 18–26 (2003)

    Google Scholar 

  28. Mehrotra, R.: Algorithmic balancing of familiarity, similarity, & discovery in music recommendations. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM 2021, pp. 3996–4005. Association for Computing Machinery, New York (2021)

    Google Scholar 

  29. Müller, M.: Fundamentals of Music Processing: Audio, Analysis, Algorithms, Applications, vol. 5. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-319-21945-5

    Book  Google Scholar 

  30. Orio, N., Rodà, A.: A measure of melodic similarity based on a graph representation of the music structure. In: Hirata, K., Tzanetakis, G., Yoshii, K. (eds.) Proceedings of the 10th International Society for Music Information Retrieval Conference, ISMIR 2009, Kobe International Conference Center, Kobe, Japan, 26–30 October 2009, pp. 543–548. International Society for Music Information Retrieval (2009)

    Google Scholar 

  31. Sakoe, H., Chiba, S.: Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Acoust. Speech Signal Process. 26(1), 43–49 (1978)

    Article  Google Scholar 

  32. Sheikh Fathollahi, M., Razzazi, F.: Music similarity measurement and recommendation system using convolutional neural networks. Int. J. Multimed. Inf. Retrieval 10(1), 43–53 (2021)

    Article  Google Scholar 

  33. Simonetta, F., Ntalampiras, S., Avanzini, F.: Multimodal music information processing and retrieval: Survey and future challenges. In: 2019 International Workshop on Multilayer Music Representation and Processing (MMRP), pp. 10–18 (2019)

    Google Scholar 

  34. Suyoto, I.S.H., Uitdenbogerd, A.L., Scholer, F.: Searching musical audio using symbolic queries. IEEE Trans. Audio Speech Lang. Process. 16(2), 372–381 (2008)

    Article  Google Scholar 

  35. Tan, H.H., Herremans, D.: Music fadernets: controllable music generation based on high-level features via low-level feature modelling. In: Cumming, J., et al. (eds.) Proceedings of the 21th International Society for Music Information Retrieval Conference, ISMIR 2020, Montreal, Canada, 11–16 October 2020, pp. 109–116 (2020)

    Google Scholar 

  36. Velardo, V., Vallati, M., Jan, S.: Symbolic melodic similarity: state of the art and future challenges. Comput. Music. J. 40(2), 70–83 (2016)

    Article  Google Scholar 

  37. Vinet, H.: The representation levels of music information. In: Wiil, U.K. (ed.) CMMR 2003. LNCS, vol. 2771, pp. 193–209. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-39900-1_17

    Chapter  Google Scholar 

  38. Wiggins, G., Miranda, E., Smaill, A., Harris, M.: A framework for the evaluation of music representation systems. Comput. Music. J. 17(3), 31–42 (1993)

    Article  Google Scholar 

  39. Zheng, E., Moh, M., Moh, T.S.: Music genre classification: a n-gram based musicological approach. In: 2017 IEEE 7th International Advance Computing Conference (IACC), pp. 671–677 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Bologna, Bologna, Italy

    Andrea Poltronieri

Authors
  1. Andrea Poltronieri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrea Poltronieri .

Editor information

Editors and Affiliations

  1. University of Lisbon, Lisbon, Portugal

    Catia Pesquita

  2. University of Nantes, Nantes, France

    Hala Skaf-Molli

  3. Foundation for Research and Technology, Heraklion, Greece

    Vasilis Efthymiou

  4. Vienna University of Economics and Business, Vienna, Austria

    Sabrina Kirrane

  5. University of Paderborn, Paderborn, Germany

    Axel Ngonga

  6. Fraunhofer IAIS, Sankt Augustin, Germany

    Diego Collarana

  7. IBM Research, Rio de Janeiro, Brazil

    Renato Cerqueira

  8. Institut Polytechnique de Paris, Palaiseau, France

    Mehwish Alam

  9. Institut de Recherche en Informatique de Toulouse, Toulouse, France

    Cassia Trojahn

  10. University of Mannheim, Mannheim, Germany

    Sven Hertling

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Poltronieri, A. (2023). Knowledge-Based Multimodal Music Similarity. In: Pesquita, C., et al. The Semantic Web: ESWC 2023 Satellite Events. ESWC 2023. Lecture Notes in Computer Science, vol 13998. Springer, Cham. https://doi.org/10.1007/978-3-031-43458-7_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43458-7_41

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43457-0

  • Online ISBN: 978-3-031-43458-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation