Melody Retrieval and Classification Using Biologically-Inspired Techniques

  • Dimitrios Bountouridis
  • Dan Brown
  • Hendrik Vincent Koops
  • Frans Wiering
  • Remco C. Veltkamp
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10198)


Retrieval and classification are at the center of Music Information Retrieval research. Both tasks rely on a method to assess the similarity between two music documents. In the context of symbolically encoded melodies, pairwise alignment via dynamic programming has been the most widely used method. However, this approach fails to scale-up well in terms of time complexity and insufficiently models the variance between melodies of the same class. Compact representations and indexing techniques that capture the salient and robust properties of music content, are increasingly important. We adapt two existing bioinformatics tools to improve the melody retrieval and classification tasks. On two datasets of folk tunes and cover song melodies, we apply the extremely fast indexing method of the Basic Local Alignment Search Tool (BLAST) and achieve comparable classification performance to exhaustive approaches. We increase retrieval performance and efficiency by using multiple sequence alignment algorithms for locating variation patterns and profile hidden Markov models for incorporating those patterns into a similarity model.


Multiple Sequence Alignment Basic Local Alignment Search Tool Mean Average Precision Profile Hide Markov Model Music Information Retrieval 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Altschul, S.F., Gish, W., Miller, W., Myers, E.W., Lipman, D.J.: Basic local alignment search tool. J. Mol. Biol. 215(3), 403–410 (1990)CrossRefGoogle Scholar
  2. 2.
    Altschul, S.F., Madden, T.L., Schäffer, A.A., Zhang, J., Zhang, Z., Miller, W., Lipman, D.J.: Gapped blast and psi-blast: a new generation of protein database search programs. Nucleic Acids Res. 25(17), 3389–3402 (1997)CrossRefGoogle Scholar
  3. 3.
    Bairoch, A.: Prosite: a dictionary of sites and patterns in proteins. Nucleic Acids Res. 19(Suppl), 2241 (1991)CrossRefGoogle Scholar
  4. 4.
    Bertin-Mahieux, T., Ellis, D.P.W.: Large-scale cover song recognition using hashed chroma landmarks. In: Applications of Signal Processing to Audio and Acoustics, pp. 117–120 (2011)Google Scholar
  5. 5.
    Boot, P., Volk, A., de Haas, W.B.: Evaluating the role of repeated patterns in folk song classification and compression. J. New Music Res. 1–16 (2016)Google Scholar
  6. 6.
    Bountouridis, D., Koops, H.V., Wiering, F., Veltkamp, R.C.: Music outlier detection using multiple sequence alignment and independent ensembles. In: Amsaleg, L., Houle, M.E., Schubert, E. (eds.) SISAP 2016. LNCS, vol. 9939, pp. 286–300. Springer, Cham (2016). doi: 10.1007/978-3-319-46759-7_22 CrossRefGoogle Scholar
  7. 7.
    Bountouridis, D., Van Balen, J.: The cover song variation dataset. In: The International Workshop on Folk Music Analysis (2014)Google Scholar
  8. 8.
    Carroll, H., Clement, M.J., Ridge, P., Snell, Q.O.: Effects of gap open and gap extension penalties. In: The Biotechnology and Bioinformatics Symposium, pp. 19–23 (2006)Google Scholar
  9. 9.
    Casey, M., Slaney, M.: Fast recognition of remixed music audio. In: Acoustics, Speech and Signal Processing, vol. 4, p. IV-1425 (2007)Google Scholar
  10. 10.
    Chai, W., Vercoe, B.: Folk music classification using hidden Markov models. In: International Conference on Artificial Intelligence, number 6 in 4. Citeseer (2001)Google Scholar
  11. 11.
    Day, W.H.E., McMorris, F.R.: Threshold consensus methods for molecular sequences. J. Theor. Biol. 159(4), 481–489 (1992)CrossRefGoogle Scholar
  12. 12.
    Dong, X.L., Berti-Equille, L., Srivastava, D.: Integrating conflicting data: the role of source dependence. Proc. VLDB Endowment 2(1), 550–561 (2009)CrossRefGoogle Scholar
  13. 13.
    Durbin, R., Eddy, S.R., Krogh, A., Mitchison, G.: Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press, Cambridge (1998)CrossRefzbMATHGoogle Scholar
  14. 14.
    Eddy, S.R.: Profile hidden Markov models. Bioinformatics 14(9), 755–763 (1998)CrossRefGoogle Scholar
  15. 15.
    Finn, R.D., Clements, J., Eddy, S.R.: Hmmer web server: interactive sequence similarity searching. Nucleic Acids Res. gkr367 (2011)Google Scholar
  16. 16.
    Gómez, E., Klapuri, A., Meudic, B.: Melody description and extraction in the context of music content processing. J. New Music Res. 32(1), 23–40 (2003)CrossRefGoogle Scholar
  17. 17.
    Hillewaere, R., Manderick, B., Conklin, D.: Alignment methods for folk tune classification. In: Spiliopoulou, M., Schmidt-Thieme, L., Janning, R. (eds.) Data Analysis, Machine Learning and Knowledge Discovery, pp. 369–377. Springer, Cham (2014)CrossRefGoogle Scholar
  18. 18.
    Hogeweg, P., Hesper, B.: The alignment of sets of sequences and the construction of phyletic trees: an integrated method. J. Mol. Evol. 20(2), 175–186 (1984)CrossRefGoogle Scholar
  19. 19.
    Hu, N., Dannenberg, R.B., Tzanetakis, G.: Polyphonic audio matching and alignment for music retrieval. Computer Science Department, p. 521 (2003)Google Scholar
  20. 20.
    Katoh, K., Misawa, K., Kuma, K., Miyata, T.: Mafft: a novel method for rapid multiple sequence alignment based on fast fourier transform. Nucleic Acids Res. 30(14), 3059–3066 (2002)CrossRefGoogle Scholar
  21. 21.
    Kemena, C., Notredame, C.: Upcoming challenges for multiple sequence alignment methods in the high-throughput era. Bioinformatics 25(19), 2455–2465 (2009)CrossRefGoogle Scholar
  22. 22.
    Kilian, J., Hoos, H.H.: Musicblast-gapped sequence alignment for MIR. In: International Society for Music Information Retrieval Conference, pp. 38–41 (2004)Google Scholar
  23. 23.
    Kim, S., Narayanan, S.: Dynamic chroma feature vectors with applications to cover song identification. In: Multimedia Signal Processing, pp. 984–987 (2008)Google Scholar
  24. 24.
    Koops, H.V., de Haas, W.B., Bountouridis, D., Volk, A.: Integration and quality assessment of heterogeneous chord sequences using data fusion. In: International Society for Music Information Retrieval Conference, pp. 178–184 (2016)Google Scholar
  25. 25.
    Krogh, A.: An introduction to hidden Markov models for biological sequences. New Compr. Biochem. 32, 45–63 (1998)CrossRefzbMATHGoogle Scholar
  26. 26.
    Krogh, A., Brown, M., Saira Mian, I., Sjölander, K., Haussler, D.: Hidden Markov models in computational biology: applications to protein modeling. J. Mol. Biol. 235(5), 1501–1531 (1994)CrossRefGoogle Scholar
  27. 27.
    Malt, B.C.: An on-line investigation of prototype and exemplar strategies in classification. J. Exp. Psychol. Learn. Mem. Cogn. 15(4), 539 (1989)CrossRefGoogle Scholar
  28. 28.
    Margulis, E.H.: Musical repetition detection across multiple exposures. Music Percept. Interdisc. J. 29(4), 377–385 (2012)CrossRefGoogle Scholar
  29. 29.
    Martin, B., Brown, D.G., Hanna, P., Ferraro, P.: Blast for audio sequences alignment: a fast scalable cover identification. In: International Society for Music Information Retrieval Conference, pp. 529–534 (2012)Google Scholar
  30. 30.
    Müller, M., Mattes, H., Kurth, F.: An efficient multiscale approach to audio synchronization. In: International Society for Music Information Retrieval Conference, pp. 192–197. Citeseer (2006)Google Scholar
  31. 31.
    Needleman, S.B., Wunsch, C.D.: A general method applicable to the search for similarities in the amino acid sequence of two proteins. J. Mol. Biol. 48(3), 443–453 (1970)CrossRefGoogle Scholar
  32. 32.
    Pampalk, E.: Computational models of music similarity and their application in music information retrieval. na (2006)Google Scholar
  33. 33.
    Ratanamahatana, C.A., Keogh, E.: Everything you know about dynamic time warping is wrong. In: Third Workshop on Mining Temporal and Sequential Data, pp. 1–11. Citeseer (2004)Google Scholar
  34. 34.
    Serra, J., Gómez, E., Herrera, P., Serra, X.: Chroma binary similarity and local alignment applied to cover song identification. IEEE Trans. Audio Speech Lang. Process. 16(6), 1138–1151 (2008)CrossRefGoogle Scholar
  35. 35.
    Smith, T.F., Waterman, M.S.: Identification of common molecular subsequences. J. Mol. Biol. 147(1), 195–197 (1981)CrossRefGoogle Scholar
  36. 36.
    Temperley, D.: Bayesian models of musical structure and cognition. Musicae Sci. 8(2), 175–205 (2004)CrossRefGoogle Scholar
  37. 37.
    Thompson, J.D., Higgins, D.G., Gibson, T.J.: Clustal w: improving the sensitivity of progressive multiple sequence alignment through sequence weighting, position-specific gap penalties and weight matrix choice. Nucleic Acids Res. 22(22), 4673–4680 (1994)CrossRefGoogle Scholar
  38. 38.
    van Kranenburg, P.: A computational approach to content-based retrieval of folk song melodies. Ph.D. thesis, Utrecht University (2010)Google Scholar
  39. 39.
    van Kranenburg, P., de Bruin, M., Grijp, L., Wiering, F.: The Meertens tune collections. In: Meertens Online Reports (2014)Google Scholar
  40. 40.
    Volk, A., Haas, W.B., Van Kranenburg, P.: Towards modelling variation in music as foundation for similarity. In: Proceedings of the 12th International Conference on Music Perception and Cognition (2012)Google Scholar
  41. 41.
    Volk, A., Van Kranenburg, P.: Melodic similarity among folk songs: an annotation study on similarity-based categorization in music. Musicae Sci. 16, 317–339 (2012). page 1029864912448329CrossRefGoogle Scholar
  42. 42.
    Wang, L., Jiang, T.: On the complexity of multiple sequence alignment. J. Comput. Biol. 1(4), 337–348 (1994)CrossRefGoogle Scholar
  43. 43.
    Wang, S., Ewert, S., Dixon, S.: Robust joint alignment of multiple versions of a piece of music. In: International Society for Music Information Retrieval, pp. 83–88 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Dimitrios Bountouridis
    • 1
  • Dan Brown
    • 2
  • Hendrik Vincent Koops
    • 1
  • Frans Wiering
    • 1
  • Remco C. Veltkamp
    • 1
  1. 1.Department of Information and Computing SciencesUtrecht UniversityUtrechtThe Netherlands
  2. 2.David R. Cheriton School of Computer ScienceUniversity of WaterlooWaterlooCanada

Personalised recommendations