Journal of Intelligent Information Systems

, Volume 41, Issue 3, pp 435–459 | Cite as

Capturing the workflows of music information retrieval for repeatability and reuse

  • Kevin R. PageEmail author
  • Ben Fields
  • David De Roure
  • Tim Crawford
  • J. Stephen Downie


Many solutions for the reuse and re-purposing of Music Information Retrieval (MIR) methods, and the tools implementing those methods, have been introduced over recent years. Proposals for achieving interoperability between systems have ranged from shared software libraries and interfaces, through common frameworks and portals, to standardised file formats and metadata. Here we assess these solutions for their suitability to be reused and combined as repurposable components within assemblies (or workflows) that can be used in novel and possibly more ambitious ways. Reuse and repeatability also have great implications for the process of MIR research: the encapsulation of any algorithm and its operation—including inputs, parameters, and outputs—is fundamental to the repeatability and reproducibility of an experiment. This is desirable both for the open and reliable evaluation of algorithms and for the advancement of MIR by building more effectively upon prior research. At present there is no clear best practice widely adopted by the field. Based upon our analysis of contemporary systems and their adoption we reflect as to whether this should be considered a failure. Are there limits to interoperability unique to MIR, and how might they be overcome? Beyond workflows how much research context can, and should, be captured? We frame our assessment within the emerging notion of Research Objects for reproducible research in other domains, and describe how their adoption could serve as a route to reuse in MIR.


Music Information Retrieval (MIR) Workflows Reproducible research Research Objects 



This work was carried out through the EPSRC funded e-Research South platform grant (Grant No. EP/F05811X/1), the EU FP7 (ICT-2009.4.1) funded Wf4Ever project (270129), and the Structural Analysis of Large Amounts of Music Information (SALAMI) project funded by the JISC Digitisation and e-Content programme and the National Science Foundation (Grant Nos. IIS 10-42727 and IIS 09-39253) and the Canadian Social Science and Humanities Research Council (SSHRC). We are extremely grateful to Kahyun Choi and other members of the International Music Information Retrieval Systems Evaluation Laboratory (IMIRSEL) at the University of Illinois for their assistance collating the MIREX programming language statistics, and the reviewers of earlier iterations of this paper for their invaluable comments.


  1. Bechhofer, S., Buchan, I., De Roure, D., Missier, P., et al. (2013). Why linked data is not enough for scientists. Future Generation Computer Systems, 29(2), 599–611.CrossRefGoogle Scholar
  2. Belhajjame, K., Corcho, O., Garijo, D., Zhao, J., et al. (2012). Workflow-centric research objects: first class citizens in scholarly discourse. In Proc. workshop on the semantic publishing (SePublica) (pp. 1–12).Google Scholar
  3. Berenzweig, A., Logan, B., D.Ellis, P., W., Whitman, B. (2004). A large-scale evaluation of acoustic and subjective music-similarity measures. Computer Music Journal, 28(2), 63–76.CrossRefGoogle Scholar
  4. Bullock, J. (2007). libXtract: a lightweight library for audio feature extraction. In Proc. international computer music conference (pp. 25–28).Google Scholar
  5. Cannam, C., Landone, C., Sandler, M., Bello, J.P. (2006). The sonic visualiser: a visualisation platform for semantic descriptors from musical signals. In Proc. 7th international conference on music information retrieval (pp. 324–327).Google Scholar
  6. Cannam, C., Sandler, M., Jewell, M.O., Rhodes, C., d’Inverno, M. (2010). Linked data and you: bringing music research software into the semantic web. Journal of New Music Research, 39(4):313–325.CrossRefGoogle Scholar
  7. Casey, M.A., Veltkamp, R., Goto, M., Leman, M., et al. (2008). Content-based music information retrieval: current directions and future challenges. Proc. IEEE, 96(4), 668–696.CrossRefGoogle Scholar
  8. Christenson, H. (2011). Hathitrust: a research library at web scale. Library Resources & Technical Services, 55(2), 93–102.CrossRefGoogle Scholar
  9. Damnjanovic, I., Davies, M., Plumbley, M. (2010). SMALLbox—an evaluation framework for sparse representations and dictionary learning algorithms. In Latent variable analysis and signal separation (pp. 418–425).Google Scholar
  10. De Roure, D., et al. (2009). The design and realisation of the virtual research environment for social sharing of workflows. FGCS, 25(5), 561–567.CrossRefGoogle Scholar
  11. Downie, J., Ehmann, A., Bay, M., Jones, M. (2010). The music information retrieval evaluation eXchange: some observations and insights. In Advances in music information retrieval (pp. 93–115).Google Scholar
  12. Downie, J.S., Byrd, D., Crawford, T. (2009). Ten years of ISMIR: reflections on challenges and opportunities. In Proc. 10th international society for music information retrieval conference (pp. 13–18).Google Scholar
  13. Downie, J.S., Ehmann, A.F., Hu, X. (2005). Music-to-knowledge (M2K): a prototyping and evaluation environment for music digital library research. In Proc. 5th ACM/IEEE joint conference on digital libraries (pp. 376–376).Google Scholar
  14. Fiebrink, R., Wang, G., Cook, P. (2008). Support for MIR prototyping and real-time applications in the ChucK programming language. In Proc. 9th international conference of music information retrieval (pp. 153–158).Google Scholar
  15. Gil, Y. (2007). Workflow composition: semantic representations for flexible automation. In Workflows for e-science (pp. 244–257).Google Scholar
  16. Hall, M., Frank, E., Holmes, G., Pfahringer, B., et al. (2009). The WEKA data mining software: an update. ACM SIGKDD Explorations Newsletter, 11(1), 10–18.CrossRefGoogle Scholar
  17. Hull, D., Wolstencroft, K., Stevens, R., Goble, C., Pocock, M., Li, P., Oinn, T. (2006). Taverna: a tool for building and running workflows of services. Nucleic Acids Research, 34(web server issue):729–732.CrossRefGoogle Scholar
  18. Janecek, A., Grotthoff, S.S., Gansterer, W.N. (2012). libNMF—a library for nonnegative matrix factorization. Computing and Informatics, 30(2), 205–224.Google Scholar
  19. Kassler, M. (1966). Toward musical information retrieval. Perspectives of New Music, 4(2), 59–67.CrossRefGoogle Scholar
  20. Lartillot, O., & Toiviainen, P. (2007). MIR in Matlab (II): a toolbox for musical feature extraction from audio. In Proc. 8th international society of music information retrieval conference (pp. 127–130).Google Scholar
  21. Laurier, C., Grivolla, J., Herrera, P. (2008). Multimodal music mood classification using audio and lyrics. In Proceedings of the 7th international conference on machine learning and applications (pp. 688–693). IEEE.Google Scholar
  22. Li, T., Ogihara, M., Li, Q. (2003). A comparative study on content-based music genre classification. In Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval (pp. 282–289). ACM.Google Scholar
  23. Llorà, X., Ács, B., Auvil, L.S., Capitanu, B., Welge, M.E., Goldberg, D.E. (2008). Meandre: semantic-driven data-intensive flows in the clouds. In Proc. IEEE fourth international conference on e-science, 2008 (pp. 238–245). IEEE.Google Scholar
  24. Mayer, R., & Rauber, A. (2012). Towards time-resilient MIR processes. In Proc. international conference on music information retrieval (pp. 337–342).Google Scholar
  25. McEnnis, D., McKay, C., Fujinaga, I. (2006). Overview of OMEN. In Proc. international conference on music information retrieval (pp. 7–12).Google Scholar
  26. McKay, C. (2010). Automatic music classification with jMIR. Ph.D. thesis, McGill University.Google Scholar
  27. McKay, C., Burgoyne, J.A., Thompson, J., Fujinaga, I. (2009). Using ACE XML 2.0 to store and share feature, instance and class data for musical classification. In Proc. international society for music information retrieval conference (pp. 303–308).Google Scholar
  28. Michener, W., Beach, J., Jones, M., Ludascher, B., Pennington, D., Pereira, R., Rajasekar, A., Schildhauer, M. (2007). A knowledge environment for the biodiversity and ecological sciences. Journal of Intelligent Information Systems, 29, 111–126.CrossRefGoogle Scholar
  29. Mitrović, D., Zeppelzauer, M., Breiteneder, C. (2010). Features for content-based audio retrieval. Advances in Computers, 78, 71–150.CrossRefGoogle Scholar
  30. Page, K.R., Fields, B., Nagel, B.J., O’Neill, G., et al. (2010). Semantics for music analysis through linked data: how country is my country? In Proc. IEEE sixth international conference on e-science (pp. 41–48).Google Scholar
  31. Pampalk, E., Rauber, A., Merkl, D. (2002). Using smoothed data histograms for cluster visualization in self-organizing maps. In Proceedings of the 12th internation conference on artificial neural networks (pp. 81–81).Google Scholar
  32. Raimond, Y., Abdallah, S., Sandler, M., Giasson, F. (2007). The music ontology. In Proc. international conference on music information retrieval (pp. 417–422).Google Scholar
  33. Rhodes, C., Crawford, T., Casey, M., d’Inverno, M. (2010). Investigating music collections at different scales with AudioDB. Journal of New Music Research, 39(4), 337–348.CrossRefGoogle Scholar
  34. Star, S.L., & Griesemer, J.R. (1989). Institutional ecology, translations and boundary objects: amateurs and professionals in Berkeley’s museum of vertebrate zoology, 1907–39. Social Studies of Science, 19(3), 387–420.CrossRefGoogle Scholar
  35. Tzanetakis, G., & Cook, P. (1999). Marsyas: a framework for audio analysis. Organised Sound, 4(3), 169–175.CrossRefGoogle Scholar
  36. Tzanetakis, G., Martins, L.G., Teixeira, L.F., Castillo, C., Jones, R., Lagrange, M. (2008). Interoperability and the Marsyas 0.2 runtime. In Proc. international computer music conference. Accessed 14 June 2013
  37. Wang, G. (2008). The ChucK audio programming language a strongly-timed and on-the-fly environ/mentality. Ph.D. thesis, Princeton University.Google Scholar
  38. West, K., Kumar, A., Shirk, A., Zhu, G., et al. (2010). The networked environment for music analysis (NEMA). In Proc. 6th IEEE world congress on services (pp. 314–317).Google Scholar
  39. Zhu, Y., & Shasha, D. (2003). Warping indexes with envelope transforms for query by humming. In Proceedings of the 2003 ACM SIGMOD international conference on management of data (pp. 181–192). ACM.Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Kevin R. Page
    • 1
    Email author
  • Ben Fields
    • 2
  • David De Roure
    • 1
  • Tim Crawford
    • 3
  • J. Stephen Downie
    • 4
  1. 1.Oxford e-Research CentreUniversity of OxfordOxfordUK
  2. 2.Musicmetric (Semetric Ltd.)LondonUK
  3. 3.Department of Computing, GoldsmithsUniversity of LondonLondonUK
  4. 4.Graduate School of Library and Information SciencesUniversity of IllinoisUrbana-ChampaignUSA

Personalised recommendations