Evaluation of the Music Ontology Framework

  • Yves Raimond
  • Mark Sandler
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7295)


The Music Ontology provides a framework for publishing structured music-related data on the Web, ranging from editorial data to temporal annotations of audio signals. It has been used extensively, for example in the DBTune project and on the BBC Music website. Until now it hasn’t been systematically evaluated and compared to other frameworks for handling music-related data. In this article, we design a ‘query-driven’ ontology evaluation framework capturing the intended use of this ontology. We aggregate a large set of real-world music-related user needs, and evaluate how much of it is expressible within our ontological framework. This gives us a quantitative measure of how well our ontology could support a system addressing these real-world user needs. We also provide some statistical insights in terms of lexical coverage for comparison with related description frameworks and identify areas within the ontology that could be improved.


Latent Dirichlet Allocation Evaluation Methodology User Query Representation Framework Music Video 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Alani, H., Brewster, C.: Metrics for ranking ontologies. In: Proceedings of the 4th Int. Workshop on Evaluation of Ontologies for the Web (2006)Google Scholar
  2. 2.
    Bainbridge, D., Cunningham, S.J., Downie, S.J.: How people describe their music information needs: A grounded theory analysis of music queries. In: Proceedings of the 4th International Conference on Music Information Retrieval (2003)Google Scholar
  3. 3.
    Baumann, S.: A music library in the palm of your hand. In: Proceedings of the Contact Forum on Digital libraries for musical audio (Perspectives and tendencies in digitalization, conservation, management and accessibility), Brussels (June 2005)Google Scholar
  4. 4.
    Baumann, S., Klüter, A., Norlien, M.: Using natural language input and audio analysis for a human-oriented MIR system. In: Proceedings of Web Delivery of Music (WEDELMUSIC) (2002)Google Scholar
  5. 5.
    Blei, D.M., Lafferty, J.D.: A correlated topic model of. The Annals of Applied Statistics 1(1), 17–35 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet Allocation. The Journal of Machine Learning Research 3(3), 993–1022 (2003)zbMATHGoogle Scholar
  7. 7.
    Brewster, C., Alani, H., Dasmahapatra, S., Wilks, Y.: Data driven ontology evaluation. In: Proceedings of the International Conference on Language Resources and Evaluation, Lisbon, Portugal, pp. 164–168 (2004)Google Scholar
  8. 8.
    Elhadad, M., Gabay, D., Netzer, Y.: Automatic Evaluation of Search Ontologies in the Entertainment Domain using Text Classification. In: Applied Semantic Technologies: Using Semantics in Intelligent Information Processing. Taylor and Francis (2011)Google Scholar
  9. 9.
    Fernández, M., Overbeeke, C., Sabou, M., Motta, E.: What Makes a Good Ontology? A Case-Study in Fine-Grained Knowledge Reuse. In: Gómez-Pérez, A., Yu, Y., Ding, Y. (eds.) ASWC 2009. LNCS, vol. 5926, pp. 61–75. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  10. 10.
    Griffiths, T., Steyvers, M.: Finding scientific topics. Proceedings of the National Academy of Sciences (2004)Google Scholar
  11. 11.
    Guarino, N., Welty, C.: Evaluating ontological decisions with ONTOCLEAN. Communications of the ACM 45(2), 61–65 (2002)CrossRefGoogle Scholar
  12. 12.
    Heinrich, G.: Parameter estimation for text analysis. Technical report, University of Leipzig & vsonix GmbH, Darmstadt, Germany (April 2008)Google Scholar
  13. 13.
    Lavbic, D., Krisper, M.: Facilitating ontology development with continuous evaluation. Informatica 21(4), 533–552 (2010)Google Scholar
  14. 14.
    Ha Lee, J., Stephen Downie, J., Cameron Jones, M.: Preliminary analyses of information features provided by users for identifying music. In: Proceedings of the International Conference on Music Information Retrieval (2007)Google Scholar
  15. 15.
    Lozano-Tello, A., Gomez-Perez, A.: ONTOMETRIC: A method to choose the appropriate ontology. Journal of Database Management 15(2), 1–18 (2004)CrossRefGoogle Scholar
  16. 16.
    Maedche, A., Staab, S.: Measuring Similarity between Ontologies. In: Gómez-Pérez, A., Benjamins, V.R. (eds.) EKAW 2002. LNCS (LNAI), vol. 2473, pp. 251–263. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  17. 17.
    IFLA Study Group on the Functional Requirements for Bibliographic Records. Functional requirements for bibliographic records - final report. UBCIM Publications - New Series, vol.19 (September 1998), (last accessed March 2012)
  18. 18.
    Porzel, R., Malaka, R.: A task-based approach for ontology evaluation. In: Proceedings of the ECAI Workshop on Ontology Learning and Population (2004)Google Scholar
  19. 19.
    Raimond, Y., Abdallah, S., Sandler, M., Giasson, F.: The music ontology. In: Proceedings of the International Conference on Music Information Retrieval, pp. 417–422 (September 2007)Google Scholar
  20. 20.
    Raimond, Y., Scott, T., Oliver, S., Sinclair, P., Smethurst, M.: Use of Semantic Web technologies on the BBC Web Sites. In: Linking Enterprise Data, pp. 263–283. Springer (2010)Google Scholar
  21. 21.
    Saxton, M.L., Richardson, J.V.: Understanding reference transactions. Academic Press (May 2002)Google Scholar
  22. 22.
    Sugimoto, C.R.: Evaluating reference transactions in academic music libraries. Master’s thesis, School of Information and Library Science of the University of North Carolina at Chapel Hill (2007)Google Scholar
  23. 23.
    Vrandečić, D., Sure, Y.: How to Design Better Ontology Metrics. In: Franconi, E., Kifer, M., May, W. (eds.) ESWC 2007. LNCS, vol. 4519, pp. 311–325. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  24. 24.
    Zhang, Y., Li, Y.: A user-centered functional metadata evaluation of moving image collections. Journal of the American Society for Information Science and Technology 59(8), 1331–1346 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Yves Raimond
    • 1
  • Mark Sandler
    • 2
  1. 1.BBC R&DUK
  2. 2.Queen Mary, University of LondonUK

Personalised recommendations