Searching for Music Using Natural Language Queries and Relevance Feedback

  • Peter Knees
  • Gerhard Widmer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4918)


We extend an approach to search inside large-scale music collections by enabling the user to give feedback on the retrieved music pieces. In the original approach, a search engine that can be queried through free-form natural language text is automatically built upon audio-based and Web-based similarity measures. Features for music pieces in the collection are derived automatically by retrieving relevant Web pages via Google queries and using the contents of these pages to construct term vectors. The additional use of information about acoustic similarity allows for reduction of the dimensionality of the vector space and characterization of audio pieces with no associated Web information. With the incorporation of relevance feedback, the retrieval of pieces can be adapted according to the preferences of the user and thus compensate for inadequately represented initial queries. The approach is evaluated on a collection comprising about 12,000 pieces by using semantic tags provided by Audioscrobbler and a user study which also gives further insights into users search behaviors.


Relevance Feedback Music Piece Music Information Retrieval Query Vector Music Collection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Knees, P., Pohle, T., Schedl, M., Widmer, G.: A Music Search Engine Built upon Audio-based and Web-based Similarity Measures. In: Proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2007), Amsterdam, The Netherlands (2007)Google Scholar
  2. 2.
    Maddage, N.C., Li, H., Kankanhalli, M.S.: Music structure based vector space retrieval. In: SIGIR 2006: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval (2006)Google Scholar
  3. 3.
    Ghias, A., Logan, J., Chamberlin, D., Smith, B.C.: Query by humming: musical information retrieval in an audio database. In: Proceedings of the 3rd ACM International Conference on Multimedia (MULTIMEDIA 1995), San Francisco, California, United State (1995)Google Scholar
  4. 4.
    Baumann, S., Klüter, A., Norlien, M.: Using natural language input and audio analysis for a human-oriented MIR system. In: Proceedings of the 2nd International Conference on Web Delivering of Music (WEDELMUSIC 2002), Darmstadt, Germany (2002)Google Scholar
  5. 5.
    Celma, O., Cano, P., Herrera, P.: Search Sounds: An audio crawler focused on weblogs. In: Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR 2006), Victoria, B.C., Canada (2006)Google Scholar
  6. 6.
    Celino, I., Della Valle, E., Cerizza, D., Turati, A.: Squiggle: A semantic search engine for indexing and retrieval of multimedia content. In: Proceedings of the 1st International Workshop on Semantic-enhanced Multimedia Presentation Systems (SEMPS 2006), Athens, Greece (2006)Google Scholar
  7. 7.
    Knopke, I.: AROOOGA: An audio search engine for the World Wide Web. In: Proceedings of the 2004 International Computer Music Conference (ICMC 2004), Miami, USA (2004)Google Scholar
  8. 8.
    Whitman, B., Lawrence, S.: Inferring Descriptions and Similarity for Music from Community Metadata. In: Proceedings of the 2002 International Computer Music Conference (ICMC 2002), Gotheborg, Sweden, pp. 591–598 (2002)Google Scholar
  9. 9.
    Knees, P., Pampalk, E., Widmer, G.: Artist Classification with Web-based Data. In: Proceedings of 5th International Conference on Music Information Retrieval (ISMIR 2004), Barcelona, Spain, pp. 517–524 (2004)Google Scholar
  10. 10.
    Salton, G., Buckley, C.: Term-weighting approaches in automatic text retrieval. Information Processing and Management 24(5), 513–523 (1988)CrossRefGoogle Scholar
  11. 11.
    Aucouturier, J.J., Pachet, F., Sandler, M.: The Way It Sounds: Timbre Models for Analysis and Retrieval of Music Signals. IEEE Transactions on Multimedia 7(6), 1028–1035 (2005)CrossRefGoogle Scholar
  12. 12.
    Pampalk, E.: Computational Models of Music Similarity and their Application to Music Information Retrieval. PhD thesis, Vienna University of Technology (2006)Google Scholar
  13. 13.
    Aucouturier, J.J., Pachet, F.: Music Similarity Measures: What’s the Use? In: Proceedings of the 3rd International Conference on Music Information Retrieval (ISMIR 2002), Paris, France, IRCAM, pp. 157–163 (2002)Google Scholar
  14. 14.
    Mandel, M., Ellis, D.: Song-Level Features and Support Vector Machines for Music Classification. In: Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR 2005), London, UK (2005)Google Scholar
  15. 15.
    Aucouturier, J.J.: Ten Experiments on the Modelling of Polyphonic Timbre. PhD thesis, University of Paris 6 (2006)Google Scholar
  16. 16.
    Pohle, T., Knees, P., Schedl, M., Widmer, G.: Automatically Adapting the Structure of Audio Similarity Spaces. In: Proceedings of 1st Workshop on Learning the Semantics of Audio Signals (LSAS 2006), Athens, Greece (2006)Google Scholar
  17. 17.
    Yang, Y., Pedersen, J.O.: A comparative study on feature selection in text categorization. In: Fisher, D.H. (ed.) Proceedings of ICML 1997, 14th International Conference on Machine Learning, Nashville, US, pp. 412–420. Morgan Kaufman Publishers, San Francisco, US (1997)Google Scholar
  18. 18.
    Rocchio, J.J.: Relevance feedback in information retrieval. In: Salton, G. (ed.) The SMART Retrieval System – Experiments in Automatic Document Processing, Prentice Hall Inc., Englewood Cliffs, NJ, USA (1971)Google Scholar
  19. 19.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley, Reading, Massachusetts (1999)Google Scholar
  20. 20.
    Harman, D.: Relevance Feedback Revisited. In: Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR 1992), Copenhagen, Denmark (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Peter Knees
    • 1
  • Gerhard Widmer
    • 1
    • 2
  1. 1.Dept. of Computational PerceptionJohannes Kepler UniversityLinzAustria
  2. 2.Austrian Research Institute for Artificial Intelligence (OFAI)Austria

Personalised recommendations