Multi-Objective Evaluation of Music Classification

  • Igor Vatolkin
Conference paper
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)


Music classification targets the management of personal music collections or recommendation of new songs. Several steps are required here: feature extraction and processing, selection of the most relevant of them, and training of classification models. The complete classification chain is evaluated by a selected performance measure. Often standard confusion matrix based metrics like accuracy are calculated. However it can be valuable to compare the methods using further metrics depending on the current application scenario. For this work we created a large empirical study for different music categories using several feature sets, processing methods and classification algorithms. The correlation between different metrics is discussed, and the ideas for better algorithm evaluation are outlined.


Mean Square Error Random Forest Music Information Retrieval Music Track Hard Category 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We thank the Klaus Tschira Foundation for financial support.


  1. AllMusicGuide (2010) Webpage. URL, cited Feb 2011
  2. Bischl B, Vatolkin I, Preuss M (2010) Selecting small audio feature sets in music classification by means of asymmetric mutation. In: Proceedings of the 11th International Conference on Parallel Problem Solving From Nature (PPSN), Krakow, pp 314–323Google Scholar
  3. Deb K (2001) Multiobjective optimization using evolutionary algorithms. Wiley-Interscience, Chichester, UKGoogle Scholar
  4. Downie JS (2008) The music information retrieval evaluation exchange (2005-2007): A window into music information retrieval research. Acoust Sci Technol 29(4):247–255CrossRefGoogle Scholar
  5. Duda RO, Hart PE, Stork DG (2001) Pattern classification. Wiley, New YorkzbMATHGoogle Scholar
  6. Hogg RV, Craig AT (1995) Introduction to mathematical statistics. Macmillan, New YorkGoogle Scholar
  7. Lartillot O, Toiviainen P, Eerola T (2008) A Matlab toolbox for music information retrieval. In: Preisach C, Burkhardt H, Schmidt-Thieme L, Decker R (eds) Data analysis, machine learning and applications, studies in classification, data analysis, and knowledge organization. Springer, Berlin, pp 261–268Google Scholar
  8. Lidy T, Rauber A (2005) Evaluation of feature extractors and psycho-acoustic transformations for music genre classification. In: Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR), London, UK, pp 34–41Google Scholar
  9. Martin R, Nagathil A (2009) Cepstral modulation ratio regression (CMRARE) parameters for audio signal analysis and classification. In: Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), Taipei, Taiwan, pp 321–324Google Scholar
  10. Music test database of the CI research group (2010)\#music\_test\_database, cited Feb 2011
  11. Paulus J, Klapuri A (2008) Music structure analysis using a probabilistic fitness measure and an integrated musicological model. In: Bello JP, Chew E, Turnbull D (eds) Proc. of the 9th International Conference on Music Information Retrieval, pp 369–374Google Scholar
  12. Ras ZW, Wieczorkowska A (2010) Advances in music information retrieval. Springer, BerlinCrossRefGoogle Scholar
  13. Sokolova M, Japkowicz N, Szpakowicz S (2006) Beyond accuracy, F-score and ROC: A family of discriminant measures for performance evaluation. In: Sattar A, Kang BH (eds) Proceedings of the AAAI’06 workshop on Evaluation Methods for Machine Learning, Springer, Berlin, pp 1015–1021Google Scholar
  14. Theimer W, Vatolkin I, Eronen A (2008) Definitions of Audio Features for Music Content Description, Algorithm Engineering Report TR08-2-001. Technische Universität Dortmund, GermanyGoogle Scholar
  15. Tzanetakis G, Cook P (2002) Musical genre classification of audio signals. IEEE Trans Speech Audio Process 10:293–302CrossRefGoogle Scholar
  16. Vatolkin I, Theimer W, Rudolph G (2009) Design and comparison of different evolution strategies for feature selection and consolidation in music classification. In: Proceedings of the 2009 IEEE Congress on Evolutionary Computation (CEC 2009), IEEE Press, Piscataway, NJ, pp 174–181Google Scholar
  17. Witten IH, Frank E (2005) Data mining. Elsevier, AmsterdamzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.TU DortmundDortmundGermany

Personalised recommendations