Advertisement

On the Estimation of Predictive Evaluation Measure Baselines for Multi-label Learning

  • Jean Metz
  • Luís F. D. de Abreu
  • Everton A. Cherman
  • Maria C. Monard
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7637)

Abstract

Machine learning research relies to a large extent on experimental observations. The evaluation of classifiers is often carried out by empirical comparison with classifiers generated by different learning algorithms, allowing the identification of the best algorithm for the problem at hand. Nevertheless, previously to this evaluation, it is important to state if the classifiers have truly learned the domain class concepts, which can be done by comparing the classifiers’ predictive measures with the ones from the baseline classifiers. A baseline classifier is the one constructed by a naïve learning algorithm which only uses the class distribution of the dataset. However, finding naïve classifiers in multi-label learning is not as straightforward as in single-label learning. This work proposes a simple way to find baseline multi-label classifiers. Three specific and one general naïve multi-label classifiers are proposed to estimate the baseline values for multi-label predictive evaluation measures. Experimental results show the suitability of our proposal in revealing the learning power of multi-label learning algorithms.

Keywords

machine learning multi-label classification baseline classifiers 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alpaydin, E.: Introduction to Machine Learning. MITP (2004)Google Scholar
  2. 2.
    Cherman, E.A., Metz, J., Monard, M.C.: Incorporating Label Dependency into the Binary Relevance Framework for Multi-Label Classification. Expert Systems with Applications 39(2), 1647–1655 (2012)CrossRefGoogle Scholar
  3. 3.
    Dembczynski, K., Waegeman, W., Cheng, W., Hüllermeier, E.: On Label Dependence in Multi-Label Classification. In: 2nd International Workshop on Learning from Multi-Label Data (MLD 2010) at the 27th International Conference of Machine Learning (ICML 2010), pp. 5–12 (2010)Google Scholar
  4. 4.
    Demžsar, J.: Statistical comparison of classifiers over multiple data sets. Journal of Machine Learning Research 7(1), 1–30 (2006)MathSciNetGoogle Scholar
  5. 5.
    Feng, S., Xu, D.: Transductive Multi-Instance Multi-Label learning algorithm with application to automatic image annotation. Expert Systems with Applications 37(1), 661–670 (2010)CrossRefGoogle Scholar
  6. 6.
    Qi, G.J., Hua, X.S., Rui, Y., Tang, J., Mei, T., Zhang, H.J.: Correlative Multi-Label Video Annotation. In: 15th International Conference on Multimedia (MULTIMEDIA 2007), pp. 17–26. ACM (2007)Google Scholar
  7. 7.
    Trohidis, K., Tsoumakas, G., Kalliris, G., Vlahavas, I.: Multilabel Classification of Music into Emotions. In: 9th International Conference on Music Information Retrieval (ISMIR 2008), pp. 325–330 (2008)Google Scholar
  8. 8.
    Tsoumakas, G., Katakis, I., Vlahavas, I.: Mining Multi-label Data. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook, pp. 667–685. Springer (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jean Metz
    • 1
  • Luís F. D. de Abreu
    • 1
  • Everton A. Cherman
    • 1
  • Maria C. Monard
    • 1
  1. 1.Institute of Mathematics and Computer Science (ICMC)University of São Paulo in São Carlos (USP/São Carlos)São CarlosBrazil

Personalised recommendations