Selecting a Multi-Label Classification Method for an Interactive System

  • Noureddine-Yassine NAIR-BENREKIAEmail author
  • Pascale Kuntz
  • Frank Meyer
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)


Interactive classification-based systems engage users to coach learning algorithms to take into account their own individual preferences. However most of the recent interactive systems limit the users to a single-label classification, which may be not expressive enough in some organization tasks such as film classification, where a multi-label scheme is required. The objective of this paper is to compare the behaviors of 12 multi-label classification methods in an interactive framework where “good” predictions must be produced in a very short time from a very small set of multi-label training examples. Experimentations highlight important performance differences for four complementary evaluation measures (Log-Loss, Ranking-Loss, Learning and Prediction Times). The best results are obtained for Multi-label k Nearest Neighbors (ML-kNN), ensemble of classifier chains (ECC), and ensemble of binary relevance (EBR).


Ensemble Method Training Subset Binary Relevance Classifier Chain Relevant Label 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Amershi, S., Fogarty, J., & Weld, D. (2012). Regroup: Interactive machine learning for on-demand group creation in social networks. In Proceeding of the SIGCHI Conference on Human Factors in Computing Systems (pp. 21–30). New York: ACM.Google Scholar
  2. Drucker, S. M., Fisher, D., & Basu, S. (2011). Helping users sort faster with adaptive machine learning recommendations. In Human-Computer Interaction-INTERACT (pp. 187–203). Berlin/Heidelberg: Springer.Google Scholar
  3. Fürnkranz, J., Hüllermeier, E., Mencía, E. L., & Brinker, K. (2008). Multilabel classification via calibrated label ranking. Machine Learning, 73(2), 133–153.CrossRefGoogle Scholar
  4. Hearst, M. A., Dumais, S. T., Osman, E., Platt, J., & Scholkopf, B. (1998). Support vector machines. IEEE Intelligent Systems and Their Applications, 13(4), 18–28.CrossRefGoogle Scholar
  5. Hüllermeier, E., Fürnkranz, J., Cheng, W., & Brinker, K. (2008). Label ranking by learning pairwise preferences. Artificial Intelligence, 172(16), 1897–1916.CrossRefzbMATHMathSciNetGoogle Scholar
  6. Madjarov, G., Kocev, D., Gjorgjevikj, D., & Džeroski, S. (2012). An extensive experimental comparison of methods for multi-label learning. Pattern Recognition, 45(9), 3084–3104.CrossRefGoogle Scholar
  7. Quinlan, J. R. (1993). C4. 5: Programs for machine learning. San Mateo: Morgan kaufmann.Google Scholar
  8. Read, J. (2010). Scalable multi-label classification. Doctoral dissertation, University of Waikato.Google Scholar
  9. Read, J., Pfahringer, B., Holmes, G., & Frank, E. (2009). Classifier chains for multi-label classification. Machine Learning, 85(3), 333–359.CrossRefMathSciNetGoogle Scholar
  10. Ritter, A., & Basu, S. (2009). Learning to generalize for complex selection tasks. In Proceeding of the 14th International Conference on Intelligent User Interfaces (pp. 167–176). New York: ACM.Google Scholar
  11. Schapire, R. E., & Singer, Y. (2000). BoosTexter: A boosting-based system for text categorization. Machine learning, 39(2–3), 135–168.CrossRefzbMATHGoogle Scholar
  12. Tsoumakas, G., & Katakis, I. (2007). Multi-label classification: An overview. International Journal of Data Warehousing (IJDWM), 3(3), 1–13.CrossRefGoogle Scholar
  13. Tsoumakas, G., Katakis, I., & Vlahavas, I. (2008). Effective and efficient multilabel classification in domains with large number of labels. In Proceedings of ECML/PKDD 2008 Workshop on Mining Multidimensional Data (MMD’08) (pp. 30–44).Google Scholar
  14. Tsoumakas, G., & Vlahavas, I. (2007). Random k-labelsets: An ensemble method for multilabel classification. In Machine learning: ECML 2007 (pp. 406–417). Berlin/Heidelberg: Springer.CrossRefGoogle Scholar
  15. Ware, M., Frank, E., Holmes, G., Hall, M., & Witten, I. H. (2001). Interactive machine learning: Letting users build classifiers. International Journal of Human-Computer Studies, 55(3), 281–292.CrossRefzbMATHGoogle Scholar
  16. Zhang, M. L., & Zhou, Z. H. (2007). ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognition, 40(7), 2038–2048.CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Noureddine-Yassine NAIR-BENREKIA
    • 1
    • 2
    Email author
  • Pascale Kuntz
    • 2
  • Frank Meyer
    • 1
  1. 1.Orange LabsLannion cedexFrance
  2. 2.LINANANTES cedexFrance

Personalised recommendations