Active Learning Algorithms for Multi-label Data

  • Everton Alvares ChermanEmail author
  • Grigorios Tsoumakas
  • Maria-Carolina Monard
Conference paper
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 475)


Active learning is an iterative supervised learning task where learning algorithms can actively query an oracle, i.e. a human annotator that understands the nature of the pro blem, for labels. As the learner is allowed to interactively choose the data from which it learns, it is expected that the learner will perform better with less training. The active learning approach is appropriate to machine learning applications where training labels are costly to obtain but unlabeled data is abundant. Although active learning has been widely considered for single-label learning, this is not the case for multi-label learning, where objects can have more than one class labels and a multi-label learner is trained to assign multiple labels simultaneously to an object. We discuss the key issues that need to be considered in pool-based multi-label active learning and discuss how existing solutions in the literature deal with each of these issues. We further empirically study the performance of the existing solutions, after implementing them in a common framework, on two multi-label datasets with different characteristics and under two different applications settings (transductive, inductive). We find out interesting results that we attribute to the properties of, mainly, the data sets, and, secondarily, the application settings.


Supervised learning Multi-label learning Active learning Pool-based strategies 



This research was supported by the São Paulo Research Foundation (FAPESP), grants 2010/15992-0 and 2011/21723-5, and Brazilian National Council for Scientific and Technological Development (CNPq), grant 644963.


  1. 1.
    Brinker, K.: On active learning in multi-label classification. In: Spiliopoulou, M., Kruse, R., Borgelt, C., Nurnberger, A., Gaul, W. (eds.) From Data and Information Analysis to Knowledge Engineering. Studies in Classification, Data Analysis, and Knowledge Organization, pp. 206–213. Springer, Heidelberg (2006)Google Scholar
  2. 2.
    Esuli, A., Sebastiani, F.: Active learning strategies for multi-label text classification. In: Boughanem, M., Berrut, C., Mothe, J., Soule-Dupuy, C. (eds.) ECIR 2009. LNCS, vol. 5478, pp. 102–113. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. 3.
    Hung, C.W., Lin, H.T.: Multi-label active learning with auxiliary learner. In: 3rd Asian Conference on Machine Learning, Taoyuan, Taiwan (2011)Google Scholar
  4. 4.
    Nowak, S., Nagel, K., Liebetrau, J.: The CLEF 2011 photo annotation and concept-based retrieval tasks. In: CLEF (Notebook Papers/Labs/Workshop), pp. 1–25 (2011)Google Scholar
  5. 5.
    Qi, G.J., Hua, X.S., Rui, Y., Tang, J., Zhang, H.J.: Two-dimensional multilabel active learning with an efficient online adaptation model for image classification. IEEE Trans. Pattern Anal. Mach. Intell. 31(10), 1880–1897 (2009). CrossRefGoogle Scholar
  6. 6.
    Settles, B.: Active learning literature survey. Technical report 1648. University of Wisconsin-Madison (2010)Google Scholar
  7. 7.
    Singh, M., Brew, A., Greene, D., Cunningham, P.: Score normalization and aggregation for active learning in multi-label classification. Technical report. University College Dublin (2010)Google Scholar
  8. 8.
    Tsoumakas, G., Spyromitros-Xioufis, E., Vilcek, J., Vlahavas, I.: Mulan: a java library for multi-label learning. J. Mach. Learn. Res. 12, 2411–2414 (2011)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Tsoumakas, G., Zhang, M.L., Zhou, Z.H.: Introduction to the special issue on learning from multi-label data. Mach. Learn. 88(1–2), 1–4 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Yang, B., Sun, J.T., Wang, T., Chen, Z.: Effective multi-label active learning for text classification. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2009, NY, USA, pp. 917–926 (2009)

Copyright information

© IFIP International Federation for Information Processing 2016

Authors and Affiliations

  • Everton Alvares Cherman
    • 1
    Email author
  • Grigorios Tsoumakas
    • 2
  • Maria-Carolina Monard
    • 1
  1. 1.Institute of Mathematics and Computer SciencesUniversity of Sao PauloSao CarlosBrazil
  2. 2.Department of InformaticsAristotle University of ThessalonikiThessalonikiGreece

Personalised recommendations