Multi-label Active Learning with Error Correcting Output Codes

  • Ningzhao Sun
  • Jincheng Shan
  • Chenping HouEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11440)


Due to the demand of practical problems, multi-label learning has become an important research where each instance belongs to multiple classes. Compared with single-label problem, the labeling cost for multi-label one is rather expensive because of the diversity and non-uniqueness of the labels. Therefore, the active learning which reduces the cost by selecting the most valuable data to query the labels attracts a lot of interests. Although several multi-label active learning (MLAL) methods were proposed, they often identify the label merely through a classifier via one-versus-all (OVA) strategy for each class, which makes the classification model very fragile, thus having a serious impact on the later selection criteria. In this paper, we utilize a new multi-label Error Correcting Output Codes (ECOC) method which determines the label of an instance on each class by combining multiple classifiers. This makes our classification model has a good ability of error-correcting and thus ensures the effectiveness of evaluation information in the selection process. Then we combine two effective selection strategies, the margin prediction uncertainty and label cardinality inconsistency, to complement each other and select the most informative instance. Based on this combination, we propose a novel MLAL framework, termed Multi-label Active Learning with Error Correcting Output Codes (MAOC). Experiments on multiple benchmark multi-label datasets demonstrate the efficacy of the combination in proposed approach.


Active learning Multi-label classification Error Correcting Output Codes 



This work was supported by the National Natural Science Foundation of China (No. 61473302, 61503396). Chenping Hou is the corresponding author of this paper.


  1. 1.
    Li, X., Guo, Y.: Active learning with multi-label SVM classification. In: Proceedings of the 23rd International Joint Conference on Artificial Intelligence, pp. 1479–1485 (2013)Google Scholar
  2. 2.
    Guo, Y., Schuurmans, D.: Discriminative batch mode active learning. In: Proceedings of Advances in Neural Information Processing Systems, pp. 593–600 (2008)Google Scholar
  3. 3.
    Yang, Y., Ma, Z., Nie, F., Chang, X., Hauptmann, A.G.: Multi-class active learning by uncertainty sampling with diversity maximization. Int. J. Comput. Vis. 113(2), 113–127 (2014)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Li, X., Wang, L., Sung, E.: Multi-label SVM active learning for image classification. In: International Conference on Image Processing, pp. 2207–2210 (2004)Google Scholar
  5. 5.
    Yang, B., Sun, J., Wang, T., Chen, Z.: Effective multi-label active learning for text classification. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 917–926 (2009)Google Scholar
  6. 6.
    Huang, S., Zhou, Z.: Active query driven by uncertainty and diversity for incremental multi-label learning. In: Proceedings of the 13th IEEE International Conference on Data Mining, pp. 1079–1084 (2013)Google Scholar
  7. 7.
    Lewis, D., Catlett, J.: Heterogeneous uncertainty sampling for supervised learning. In: Proceedings of the International Conference on Machine Learning, pp. 148–156 (1994)Google Scholar
  8. 8.
    Brinker, K.: On active learning in multi-label classification. In: Spiliopoulou, M., Kruse, R., Borgelt, C., Nürnberger, A., Gaul, W. (eds.) From Data and Information Analysis to Knowledge Engineering. STUDIES CLASS, pp. 206–213. Springer, Heidelberg (2006). Scholar
  9. 9.
    Singh, M., Curran, E., Cunningham, P.: Active learning for multi-label image annotation. In: Proceedings of the 19th Irish Conference on Artificial Intelligence and Cognitive Science, pp. 173–182 (2009)Google Scholar
  10. 10.
    Duygulu, P., Barnard, K., de Freitas, J.F.G., Forsyth, D.A.: Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 97–112. Springer, Heidelberg (2002). Scholar
  11. 11.
    Zhang, M., Zhou, Z.: ML-KNN: a lazy learning approach to multi-label learning. In: Pattern Recognition, pp. 2038–2048 (2007)Google Scholar
  12. 12.
    Pestian, J.P., et al.: A shared task involving multi-label classification of clinical free text. In: Proceedings of the Workshop on BioNLP 2007: Biological, Translational, and Clinical Language Processing, pp. 97–104 (2007)Google Scholar
  13. 13.
    Boutell, M., Luo, J., Shen, X., Brown, C.: Learning multi-label scene classification. Pattern Recogn. 37, 1757–1771 (2004)CrossRefGoogle Scholar
  14. 14.
    Srivastava, A., Zane-Ulman, B.: Discovering recurring anomalies in text reports regarding complex space systems. In: IEEE Aerospace Conference (2005)Google Scholar
  15. 15.
    Elisseeff, A., Weston, J.: A kernel method for multi-labelled classification. In: Advances in Neural Information Processing Systems, pp. 681–687 (2001)Google Scholar
  16. 16.
    Huang, S., Jin, R., Zhou, Z.: Active learning by querying informative and representative examples. IEEE Trans. Pattern Anal. Mach. Intell. 36(10), 1936–1949 (2014)CrossRefGoogle Scholar
  17. 17.
    Fan, R., Chang, K., Hsieh, C., Wang, X., Lin, C.: LIBLINEAR: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)zbMATHGoogle Scholar
  18. 18.
    Zhang, M., Zhou, Z.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.National University of Defense TechnologyChangshaChina

Personalised recommendations