Learning Tree Structure of Label Dependency for Multi-label Learning

  • Bin Fu
  • Zhihai Wang
  • Rong Pan
  • Guandong Xu
  • Peter Dolog
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7301)


There always exists some kind of label dependency in multi-label data. Learning and utilizing those dependencies could improve the learning performance further. Therefore, an approach for multi-label learning is proposed in this paper, which quantifies the dependencies of pairwise labels firstly, and then builds a tree structure of the labels to describe them. Thus the approach could find out potential strong label dependencies and produce more generalized dependent relationships. The experimental results have validated that compared with other state-of-the-art algorithms, the method is not only a competitive alternative, but also has shown better performance after ensemble learning especially.


classification multi-label instance multi-label learning label dependency 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cheng, W., Hullermeier, E.: Combining Instance-Based Learning and Logistic Regression for Multilabel Classification. Machine Learning 76(2-3), 211–225 (2009)CrossRefGoogle Scholar
  2. 2.
    Tsoumakas, G., Katakis, I., Vlahavas, I.: Mining Multi-label Data. In: Oded, M., Lior, R. (eds.) Data Mining and Knowledge Discovery Handbook, pp. 667–685. Springer, New York (2010)Google Scholar
  3. 3.
    McCallum, A.K.: Multi-label Text Classification with a Mixture Model Trained by EM. In: Proceedings of AAAI 1999 Workshop on Text Learning (1999)Google Scholar
  4. 4.
    Schapire, R.E., Singer, Y.: Boostexter: a Boosting-Based System for Text Categorization. Machine Learning 39(2-3), 135–168 (2000)zbMATHCrossRefGoogle Scholar
  5. 5.
    Clare, A.J., King, R.D.: Knowledge Discovery in Multi-label Phenotype Data. In: Siebes, A., De Raedt, L. (eds.) PKDD 2001. LNCS (LNAI), vol. 2168, pp. 42–53. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  6. 6.
    Thabtah, F.A., Cowling, P., Peng, Y.: MMAC: a New Multi-class, Multi-label Associative Classification Approach. In: Proceedings of the 4th International Conference on Data Mining, pp. 217–224 (2004)Google Scholar
  7. 7.
    Zhang, M., Zhou, Z.: ML-KNN: A Lazy Learning Approach to Multi-label Learning. Pattern Recognition 7(40), 2038–2048 (2007)CrossRefGoogle Scholar
  8. 8.
    Read, J.: Multi-label Classification using Ensembles of Pruned Sets. In: Proceedings of the IEEE International Conference on Data Mining, pp. 995–1000. IEEE Computer Society, Washington, DC (2008)Google Scholar
  9. 9.
    Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier Chains for Multi-label Classification. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds.) ECML PKDD 2009, Part II. LNCS, vol. 5782, pp. 254–269. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  10. 10.
    Dembczynski, K., Cheng, W., Hullermeier, E.: Bayes Optimal Multilabel Classification via Probabilistic Classifier Chains. In: Proceedings of the 27th International Conference on Machine Learning, pp. 279–286. Omnipress (2010)Google Scholar
  11. 11.
    Zhang, M., Zhang, K.: Multi-label Learning by Exploiting Label Dependency. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 999–1000. ACM Press, Washington, DC (2010)CrossRefGoogle Scholar
  12. 12.
    Zhang, Y., Zhou, Z.: Multi-label Dimensionality Reduction via Dependence Maximization. ACM Transactions on Knowledge Discovery from Data 4(3), 1–21 (2010)CrossRefGoogle Scholar
  13. 13.
    Boutell, M.R., Luo, J., Shen, X.: Learning Multi-label Scene Classification. Pattern Recognition 37(9), 1757–1771 (2004)CrossRefGoogle Scholar
  14. 14.
    Tsoumakas, G., Katakis, I., Vlahavas, I.: Effective and Efficient Multilabel Classification in Domains with Large Number of Labels. In: Proceedings of ECML/PKDD 2008 Workshop on Mining Multidimensional Data, pp. 30–44 (2008)Google Scholar
  15. 15.
    Hullermeier, E., Furnkranz, J., Cheng, W.: Label Ranking by Learning Pairwise Preferences. Artificial Intelligence 172(16-17), 1897–1916 (2008)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Furnkranz, J., Hullermeier, E., Mencia, E.L.: Multilabel Classification via Calibrated Label Ranking. Machine Learning 2(73), 133–153 (2008)CrossRefGoogle Scholar
  17. 17.
    Madjarov, G., Gjorgjevikj, D., Dzeroski, S.: Two Stage Architecture for Multi-label learning. Pattern Recognition 45(3), 1019–1034 (2011)CrossRefGoogle Scholar
  18. 18.
    Tsoumakas, G., Katakis, I., Vlahavas, I.: Random k-labelsets for Multi-label Classification. IEEE Transactions On Knowledge and Data Engineering 23(7), 1079–1089 (2011)CrossRefGoogle Scholar
  19. 19.
    Tsoumakas, G., Dimou, A., Spyromitros, E.: Correlation-Based Pruning of Stacked Binary Relevance Models for Multi-Label Learning. In: Proceeding of ECML/PKDD 2009 Workshop on Learning from Multi-Label Data, Bled, Slovenia, pp. 101–116 (2009)Google Scholar
  20. 20.
    Gaag, L., Waal, P.: Multi-dimensional Bayesian Network Classifiers. In: Third European Workshop on Probabilistic Graphical Models, pp. 107–114 (2006)Google Scholar
  21. 21.
    Bielza, C., Li, G., Larranage, P.: Multi-dimensional Classification with Bayesian Networks. International Journal of Approximate Reasoning 52(6), 705–727 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Guo, Y., Gu, S.: Multi-label Classification using Conditional Dependency Networks. In: Proceedings of the 22nd International Joint Conference on Artificial Intelligence, pp. 1300–1305 (2011)Google Scholar
  23. 23.
    Ghamrawi, N., McCallum, A.K.: Collective Multi-label Classification. In: Proceedings of the 2005 ACM Conference on Information and Knowledge Management, pp. 195–200 (2005)Google Scholar
  24. 24.
    Dembczynski, K., Waegeman, W., Cheng, W.: On Label Dependence in Multi-label Classification. In: Proceedings of the 2nd International Workshop on Learning From Multi-label Data, pp. 5–12 (2010)Google Scholar
  25. 25.
    Chow, C.K., Liu, C.N.: Approximating Discrete Probability Distributions with Dependency Trees. IEEE Transactions on Information Theory 14(3), 462–467 (1968)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Tsoumakas, G., Spyromitros-Xioufis, E., Vilcek, J., Vlahavas, I.: Mulan: A Java Library for Multi-Label Learning. Journal of Machine Learning Research 12, 2411–2414 (2011)MathSciNetGoogle Scholar
  27. 27.
    Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Bin Fu
    • 1
  • Zhihai Wang
    • 1
  • Rong Pan
    • 2
  • Guandong Xu
    • 3
  • Peter Dolog
    • 2
  1. 1.School of Computer and Information TechnologyBeijing Jiaotong UniversityBeijingChina
  2. 2.Department of Computer ScienceAalborg UniversityDenmark
  3. 3.School of Engineering & ScienceVictoria UniversityAustralia

Personalised recommendations