Stacking Label Features for Learning Multilabel Rules

  • Eneldo Loza Mencía
  • Frederik Janssen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8777)


Dependencies between the labels is commonly regarded as the crucial issue in multilabel classification. Rules provide a natural way for symbolically describing such relationships, for instance, rules with label tests in the body allow for representing directed dependencies like implications, subsumptions, or exclusions. Moreover, rules naturally allow to jointly capture both local and global label dependencies.

We present a bootstrapped stacking approach which uses a common rule learner in order to induce label-dependent rules. For this, we learn for each label a separate ruleset, but we include the remaining labels as additional attributes in the training instances. Proceeding this way, label dependencies can be made explicit in the rules. Our experiments show competitive results in terms of the standard multilabel evaluation measures. But more importantly, using these additional attributes is shown to allow to discover and consider label relations as well as to better comprehend the available multilabel datasets.

However, this approach is only a first step towards integrating the multilabel rule learning directly in the rule induction process, e.g., in typical separate-and-conquer rule learners. We present future perspectives, advantages, and arising issues in this regard.


Association Rule Frequent Itemsets Inductive Logic Programming Rule Induction Instance Feature 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allamanis, M., Tzima, F.A., Mitkas, P.A.: Effective Rule-Based Multi-label Classification with Learning Classifier Systems. In: Tomassini, M., Antonioni, A., Daolio, F., Buesser, P. (eds.) ICANNGA 2013. LNCS, vol. 7824, pp. 466–476. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  2. 2.
    Ávila-Jiménez, J.L., Gibaja, E., Ventura, S.: Evolving Multi-label Classification Rules with Gene Expression Programming: A Preliminary Study. In: Corchado, E., Graña Romay, M., Manhaes Savio, A. (eds.) HAIS 2010, Part II. LNCS, vol. 6077, pp. 9–16. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    Cohen, W.W.: Fast Effective Rule Induction. In: Proceedings of the 12th International Conference on Machine Learning (ICML 1995), pp. 115–123 (1995)Google Scholar
  4. 4.
    Malerba, D., Semeraro, G., Esposito, F.: A multistrategy approach to learning multiple dependent concepts. In: Machine Learning and Statistics: The Interface, ch. 4, pp. 87–106 (1997)Google Scholar
  5. 5.
    Dembczyński, K., Waegeman, W., Cheng, W., Hüllermeier, E.: On label dependence and loss minimization in multi-label classification. Machine Learning 88(1-2), 5–45 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Fürnkranz, J.: Separate-and-Conquer Rule Learning. Artificial Intelligence Review 13(1), 3–54 (1999)CrossRefzbMATHGoogle Scholar
  7. 7.
    Ghamrawi, N., McCallum, A.: Collective multi-label classification. In: CIKM 2005: Proceedings of the 14th ACM International Conference on Information and Knowledge Management, pp. 195–200. ACM (2005)Google Scholar
  8. 8.
    Godbole, S., Sarawagi, S.: Discriminative methods for multi-labeled classification. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 22–30. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Guo, Y., Gu, S.: Multi-label classification using conditional dependency networks. In: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, IJCAI 2011, vol. 2, pp. 1300–1305. AAAI Press (2011)Google Scholar
  10. 10.
    Li, B., Li, H., Wu, M., Li, P.: Multi-label Classification based on Association Rules with Application to Scene Classification. In: Proceedings of the 2008 the 9th International Conference for Young Computer Scientists, pp. 36–41. IEEE Computer Society (2008)Google Scholar
  11. 11.
    Loza Mencía, E., Janssen, F.: Towards multilabel rule learning. In: Proceedings of the German Workshop on Lernen, Wissen, Adaptivität - LWA 2013, pp. 155–158 (2013)Google Scholar
  12. 12.
    McCallum, A.K.: Multi-label text classification with a mixture model trained by EM. In: AAAI 1999 Workshop on Text Learning (1999)Google Scholar
  13. 13.
    Montañés, E., Senge, R., Barranquero, J., Quevedo, J.R., del Coz, J.J., Hüllermeier, E.: Dependent binary relevance models for multi-label classification. Pattern Recognition 47(3), 1494–1508 (2014)CrossRefGoogle Scholar
  14. 14.
    Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. Machine Learning 85(3), 333–359 (2011)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Thabtah, F., Cowling, P., Peng, Y.: MMAC: A New Multi-Class, Multi-Label Associative Classification Approach. In: Proceedings of the 4th IEEE ICDM, pp. 217–224 (2004)Google Scholar
  16. 16.
    Tsoumakas, G., Katakis, I., Vlahavas, I.P.: Mining Multi-label Data. In: Data Mining and Knowledge Discovery Handbook, pp. 667–685 (2010)Google Scholar
  17. 17.
    Zhu, S., Ji, X., Xu, W., Gong, Y.: Multi-labelled classification using maximum entropy method. In: SIGIR 2005: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 274–281. ACM (2005)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Eneldo Loza Mencía
    • 1
  • Frederik Janssen
    • 1
  1. 1.Knowledge Engineering GroupTechnische Universität DarmstadtGermany

Personalised recommendations