Multi-label LeGo — Enhancing Multi-label Classifiers with Local Patterns

  • Wouter Duivesteijn
  • Eneldo Loza Mencía
  • Johannes Fürnkranz
  • Arno Knobbe
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7619)


The straightforward approach to multi-label classification is based on decomposition, which essentially treats all labels independently and ignores interactions between labels. We propose to enhance multi-label classifiers with features constructed from local patterns representing explicitly such interdependencies. An Exceptional Model Mining instance is employed to find local patterns representing parts of the data where the conditional dependence relations between the labels are exceptional. We construct binary features from these patterns that can be interpreted as partial solutions to local complexities in the data. These features are then used as input for multi-label classifiers. We experimentally show that using such constructed features can improve the classification performance of decompositive multi-label learning techniques.


Exceptional Model Mining Multi-Label Classification LeGo 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tsoumakas, G., Katakis, I., Vlahavas, I.P.: Mining Multi-label Data. In: Data Mining and Knowledge Discovery Handbook, pp. 667–685. Springer (2010)Google Scholar
  2. 2.
    Paine, R.T.: Food Web Complexity and Species Diversity. The American Naturalist 100(910), 65–75 (1966)CrossRefGoogle Scholar
  3. 3.
    Fürnkranz, J., Knobbe, A. (eds.): Special Issue: Global Modeling Using Local Patterns. Data Mining and Knowledge Discovery Journal 20 (1) (2010)Google Scholar
  4. 4.
    Leman, D., Feelders, A., Knobbe, A.: Exceptional Model Mining. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part II. LNCS (LNAI), vol. 5212, pp. 1–16. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Duivesteijn, W., Knobbe, A., Feelders, A., van Leeuwen, M.: Subgroup Discovery meets Bayesian networks – an Exceptional Model Mining approach. In: Proc. ICDM, pp. 158–167 (2010)Google Scholar
  6. 6.
    Duivesteijn, W., Loza Mencía, E., Fürnkranz, J., Knobbe, A.: Multi-label LeGo — Enhancing Multi-label Classifiers with Local Patterns. Technical Report, Technische Universität Darmstadt, TUD-KE-2012-02 (2012),
  7. 7.
    Knobbe, A., Valkonet, J.: Building Classifiers from Pattern Teams. In: From Local Patterns to Global Models: Proc. ECML PKDD 2009 Workshop, Slovenia (2009)Google Scholar
  8. 8.
    Sulzmann, J.-N., Fürnkranz, J.: A Comparison of Techniques for Selecting and Combining Class Association Rules. In: From Local Patterns to Global Models: Proc. ECML PKDD 2008 Workshop, Belgium (2008)Google Scholar
  9. 9.
    Cheng, W., Hüllermeier, E.: Combining instance-based learning and logistic regression for multilabel classification. Machine Learning 76(2-3), 211–225 (2009)CrossRefGoogle Scholar
  10. 10.
    Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier Chains for Multi-label Classification. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds.) ECML PKDD 2009, Part II. LNCS, vol. 5782, pp. 254–269. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  11. 11.
    Fürnkranz, J., Hüllermeier, E., Loza Mencía, E., Brinker, K.: Multilabel Classification via Calibrated Label Ranking. Machine Learning 73(2), 133–153 (2008)CrossRefGoogle Scholar
  12. 12.
    Shapiro, L.G., Haralick, R.M.: A Metric for Comparing Relational Descriptions. IEEE Trans. Pattern Anal. Mach. Intell. 7, 90–94 (1985)CrossRefGoogle Scholar
  13. 13.
    Verma, T., Pearl, J.: Equivalence and Synthesis of Causal Models. In: Proc. UAI, pp. 255–270 (1990)Google Scholar
  14. 14.
    Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar
  15. 15.
    Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines, Software,
  16. 16.
    Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., Lin, C.-J.: LIBLINEAR: A Library for Large Linear Classification. Journal of Machine Learning Research 9, 1871–1874 (2008)zbMATHGoogle Scholar
  17. 17.
    Demšar, J.: Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research 7, 1–30 (2006)zbMATHGoogle Scholar
  18. 18.
    van Leeuwen, M., Knobbe, A.: Non-redundant Subgroup Discovery in Large and Complex Data. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011, Part III. LNCS, vol. 6913, pp. 459–474. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  19. 19.
    Sechidis, K., Tsoumakas, G., Vlahavas, I.: On the Stratification of Multi-label Data. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011, Part III. LNCS, vol. 6913, pp. 145–158. Springer, Heidelberg (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Wouter Duivesteijn
    • 1
  • Eneldo Loza Mencía
    • 2
  • Johannes Fürnkranz
    • 2
  • Arno Knobbe
    • 1
  1. 1.LIACSLeiden UniversityThe Netherlands
  2. 2.Knowledge Engineering GroupTU DarmstadtGermany

Personalised recommendations