DS 2012: Discovery Science pp 254-267 | Cite as

Error-Correcting Output Codes as a Transformation from Multi-Class to Multi-Label Prediction

  • Johannes Fürnkranz
  • Sang-Hyeun Park
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7569)

Abstract

In this paper, we reinterpret error-correcting output codes (ECOCs) as a framework for converting multi-class classification problems into multi-label prediction problems. Different well-known multi-label learning approaches can be mapped upon particular ways of dealing with the original multi-class problem. For example, the label powerset approach obviously constitutes the inverse transformation from multi-label back to multi-class, whereas binary relevance learning may be viewed as the conventional way of dealing with ECOCs, in which each classifier is learned independently of the others. Consequently, we evaluate whether alternative choices for solving the multi-label problem may result in improved performance. This question is interesting because it is not clear whether approaches that do not treat the bits of the code words independently have sufficient error-correcting properties. Our results indicate that a slight but consistent advantage can be obtained with the use of multi-label methods, in particular when longer codes are employed.

This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research 1, 113–141 (2000)MathSciNetGoogle Scholar
  2. 2.
    Armano, G., Chira, C., Hatami, N.: Error-correcting output codes for multi-label text categorization. In: Proceedings of the 3rd Italian Information Retrieval Workshop, Bari, Italy, pp. 26–37 (2012)Google Scholar
  3. 3.
    Berger, A.: Error-correcting output coding for text classification. In: Proceedings of the IJCAI 1999 Workshop on Machine Learning for Information Filtering, Stockholm, Sweden (1999)Google Scholar
  4. 4.
    Bose, R.C., Ray-Chaudhuri, D.K.: On a class of error correcting binary group codes. Information and Control 3(1), 68–79 (1960)MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Bouckaert, R.R., Frank, E., Hall, M., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: WEKA — Experiences with a Java open-source project. Journal of Machine Learning Research 11, 2533–2541 (2010)MATHGoogle Scholar
  6. 6.
    Boutell, M.R., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recognition 37(9), 1757–1771 (2004)CrossRefGoogle Scholar
  7. 7.
    Brinker, K., Hüllermeier, E.: Case-based multilabel ranking. In: Veloso, M.M. (ed.) Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007), Hyderabad, India, pp. 702–707 (2007)Google Scholar
  8. 8.
    Dembczynski, K., Cheng, W., Hüllermeier, E.: Bayes optimal multilabel classification via probabilistic classifier chains. In: Fürnkranz, J., Joachims, T. (eds.) Proceedings of the 27th International Conference on Machine Learning (ICML 2010), Haifa, Israel, pp. 279–286 (2010)Google Scholar
  9. 9.
    Dembczyński, K., Waegeman, W., Cheng, W., Hüllermeier, E.: On label dependence in multi-label classification. In: Zhang, M.L., Tsoumakas, G., Zhou, Z.H. (eds.) Proceedings of the ICML 2010 Workshop on Learning from Multi-Label Data, Haifa, Israel, pp. 5–12 (2010)Google Scholar
  10. 10.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)MATHGoogle Scholar
  11. 11.
    Diplaris, S., Tsoumakas, G., Mitkas, P.A., Vlahavas, I.P.: Protein Classification with Multiple Algorithms. In: Bozanis, P., Houstis, E.N. (eds.) PCI 2005. LNCS, vol. 3746, pp. 448–456. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Elisseeff, A., Weston, J.: A kernel method for multi-labelled classification. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 14 (NIPS 2001), pp. 681–687 (2002)Google Scholar
  13. 13.
    Ferng, C.-S., Lin, H.-T.: Multi-label classification with error-correcting codes. Journal of Machine Learning Research – Proceedings Track 20, 281–295 (2011)Google Scholar
  14. 14.
    Fürnkranz, J., Hüllermeier, E., Loza Mencía, E., Brinker, K.: Multilabel classification via calibrated label ranking. Machine Learning 73(2), 133–153 (2008)CrossRefGoogle Scholar
  15. 15.
    Gama, J., Brazdil, P.: Cascade generalization. Machine Learning 41(3), 315–343 (2000)MATHCrossRefGoogle Scholar
  16. 16.
    Ghani, R.: Using error-correcting codes for text classification. In: Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 303–310. Morgan Kaufmann (2000)Google Scholar
  17. 17.
    Hocquenghem, A.: Codes correcteurs d’erreurs. Chiffres 2, 147–156 (1959) (in French)MathSciNetMATHGoogle Scholar
  18. 18.
    Ji, S., Sun, L., Jin, R., Ye, J.: Multi-label multiple kernel learning. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems 21 (NIPS 2008), pp. 777–784. Curran Associates, Inc., Vancouver (2009)Google Scholar
  19. 19.
    Kang, F., Jin, R., Sukthankar, R.: Correlated label propagation with application to multi-label learning. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), pp. 1719–1726. IEEE Computer Society (2006)Google Scholar
  20. 20.
    Kittler, J., Ghaderi, R., Windeatt, T., Matas, J.: Face verification via error correcting output codes. Image and Vision Computing 21(13-14), 1163–1169 (2003)CrossRefGoogle Scholar
  21. 21.
    Kong, E.B., Dietterich, T.G.: Error-correcting output coding corrects bias and variance. In: Proceedings of the 12th International Conference on Machine Learning (ICML 1995), pp. 313–321. Morgan Kaufmann (1995)Google Scholar
  22. 22.
    Lewis, D.D., Yang, Y., Rose, T.G., Li, F.: RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research 5, 361–397 (2004)Google Scholar
  23. 23.
    Loza Mencía, E., Fürnkranz, J.: Efficient Pairwise Multilabel Classification for Large-Scale Problems in the Legal Domain. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part II. LNCS (LNAI), vol. 5212, pp. 50–65. Springer, Heidelberg (2008), accompanying EUR-Lex dataset available at http://www.ke.tu-darmstadt.de/resources/eurlex CrossRefGoogle Scholar
  24. 24.
    MacWilliams, F.J., Sloane, N.J.A.: The Theory of Error-Correcting Codes. North-Holland Mathematical Library. North Holland (January 1983)Google Scholar
  25. 25.
    Madjarov, G., Gjorgjevikj, D., Deroski, S.: Two stage architecture for multi-label learning. Pattern Recognition 45(3), 1019–1034 (2012)CrossRefGoogle Scholar
  26. 26.
    Melvin, I., Ie, E., Weston, J., Noble, W.S., Leslie, C.: Multi-class protein classification using adaptive codes. Journal of Machine Learning Research 8, 1557–1581 (2007)MathSciNetMATHGoogle Scholar
  27. 27.
    Park, S.H., Fürnkranz, J.: Multi-label classification with label constraints. In: Hüllermeier, E., Fürnkranz, J. (eds.) Proceedings of the ECML PKDD 2008 Workshop on Preference Learning (PL 2008), Antwerp, Belgium, pp. 157–171 (2008)Google Scholar
  28. 28.
    Park, S.-H., Weizsäcker, L., Fürnkranz, J.: Exploiting Code Redundancies in ECOC. In: Pfahringer, B., Holmes, G., Hoffmann, A. (eds.) DS 2010. LNCS, vol. 6332, pp. 266–280. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  29. 29.
    Park, S.H., Fürnkranz, J.: Efficient prediction algorithms for binary decomposition techniques. Data Mining and Knowledge Discovery 24(1), 40–77 (2012)MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. Machine Learning 85(3), 333–359 (2011)CrossRefGoogle Scholar
  31. 31.
    Snoek, C.G., Worring, M., van Gemert, J.C., Geusebroek, J.M., Smeulders, A.W.: The challenge problem for automated detection of 101 semantic concepts in multimedia. In: Proceedings of ACM Multimedia, Santa Barbara, CA, pp. 421–430 (2006)Google Scholar
  32. 32.
    Tsoumakas, G., Katakis, I.: Multi-label classification: An overview. International Journal of Data Warehousing and Mining 3(3), 1–17 (2007)CrossRefGoogle Scholar
  33. 33.
    Tsoumakas, G., Katakis, I., Vlahavas, I.P.: Mining multi-label data. In: Maimon, O., Rokach, L. (eds.) Data Mining and Knowledge Discovery Handbook, 2nd edn., pp. 667–685. Springer (2010)Google Scholar
  34. 34.
    Tsoumakas, G., Katakis, I., Vlahavas, I.P.: Random k-labelsets for multilabel classification. IEEE Transactions on Knowledge and Data Engineering 23(7), 1079–1089 (2011)CrossRefGoogle Scholar
  35. 35.
    Tsoumakas, G., Spyromitros Xioufis, E., Vilcek, J., Vlahavas, I.P.: Mulan: A Java library for multi-label learning. Journal of Machine Learning Research 12, 2411–2414 (2011), http://mulan.sourceforge.net/ MathSciNetGoogle Scholar
  36. 36.
    Windeatt, T., Ghaderi, R.: Coding and decoding strategies for multi-class learning problems. Information Fusion 4(1), 11–21 (2003)CrossRefGoogle Scholar
  37. 37.
    Wolpert, D.H.: Stacked generalization. Neural Networks 5(2), 241–260 (1992)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Zaragoza, J.H., Sucar, J.E., Morales, E.F., Bielza, C.: Larrañaga: Bayesian chain classifiers for multidimensional classification. In: Walsh, T. (ed.) Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI 2011), Barcelona, Spain, pp. 2192–2197 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Johannes Fürnkranz
    • 1
  • Sang-Hyeun Park
    • 1
  1. 1.Department of Computer ScienceTU DarmstadtGermany

Personalised recommendations