Bi-directional Representation Learning for Multi-label Classification

  • Xin Li
  • Yuhong Guo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8725)


Multi-label classification is a central problem in many application domains. In this paper, we present a novel supervised bi-directional model that learns a low-dimensional mid-level representation for multi-label classification. Unlike traditional multi-label learning methods which identify intermediate representations from either the input space or the output space but not both, the mid-level representation in our model has two complementary parts that capture intrinsic information of the input data and the output labels respectively under the autoencoder principle while augmenting each other for the target output label prediction. The resulting optimization problem can be solved efficiently using an iterative procedure with alternating steps, while closed-form solutions exist for one major step. Our experiments conducted on a variety of multi-label data sets demonstrate the efficacy of the proposed bi-directional representation learning model for multi-label classification.


  1. 1.
    Chen, Y., Lin, H.: Feature-aware label space dimension reduction for multi-label classification. In: Proceedings of NIPS (2012)Google Scholar
  2. 2.
    Diplaris, S., Tsoumakas, G., Mitkas, P.A., Vlahavas, I.P.: Protein classification with multiple algorithms. In: Bozanis, P., Houstis, E.N. (eds.) PCI 2005. LNCS, vol. 3746, pp. 448–456. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  3. 3.
    Duygulu, P., Barnard, K., de Freitas, J.F.G., Forsyth, D.: Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part IV. LNCS, vol. 2353, pp. 97–112. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  4. 4.
    Elisseeff, A., Weston, J.: A kernel method for multi-labelled classification. In: Proceedings of NIPS (2001)Google Scholar
  5. 5.
    Godbole, S., Sarawagi, S.: Discriminative methods for multi-labeled classification. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 22–30. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Guo, Y., Schuurmans, D.: Adaptive large margin training for multilabel classification. In: Proceedings of AAAI (2011)Google Scholar
  7. 7.
    Guo, Y., Schuurmans, D.: Multi-label classification with output kernels. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds.) ECML PKDD 2013, Part II. LNCS, vol. 8189, pp. 417–432. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  8. 8.
    Guo, Y., Xue, W.: Probablistic mult-label classification with sparse feature learning. In: Proceedings of IJCAI (2013)Google Scholar
  9. 9.
    Hinton, G., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Ho, T.: The random subspace method for constructing decision forests. IEEE TPAMI 20(8) (August 1998)Google Scholar
  11. 11.
    Hsu, D., Kakade, S., Langford, J., Zhang, T.: Multi-label prediction via compressed sensing. In: Proceedings of NIPS (2009)Google Scholar
  12. 12.
    Huang, S., Zhou, Z.: Multi-label learning by exploiting label correlations locally. In: Proceedings of AAAI (2012)Google Scholar
  13. 13.
    Huiskes, M., Lew, M.: The MIR Flickr retrieval evaluation. In: Proceedings of ACM MIR (2008)Google Scholar
  14. 14.
    Ji, S., Tang, L., Yu, S., Ye, J.: Extracting shared subspace for multi-label classification. In: Proceedings of KDD (2008)Google Scholar
  15. 15.
    Lastra, G., Luaces, O., Quevedo, J.R., Bahamonde, A.: Graphical feature selection for multilabel classification tasks. In: Gama, J., Bradley, E., Hollmén, J. (eds.) IDA 2011. LNCS, vol. 7014, pp. 246–257. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  16. 16.
    Lewis, D., Yang, Y., Rose, T., Li, F.: RCV1: A new benchmark collection for text categorization research. JMLR 5, 361–397 (2004)Google Scholar
  17. 17.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (2006)zbMATHGoogle Scholar
  18. 18.
    Sharmanska, V., Quadrianto, N., Lampert, C.H.: Augmented attribute representations. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part V. LNCS, vol. 7576, pp. 242–255. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  19. 19.
    Tai, F., Lin, H.: Multilabel classification with principal label space transformation. In: Proceedings of Inter. Workshop on Learning from Multi-Label Data (2010)Google Scholar
  20. 20.
    Tsoumakas, G., Katakis, I., Vlahavas, I.: Effective and efficient multilabel classification in domains with large number of labels. In: ECML/PKDD Workshop on Mining Multidimensional Data (2008)Google Scholar
  21. 21.
    Yan, R., Tesic, J., Smith, J.: Model-shared subspace boosting for multi-label classification. In: Proceedings of KDD (2007)Google Scholar
  22. 22.
    Yu, K., Yu, S., Tresp, V.: Multi-label informed latent semantic indexing. In: Proceedings of the Annual ACM SIGIR Conference (2005)Google Scholar
  23. 23.
    Zhang, M., Peña, J., Robles, V.: Feature selection for multi-label naive bayes classification. Inf. Sci. 179(19) (September 2009)Google Scholar
  24. 24.
    Zhang, Y., Schneider, J.: Multi-label output codes using canonical correlation analysis. In: Proceedings of AISTATS (2011)Google Scholar
  25. 25.
    Zhang, Y., Schneider, J.: Maximum margin output coding. In: Proceedings of ICML (2012)Google Scholar
  26. 26.
    Zhang, Y., Zhou, Z.: Multilabel dimensionality reduction via dependence maximization. In: Proceedings of AAAI (2008)Google Scholar
  27. 27.
    Zhou, T., Tao, D., Wu, X.: Compressed labeling on distilled lablsets for multi-label learning. Machine Learning 88, 69–126 (2012)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Xin Li
    • 1
  • Yuhong Guo
    • 1
  1. 1.Department of Computer and Information SciencesTemple UniversityPhiladelphiaUSA

Personalised recommendations