Semi-supervised Multi-label Linear Discriminant Analysis

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10634)


Multi-label dimensionality reduction methods often ask for sufficient labeled samples and ignore abundant unlabeled ones. To leverage abundant unlabeled samples and scarce labeled ones, we introduce a method called Semi-supervised Multi-label Linear Discriminant Analysis (SMLDA). SMLDA measures the dependence between pairwise samples in the original space and in the projected subspace to utilize unlabeled samples. After that, it optimizes the target projective matrix by minimizing the distance of within-class samples, whilst maximizing the distance of between-class samples and the dependence term. Extensive empirical study on multi-label datasets shows that SMLDA outperforms other related methods across various evaluation metrics, and the dependence term is an effective alternative to the widely-used smoothness term.


Dimensionality reduction Semi-supervised learning Multi-label learning Dependence maximization 



This work is supported by Natural Science Foundation of China (61402378), Natural Science Foundation of CQ CSTC (cstc2016jcyjA0351), Fundamental Research Funds for the Central Universities of China (XDJK2362015XK07 and XDJK2017D067).


  1. 1.
    Zhang, M., Zhou, Z.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014)CrossRefGoogle Scholar
  2. 2.
    Bellman, R.: Dynamic programming and lagrange multipliers. Proc. Natl. Acad. Sci. 42, 767–769 (1956)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Wang, H., Ding, C., Huang, H.: Multi-label linear discriminant analysis. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 126–139. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15567-3_10 CrossRefGoogle Scholar
  4. 4.
    Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Hum. Genet. 7(2), 179–188 (1936)Google Scholar
  5. 5.
    Zhang, Y., Zhou, Z.: Multilabel dimensionality reduction via dependence maximization. ACM Tran. Knowl. Discov. Data 4(3), 14 (2010)Google Scholar
  6. 6.
    Gretton, A., Bousquet, O., Smola, A., Schölkopf, B.: Measuring statistical dependence with Hilbert-Schmidt norms. In: Jain, S., Simon, H.U., Tomita, E. (eds.) ALT 2005. LNCS, vol. 3734, pp. 63–77. Springer, Heidelberg (2005). doi: 10.1007/11564089_7 CrossRefGoogle Scholar
  7. 7.
    Ji, S., Tang, L., Yu, S., Ye, J.: A shared-subspace learning framework for multi-label classification. ACM Trans. Knowl. Discov. Data 4(2), 8 (2010)CrossRefGoogle Scholar
  8. 8.
    Hotelling, H.: Relations between two sets of variates. Biometrika 28(3–4), 321–377 (1936)CrossRefzbMATHGoogle Scholar
  9. 9.
    Sun, L., Ji, S., Yu, S., Ye, J.: On the equivalence between canonical correlation analysis and orthonormalized partial least squares. In: International Joint Conference on Artificial Intelligence, pp. 1230–1235 (2009)Google Scholar
  10. 10.
    Qian, B., Davidson, I.: Semi-supervised dimension reduction for multi-Label classification. In: AAAI Conference on Artificial Intelligence, vol. 10, pp. 569–574 (2010)Google Scholar
  11. 11.
    Yuan, Y., Zhao, K., Lu, H.: Multi-label linear discriminant analysis with locality consistency. In: Loo, C.K., Yap, K.S., Wong, K.W., Teoh, A., Huang, K. (eds.) ICONIP 2014. LNCS, vol. 8835, pp. 386–394. Springer, Cham (2014). doi: 10.1007/978-3-319-12640-1_47 Google Scholar
  12. 12.
    Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7, 2399–2434 (2006)zbMATHMathSciNetGoogle Scholar
  13. 13.
    Guo, B., Hou, C., Nie, F., Yi, D.: Semi-supervised multi-label dimensionality reduction. In: IEEE International Conference on Data Mining, pp. 919–924 (2016)Google Scholar
  14. 14.
    Tan, Q., Liu, Y., Chen, X., Yu, G.: Multi-label classification based on low rank representation for image annotation. Remote Sens. 9(2), 109 (2017)CrossRefGoogle Scholar
  15. 15.
    Wang, C., Yan, S., Zhang, L., Zhang, H.: Multi-label sparse coding for automatic image annotation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1643–1650 (2009)Google Scholar
  16. 16.
    Yu, G., Fu, G., Wang, J., Zhu, H.: Predicting protein function via semantic integration of multiple networks. IEEE/ACM Trans. Comput. Biol. Bioinform. 13(2), 220–232 (2016)CrossRefGoogle Scholar
  17. 17.
    Cai, D., He, X., Han, J.: Semi-supervised discriminant analysis. In: IEEE International Conference on Computer Vision, pp. 1–7 (2007)Google Scholar
  18. 18.
    Yu, G., Zhang, G., Domeniconi, C., Yu, Z., You, J.: Semi-supervised classification based on random subspace dimensionality reduction. Pattern Recognit. 45(3), 1119–1135 (2012)CrossRefzbMATHGoogle Scholar
  19. 19.
    Zhang, D., Zhou, Z., Chen, S.: Semi-supervised dimensionality reduction. In: SIAM International Conference on Data Mining, pp. 629–634 (2007)Google Scholar
  20. 20.
    Yu, G., Domeniconi, C., Rangwala, H., Zhang, G.: Protein function prediction using dependence maximization. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds.) ECML PKDD 2013. LNCS, vol. 8188, pp. 574–589. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40988-2_37 CrossRefGoogle Scholar
  21. 21.
    Zhang, M., Zhou, Z.: ML-\(k\)NN: a lazy learning approach to multi-label learning. Pattern Recognit. 40(7), 2038–2048 (2007)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of Computer and Information ScienceSouthwest UniversityChongqingChina
  2. 2.School of Computer Science and Engineering, Big Data Research CenterUniversity of Electronic Science and Technology of ChinaChengduChina

Personalised recommendations