Sparse Multi-label Bilinear Embedding on Stiefel Manifolds

  • Yang LiuEmail author
  • Guohua Dong
  • Zhonglei Gu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11177)


Dimensionality reduction plays an important role in various machine learning tasks. In this paper, we propose a novel method dubbed Sparse Multi-label bILinear Embedding (SMILE) on Stiefel manifolds for supervised dimensionality reduction on multi-label data. Unlike the traditional multi-label dimensionality reduction algorithms that work on the vectorized data, the proposed SMILE directly takes the second-order tensor data as the input, and thus characterizes the spatial structure of the tensor data in an efficient way. Differentiating from the existing tensor-based dimensionality reduction methods that perform the eigen-decomposition in each iteration, SMILE utilizes a gradient ascent strategy to optimize the objective function in each iteration, and thus is more efficient. Moreover, we introduce column-orthonormal constraints to transformation matrices to eliminate the redundancy between the projection directions of the learned subspace and add an \(L_1\)-norm regularization term to the objective function to enhance the interpretability of the learned subspace. Experiments on a standard image dataset validate the effectiveness of the proposed method.


Sparse multi-label bilinear embedding Dimensionality reduction Second-order tensor Column-orthonormal constraints Stiefel manifolds \(L_1\)-norm regularization 



This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant 61503317, in part by the General Research Fund (GRF) from the Research Grant Council (RGC) of Hong Kong SAR under Project HKBU12202417, and in part by the SZSTI Grant with the Project Code JCYJ20170307161544087.


  1. 1.
    Boothby, W.M.: An Introduction to Differentiable Manifolds and Riemannian Geometry. Academic Press, New York (2002)zbMATHGoogle Scholar
  2. 2.
    Cunningham, J.P., Ghahramani, Z.: Linear dimensionality reduction: survey, insights, and generalizations. J. Mach. Learn. Res. 16, 2859–2900 (2015)MathSciNetzbMATHGoogle Scholar
  3. 3.
    He, X., Yan, S., Hu, Y., Niyogi, P., Zhang, H.J.: Face recognition using laplacianfaces. IEEE Trans. Pattern Anal. Mach. Intell. 27(3), 328–340 (2005)CrossRefGoogle Scholar
  4. 4.
    Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Edu. Psychol. 24(417–441), 498–520 (1933)CrossRefGoogle Scholar
  5. 5.
    Huang, J., Nie, F., Huang, H., Ding, C.: Supervised and projected sparse coding for image classification. In: Proceedings of 27th AAAI, pp. 438–444 (2013)Google Scholar
  6. 6.
    Lee, J.M.: Introduction to Smooth Manifolds. Graduate Texts in Mathematics, 2nd edn. Springer, New York (2012). Scholar
  7. 7.
    Liu, Y., Liu, Y., Chan, K.C.C., Hua, K.A.: Hybrid manifold embedding. IEEE Trans. Neural Netw. Learn. Syst. 25(12), 2295–2302 (2014)CrossRefGoogle Scholar
  8. 8.
    Wang, C., et al.: What strikes the strings of your heart? - multi-label dimensionality reduction for music emotion analysis via brain imaging. IEEE Trans. Auton. Mental Develop. 7(3), 176–188 (2015)CrossRefGoogle Scholar
  9. 9.
    Liu, Y., Liu, Y., Zhao, Y., Hua, K.A.: What strikes the strings of your heart? - feature mining for music emotion analysis. IEEE Trans. Affect. Comput. 6(3), 247–260 (2015)CrossRefGoogle Scholar
  10. 10.
    Maron, O., Ratan, A.L.: Multiple-instance learning for natural scene classification. In: Proceedings of 15th ICML, pp. 341–349 (1998)Google Scholar
  11. 11.
    Panagakis, I., Benetos, E., Kotropoulos, C.: Music genre classification: a multilinear approach. In: Proceedings of 9th ISMIR, pp. 583–588 (2008)Google Scholar
  12. 12.
    Qi, G.J., Hua, X.S., Rui, Y., Tang, J., Zhang, H.J.: Two-dimensional multilabel active learning with an efficient online adaptation model for image classification. IEEE Trans. Pattern Anal. Mach. Intell. 31(10), 1880–1897 (2009)CrossRefGoogle Scholar
  13. 13.
    Sun, L., Ji, S., Ye, J.: Multi-Label Dimensionality Reduction. Chapman and Hall/CRC Machine Learning & Pattern Recognition, Chapman & Hall/CRC (2013)Google Scholar
  14. 14.
    Symeonidis, P., Nanopoulos, A., Manolopoulos, Y.: Tag recommendations based on tensor dimensionality reduction. In: Proceedings of 2nd ACM RecSys, pp. 43–50 (2008)Google Scholar
  15. 15.
    Ueda, N., Saito, K.: Parametric mixture models for multi-labeled text. In: NIPS, vol. 15, pp. 737–744 (2003)Google Scholar
  16. 16.
    Venna, J., Peltonen, J., Nybo, K., Aidos, H., Kaski, S.: Information retrieval perspective to nonlinear dimensionality reduction for data visualization. J. Mach. Learn. Res. 11, 451–490 (2010)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Wang, H., Ding, C., Huang, H.: Multi-label linear discriminant analysis. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 126–139. Springer, Heidelberg (2010). Scholar
  18. 18.
    Yu, K., Yu, S., Tresp, V.: Multi-label informed latent semantic indexing. In: Proceedings of 28th ACM SIGIR, pp. 258–265 (2005)Google Scholar
  19. 19.
    Zeng, J., Liu, Y., Leng, B., Xiong, Z., Cheung, Y.: Dimensionality reduction in multiple ordinal regression. IEEE Trans. Neural Netw. Learn. Syst. 29(9), 4088–4101 (2018). Scholar
  20. 20.
    Zhang, M.L., Zhou, Z.H.: Ml-knn: A lazy learning approach to multi-label learning. Pattern Recogn. 40(7), 2038–2048 (2007)CrossRefGoogle Scholar
  21. 21.
    Zhang, M.L., Zhou, Z.H.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014)CrossRefGoogle Scholar
  22. 22.
    Zhang, Y., Zhou, Z.H.: Multilabel dimensionality reduction via dependence maximization. ACM Trans. Knowl. Discov. Data 4(3), 14:1–14:21 (2010)CrossRefGoogle Scholar
  23. 23.
    Zhou, Z.H., Zhang, M.L.: Multi-instance multi-label learning with application to scene classification. In: Schölkopf, B., Platt, J.C., Hoffman, T. (eds.) NIPS 19, pp. 1609–1616 (2007)Google Scholar
  24. 24.
    Zhu, X., Li, X., Zhang, S.: Block-row sparse multiview multilabel learning for image classification. IEEE Trans. Cybern. 46(2), 450–461 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Computer ScienceHong Kong Baptist UniversityKowloonHong Kong SAR
  2. 2.HKBU Institute for Research and Continuing EducationShenzhenPeople’s Republic of China
  3. 3.Songshan Lake TechX InstituteDongguanPeople’s Republic of China

Personalised recommendations