Multiview Dimension Reduction Based on Sparsity Preserving Projections

  • Haohao Li
  • Yu Cai
  • Guohui Zhao
  • Hu Lin
  • Zhixun SuEmail author
  • Ximin Liu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11854)


In this paper, we focus on boosting the subspace learning by exploring the complimentary and compatible information from multi-view features. A novel multi-view dimension reduction method is proposed named Multiview Sparsity Preserving Projection (MSPP) for this task. MSPP aims to seek a set of linear transforms to project multi-view features into subspace where the sparse reconstructive weights of multi-view features are preserved as much as possible. And the Hilbert Schmidt Independence Criterion (HSIC) is utilized as a dependence term to explore the compatible and complementary information from multi-view features. An efficient alternative iterating optimization is presented to obtain the optimal solution of MSPP. Experiments on image datasets and multi-view textual datasets well demonstrate the excellent performance of MSPP.


Dimension reduction Multi-view learning Co-regularization 



This work is supported by the Natural Science Foundation of China [No. 61572099]; Major National Science and Technology of China 2018ZX04011001-007, 2018ZX04016001-011.


  1. 1.
    Balakrishnama, S., Ganapathiraju, A.: Linear discriminant analysis-a brief tutorial. Inst. Signal Inf. Process. 18, 1–8 (1998)Google Scholar
  2. 2.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)CrossRefGoogle Scholar
  3. 3.
    Cai, D., He, X., Han, J.: Spectral regression: a unified approach for sparse subspace learning. In: Seventh IEEE International Conference on Data Mining (ICDM 2007), pp. 73–82. IEEE (2007)Google Scholar
  4. 4.
    Cai, D., He, X., Zhou, K., Han, J., Bao, H.: Locality sensitive discriminant analysis. In: IJCAI, vol. 2007, pp. 1713–1726 (2007)Google Scholar
  5. 5.
    Cao, X., Zhang, C., Fu, H., Liu, S., Zhang, H.: Diversity-induced multi-view subspace clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–594 (2015)Google Scholar
  6. 6.
    Chapelle, O., Haffner, P., Vapnik, V.: Support vector machines for histogram-based image classification. IEEE Trans. Neural Networks 10(5), 1055–1064 (1999)CrossRefGoogle Scholar
  7. 7.
    Dhillon, P., Foster, D.P., Ungar, L.H.: Multi-view learning of word embeddings via CCA. In: Advances in Neural Information Processing Systems, pp. 199–207 (2011)Google Scholar
  8. 8.
    Gao, X., Xiao, B., Tao, D., Li, X.: Image categorization: graph edit direction histogram. Pattern Recogn. 41(10), 3179–3191 (2008)CrossRefGoogle Scholar
  9. 9.
    Gretton, A., Bousquet, O., Smola, A., Schölkopf, B.: Measuring statistical dependence with Hilbert-Schmidt norms. In: Jain, S., Simon, H.U., Tomita, E. (eds.) ALT 2005. LNCS (LNAI), vol. 3734, pp. 63–77. Springer, Heidelberg (2005). Scholar
  10. 10.
    He, X., Cai, D., Yan, S., Zhang, H.J.: Neighborhood preserving embedding. In: Tenth IEEE International Conference on Computer Vision (ICCV 2005) Volume 1, vol. 2, pp. 1208–1213. IEEE (2005)Google Scholar
  11. 11.
    He, X., Niyogi, P.: Locality preserving projections. In: Advances in Neural Information Processing Systems, pp. 153–160 (2004)Google Scholar
  12. 12.
    Hu, H., Lin, Z., Feng, J., Zhou, J.: Smooth representation clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3834–3841 (2014)Google Scholar
  13. 13.
    Joachims, T.: Transductive inference for text classification using support vector machines (1999)Google Scholar
  14. 14.
    Kan, M., Shan, S., Zhang, H., Lao, S., Chen, X.: Multi-view discriminant analysis. IEEE Trans. Pattern Anal. Mach. Intell. 38(1), 188–194 (2015)CrossRefGoogle Scholar
  15. 15.
    Kumar, A., Rai, P., Daume, H.: Co-regularized multi-view spectral clustering. In: Advances in Neural Information Processing Systems, pp. 1413–1421 (2011)Google Scholar
  16. 16.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  17. 17.
    Lu, D., Weng, Q.: A survey of image classification methods and techniques for improving classification performance. Int. J. Remote Sensing 28(5), 823–870 (2007)CrossRefGoogle Scholar
  18. 18.
    Nie, F., Cai, G., Li, J., Li, X.: Auto-weighted multi-view learning for image clustering and semi-supervised classification. IEEE Trans. Image Process. 27(3), 1501–1511 (2017)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002). Scholar
  20. 20.
    Qiao, L., Chen, S., Tan, X.: Sparsity preserving projections with applications to face recognition. Pattern Recogn. 43(1), 331–341 (2010)CrossRefGoogle Scholar
  21. 21.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  22. 22.
    Tao, D., Jin, L.: Discriminative information preservation for face recognition. Neurocomputing 91(2), 11–20 (2012)CrossRefGoogle Scholar
  23. 23.
    Wang, H., Feng, L., Zhang, J., Liu, Y.: Semantic discriminative metric learning for image similarity measurement. IEEE Trans. Multimedia 18(8), 1579–1589 (2016). Scholar
  24. 24.
    Wang, H., Feng, L., Yu, L., Zhang, J.: Multi-view sparsity preserving projection for dimension reduction. Neurocomputing 216, 286–295 (2016)CrossRefGoogle Scholar
  25. 25.
    Wang, H., Li, H., Fu, X.: Auto-weighted mutli-view sparse reconstructive embedding. Multimedia Tools Appl. 1–15 (2019) Google Scholar
  26. 26.
    Wang, H., Li, H., Peng, J., Fu, X.: Multi-feature distance metric learning for non-rigid 3D shape retrieval. Multimedia Tools Appl. 1–16 (2019)Google Scholar
  27. 27.
    Wang, H., Lin, F., Meng, X., Chen, Z., Yu, L., Zhang, H.: Multi-view metric learning based on KL-divergence for similarity measurement. Neurocomputing 238(C), 269–276 (2017)CrossRefGoogle Scholar
  28. 28.
    Wang, H., Peng, J., Fu, X.: Co-regularized multi-view sparse reconstruction embedding for dimension reduction. Neurocomputing 347, 191–199 (2019)CrossRefGoogle Scholar
  29. 29.
    Wold, S.: Principal component analysis. Chemometr. Intell. Lab. Syst. 2(1), 37–52 (1987)CrossRefGoogle Scholar
  30. 30.
    Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2008)CrossRefGoogle Scholar
  31. 31.
    Xia, T., Tao, D., Mei, T., Zhang, Y.: Multiview spectral embedding. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 40(6), 1438–1446 (2010)CrossRefGoogle Scholar
  32. 32.
    Xu, C., Tao, D., Xu, C.: A survey on multi-view learning. arXiv preprint arXiv:1304.5634 (2013)
  33. 33.
    Xu, D., Yan, S., Tao, D., Lin, S., Zhang, H.J.: Marginal fisher analysis and its variants for human gait recognition and content-based image retrieval. IEEE Trans. Image Process. 16(11), 2811–2821 (2007)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Yang, J., Kai, Y., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: CVPR, pp. 1794–1801 (2009)Google Scholar
  35. 35.
    Zhang, Z., Zha, H.: Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J. Sci. Comput. 26(1), 313–338 (2004)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Zhao, J., Xie, X., Xu, X., Sun, S.: Multi-view learning overview: recent progress and new challenges. Inf. Fusion 38, 43–54 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Haohao Li
    • 1
  • Yu Cai
    • 1
  • Guohui Zhao
    • 1
  • Hu Lin
    • 2
  • Zhixun Su
    • 1
    Email author
  • Ximin Liu
    • 1
  1. 1.School of Mathematical SciencesDalian University of TechnologyDalianChina
  2. 2.Dalian Shipbuilding Industry Co.DalianChina

Personalised recommendations