Multimedia Systems

, Volume 21, Issue 2, pp 147–157 | Cite as

Large-margin multi-view Gaussian process

Special Issue Paper


In image classification, the goal was to decide whether an image belongs to a certain category or not. Multiple features are usually employed to comprehend the contents of images substantially for the improvement of classification accuracy. However, it also brings in some new problems that how to effectively combine multiple features together and how to handle the high-dimensional features from multiple views given the small training set. In this paper, we integrate the large-margin idea into the Gaussian process to discover the latent subspace shared by multiple features. Therefore, our approach inherits all the advantages of Gaussian process and large-margin principle. A probabilistic explanation is provided by Gaussian process to embed multiple features into the shared low-dimensional subspace, which derives a strong discriminative ability from the large-margin principle, and thus, the subsequent classification task can be effectively accomplished. Finally, we demonstrate the advantages of the proposed algorithm on real-world image datasets for discovering discriminative latent subspace and improving the classification performance.


Multi-view learning Large margin Gaussian process 



The work was supported in part by ARC FT130101457, NBRPC 2011CB302400, NSFC 61121002, 61375026, and JCYJ 20120614152136201.


  1. 1.
    Akaho, S.: A kernel method for canonical correlation analysis. In: IMPS’2001, (2001)Google Scholar
  2. 2.
    Bach, F.R., Lanckriet, G.R.G., Jordan, M.I.: Multiple kernel learning, conic duality, and the smo algorithm. In: ICML’2004, (2004)Google Scholar
  3. 3.
    Bishop, C.M., et al.: Pattern Recognition and Machine Learning, vol. 1. Springer, New York (2006)MATHGoogle Scholar
  4. 4.
    Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: COLT’1998, (1998)Google Scholar
  5. 5.
    Chapelle, O., Weston, J., Schölkopf, B.: Cluster kernels for semi-supervised, learning. In: NIPS’2002, (2002)Google Scholar
  6. 6.
    Chaudhuri, K., Kakade, S.M., Livescu, K., Sridharan, K.: Multi-view clustering via canonical correlation analysis. In: ICML’2009, (2009)Google Scholar
  7. 7.
    Chen, N., Zhu, J., Xing, E.P.: Predictive subspace learning for multi-view data: a large margin approach. In: NIPS’2010, (2010)Google Scholar
  8. 8.
    Chen, Ning, Zhu, Jun, Sun, Fuchun: Large-margin predictive latent subspace learning for multiview data analysis. IEEE. Trans. Pattern Anal. Mach. Intell. 34(12), 2365–2378 (2012)CrossRefGoogle Scholar
  9. 9.
    Christoudias, C.M., Urtasun, R., Darrell, T.: Multi-view learning in the presence of view disagreementGoogle Scholar
  10. 10.
    Diethe, T., Hardoon, D.R., Shawe-Taylor, J.: Multiview fisher discriminant analysis. In: NIPS workshop on learning from multiple sources (2008)Google Scholar
  11. 11.
    Guillaumin, M., Verbeek, J., Schmid, C.: Multimodal semi-supervised learning for image classification. In: CVPR’2010, (2010)Google Scholar
  12. 12.
    Jia, Y., Salzmann, M., Darrell, T.: Factorized latent spaces with structured sparsity. In: NIPS’2010, (2010)Google Scholar
  13. 13.
    Kakade, S.M., Foster, D.P.: Multi-view regression via canonical correlation analysis. In: Learning Theory, pp. 82–96. Springer (2007)Google Scholar
  14. 14.
    Lanckriet, G.R.G., Cristianini, N., El Ghaoui, L., Bartlett, P., Jordan, M.I.: Learning the kernel matrix with semi-definite programming. Computer (2002)Google Scholar
  15. 15.
    Lawrence, D.N.: Gaussian process models for visualisation of high dimensional data (2004)Google Scholar
  16. 16.
    Lawrence, N.D., Quiñonero-Candela, J.: Local distance preservation in the gp-lvm through back constraints. In: ICML’2006, (2006)Google Scholar
  17. 17.
    Luo, Y., Tao, D., Xu, C., Liu, H., Wen, Y.: Multiview vector-valued manifold regularization for multilabel image classification. IEEE Trans. Neural Netw. Learn. Syst. 24(709—-722), 2676–2687 (2013)Google Scholar
  18. 18.
    Memisevic, R.: Kernel information embeddings. In: ICML’2006, (2006)Google Scholar
  19. 19.
    Memisevic, Roland, Sigal, Leonid, Fleet, David J.: Shared kernel information embedding for discriminative inference. IEEE. Trans. Pattern Anal. Mach. Intell. 34(4), 778–790 (2012)CrossRefGoogle Scholar
  20. 20.
    Møller, Martin Fodslette: A scaled conjugate gradient algorithm for fast supervised learning. Neural Netw. 6(4), 525–533 (1993)CrossRefGoogle Scholar
  21. 21.
    Nigam, K., Ghani, R.: Analyzing the effectiveness and applicability of co-training. In: CIKM’2000, (2000)Google Scholar
  22. 22.
    Rakotomamonjy, A., Bach, F., Canu, S., Grandvalet, Y.: Simplemkl. J. Mach. Learn. Res. 9, 2491–2521 (2008)MATHMathSciNetGoogle Scholar
  23. 23.
    Rasmussen, C.E. Gaussian processes for machine learning (2006)Google Scholar
  24. 24.
    Shon, A.P., Grochow, K., Hertzmann, A., Rao, R.P.: Learning shared latent structure for image synthesis and robotic imitation. Adv. Neural Inf. Process. Syst. 18, 1233 (2006)Google Scholar
  25. 25.
    Sigal, L., Memisevic, R., Fleet, D.J.: Shared kernel information embedding for discriminative inference. In: CVPR’2009, (2009)Google Scholar
  26. 26.
    Sindhwani, V., Niyogi, P., Belkin, M.: A co-regularization approach to semi-supervised learning with multiple views. In: ICML Workshop on Learning with Multiple Views (2005)Google Scholar
  27. 27.
    Sonnenburg, Sören, Rätsch, Gunnar, Schäfer, Christin, Schölkopf, Bernhard: Large scale multiple kernel learning. J. Mach. Learn. Res. 7, 1531–1565 (2006)MATHMathSciNetGoogle Scholar
  28. 28.
    Tao, D., Liu, W.: Multiview hessian regularization for image annotation. IEEE Trans. Syst. Man Cybern. Part B. Cybern. 22(7), 2676–2687 (2013)MathSciNetGoogle Scholar
  29. 29.
    Tao, D., Wang, X., Bian, W.: Grassmannian regularized structured multi-view embedding for image classification. IEEE Trans. Image Process. (7):2646–2660Google Scholar
  30. 30.
    Tuytelaars, Tinne, Mikolajczyk, Krystian: Local invariant feature detectors: a survey. Found. Trends Comput. Graph. Vis. 3(3), 177–280 (2008)CrossRefGoogle Scholar
  31. 31.
    Urtasun, R., Darrell, T.: Discriminative gaussian process latent variable model for classification. In: ICML’2007, (2007)Google Scholar
  32. 32.
    Wang, Meng, Hua, Xian-Sheng, Hong, Richang, Tang, Jinhui, Qi, Guo-Jun, Song, Yan: Unified video annotation via multigraph learning. IEEE. Trans. Circuits Syst. Video Technol. 19(5), 733–746 (2009)CrossRefGoogle Scholar
  33. 33.
    Wang, Meng, Ni, Bingbing, Hua, Xian-Sheng, Chua, Tat-Seng: Assistive tagging: a survey of multimedia tagging with human-computer joint exploration. ACM Comput. Surv. (CSUR) 44(4), 25 (2012)CrossRefGoogle Scholar
  34. 34.
    Wang, M., Gao, Y., Lu, K., Rui, Y.: View-based discriminative probabilistic modeling for 3d object retrieval and recognition. IEEE Trans. Image Process. 22(4), 1395–1407 (2013)CrossRefMathSciNetGoogle Scholar
  35. 35.
    Xu, C., Tao, D., Xu, C.: Large-margin multi-view information bottleneck. IEEE Trans. Pattern Anal. Mach. Intell.Google Scholar
  36. 36.
    Xu, C., Tao, D., Li, Y., Xu, C.: Large-margin multi-view gaussian process for image classification. In: ICIMCS’ 2013, (2013a)Google Scholar
  37. 37.
    Xu, C., Tao, D., Xu, C.: A survey on multi-view learning. arXiv, preprint arXiv:1304.5634, (2013b)
  38. 38.
    Yu, J., Liu, D., Tao, D., Seah, HS.: On combining multiple features for cartoon character retrieval and clip synthesis. IEEE Trans. Syst. Man Cybern. Part B Cybern, (2012a)Google Scholar
  39. 39.
    Yu, J., Tao, D.: Modern Machine Learning Techniques and Their Applications in Cartoon Animation Research, vol. 1. Wiley, New York (2013)CrossRefGoogle Scholar
  40. 40.
    Yu, J., Tao, D., Rui, Y., Cheng, J.: Pairwise constraints based multiview features fusion for scene classification. Pattern Recogn. (2012b)Google Scholar
  41. 41.
    Jun, Yu., Wang, Meng, Tao, Dacheng: Semi-supervised multiview distance metric learning for cartoon synthesis. IEEE Trans. Image Process. 1, 4636–4648 (2012c)CrossRefGoogle Scholar
  42. 42.
    Jun, Y., Rui, Y., Chen, B.: Exploiting click constraints and multi-view features for image re-ranking. IEEE Trans, Multimedia (2013)Google Scholar
  43. 43.
    Zha, Z.-J., Hua, X.-S., Mei, T., Wang, J., Qi, G.-J., Wang, Z.: Joint multi-label multi-instance learning for image classification. In: CVPR’2008, (2008)Google Scholar
  44. 44.
    Zha, Z.-J., Yang, L., Mei, T., Wang, M., Wang, Z.: Visual query suggestion. In: MM’2009, (2009)Google Scholar
  45. 45.
    Zha, Z.-J., Yang, L., Mei, T., Wang, M., Wang, Z., Chua, T.-S., Hua, X.-S.: Visual query suggestion: Towards capturing user intent in internet image search. ACM Trans. Multimedia Comput. Commun. Appl. 6(3), 13 (2010)CrossRefGoogle Scholar
  46. 46.
    Zha, Zheng-Jun, Wang, Meng, Zheng, Yan-Tao, Yang, Yi, Hong, Richang, Chua, Tat-Seng: Interactive video indexing with statistical active learning. IEEE Trans. Multimedia 14(1), 17–27 (2012)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Key Laboratory of Machine Perception (Ministry of Education)Peking UniversityBeijingChina
  2. 2.Centre for Quantum Computation & Intelligent Systems, Faculty of Engineering and Information TechnologyUniversity of Technology, SydneyUltimoAustralia
  3. 3.National Computer Network Emergency Response Technical Team/Coordination Center of China (CNCERT/CC) BeijingChina

Personalised recommendations