Support Vector Machine to Synthesise Kernels

  • Hongying Meng
  • John Shawe-Taylor
  • Sandor Szedmak
  • Jason D. R. Farquhar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3635)


In this paper, we introduce a new method (SVM_2K) which amalgamates the capabilities of the Support Vector Machine (SVM) and Kernel Canonical Correlation Analysis (KCCA) to give a more sophisticated combination rule that the boosting framework allows. We show how this combination can be achieved within a unified optimisation model to create a consistent learning rule which combines the classification abilities of the individual SVMs with the synthesis abilities of KCCA. To solve the unified problem, we present an algorithm based on the Augmented Lagrangian Method. Experiments show that SVM_2K performs well on generic object recognition problems in computer vision.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines (and other kernel-based learning methods). Cambridge University Press, Cambridge (2000)Google Scholar
  2. 2.
    Kolenda, T., Hansen, L.K., Larsen, J., Winther, O.: Independent component analysis for understanding multimedia content. In: Bourlard, H., Adali, T., Bengio, S., Larsen, J., Douglas, S. (eds.) Proceedings of IEEE Workshop on Neural Networks for Signal Processing XII, pp. 757–766. IEEE Press, Los Alamitos (2002)CrossRefGoogle Scholar
  3. 3.
    Hardoon, D.R., Shawe-Taylor, J.: KCCA for different level precision in content-based image retrieval. In: Proceedings of Third International Workshop on Content-Based Multimedia Indexing, IRISA, Rennes, France (2003)Google Scholar
  4. 4.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)Google Scholar
  5. 5.
    Vinokourov, A., Shawe-Taylor, J., Cristianini, N.: Inferring a semantic representation of text via cross-language correlation analysis. In: Advances of Neural Information Processing Systems, vol. 15 (2002)Google Scholar
  6. 6.
    Vinokourov, A., Hardoon, D.R., Shawe-Taylor, J.: Learning the semantics of multimedia content with application to web image retrieval and classification. In: Proceedings of Fourth International Symposium on Independent Component Analysis and Blind Source Separation, Nara, Japan (2003)Google Scholar
  7. 7.
    Meng, H., Hardoon, D.R., Shawe-Taylor, J., Szedmak, S.: Generic object recognition by distinct features combination in machine learning. In: Proceedings of SPIE, vol. 5673 (January 2005)Google Scholar
  8. 8.
    Lanckriet, G.R., Cristianini, N., Ghaoui, P.B.L.E., Jordan, M.I.: Learning the kernel matrix with semidefinite programming. Journal of machine learning research, 27–72 (2004)Google Scholar
  9. 9.
    Dasgupta, S., Littman, M.L., McAllester, D.: PAC generalization bounds for co-training. Advances in Neural Information Processing Systems, NIPS (2001)Google Scholar
  10. 10.
    Lanckriet, G., Deng, M., Cristianini, N., Jordan, M., Noble, W.: Kernel-based data fusion and its application to protein function prediction in yeast. In: Proceedings of the Pacific Symposium on Biocomputing, pp. 300–311 (2004)Google Scholar
  11. 11.
    Bach, F., Lanckriet, G., Jordan, M.: Multiple kernel learning, conic duality, and the smo algorithm. In: Proceedings of the 21st International Conference on Machine Learning, Canada (2004)Google Scholar
  12. 12.
    Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: 15th Advances in Neural Information Processing Systems (2002)Google Scholar
  13. 13.
    Bertsekas, D.: Nonlinear Programming, 2nd edn. Athena Scientific (1999)Google Scholar
  14. 14.
    Opelt, A., Fussenegger, M., Pinz, A., Auer, P.: Weak hypotheses and boosting for generic object detection and recognition. In: Proceedings of the 2004 European Conference on Computer vision. Prague Czech Republic, pp. 71–84 (2004)Google Scholar
  15. 15.
    Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2003)Google Scholar
  16. 16.
    Mikolajczyk, K., Schmid, C.: An affine invariant interest point detector. In: Proceedings of the 2002 European Conference on Computer vision. Copenhagen Denmark, pp. 128–142 (2002)Google Scholar
  17. 17.
    Lowe, D.: Object recognition from local scale-invariant features. In: Proceedings of the 7th IEEE International Conference on Computer vision. Kerkyra Greece, pp. 1150–1157 (1999)Google Scholar
  18. 18.
    Blum, A., Mitchell, T.: Combining Labeled and Unlabeled Data with Co-Training. In: Proceedings of the 11th Annual Conference on Computational Learning Theory, pp. 92–100 (1998)Google Scholar
  19. 19.
    Evgeniou, T., Micchelli, C.A., Pontil, M.: Learning multiple tasks with kernel methods. Journal of Machine Learning Research 6, 615–637 (2005)MathSciNetGoogle Scholar
  20. 20.
    Muslea, I., Minton, S., Knoblock, C.A.: Active + semi-supervised learning = robust multi-view learning. In: Proceedings of the 19th International Conference on Machine Learning (ICML 2002), Sydney, Australia, pp. 435–442 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Hongying Meng
    • 1
  • John Shawe-Taylor
    • 1
  • Sandor Szedmak
    • 1
  • Jason D. R. Farquhar
    • 1
  1. 1.School of Electronics and Computer ScienceUniversity of SouthamptonSouthamptonUK

Personalised recommendations