Improving Support Vector Classification via the Combination of Multiple Sources of Information

  • Javier M. Moguerza
  • Alberto Muñoz
  • Isaac Martín de Diego
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3138)


In this paper we describe several new methods to build a kernel matrix from a collection of kernels. This kernel will be used for classification purposes using Support Vector Machines (SVMs). The key idea is to extend the concept of linear combination of kernels to the concept of functional (matrix) combination of kernels. The functions involved in the combination take advantage of class conditional probabilities and nearest neighbour techniques. The proposed methods have been successfully evaluated on a variety of real data sets against a battery of powerful classifiers and other kernel combination techniques.


Support Vector Machine Kernel Matrix Multivariate Adaptative Regression Spline Machine Learn Research Handwritten Digit Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Alpaydin, E., Kaynak, C.: Cascading Classifiers. Kybernetika 34(4), 369–374 (1998)Google Scholar
  2. 2.
    Bennett, K., Momma, M., Embrechts, J.: MARK: A Boosting Algorithm for Heterogeneous Kernel Models. In: Proceedings of SIGKDD International Conference on Knowledge Discovery and Data Mining (2002)Google Scholar
  3. 3.
    Cortes, C., Vapnik, V.: Support Vector Networks. Machine Learning 20, 273–297 (1995)zbMATHGoogle Scholar
  4. 4.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)Google Scholar
  5. 5.
    Domeniconi, C., Peng, J., Gunopulos, D.: Adaptive Metric Nearest Neighbor Classification. In: Proc. of IEEE Conf. on CVPR, pp. 517–522 (2000)Google Scholar
  6. 6.
    Friedman, J.: Multivariate adaptative regression splines (with discussion). Annals of Statistics 19(1), 1–141 (1991)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Frey, P.W., Slate, D.J.: Letter Recognition Using Holland-Style Adaptive Classifiers. Machine Learning 6(2), 161–182 (1991)Google Scholar
  8. 8.
    Fukushima, K., Imagawa, T.: Recognition and segmentation of connected characters with selective attention. Neural Networks 6, 33–41 (1993)CrossRefGoogle Scholar
  9. 9.
    Joachims, T.: Learning to Classify Text using Support Vector Machines. Kluwer, Dordrecht (2002)Google Scholar
  10. 10.
    Lanckriet, G.R.G., Cristianini, N., Barlett, P., El Ghaoui, L., Jordan, M.I.: Learning the kernel matrix with semi-definite programming. Journal of Machine Learning Research 5, 27–72 (2004)Google Scholar
  11. 11.
    Mangasarian, O.L., Wolberg, W.H.: Cancer diagnosis via linear programming. SIAM News 23(5), 1–18 (1990)Google Scholar
  12. 12.
    Muñoz, A., Villagarcía, T.: Unsupervised neural networks for variable selection with mixed covariates. Analyse Multidimensionelle des Donnes, CSIA Ceresta, 217–227 (1998)Google Scholar
  13. 13.
    Muñoz, A., Martín de Diego, I., Moguerza, J.M.: Support Vector Machine Classifiers for Assymetric Proximities. In: Kaynak, O., Alpaydın, E., Oja, E., Xu, L. (eds.) ICANN 2003 and ICONIP 2003. LNCS, vol. 2714, pp. 217–224. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  14. 14.
    Pekalska, E., Paclík, P., Duin, R.P.W.: A Generalize Kernel Approach to Dissimilarity-based Classification. Journal of Machine Learning Research, Special Issue on Kernel Methods 2(2), 175–211 (2002)zbMATHGoogle Scholar
  15. 15.
    Poggio, T., Girosi, F.: Networks for Approximation and Learning. Proceedings of the IEEE 78(10), 1481–1497 (1990)CrossRefGoogle Scholar
  16. 16.
    Schölkopf, B., Herbrich, R., Smola, A., Williamson, R.: A Generalized Representer Theorem. NeuroCOLT2 TR Series, NC2-TR2000-81 (2000)Google Scholar
  17. 17.
    Schölkopf, B., Mika, S., Burges, C., Knirsch, P., Müller, K., Rätsch, G., Smola, A.: Input Space versus Feature Space in Kernel-based Methods. IEEE Transactions on Neural Networks 10(5), 1000–1017 (1999)CrossRefGoogle Scholar
  18. 18.
    Schölkopf, B., Platt, J.C., Shawe-Taylor, J., Smola, A.J., Williamson, R.C.: Estimating the Support of a High Dimensional Distribution. Neural Computation 13(7), 1443–1471 (2001)zbMATHCrossRefGoogle Scholar
  19. 19.
    Vandenberghe, L., Boyd, S.: Semidefinite programming. SIAM Review 38(1), 49–95 (1996)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Javier M. Moguerza
    • 1
  • Alberto Muñoz
    • 2
  • Isaac Martín de Diego
    • 2
  1. 1.University Rey Juan CarlosMóstolesSpain
  2. 2.University Carlos III de MadridGetafeSpain

Personalised recommendations