Functional Learning of Kernels for Information Fusion Purposes

  • Alberto Muñoz
  • Javier González
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5197)

Abstract

When there are several sources of information available in pattern recognition problems, the task of combining them is most interesting. In the context of kernel methods it means to design a single kernel function that collects all the relevant information of each kernel for the classification task at hand. The problem is then solved by training a Support Vector Machine (SVM) on the resulting kernel. Here we propose a consistent method to produce kernel functions from kernel matrices created by any given kernel combination technique. Once this fusion kernel function is available, it will be possible to evaluate the kernel at any data point. The performance of the proposed fusion Kernel is illustrated on several classification and visualization tasks.

Keywords

Mercer Kernel Support Vector Machines Kernel Combination Classification problems 

References

  1. 1.
    Aroszajn, N.: Theory of Reproducing Kernels. Transactions of the American Mathematical Society 68(3), 337–404 (1950)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bengio, Y., Delalleau, O., Le Roux, N., Paiement, J.-F., Vincent, P., Ouimet, M.: Learning eigenfunctions links spectral embedding and kernel PCA. Neural Computation 16, 2197–2219 (2004)CrossRefMATHGoogle Scholar
  3. 3.
    Bengio, Y., Vincent, P., Paiement, J.: Learning eigenfunctions of similarity: Linking spectral clustering and kernel PCA. Technical Report 1232, Département d’informatique et recherche opérationnelle, Université de Montréal (2003)Google Scholar
  4. 4.
    Burges, C.J.C., Platt, J.C., Jana, S.: Distortion discriminant analysis for audio fingerprinting. EEE Transactions on Speech and Audio (2003)Google Scholar
  5. 5.
    Cucker, F., Smale, S.: On the Mathematical Foundations of Learning. Bulletin of the American Mathematical Society 39(1), 1–49 (2002)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine intelligence 20(3), 226–239 (1998)CrossRefGoogle Scholar
  7. 7.
    Lanckriet, G.R.G., Cristianini, N., Bartlett, P., El Ghaoui, L., Jordan, M.I.: Learning the Kernel Matrix with Semidefinite Programming. Journal of Machine Learning Research 5, 27–72 (2004)MathSciNetMATHGoogle Scholar
  8. 8.
    Mikio, L.B.: Accurate Error Bounds for the Eigenvalues of the Kernel Matrix. Journal of Machine Learning Research 7, 2303–2328 (2006)MathSciNetMATHGoogle Scholar
  9. 9.
    Martin de Diego, I., Moguerza, J.M., Muñoz, A.: Combining Kernel Information for Support Vector Classification. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 102–111. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  10. 10.
    Moguerza, J., Muñoz, A.: Support Vector Machines with Applications. Statistical Science 21(3), 322–336 (2006)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Moguerza, J.M., Muñoz, A., de Diego, I.M.: Improving Support Vector Classification via the Combination of Multiple Sources of Information. In: Fred, A., Caelli, T.M., Duin, R.P.W., Campilho, A.C., de Ridder, D. (eds.) SSPR&SPR 2004. LNCS, vol. 3138, pp. 592–600. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. 12.
    Muñoz, A., de Diego, I.M.: From indefinite to Semi-Definite Matrices. In: Yeung, D.-Y., Kwok, J.T., Fred, A., Roli, F., de Ridder, D. (eds.) SSPR 2006 and SPR 2006. LNCS, vol. 4109, pp. 764–772. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  13. 13.
    Muñoz, A., González, J.: Joint Diagonalization of Kernels for Information Fusion. In: Rueda, L., Mery, D., Kittler, J. (eds.) CIARP 2007. LNCS, vol. 4756, pp. 556–563. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  14. 14.
    Schlesinger, S.: Approximating Eigenvalues and Eigenfunctions of Symmetric Kernels. Journal of the Society for Industrial and Applied Mathematics 6(1), 1–14 (1957)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Sigillito, V.G., Wing, S.P., Hutton, L.V., Baker, K.B.: Classification of radar returns from the ionosphere using neural networks. Johns Hopkins APL Technical Digest 10, 262–266 (1989)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Alberto Muñoz
    • 1
  • Javier González
    • 1
  1. 1.Universidad Carlos III de MadridGetafeSpain

Personalised recommendations