Optimal Kernel in a Class of Kernels with an Invariant Metric

  • Akira Tanaka
  • Hideyuki Imai
  • Mineichi Kudo
  • Masaaki Miyakoshi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5342)


Learning based on kernel machines is widely known as a powerful tool for various fields of information science such as pattern recognition and regression estimation. One of central topics of kernel machines is model selection, especially selection of a kernel or its parameters. In this paper, we consider a class of kernels that forms a monotonic classes of reproducing kernel Hilbert spaces with an invariant metric and show that the kernel corresponding to the smallest reproducing kernel Hilbert space including an unknown true function gives the optimal model for the unknown true function.


  1. 1.
    Muller, K., Mika, S., Ratsch, G., Tsuda, K., Scholkopf, B.: An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks 12, 181–201 (2001)CrossRefGoogle Scholar
  2. 2.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1999)zbMATHGoogle Scholar
  3. 3.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Recognition. Cambridge University Press, Cambridge (2004)zbMATHGoogle Scholar
  4. 4.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge University Press, Cambridge (2000)CrossRefzbMATHGoogle Scholar
  5. 5.
    Sugiyama, M., Ogawa, H.: Subspace Information Criterion for Model Selection. Neural Computation 13, 1863–1889 (2001)CrossRefzbMATHGoogle Scholar
  6. 6.
    Sugiyama, M., Kawanage, M., Muller, K.: Trading variance reduction with unbiasedness: The regularized subspace information criterion for robust model selection in kernel regression. Neural Computation 16, 1077–1104 (2004)CrossRefzbMATHGoogle Scholar
  7. 7.
    Aronszajn, N.: Theory of Reproducing Kernels. Transactions of the American Mathematical Society 68, 337–404 (1950)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Mercer, J.: Functions of Positive and Negative Type and Their Connection with The Theory of Integral Equations. Transactions of the London Philosophical Society A, 415–446 (1909)CrossRefzbMATHGoogle Scholar
  9. 9.
    Tanaka, A., Sugiyama, M., Imai, H., Kudo, M., Miyakoshi, M.: Model selection using a class of kernels with an invarinat matric. In: Joint IAPR Internatioanl Workshops SSPR 2006 and SPR 2006, pp. 862–870. Springer, Heidelberg (2006)Google Scholar
  10. 10.
    Schatten, R.: Norm Ideals of Completely Continuous Operators. Springer, Berlin (1960)CrossRefzbMATHGoogle Scholar
  11. 11.
    Ogawa, H.: Neural Networks and Generalization Ability. IEICE Technical Report NC95-8, 57–64 (1995)Google Scholar
  12. 12.
    Rao, C.R., Mitra, S.K.: Generalized Inverse of Matrices and its Applications. John Wiley & Sons, Chichester (1971)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Akira Tanaka
    • 1
  • Hideyuki Imai
    • 1
  • Mineichi Kudo
    • 1
  • Masaaki Miyakoshi
    • 1
  1. 1.Division of Computer Science, Graduate School of Information Science and TechnologyHokkaido UniversitySapporoJapan

Personalised recommendations