Advertisement

Geodesic Analysis on the Gaussian RKHS Hypersphere

  • Nicolas Courty
  • Thomas Burger
  • Pierre-François Marteau
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7523)

Abstract

Using kernels to embed non linear data into high dimensional spaces where linear analysis is possible has become utterly classical. In the case of the Gaussian kernel however, data are distributed on a hypersphere in the corresponding Reproducing Kernel Hilbert Space (RKHS). Inspired by previous works in non-linear statistics, this article investigates the use of dedicated tools to take into account this particular geometry. Within this geometrical interpretation of the kernel theory, Riemannian distances are preferred over Euclidean distances. It is shown that this amounts to consider a new kernel and its corresponding RKHS. Experiments on real publicly available datasets show the possible benefits of the method on clustering tasks, notably through the definition of a new variant of kernel k-means on the hypersphere. Classification problems are also considered in a classwise setting. In both cases, the results show improvements over standard techniques.

Keywords

Riemannian Manifold Gaussian Kernel Spectral Cluster Geodesic Distance Reproduce Kernel Hilbert Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Cortes, C., Vapnik, V.: Support vector machine. Machine Learning 20(3), 273–297 (1995)zbMATHGoogle Scholar
  2. 2.
    Schölkopf, B., Smola, A., Müller, K.R.: Kernel Principal Component Analysis. In: Gerstner, W., Hasler, M., Germond, A., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 583–588. Springer, Heidelberg (1997)Google Scholar
  3. 3.
    Schölkopf, B., Smola, A.J.: Learning with kernels: Support vector machines, regularization, optimization, and beyond. The MIT Press (2002)Google Scholar
  4. 4.
    Lafferty, J., Lebanon, G.: Diffusion kernels on statistical manifolds. Journal of Machine Learning Research 6, 129–163 (2005)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Fletcher, T., Lu, C., Pizer, S., Joshi, S.: Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Trans. Med. Imaging 23(8), 995–1005 (2004)CrossRefGoogle Scholar
  6. 6.
    Said, S., Courty, N., LeBihan, N., Sangwine, S.J.: Exact principal geodesic analysis for data on so(3). In: Proceedings of EUSIPCO 2007, Poznan, Poland (2007)Google Scholar
  7. 7.
    Sommer, S., Lauze, F., Nielsen, M.: The differential of the exponential map, jacobi fields and exact principal geodesic analysis. CoRR, abs/1008.1902 (2010)Google Scholar
  8. 8.
    Karcher, H.: Riemannian center of mass and mollifier smoothing. Communications on Pure and Applied Mathematics 30(5), 509–541 (1977)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Sommer, S., Lauze, F., Hauberg, S., Nielsen, M.: Manifold Valued Statistics, Exact Principal Geodesic Analysis and the Effect of Linear Approximations. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 43–56. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
    Kendall, W.S.: Convexity and the hemisphere. Journal of the London Mathematical Society 2(3), 567 (1991)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Mika, S., Schölkopf, B., Smola, A.J., Müller, K.R., Scholz, M., Rätsch, G.: Kernel pca and de-noising in feature spaces. In: Advances in Neural Information Processing Systems, pp. 536–542. MIT Press (1999)Google Scholar
  12. 12.
    Kwok, J., Tsang, I.: The pre-image problem in kernel methods. IEEE Trans. on Neural Networks 15(6), 1517–1525 (2004)CrossRefGoogle Scholar
  13. 13.
    Huang, D., Tian, Y., De la Torre, F.: Local isomorphism to solve the pre-image problem in kernel methods. In: CVPR 2011, pp. 2761–2768 (2011)Google Scholar
  14. 14.
    Amari, S.I., Wu, S.: Improving support vector machine classifiers by modifying kernel functions. Neural Networks 12(6), 783–789 (1999)CrossRefGoogle Scholar
  15. 15.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010)Google Scholar
  16. 16.
    Dhillon, I., Guan, Y., Kulis, B.: Kernel k-means: spectral clustering and normalized cuts. In: KDD, pp. 551–556 (2004)Google Scholar
  17. 17.
    Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: Analysis and an algorithm. Advances in Neural Information Processing Systems 2, 849–856 (2002)Google Scholar
  18. 18.
    Courty, N., Burger, T., Laurent, J.: PerTurbo: A New Classification Algorithm Based on the Spectrum Perturbations of the Laplace-Beltrami Operator. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011, Part I. LNCS, vol. 6911, pp. 359–374. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  19. 19.
    Chi-Yuan, Y., Zhi-Ying, L., Shie-Jue, L.: Boosting one-class support vector machines for multi-class classification. Applied Artificial Intelligence 23(4), 297–315 (2009)CrossRefGoogle Scholar
  20. 20.
    Coifman, R.R., Lafon, S.: Diffusion maps. Applied and Computational Harmonic Analysis 21(1), 5–30 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Öztireli, C., Alexa, M., Gross, M.: Spectral sampling of manifolds. ACM Transaction on Graphics, Siggraph Asia (December 2010)Google Scholar
  22. 22.
    Cevikalp, H., Larlus, D., Neamtu, M., Triggs, B., Jurie, F.: Manifold based local classifiers: Linear and nonlinear approaches. Journal of Signal Processing Systems 61(1), 61–73 (2010)CrossRefGoogle Scholar
  23. 23.
    Karatzoglou, A., Smola, A., Hornik, K., Zeileis, A.: kernlab-an s4 package for kernel methods in r (2004)Google Scholar
  24. 24.
    Gong, Y., Lazebnik, S.: Comparing data-dependent and data-independent embeddings for classification and ranking of internet images. In: CVPR, pp. 2633–2640. IEEE (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Nicolas Courty
    • 1
    • 3
  • Thomas Burger
    • 2
  • Pierre-François Marteau
    • 1
  1. 1.IRISA, Université de Bretagne SudVannesFrance
  2. 2.iRTSV (FR3425) / BGE (U1038), CNRS/CEA/UJF/INSERMGrenobleFrance
  3. 3.Institute of AutomationChinese Academy of ScienceBeijingChina

Personalised recommendations