Nonparametric Fisher Kernel Using Fuzzy Clustering

  • Ryo Inokuchi
  • Sadaaki Miyamoto
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4252)


The Fisher kernel, which refers to the inner product in the feature space of the Fisher score, has been known to be a successful tool for feature extraction using a probabilistic model. If an appropriate probabilistic model for given data is known, the Fisher kernel provides a discriminative classifier such as support vector machines with good generalization. However, if the distribution is unknown, it is difficult to obtain an appropriate Fisher kernel. In this paper, we propose a new nonparametric Fisher-like kernel derived from fuzzy clustering instead of a probabilistic model, noting that fuzzy clustering methods such as a family of fuzzy c-means are highly related to probabilistic models, e.g., entropy-based fuzzy c-means and a Gaussian mixture distribution model. The proposed kernel is derived from observing the last relationship. Numerical examples show the effectiveness of the proposed method.


Support Vector Machine Feature Space Probabilistic Model Fuzzy Cluster Neural Information Processing System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum, New York (1981)MATHGoogle Scholar
  2. 2.
    Chapelle, O., Weston, J., Schölkopf, B.: Cluster kernels for semi-supervised learning. Advances in Neural Information Processing Systems 15, 585–592 (2003)Google Scholar
  3. 3.
    Ichihashi, H., Honda, K., Tani, N.: Gaussian mixture PDF approximation and fuzzy c-means clustering with entropy regularization. In: Proc. of the 4th Asian Fuzzy System Symposium, Tsukuba, Japan, May 31-June 3, pp. 217–221 (2000)Google Scholar
  4. 4.
    Jaakkola, T., Haussler, D.: Exploiting generative models in discriminative classifiers. In: Proc. of Neural Information Processing Systems. NIPS (1998)Google Scholar
  5. 5.
    Miyamoto, S., Mukaidono, M.: Fuzzy c-means as a regularization and maximum entropy approach. In: Proc. of the 7th International Fuzzy Systems Association World Congress (IFSA 1997), Prague, Czech, June 25-30, vol. II, pp. 86–92 (1997)Google Scholar
  6. 6.
    Miyamoto, S.: Introduction to Cluster Analysis: Theory and Applications of Fuzzy Clustering. Morikita-Shuppan, Tokyo (1999) (in Japanese)Google Scholar
  7. 7.
    Seeger, M.: Covariance kernels from bayesian generative models. Advances in Neural Information Processing Systems 14, 905–912 (2001)Google Scholar
  8. 8.
    Miyamoto, S., Suizu, D.: Fuzzy c-means clustering using kernel functions in support vector machines. J. of Advanced Computational Intelligence and Intelligent Informatics 7(1), 25–30 (2003)Google Scholar
  9. 9.
    Tsuda, K., Kawanabe, M., Muller, K.R.: Clustering with the Fisher score. Advances in Neural Information Processing Systems 15, 729–736 (2003)Google Scholar
  10. 10.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ryo Inokuchi
    • 1
  • Sadaaki Miyamoto
    • 2
  1. 1.Doctoral Program in Risk EngineeringUniversity of TsukubaIbarakiJapan
  2. 2.Department of Risk EngineeringUniversity of TsukubaIbarakiJapan

Personalised recommendations