Possibilistic Approach to Kernel-Based Fuzzy c-Means Clustering with Entropy Regularization

  • Kiyotaka Mizutani
  • Sadaaki Miyamoto
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3558)


The fuzzy c-means (FCM) is sensitive to noise or outliers because this method has the probabilistic constraint that the memberships of a data point across classes sum to one. To solve the problem, a possibilistic c-means clustering (PCM) has been proposed by Krishnapuram and Keller. An advantage of PCM is highly robust in a noisy environment. On the other hand, some clustering algorithms using the kernel trick, e.g., kernel-based FCM and kernel-based LVQ clustering, have been studied to obtain nonlinear classification boundaries. In this paper, an entropy-based possibilistic c-means clustering using the kernel trick has been proposed as more robust method. Numerical examples are shown and effect of the kernel method is discussed.


Probabilistic Constraint Kernel Principal Component Analysis Kernel Trick Fisher Discriminant Analysis Possibilistic Approach 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum, New York (1981)zbMATHGoogle Scholar
  2. 2.
    Davé, R.N., Krishnapuram, R.: Robust clustering methods: a unified view. IEEE Trans. Fuzzy Syst. 5(2), 270–293 (1997)CrossRefGoogle Scholar
  3. 3.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York (2001)zbMATHGoogle Scholar
  4. 4.
    Girolami, M.: Mercer kernel based clustering in feature space. IEEE Trans. on Neural Networks 13(13), 780–784 (2002)CrossRefGoogle Scholar
  5. 5.
    Höppner, F., Klawonn, F., Kruse, R., Runkler, T.: Fuzzy Cluster Analysis. Wiley, Chichester (1999)zbMATHGoogle Scholar
  6. 6.
    Ichihashi, H., Honda, K., Tani, N.: Gaussian mixture PDF approximation and fuzzy c-means clustering with entropy regularization. In: Proc. of the 4th Asian Fuzzy System Symposium, Tsukuba, Japan, May 31-June 3, pp. 212–221 (2000)Google Scholar
  7. 7.
    Inokuchi, R., Miyamoto, S.: LVQ clustering and SOM using a kernel function. In: Proc. of the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2004), Budapest, Hungary, July 25-29, pp. 182–185 (2004)Google Scholar
  8. 8.
    Krishnapuram, R., Keller, J.M.: A possibilistic approach to fuzzy clustering. IEEE Trans. on Fuzzy Syst. 1(2), 98–110 (1993)CrossRefGoogle Scholar
  9. 9.
    Liu, Z.Q., Miyamoto, S. (eds.): Soft Computing and Human-Centered Machines. Springer, Tokyo (2000)Google Scholar
  10. 10.
    Mika, S., Rätsch, G., Weston, J., Schölkopf, B., Müller, K.-R.: Fisher discriminant analysis with kernels. In: Hu, Y.-H., et al. (eds.) Neural Network for Signal Processing IX, pp. 41–48. IEEE, Los Alamitos (1999)Google Scholar
  11. 11.
    Miyamoto, S., Mukaidono, M.: Fuzzy c - means as a regularization and maximum entropy approach. In: Proc. of the 7th International Fuzzy Systems Association World Congress (IFSA 1997), Prague, Czech, June 25-30, vol. II, pp. 86–92 (1997)Google Scholar
  12. 12.
    Miyamoto, S., Suizu, D.: Fuzzy c-means clustering using kernel functions in support vector machines. J. of Advanced Computational Intelligence and Intelligent Informatics 7(1), 25–30 (2003)Google Scholar
  13. 13.
    Miyamoto, S., Suizu, D., Takata, O.: Methods of fuzzy c-means and possibilistic clustering using quadratic term. Scientiae Mathematicae Japonicae 60(2), 217–233 (2004)zbMATHMathSciNetGoogle Scholar
  14. 14.
    Schölkopf, B., Smola, A., Müller, K.-R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)CrossRefGoogle Scholar
  15. 15.
    Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Kiyotaka Mizutani
    • 1
  • Sadaaki Miyamoto
    • 2
  1. 1.Graduate School of Systems and Information EngineeringUniversity of TsukubaIbarakiJapan
  2. 2.Department of Risk Engineering, School of Systems and Information EngineeringUniversity of TsukubaIbarakiJapan

Personalised recommendations