Objective Functions for Fuzzy Clustering

Conference paper
Part of the Studies in Computational Intelligence book series (SCI, volume 445)

Abstract

Fuzzy clustering comprises a family of prototype-based clustering methods that can be formulated as the problem of minimizing an objective function. These methods can be seen as “fuzzifications” of, for example, the classical c-means algorithm, which strives to minimize the sum of the (squared) distances between the data points and the cluster centers to which they are assigned. However, it is well known that in order to “fuzzify” such a crisp clustering approach, it is not enough to merely allow values from the unit interval for the variables encoding the assignments of the data points to the clusters (that is, for the elements of the partition matrix): the minimum is still obtained for a crisp data point assignment. As a consequence, additional means have to be employed in the objective function in order to obtain actual degrees of membership. This paper surveys the most common fuzzification means and examines and compares their properties.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ball, G.H., Hall, D.J.: A clustering technique for summarizing multivariate data. Behavioral Science 12(2), 153–155 (1967)CrossRefGoogle Scholar
  2. 2.
    Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York (1981)MATHCrossRefGoogle Scholar
  3. 3.
    Bezdek, J.C., Hathaway, R.J.: Visual cluster validity (VCV) displays for prototype generator clustering methods. In: Proc. 12th IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE 2003, Saint Louis, MO, vol. 2, pp. 875–880. IEEE Press, Piscataway (2003)CrossRefGoogle Scholar
  4. 4.
    Bezdek, J.C., Pal, N.: Fuzzy Models for Pattern Recognition. IEEE Press, New York (1992)Google Scholar
  5. 5.
    Bezdek, J.C., Keller, J., Krishnapuram, R., Pal, N.: Fuzzy Models and Algorithms for Pattern Recognition and Image Processing. Kluwer, Dordrecht (1999)MATHGoogle Scholar
  6. 6.
    Bilmes, J.: A gentle tutorial on the EM algorithm and its application to parameter estimation for gaussian mixture and hidden markov models. Tech. Rep. ICSI-TR-97-021, University of Berkeley, CA, USA (1997)Google Scholar
  7. 7.
    Borgelt, C.: Prototype-based classification and clustering. Habilitationsschrift, Otto-von-Guericke University of Magdeburg, Germany (2005)Google Scholar
  8. 8.
    Boujemaa, N.: Generalized competitive clustering for image segmentation. In: Proc. 19th Int. Meeting North American Fuzzy Information Processing Society, NAFIPS 2000, Atlanta, GA, pp. 133–137. IEEE Press, Piscataway (2000)Google Scholar
  9. 9.
    Daróczy, Z.: Generalized information functions. Information and Control 16(1), 36–51 (1970)MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Davé, R.N., Krishnapuram, R.: Robust clustering methods: A unified view. IEEE Trans on Fuzzy Systems 5(1997), 270–293 (1997)CrossRefGoogle Scholar
  11. 11.
    Dempster, A.P., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society (Series B) 39, 1–38 (1977)MathSciNetMATHGoogle Scholar
  12. 12.
    Döring, C., Borgelt, C., Kruse, R.: Effects of irrelevant attributes in fuzzy clustering. In: Proc. 14th IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE 2005, Reno, NV, pp. 862–866. IEEE Press, Piscataway (2005)CrossRefGoogle Scholar
  13. 13.
    Dunn, J.C.: A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters. Journal of Cybernetics 3(3), 32–57 (1973); reprinted in [4], 82–101MathSciNetMATHCrossRefGoogle Scholar
  14. 14.
    Everitt, B.S.: Cluster Analysis. Heinemann, London (1981)Google Scholar
  15. 15.
    Everitt, B.S., Hand, D.J.: Finite Mixture Distributions. Chapman & Hall, London (1981)MATHCrossRefGoogle Scholar
  16. 16.
    Frigui, H., Krishnapuram, R.: Clustering by competitive agglomeration. Pattern Recognition 30(7), 1109–1119 (1997)CrossRefGoogle Scholar
  17. 17.
    Gath, I., Geva, A.B.: Unsupervised optimal fuzzy clustering. IEEE Trans Pattern Analysis and Machine Intelligence (PAMI) 11, 773–781 (1989); reprinted in [4], 211–218CrossRefGoogle Scholar
  18. 18.
    Gustafson, E.E., Kessel, W.C.: Fuzzy clustering with a fuzzy covariance matrix. In: Proc. of the IEEE Conf. on Decision and Control, CDC 1979, San Diego, CA, pp. 761–766. IEEE Press, Piscataway (1979); reprinted in [4], 117–122Google Scholar
  19. 19.
    Hartigan, J.A., Wong, M.A.: A k-means clustering algorithm. Applied Statistics 28, 100–108 (1979)MATHCrossRefGoogle Scholar
  20. 20.
    Honda, K., Ichihashi, H.: Regularized linear fuzzy clustering and probabilistic PCA mixture models. IEEE Trans Fuzzy Systems 13(4), 508–516 (2005)CrossRefGoogle Scholar
  21. 21.
    Höppner, F., Klawonn, F., Kruse, R., Runkler, T.: Fuzzy Cluster Analysis. John Wiley & Sons, Ltd., Chichester (1999)MATHGoogle Scholar
  22. 22.
    Ichihashi, H., Miyagishi, K., Honda, K.: Fuzzy c-means clustering with regularization by K-L information. In: Proc. 10th IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE 2001, Melbourne, Australia, pp. 924–927. IEEE Press, Piscataway (2001)Google Scholar
  23. 23.
    Jain, A.K., Dubes, R.C.: Algorithms for Clustering Data. Prentice-Hall, Englewood Cliffs (1988)MATHGoogle Scholar
  24. 24.
    Jajuga, K.: l 1-norm based fuzzy clustering. Fuzzy Sets and Systems 39(1), 43–50 (2003)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Karayiannis, N.B.: MECA: maximum entropy clustering algorithm. In: Proc. 3rd IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE 1994, Orlando, FL, vol. I, pp. 630–635. IEEE Press, Piscataway (1994)CrossRefGoogle Scholar
  26. 26.
    Kaufman, L., Rousseeuw, P.: Finding Groups in Data: An Introduction to Cluster Analysis. John Wiley & Sons, Ltd., New York (1990)CrossRefGoogle Scholar
  27. 27.
    Klawonn, F., Höppner, F.: What Is Fuzzy about Fuzzy Clustering? Understanding and Improving the Concept of the Fuzzifier. In: Berthold, M., Lenz, H.-J., Bradley, E., Kruse, R., Borgelt, C. (eds.) IDA 2003. LNCS, vol. 2810, pp. 254–264. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  28. 28.
    Krishnapuram, R., Keller, J.M.: A possibilistic approach to clustering. IEEE Trans on Fuzzy Systems 1(2), 98–110 (1993)CrossRefGoogle Scholar
  29. 29.
    Krishnapuram, R., Keller, J.M.: The possibilistic c-means algorithm: Insights and recommendations. IEEE Trans on Fuzzy Systems 4(3), 385–393 (1996)CrossRefGoogle Scholar
  30. 30.
    Kullback, S., Leibler, R.A.: On information and sufficiency. Annals of Mathematical Statistics 22, 79–86 (1951)MathSciNetMATHCrossRefGoogle Scholar
  31. 31.
    Li, R.P., Mukaidono, M.: A maximum entropy approach to fuzzy clustering. In: Proc. 4th IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE 1994, Yokohama, Japan, pp. 2227–2232. IEEE Press, Piscataway (1995)Google Scholar
  32. 32.
    Lloyd, S.: Least squares quantization in PCM. IEEE Trans Information Theory 28, 129–137 (1982)MathSciNetMATHCrossRefGoogle Scholar
  33. 33.
    Miyamoto, S., Mukaidono, M.: Fuzzy c-means as a regularization and maximum entropy approach. In: Proc. 7th Int. Fuzzy Systems Association World Congress, IFSA 1997, Prague, Czech Republic, vol. II, pp. 86–92 (1997)Google Scholar
  34. 34.
    Miyamoto, S., Umayahara, K.: Fuzzy clustering by quadratic regularization. In: Proc. IEEE Int. Conf. on Fuzzy Systems/IEEE World Congress on Computational Intelligence, WCCI 1998, Anchorage, AK, vol. 2, pp. 1394–1399. IEEE Press, Piscataway (1998)Google Scholar
  35. 35.
    Mori, Y., Honda, K., Kanda, A., Ichihashi, H.: A unified view of probabilistic PCA and regularized linear fuzzy clustering. In: Proc. Int. Joint Conf. on Neural Networks, IJCNN 2003, Portland, OR, pp. 541–546. IEEE Press, Piscataway (2003)Google Scholar
  36. 36.
    Özdemir, D., Akarun, L.: A fuzzy algorithm for color quantization of images. Pattern Recognition 35, 1785–1791 (2002)MATHCrossRefGoogle Scholar
  37. 37.
    Ruspini, E.H.: A new approach to clustering. Information and Control 15(1), 22–32 (1969); reprinted in [4], 63–70MATHCrossRefGoogle Scholar
  38. 38.
    Shannon, C.E.: The mathematical theory of communication. The Bell System Technical Journal 27, 379–423 (1948)MathSciNetMATHGoogle Scholar
  39. 39.
    Timm, H., Borgelt, C., Döring, C., Kruse, R.: An extension to possibilistic fuzzy cluster analysis. Fuzzy Sets and Systems 147, 3–16 (2004)MathSciNetMATHCrossRefGoogle Scholar
  40. 40.
    Wei, C., Fahn, C.: The multisynapse neural network and its application to fuzzy clustering. IEEE Trans Neural Networks 13(3), 600–618 (2002)CrossRefGoogle Scholar
  41. 41.
    Yang, M.S.: On a class of fuzzy classification maximum likelihood procedures. Fuzzy Sets and Systems 57, 365–375 (2004)CrossRefGoogle Scholar
  42. 42.
    Yasuda, M., Furuhashi, T., Matsuzaki, M., Okuma, S.: Fuzzy clustering using deterministic annealing method and its statistical mechanical characteristics. In: Proc. 10th IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE 2001, Melbourne, Australia, vol. 2, pp. 797–800. IEEE Press, Piscataway (2001)Google Scholar
  43. 43.
    Yu, J., Yang, M.S.: A generalized fuzzy clustering regularization model with optimality tests and model complexity analysis. IEEE Trans Fuzzy Systems 15(5), 904–915 (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Edificio de InvestigaciónEuropean Centre for Soft ComputingMieresSpain

Personalised recommendations