Skip to main content
SpringerLink
Log in

Objective Functions for Fuzzy Clustering

  • Conference paper
Computational Intelligence in Intelligent Data Analysis

Part of the book series: Studies in Computational Intelligence ((SCI,volume 445))

Abstract

Fuzzy clustering comprises a family of prototype-based clustering methods that can be formulated as the problem of minimizing an objective function. These methods can be seen as “fuzzifications” of, for example, the classical c-means algorithm, which strives to minimize the sum of the (squared) distances between the data points and the cluster centers to which they are assigned. However, it is well known that in order to “fuzzify” such a crisp clustering approach, it is not enough to merely allow values from the unit interval for the variables encoding the assignments of the data points to the clusters (that is, for the elements of the partition matrix): the minimum is still obtained for a crisp data point assignment. As a consequence, additional means have to be employed in the objective function in order to obtain actual degrees of membership. This paper surveys the most common fuzzification means and examines and compares their properties.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ball, G.H., Hall, D.J.: A clustering technique for summarizing multivariate data. Behavioral Science 12(2), 153–155 (1967)

    Article  Google Scholar 

  2. Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York (1981)

    Book  MATH  Google Scholar 

  3. Bezdek, J.C., Hathaway, R.J.: Visual cluster validity (VCV) displays for prototype generator clustering methods. In: Proc. 12th IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE 2003, Saint Louis, MO, vol. 2, pp. 875–880. IEEE Press, Piscataway (2003)

    Chapter  Google Scholar 

  4. Bezdek, J.C., Pal, N.: Fuzzy Models for Pattern Recognition. IEEE Press, New York (1992)

    Google Scholar 

  5. Bezdek, J.C., Keller, J., Krishnapuram, R., Pal, N.: Fuzzy Models and Algorithms for Pattern Recognition and Image Processing. Kluwer, Dordrecht (1999)

    MATH  Google Scholar 

  6. Bilmes, J.: A gentle tutorial on the EM algorithm and its application to parameter estimation for gaussian mixture and hidden markov models. Tech. Rep. ICSI-TR-97-021, University of Berkeley, CA, USA (1997)

    Google Scholar 

  7. Borgelt, C.: Prototype-based classification and clustering. Habilitationsschrift, Otto-von-Guericke University of Magdeburg, Germany (2005)

    Google Scholar 

  8. Boujemaa, N.: Generalized competitive clustering for image segmentation. In: Proc. 19th Int. Meeting North American Fuzzy Information Processing Society, NAFIPS 2000, Atlanta, GA, pp. 133–137. IEEE Press, Piscataway (2000)

    Google Scholar 

  9. Daróczy, Z.: Generalized information functions. Information and Control 16(1), 36–51 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  10. Davé, R.N., Krishnapuram, R.: Robust clustering methods: A unified view. IEEE Trans on Fuzzy Systems 5(1997), 270–293 (1997)

    Article  Google Scholar 

  11. Dempster, A.P., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society (Series B) 39, 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  12. Döring, C., Borgelt, C., Kruse, R.: Effects of irrelevant attributes in fuzzy clustering. In: Proc. 14th IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE 2005, Reno, NV, pp. 862–866. IEEE Press, Piscataway (2005)

    Chapter  Google Scholar 

  13. Dunn, J.C.: A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters. Journal of Cybernetics 3(3), 32–57 (1973); reprinted in [4], 82–101

    Article  MathSciNet  MATH  Google Scholar 

  14. Everitt, B.S.: Cluster Analysis. Heinemann, London (1981)

    Google Scholar 

  15. Everitt, B.S., Hand, D.J.: Finite Mixture Distributions. Chapman & Hall, London (1981)

    Book  MATH  Google Scholar 

  16. Frigui, H., Krishnapuram, R.: Clustering by competitive agglomeration. Pattern Recognition 30(7), 1109–1119 (1997)

    Article  Google Scholar 

  17. Gath, I., Geva, A.B.: Unsupervised optimal fuzzy clustering. IEEE Trans Pattern Analysis and Machine Intelligence (PAMI) 11, 773–781 (1989); reprinted in [4], 211–218

    Article  Google Scholar 

  18. Gustafson, E.E., Kessel, W.C.: Fuzzy clustering with a fuzzy covariance matrix. In: Proc. of the IEEE Conf. on Decision and Control, CDC 1979, San Diego, CA, pp. 761–766. IEEE Press, Piscataway (1979); reprinted in [4], 117–122

    Google Scholar 

  19. Hartigan, J.A., Wong, M.A.: A k-means clustering algorithm. Applied Statistics 28, 100–108 (1979)

    Article  MATH  Google Scholar 

  20. Honda, K., Ichihashi, H.: Regularized linear fuzzy clustering and probabilistic PCA mixture models. IEEE Trans Fuzzy Systems 13(4), 508–516 (2005)

    Article  Google Scholar 

  21. Höppner, F., Klawonn, F., Kruse, R., Runkler, T.: Fuzzy Cluster Analysis. John Wiley & Sons, Ltd., Chichester (1999)

    MATH  Google Scholar 

  22. Ichihashi, H., Miyagishi, K., Honda, K.: Fuzzy c-means clustering with regularization by K-L information. In: Proc. 10th IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE 2001, Melbourne, Australia, pp. 924–927. IEEE Press, Piscataway (2001)

    Google Scholar 

  23. Jain, A.K., Dubes, R.C.: Algorithms for Clustering Data. Prentice-Hall, Englewood Cliffs (1988)

    MATH  Google Scholar 

  24. Jajuga, K.: l 1-norm based fuzzy clustering. Fuzzy Sets and Systems 39(1), 43–50 (2003)

    Article  MathSciNet  Google Scholar 

  25. Karayiannis, N.B.: MECA: maximum entropy clustering algorithm. In: Proc. 3rd IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE 1994, Orlando, FL, vol. I, pp. 630–635. IEEE Press, Piscataway (1994)

    Chapter  Google Scholar 

  26. Kaufman, L., Rousseeuw, P.: Finding Groups in Data: An Introduction to Cluster Analysis. John Wiley & Sons, Ltd., New York (1990)

    Book  Google Scholar 

  27. Klawonn, F., Höppner, F.: What Is Fuzzy about Fuzzy Clustering? Understanding and Improving the Concept of the Fuzzifier. In: Berthold, M., Lenz, H.-J., Bradley, E., Kruse, R., Borgelt, C. (eds.) IDA 2003. LNCS, vol. 2810, pp. 254–264. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  28. Krishnapuram, R., Keller, J.M.: A possibilistic approach to clustering. IEEE Trans on Fuzzy Systems 1(2), 98–110 (1993)

    Article  Google Scholar 

  29. Krishnapuram, R., Keller, J.M.: The possibilistic c-means algorithm: Insights and recommendations. IEEE Trans on Fuzzy Systems 4(3), 385–393 (1996)

    Article  Google Scholar 

  30. Kullback, S., Leibler, R.A.: On information and sufficiency. Annals of Mathematical Statistics 22, 79–86 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  31. Li, R.P., Mukaidono, M.: A maximum entropy approach to fuzzy clustering. In: Proc. 4th IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE 1994, Yokohama, Japan, pp. 2227–2232. IEEE Press, Piscataway (1995)

    Google Scholar 

  32. Lloyd, S.: Least squares quantization in PCM. IEEE Trans Information Theory 28, 129–137 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  33. Miyamoto, S., Mukaidono, M.: Fuzzy c-means as a regularization and maximum entropy approach. In: Proc. 7th Int. Fuzzy Systems Association World Congress, IFSA 1997, Prague, Czech Republic, vol. II, pp. 86–92 (1997)

    Google Scholar 

  34. Miyamoto, S., Umayahara, K.: Fuzzy clustering by quadratic regularization. In: Proc. IEEE Int. Conf. on Fuzzy Systems/IEEE World Congress on Computational Intelligence, WCCI 1998, Anchorage, AK, vol. 2, pp. 1394–1399. IEEE Press, Piscataway (1998)

    Google Scholar 

  35. Mori, Y., Honda, K., Kanda, A., Ichihashi, H.: A unified view of probabilistic PCA and regularized linear fuzzy clustering. In: Proc. Int. Joint Conf. on Neural Networks, IJCNN 2003, Portland, OR, pp. 541–546. IEEE Press, Piscataway (2003)

    Google Scholar 

  36. Özdemir, D., Akarun, L.: A fuzzy algorithm for color quantization of images. Pattern Recognition 35, 1785–1791 (2002)

    Article  MATH  Google Scholar 

  37. Ruspini, E.H.: A new approach to clustering. Information and Control 15(1), 22–32 (1969); reprinted in [4], 63–70

    Article  MATH  Google Scholar 

  38. Shannon, C.E.: The mathematical theory of communication. The Bell System Technical Journal 27, 379–423 (1948)

    MathSciNet  MATH  Google Scholar 

  39. Timm, H., Borgelt, C., Döring, C., Kruse, R.: An extension to possibilistic fuzzy cluster analysis. Fuzzy Sets and Systems 147, 3–16 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  40. Wei, C., Fahn, C.: The multisynapse neural network and its application to fuzzy clustering. IEEE Trans Neural Networks 13(3), 600–618 (2002)

    Article  Google Scholar 

  41. Yang, M.S.: On a class of fuzzy classification maximum likelihood procedures. Fuzzy Sets and Systems 57, 365–375 (2004)

    Article  Google Scholar 

  42. Yasuda, M., Furuhashi, T., Matsuzaki, M., Okuma, S.: Fuzzy clustering using deterministic annealing method and its statistical mechanical characteristics. In: Proc. 10th IEEE Int. Conf. on Fuzzy Systems, FUZZ-IEEE 2001, Melbourne, Australia, vol. 2, pp. 797–800. IEEE Press, Piscataway (2001)

    Google Scholar 

  43. Yu, J., Yang, M.S.: A generalized fuzzy clustering regularization model with optimality tests and model complexity analysis. IEEE Trans Fuzzy Systems 15(5), 904–915 (2007)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Edificio de Investigación, European Centre for Soft Computing, 33600, Mieres, Asturias, Spain

    Christian Borgelt

Authors
  1. Christian Borgelt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Borgelt .

Editor information

Editors and Affiliations

  1. Faculty of Computer Science, Otto-von-Guericke-University Magdeburg, Universitätsplatz 2, Magdeburg, 39106, Germany

    Christian Moewes

  2. , Faculty of Computer Science, Otto-von-Guericke University Magdeburg, Universitätsplatz 2, Magdeburg, 39106, Germany

    Andreas Nürnberger

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Borgelt, C. (2013). Objective Functions for Fuzzy Clustering. In: Moewes, C., Nürnberger, A. (eds) Computational Intelligence in Intelligent Data Analysis. Studies in Computational Intelligence, vol 445. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32378-2_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-32378-2_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-32377-5

  • Online ISBN: 978-3-642-32378-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation