Skip to main content
SpringerLink
Log in
Book cover

Optimization and Its Applications in Control and Data Sciences pp 353–380Cite as

Applying the Gradient Projection Method to a Model of Proportional Membership for Fuzzy Cluster Analysis

Part of the Springer Optimization and Its Applications book series (SOIA,volume 115)

Abstract

This paper presents a fuzzy proportional membership model for clustering (FCPM). Unlike the other clustering models, FCPM requires that each entity may express an extent of each prototype, which makes its criterion to loose the conventional prototype-additive structure. The methods for fitting the model at different fuzziness parameter values are presented. Because of the complexity of the clustering criterion, minimization of the errors requires the gradient projection method (GPM). We discuss how to find the projection of a vector on the simplex of the fuzzy membership vectors and how the stepsize length of the GPM had been fixed. The properties of the clusters found with the FCPM are discussed. Especially appealing seems the property to keep the extremal cluster prototypes stable even after addition of many entities around the grand mean.

Keywords

  • Fuzzy proportional membership
  • Gradient projection method
  • Extremal cluster prototype

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    In the follow up text the number of clusters is denoted by K instead of c as in the FCM.

References

  1. Abonyi, J., Feil, B.: Cluster Analysis for Data Mining and System Identification. Springer, Berlin (2007)

    MATH  Google Scholar 

  2. Abu-Jamous, B., Fa, R., Nandi, A.K.: Integrative Cluster Analysis in Bioinformatics. Wiley, New York (2015)

    CrossRef  Google Scholar 

  3. Ada, I., Berthold, M.R.: The new iris data: modular data generators. In: KDD’10 Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 413–422 (2010)

    Google Scholar 

  4. Berthold, M.R., Cebron, N., Dill, F., Gabriel, T.R., Kotter, T., Meinl, T., Ohl, P., Sieb, C., Thiel, K., Wiswedel, B.: KNIME: the Konstanz information miner. In: Studies in Classification, Data Analysis, and Knowledge Organization (GfKL 2007). Springer, Berlin (2007)

    Google Scholar 

  5. Bertsekas, D.: On the Goldstein-Levitin-Polyak gradient projection method. IEEE Trans. Autom. Control 21 (2), 174–184 (1976)

    CrossRef  MathSciNet  MATH  Google Scholar 

  6. Bertsekas, D.: Nonlinear Programming. Athena Scientific, Belmont (1995)

    MATH  Google Scholar 

  7. Bezdek, J.: Fuzzy mathematics in pattern classification. Ph.D. Thesis, Applied Mathematics Center, Cornell University, Ithaca (1973)

    Google Scholar 

  8. Bezdek, J.: Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York (1981)

    CrossRef  MATH  Google Scholar 

  9. Bezdek, J.C., Pal, S.K. (eds.): Fuzzy Models for Pattern Recognition. IEEE Press, New York (1992)

    Google Scholar 

  10. Bezdek, J., Keller, J., Krishnapuram, R., Pal, T.: Fuzzy Models and Algorithms for Pattern Recognition and Image Processing. Kluwer Academic, Norwell (1999)

    CrossRef  MATH  Google Scholar 

  11. Bowers, C., Beale, R., Hendley, R.: Identifying archetypal perspectives in news articles. In: Proceedings of the 26th Annual BCS Interaction Specialist Group Conference on People and Computers (BCS-HCI’12), pp. 327–332 (2012)

    Google Scholar 

  12. Cannon, R., Dave, J., Bezdek, J.: Efficient implementation of the fuzzy c-means clustering algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 8, 248–255 (1986)

    CrossRef  MATH  Google Scholar 

  13. Chan, B., Mitchell, D., Cram, L.: Archetypal analysis of galaxy spectra. Mon. Not. R. Astron. Soc. 338, 790–795 (2003)

    CrossRef  Google Scholar 

  14. Chiang, M.M.-T., Mirkin, B.: Intelligent choice of the number of clusters in K-means clustering: an experimental study with different cluster spreads. J. Classif. 27 (1), 3–40 (2010)

    CrossRef  MathSciNet  MATH  Google Scholar 

  15. Cutler, A., Breiman, L.: Archetypal analysis. Technometrics 36 (4), 338–347 (1994)

    CrossRef  MathSciNet  MATH  Google Scholar 

  16. Dunn, J.: A fuzzy relative of the ISODATA process and its use in detecting compact, well-separated clusters. J. Cybern 3, 32–57 (1974)

    CrossRef  MathSciNet  MATH  Google Scholar 

  17. Eisenack, K., Lüdeke, M., Kropp, J.: Construction of archetypes as a formal method to analyze social-ecological systems. In: Proceedings of the Institutional Dimensions of Global Environmental Change Synthesis Conference (2006)

    Google Scholar 

  18. Elder, A., Pinnel, J.: Archetypal analysis: an alternative approach to finding defining segments. In: 2003 Sawtooth Software Conference Proceedings, pp. 113–129. Sequim, WA (2003)

    Google Scholar 

  19. Eugster, M.: Performance profiles based on archetypal athletes. Int. J. Perform. Anal. Sport 12 (1), 166–187 (2012)

    Google Scholar 

  20. Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P.: From data mining to knowledge discovery: an overview. In: Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining. AAAI Press/The MIT Press, Menlo Park (1996)

    Google Scholar 

  21. Fu, L., Medico, E.: FLAME, a novel fuzzy clustering method for the analysis of DNA microarray data. BMC Bioinf. 8, 3 (2007)

    CrossRef  Google Scholar 

  22. Geng, L., Hamilton, H.J.: Interestingness measures for data mining: a survey. ACM Comput. Surv. 38 (3), 9 (2006)

    CrossRef  Google Scholar 

  23. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.: The WEKA data mining software: an update. ACM SIGKDD Explor. Newslett. 11 (1):10–18 (2009)

    CrossRef  Google Scholar 

  24. Höppner, F., Klawonn, F., Kruse, R., Runkler, T.: Fuzzy Cluster Analysis: Methods for Classification, Data Analysis and Image Recognition. Wiley, New York (1999)

    MATH  Google Scholar 

  25. Jain, A.K., Dubes, R.C.: Algorithms for Clustering Data. Prentice Hall, Englewood Cliffs (1988)

    MATH  Google Scholar 

  26. Jimenez, L., Landgrebe, D.: Supervised classification in high-dimensional space: geometrical, statistical and asymptotical properties of multivariate data. IEEE Trans. Syst. Man Cybern. Part C 28 (1):39–54 (1998)

    CrossRef  Google Scholar 

  27. Kriegel, H., Kröger, P., Zimek, A.: Clustering high-dimensional data: a survey on subspace clustering, pattern-based clustering, and correlation clustering. ACM Trans. Knowl. Discov. Data 3 (1), 1–58 (2009)

    CrossRef  Google Scholar 

  28. Levitin, E.S., Polyak, B.T.: Constrained minimization problems. USSR Comput. Math. Math. Phys. 6 (5), 1–50 (1966) (English transl. of paper in Zh. Vychisl. Mat. i Mat. Fiz., 6 (5), 787–823, 1965)

    Google Scholar 

  29. Li, S., Wang, P., Louviere, J., Carson, R.: Archetypal analysis: a new way to segment markets based on extreme individuals. In: A Celebration of Ehrenberg and Bass: Marketing Knowledge, Discoveries and Contribution, ANZMAC 2003 Conference Proceedings, pp. 1674–1679, Adelaide (2003)

    Google Scholar 

  30. Lin, C.-J.: Projected gradient methods for nonnegative matrix factorization. Neural Comput. 19 (10), 2756–2779 (2007)

    CrossRef  MathSciNet  MATH  Google Scholar 

  31. Mezzich, J.E., Solomon, H.: Taxonomy and Behavioral Science. Academic Press, London (1980)

    Google Scholar 

  32. Milligan, G.W.: An algorithm for generating artificial test clusters. Psychometrika 50 (1), 123–127 (1985)

    CrossRef  Google Scholar 

  33. Mirkin, B.: Mathematical classification and clustering. Nonconvex Optimization and Its Applications. Kluwer Academic, Dordrecht (1996)

    CrossRef  MATH  Google Scholar 

  34. Mirkin, B.: Clustering: A Data Recovery Approach, 2nd edn. Chapman & Hall/CRC Press, London/Boca Raton (2012)

    CrossRef  MATH  Google Scholar 

  35. Mirkin, B., Satarov, G: Method of fuzzy additive types for analysis of multidimensional data. Autom. Remote. Control. 51 (5,6), 1:683–688, 2:817–821 (1990)

    Google Scholar 

  36. Mørup, M., Hansen, L.: Archetypal analysis for machine learning and data mining. Neurocomputing 80, 54–63 (2012)

    CrossRef  Google Scholar 

  37. Narayanan, S.J., Bhatt, R.B., Paramasivam, I., Khalid, M., Tripathy, B.K.: Induction of fuzzy decision trees and its refinement using gradient projected-neuro-fuzzy decision tree. Int. J. Adv. Intell. Paradigms 6 (4), 346–369 (2014)

    CrossRef  Google Scholar 

  38. Nascimento, S.: Fuzzy clustering via proportional membership model. Frontiers of Artificial Intelligence and Applications, vol. 119. IOS Press, Amsterdam (2005). ISBN: 1 58603 489 8 (reprinted 2006)

    Google Scholar 

  39. Nascimento, S., Mirkin, B., Moura-Pires, F.: A fuzzy clustering model of data and fuzzy c-means. In: Langari, R. (ed.) Proceedings of The 9th IEEE International Conference on Fuzzy Systems, Fuzz-IEEE 2000. Soft Computing in the Information Age, IEEE Neural Networks Council, pp. 302–307. IEEE Press, New York (ISSN 1098-7584) (2000)

    Google Scholar 

  40. Nascimento, S., Mirkin, B., Moura-Pires, F.: Modeling proportional membership in fuzzy clustering. IEEE Trans. Fuzzy Syst. 11 (2), 173–186 (2003)

    CrossRef  MATH  Google Scholar 

  41. Oliveira, J.V., Pedrycz, W.: Advances in Fuzzy Clustering and Its Applications. Wiley, New York (2007)

    CrossRef  Google Scholar 

  42. Omari, A., Langer, R., Conrad, S.: TARtool: a temporal dataset generator for market basket analysis. In: Advanced Data Mining and Applications, pp. 400–410. Springer, Berlin (2008)

    Google Scholar 

  43. Pal, N.R., Bezdek, J.: On cluster validity for the fuzzy c-means model. IEEE Trans. Fuzzy Syst. 3, 370–379 (1995)

    CrossRef  Google Scholar 

  44. Pei, Y., Zaiane, O.: A synthetic data generator for clustering and outlier analysis, Technical Report ID:TR06-15, 33 p. (2006)

    Google Scholar 

  45. Polyak, B.: A general method for solving extremum problems. Sov. Math. Dokl. 8 (3), 593–597 (1967)

    MATH  Google Scholar 

  46. Polyak, B.: Introduction to Optimization. Optimization Software, New York (1987)

    MATH  Google Scholar 

  47. Porzio, G., Ragozini, G., Vistocco, D.: On the use of archetypes as benchmarks. Appl. Stoch. Model. Bus. Ind. 24 (5), 419–437 (2008)

    CrossRef  MathSciNet  MATH  Google Scholar 

  48. Runkler, T.A.: Data Analytics, Models and Algorithms for Intelligent Data Analysis. Springer, Berlin (2012)

    CrossRef  Google Scholar 

  49. Ruspini, E.: A new approach to clustering. Inf. Control. 15, 22–32 (1969)

    CrossRef  MATH  Google Scholar 

  50. Schmidt, M., Kim, D., Sra, S.: Projected Newton-type methods in machine learning. In: Optimization for Machine Learning, pp. 305–330. MIT Press, Cambridge (2011)

    Google Scholar 

  51. Steinbach, M., Ertoz, L., Kumar, V.: The challenges of clustering high dimensional data. In: Wille, L.T. (ed.) New Directions in Statistical Physics, pp. 273–309. Springer, Berlin (2004)

    CrossRef  Google Scholar 

  52. Steinley, D., Brusco, M.: Initializing K-means batch clustering: a critical evaluation of several techniques. J. Classif. 24 (1), 99–121 (2007)

    CrossRef  MathSciNet  MATH  Google Scholar 

  53. Sun, J., Zhao, W., Xue, J., Shen, Z. Shen, Y.: Clustering with feature order preferences. Intell. Data Anal. 14, 479–495 (2010)

    Google Scholar 

  54. Vendramin, L., Naldi, M.C., Campello, R.J.G.B.: Fuzzy clustering algorithms and validity indices for distributed data. In: Emre Celebi, M. (ed.) Partitional Clustering Algorithms. Springer, Berlin (2015)

    Google Scholar 

  55. Yeh, C-Y, Huang, C.-W., Lee, S.-J.: Multi-Kernel support vector clustering for multi-class classification. In: Proceedings of The 3rd International Conference on Innovative Computing Information and Control (ICICIC’08). IEEE, New York (2008)

    Google Scholar 

Download references

Acknowledgements

I wish to thank Professor Boris Mirkin for having introduce me to fuzzy clustering within the Data Recovery paradigm, and for all the years of discussion that we have been sharing ever since. The comments from the anonymous Reviewers are greatly acknowledged as they contributed to improve the paper.

Author information

Authors and Affiliations

  1. Department of Computer Science and NOVA Laboratory for Computer Science and Informatics (NOVA LINCS), Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa, 2829-516, Caparica, Portugal

    Susana Nascimento

Authors
  1. Susana Nascimento
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Susana Nascimento .

Editor information

Editors and Affiliations

  1. Department of Industrial and Systems Engineering, Ohio University, Athens, Ohio, USA

    Boris Goldengorin

Rights and permissions

Reprints and Permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Nascimento, S. (2016). Applying the Gradient Projection Method to a Model of Proportional Membership for Fuzzy Cluster Analysis. In: Goldengorin, B. (eds) Optimization and Its Applications in Control and Data Sciences. Springer Optimization and Its Applications, vol 115. Springer, Cham. https://doi.org/10.1007/978-3-319-42056-1_13

Download citation

search

Navigation