Skip to main content
SpringerLink
Log in

Discriminatively embedded fuzzy K-Means clustering with feature selection strategy

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Fuzzy K-Means clustering (FKM) is one of the most popular methods to partition data into clusters. Traditional FKM and its extensions perform fuzzy clustering based on original high-dimensional features. However, the presence of noisy and redundant features would cause the degradation of clustering performance. To avoid this problem, we integrate fuzzy clustering and feature selection into a unified model where the structured sparsity-inducing norm is imposed on the transformation matrix to determine the valuable feature subse adaptively. The clustering task and feature selection process are promoted mutually. To solve this model, an iterative algorithm is developed. Extensive experiments conducted on benchmark data sets demonstrate the effectiveness of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://www.cs.toronto.edu/~kriz/cifar.html.

  2. https://jundongl.github.io/scikit-feature/datasets.html.

  3. https://archive.ics.uci.edu/ml/datasets.php.

  4. https://archive.ics.uci.edu/ml/index.php.

References

  1. Cai D, He X, Han J (2005) Document clustering using locality preserving indexing. IEEE Trans Knowl Data Eng 17(12):1624–1637

    Article  Google Scholar 

  2. Cai W, Chen S, Zhang D (2007) Fast and robust fuzzy c-means clustering algorithms incorporating local information for image segmentation. Pattern Recogn 40(3):825–838

    Article  MATH  Google Scholar 

  3. Cannon RL, Dave JV, Bezdek JC (1986) Efficient implementation of the fuzzy c-means clustering algorithms. IEEE Trans Pattern Anal Mach Intell (2):248–255

  4. Chang X, Wang Q, Liu Y, Wang Y (2016) Sparse regularization in fuzzy c-means for high-dimensional data clustering. IEEE Trans Cybern 47(9):2616–2627

    Article  Google Scholar 

  5. Chatzis S, Varvarigou T (2008) Robust fuzzy clustering using mixtures of student’s-t distributions. Pattern Recogn Lett 29(13):1901–1905

    Article  Google Scholar 

  6. Chen X, Cai D (2011) Large scale spectral clustering with landmark-based representation. In: Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence. AAAI Press, pp 313–318

  7. Chen X, Hong W, Nie F, He D, Yang M, Huang JZ (2018) Spectral clustering of large-scale data by directly solving normalized cut. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery and data mining. Association for computing machinery, New York, pp 1206–1215

  8. Chuang KS, Tzeng HL, Chen S, Wu J, Chen TJ (2006) Fuzzy c-means clustering with spatial information for image segmentation. Comput Med Imaging Graph 30(1):9–15

    Article  Google Scholar 

  9. Dietterich TG, Bakiri G (1994) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286

    Article  MATH  Google Scholar 

  10. Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660

    Article  Google Scholar 

  11. Gorodnitsky I, Rao B (1997) Sparse signal reconstruction from limited data using focuss: a re-weighted minimum norm algorithm. IEEE Trans Signal Process 45(3):600–616

    Article  Google Scholar 

  12. Graham DB, Allinson NM (1998) Characterising virtual eigensignatures for general purpose face recognition. In: Face recognition. Springer, pp 446–456

  13. Guo L, Chen L, Lu X, Chen CP (2019) Membership affinity lasso for fuzzy clustering. IEEE Trans Fuzzy Syst 28(2):294–307

    Article  Google Scholar 

  14. Huang J, Nie F, Huang H (2015) A new simplex sparse learning model to measure data similarity for clustering. In: Proceedings of the 24th international conference on artificial intelligence. AAAI Press, pp 3569–3575

  15. Huang JZ, Ng MK, Rong H, Li Z (2005) Automated variable weighting in k-means type clustering. IEEE Trans Pattern Anal Mach Intell 27(5):657–668

    Article  Google Scholar 

  16. Ji S, Ye J (2008) Generalized linear discriminant analysis: a unified framework and efficient model selection. IEEE Trans Neural Netw 19(10):1768–1782

    Article  Google Scholar 

  17. Ji Z, Xia Y, Chen Q, Sun Q, Xia D, Feng DD (2012) Fuzzy c-means clustering with weighted image patch for image segmentation. Appl Soft Comput 12(6):1659–1667

    Article  Google Scholar 

  18. Jing L, Ng MK, Huang JZ (2007) An entropy weighting k-means algorithm for subspace clustering of high-dimensional sparse data. IEEE Trans Knowl Data Eng 19(8):1026–1041

    Article  Google Scholar 

  19. Li MJ, Ng MK, Cheung Ym, Huang J (2008) Agglomerative fuzzy k-means clustering algorithm with selection of number of clusters. IEEE Trans Knowl Data Eng 20(11):1519–1534

    Article  Google Scholar 

  20. Li Y, Hu P, Liu Z, Peng D, Zhou JT, Peng X (2021) Contrastive clustering. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 8547–8555

  21. Li Y, Yang M, Peng D, Li T, Huang J, Peng X (2022) Twin contrastive learning for online clustering. Int J Comput Vis:1–17

  22. Liu H, Han J, Nie F, Li X (2017) Balanced clustering with least square regression. In: AAAI

  23. Martinez A, Benavente R (1998) The ar face database CVC technical Report24

  24. Ng AY, Jordan MI, Weiss Y (2001) On spectral clustering: analysis and an algorithm. In: Proceedings of the 14th international conference on neural information processing systems: natural and synthetic. MIT Press, Cambridge, pp 849–856

  25. Nie F, Huang H, Cai X, Ding C (2010) Efficient and robust feature selection via joint ?<sub> 2,1</sub>-norms minimization. In: Proceedings of the 23rd International Conference on Neural Information Processing Systems - Volume 2. Curran Associates Inc., Red Hook, pp 1813–1821

  26. Nie F, Xue J, Wu D, Wang R, Li H, Li X (2021) Coordinate descent method for k k-means. IEEE Trans Pattern Anal Mach Intell 44(5):2371–2385

    Google Scholar 

  27. Nie F, Zeng Z, Tsang IW, Xu D, Zhang C (2011) Spectral embedded clustering: a framework for in-sample and out-of-sample spectral clustering. IEEE Trans Neural Netw 22(11):1796–1808

    Article  Google Scholar 

  28. Nie F, Zhao X, Wang R, Li X, Li Z (2022) Fuzzy k-means clustering with discriminative embedding. IEEE Trans Knowl Data Eng 34(03):1221–1230

    Article  Google Scholar 

  29. Nie F, Zhao X, Wang R, Li X, Li Z (2022) Fuzzy k-means clustering with discriminative embedding. IEEE Trans Knowl Data Eng 34(3):1221–1230

    Article  Google Scholar 

  30. Pal NR, Bezdek JC (1995) On cluster validity for the fuzzy c-means model. IEEE Trans Fuzzy Syst 3(3):370–379

    Article  Google Scholar 

  31. Passerini A, Pontil M, Frasconi P (2004) New results on error correcting output codes of kernel machines. IEEE Trans Neural Netw 15(1):45–54

    Article  Google Scholar 

  32. Peng X, Li Y, Tsang IW, Zhu H, Lv J, Zhou JT (2022) Xai beyond classification: interpretable neural clustering. J Mach Learn Res 23:6–1

    MathSciNet  MATH  Google Scholar 

  33. Qian P, Jiang Y, Deng Z, Hu L, Sun S, Wang S, Muzic RF (2016) Cluster prototypes and fuzzy memberships jointly leveraged cross-domain maximum entropy clustering. IEEE Trans Cybern 46(1):181–193. 10.1109/TCYB.2015.2399351

    Article  Google Scholar 

  34. Wang J, Su X, An improved k-means clustering algorithm (2011). In: 2011 IEEE 3rd international conference on communication software and networks, pp 44–46. https://doi.org/10.1109/ICCSN.2011.6014384https://doi.org/10.1109/ICCSN.2011.6014384

  35. Wang X, Wu P, Xu Q, Zeng Z, Xie Y (2021) Joint image clustering and feature selection with auto-adjoined learning for high-dimensional data. Knowl-Based Syst 232:107443

    Article  Google Scholar 

  36. Wang XD, Chen RC, Yan F, Zeng ZQ, Hong CQ (2019) Fast adaptive k-means subspace clustering for high-dimensional data. IEEE Access 7:42639–42651

    Article  Google Scholar 

  37. Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemom Intell Lab Syst 2(1):37–52. Proceedings of the Multivariate Statistical Workshop for Geologists and Geochemists

    Article  Google Scholar 

  38. Wu KL, Yu J, Yang MS (2005) A novel fuzzy clustering algorithm based on a fuzzy scatter matrix with optimality tests. Pattern Recogn Lett 26(5):639–652

    Article  Google Scholar 

  39. Xu J, Han J, Xiong K, Nie F (2016) Robust and sparse fuzzy k-means clustering. In: IJCAI, pp 2224–2230

  40. Yu J, Yang MS (2005) Optimality test for generalized fcm and its application to parameter selection. IEEE Trans Fuzzy Syst 13(1):164–176

    Article  Google Scholar 

  41. Yu J, Yang MS (2007) A generalized fuzzy clustering regularization model with optimality tests and model complexity analysis. Fuzzy Syst, IEEE Trans 15:904 – 915. https://doi.org/10.1109/TFUZZ.2006.889957https://doi.org/10.1109/TFUZZ.2006.889957

    Article  Google Scholar 

  42. Zhang H, Li H, Chen N, Chen S, Liu J (2022) Novel fuzzy clustering algorithm with variable multi-pixel fitting spatial information for image segmentation. Pattern Recogn 121:108201

    Article  Google Scholar 

  43. Zhang R, Li X, Zhang H, Nie F (2019) Deep fuzzy k-means with adaptive loss and entropy regularization. IEEE Trans Fuzzy Syst 28(11):2814–2824

    Article  Google Scholar 

  44. Zhang R, Nie F, Guo M, Wei X, Li X (2018) Joint learning of fuzzy k-means and nonnegative spectral clustering with side information. IEEE Trans Image Process 28(5):2152–2162

    Article  MathSciNet  MATH  Google Scholar 

  45. Zhang R, Tong H, Xia Y, Zhu Y (2019) Robust embedded deep k-means clustering. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 1181–1190

  46. Zhang Y, Bai X, Fan R, Wang Z (2019) Deviation-sparse fuzzy c-means with neighbor information constraint. IEEE Trans Fuzzy Syst 27(1):185–199. https://doi.org/10.1109/TFUZZ.2018.2883033https://doi.org/10.1109/TFUZZ.2018.2883033

    Article  Google Scholar 

  47. hua Zhao Q, li Li X, Li Y, mei Zhao X (2017) A fuzzy clustering image segmentation algorithm based on hidden markov random field models and voronoi tessellation. Pattern Recogn Lett 85:49–55

    Article  Google Scholar 

  48. Zhao X, Nie F, Wang R, Li X (2021) Robust fuzzy k-means clustering with shrunk patterns learning. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2021.3116257

  49. Zhao X, Nie F, Wang R, Li X (2022) Improving projected fuzzy k-means clustering via robust learning. Neurocomputing 491:34–43. https://doi.org/10.1016/j.neucom.2022.03.043

    Article  Google Scholar 

  50. Zhi Xb, Fan Jl, Zhao F (2013) Fuzzy linear discriminant analysis-guided maximum entropy fuzzy clustering algorithm. Pattern Recogn 46(6):1604–1615

    Article  MATH  Google Scholar 

Download references

Acknowledgments

This work is supported by the Program for Innovative Research Team (in Science and Technology) in University of Henan Province (No. 22IRTSTHN016), the funding scheme of Key scientific research of Henan’s higher education institutions (No. 23A520010), the Key R & D and promotion Special Project of Science and Technology Department of Henan Province (No. 222102210104) and the teaching reform research and practice project of higher education in Henan Province in (No.2021SJGLX502).

Author information

Authors and Affiliations

  1. Luoyang Normal University, Luoyang, 471022, China

    Peng Zhao, Yongxin Zhang & Youzhong Ma

  2. School of Computer Science and School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an, 710072, Shaanxi, People’s Republic of China

    Xiaowei Zhao

  3. Northwest University, Xi’an, 710072, China

    Xunli Fan

Authors
  1. Peng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongxin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Youzhong Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaowei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xunli Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yongxin Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, P., Zhang, Y., Ma, Y. et al. Discriminatively embedded fuzzy K-Means clustering with feature selection strategy. Appl Intell 53, 18959–18970 (2023). https://doi.org/10.1007/s10489-022-04376-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04376-5

Keywords

Search

Navigation