Skip to main content
SpringerLink
Log in

Multiple kernel k-means clustering with block diagonal property

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Multiple kernel k-means clustering (MKKC) is proposed to efficiently incorporate multiple base kernels to generate an optimal kernel. However, many existing MKKC methods all involve two stages: learning a clustering indicator matrix and performing clustering on it. This cannot ensure the ultimate clustering results are optimal because the optimal values of two steps are not equivalent to those of the original problem. To address this issue, in this paper, we propose a novel method named multiple kernel k-means clustering with block diagonal property (MKKC-BD). It is the first time to find the relationship between an indicator matrix and Laplacian matrix of the graph theory and get a block diagonal (BD) representation of the indicator matrix. By imposing the BD constraint on the indicator matrix, the BD property of the indicator matrix is ensured. Further, the explicit clustering results are generated directly from the unified framework integrating the three processes of learning an optimal kernel, an indicator matrix and clustering results, which shows the clustering task is executed just by one step. In addition, a simple kernel weight strategy is used in this framework to obtain the optimal kernel, where the value of each kernel weight directly reveals the relationship of each base kernel and the optimal kernel. Finally, by extensive experiments on ten data sets and comparison of clustering results with eight state-of-the-art multiple kernel clustering methods, it is concluded that MKKC-BD is effective. Our code is available at https://github.com/mathchen-git/MKKC-BD.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

The datasets used in the experiment are available in the corresponding website at http://featureselection.asu.edu/data sets.php (i.e., AR), https://jundongl.github.io/scikit-feature/datasets.html (i.e., ORL, COIL20, USPS, YALE), https://archive-beta.ics.uci.edu/ml/datasets (i.e., MNIST, MSRA25, SEGMENT), http://www.cs.nyu.edu/ roweis/data.html (i.e.,BA),and http://www.cad.zju.edu.cn/home/dengcai/Data/TextData.html (i.e., TR11), where a subset of MNIST is used in our experiment because of the limit of the memory space. This subset is also used in [31].

References

  1. Hartigan JA (1975) Clustering Algorithm. Wiley, New York

    MATH  Google Scholar 

  2. MacQueen J (1967) Some methods for classification and analysis of multi-variate observations. In: Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability,Vol 1, pp 281-297

  3. Andrew YN, Jordan MI, Weiss Y (2001) On spectral clustering: analysis and an algorithm. Adv Neural Inf Process Syst 14:849–856

    Google Scholar 

  4. Nie FP, Wang XQ, Huang H (2014) Clustering and projected clustering with adaptive neighbors. In: SIGKDD, pp 977-986

  5. Girolami MA (2002) Mercer kernel-based clustering in feature space. IEEE Trans Neural Netw 13(3):780–784

    Article  Google Scholar 

  6. Huang HC, Chuang YY, Chen CS (2012) Multiple kernel fuzzy clustering. IEEE Trans Fuzzy Syst 20(1):120–134

    Article  Google Scholar 

  7. Du L, Zhou P, Shi L et al (2015) Robust multiple kernel \(k\)-means using \(\ell _{2,1}\)-norm. In: IJCAI, pp 3476-3482

  8. Zhou SH, Zhu E, Liu XW et al (2020) Subspace segmentation-based robust multiple kernel clustering. Inf Fus 53:145–154

    Article  Google Scholar 

  9. Lu JT, Lu YH, Wang R, Nie FP et al (2022) Multiple kernel \(k\)-means clustering with simultaneous spectral rotation. In: ICASSP, pp 4143-4147

  10. Liu XW, Zhou SH, Liu L et al (2021) Localized simple multiple kernel \(k\)-means. In: ICCV, pp 9273-9281

  11. Zhu XZ, Liu XW, Li MM et al (2018) Localized incomplete multiple kernel \(k\)-means. In: IJCAI, pp 3271-3277

  12. Liu XW, Dou Y, Yin JP et al (2016) Multiple kernel \(k\)-means clustering with matrix-induced regularization. In: AAAI, pp 1888-1894

  13. Yao YQ, Li Y, Jiang BB et al (2021) Multiple kernel \(k\)-means clustering by selecting representative kernels. IEEE Trans Neural Netw Learn Syst 32(11):4983–4996

    Article  MathSciNet  Google Scholar 

  14. Liu XW, Zhou SH, Wang YQ et al (2017) Optimal neighborhood kernel clustering with multiple kernels. In: AAAI, pp 2266-2272

  15. Zhou SH, Liu XW, Li MM et al (2020) Multiple kernel clustering with neighbor-kernel subspace segmentation. IEEE Trans Neural Netw Learn Syst 31(4):1351–1362

    Article  MathSciNet  Google Scholar 

  16. Ren ZW, Li HR, Yang C et al (2020) Multiple kernel subspace clustering with local structural graph and low-rank consensus kernel learning. Knowl Based Syst 188:1–9

    Article  Google Scholar 

  17. Ren ZW, Sun QS (2021) Simultaneous global and local graph structure preserving for multiple kernel clustering. IEEE Trans Neural Netw Learn Syst 32(5):1839–1851

    Article  MathSciNet  Google Scholar 

  18. Liu JY, Liu XW, Xiong J et al (2022) Optimal neighborhood multiple kernel clustering with adaptive local kernels. IEEE Trans Knowl Data Eng 34(6):2872–2885

    Google Scholar 

  19. Wang R, Lu JT, Lu YH et al (2021) Discrete multiple kernel \(k\)-means. In: IJCAI, pp 3111-3117

  20. Lu CY, Feng JS, Lin ZC et al (2019) Subspace clustering by block diagonal representation. IEEE Trans Pattern Anal Mach Intell 41(2):487–501

    Article  MathSciNet  Google Scholar 

  21. Elhamifar E, Vidal R (2013) Sparse subspace clustering: Algorithm, theory, and applications. IEEE Trans Pattern Anal Mach Intell 35(11):2765–2781

    Article  Google Scholar 

  22. Liu GC, Lin ZC, Yan SC et al (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184

    Article  Google Scholar 

  23. Feng JS, Lin ZC, Xu H et al (2014) Robust subspace segmentation with block-diagonal prior. In: CVPR, pp 3818-3825

  24. Kang Z, Lu X, Yi JF et al (2018) Self-weighted multiple kernel learning for graph-based clustering and semi-supervised classification. In: IJCAI, pp 2312-2318

  25. Dattorro J (2010) Convex optimization and euclidean distance geometry http://meboo.convexoptimization.com/Meboo.html

  26. Nie FP, Zhang R, Li XL (2017) A generalized power iteration method for solving quadratic problem on the Stiefel manifold. Sci China Inf Sci 60(11):112101:1-112101:10

    Article  MathSciNet  Google Scholar 

  27. Schölkopf B, Smola AJ, Müller KR (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5):1299–1319

    Article  Google Scholar 

  28. Liu XW, Zhu E, Liu JY et al (2020) Simplemkkm: Simple multiple kernel k-means. CoRR, vol. abs/2005.04975 https://arxiv.org/abs/2005.04975

  29. Sun MJ, Wang SW, Zhang P et al (2022) Projective multiple kernel subspace clustering. IEEE Trans Multim 24:2567–2579

    Article  Google Scholar 

  30. Zhan K, Nie FP, Wang J et al (2019) Multiview consensus graph clustering. IEEE Trans Image Process 28(3):1261–1270

    Article  MathSciNet  Google Scholar 

  31. Shi ZQ, Liu JL (2023) Noise-tolerant clustering via joint doubly stochastic matrix regularization and dual sparse coding. Expert Syst Appl 222:119814

    Article  Google Scholar 

Download references

Acknowledgments

This work is partially supported by the Research Fund of Guangxi Key Lab of Multi-source Information Mining and Security (Grant No: MIMS22-03, MIMS21-M-01), the National Natural Science Foundation of China (Grant No: 61862009).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Guangxi Normal University, 15 Yucai Road, Guilin, 541004, Guangxi, China

    Cuiling Chen & Zhi Li

  2. Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University, 15 Yucai Road, Guilin, 541004, Guangxi, China

    Jian Wei

Authors
  1. Cuiling Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhi Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest or personal relationships related to the work in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, C., Wei, J. & Li, Z. Multiple kernel k-means clustering with block diagonal property. Pattern Anal Applic 26, 1515–1526 (2023). https://doi.org/10.1007/s10044-023-01183-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-023-01183-7

Keywords

Search

Navigation