Skip to main content
SpringerLink
Log in

Optimal neighborhood kernel clustering with adaptive local kernels and block diagonal property

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The purpose of multiple kernel clustering (MKC) is usually to generate an optimal kernel by fusing the information of multiple base kernels. Among the methods of generating the optimal kernel, a neighborhood kernel is usually used to enlarge the search range of the optimal kernel, or local base kernels are selected to avoid the redundancy of base kernels. However, few studies combine both methods simultaneously; then, the quality of the optimal kernel cannot be improved very well. Furthermore, most MKC methods require two-step strategy to cluster, that is, first generate clustering indicator matrix, and then execute clustering. This does not guarantee that the final clustering results are optimal. In order to overcome the above drawbacks, an optimal neighborhood kernel clustering with adaptive local kernel and block diagonal property (ONKC-ALK-BD) is proposed in this paper. In our proposed method, a simple weight strategy of selecting local base kernels is used to produce a consensus kernel, a neighborhood kernel of which is chosen as the optimal kernel. And a block diagonal (BD) regularizer imposed on the clustering indicator matrix encourages the matrix to be BD. On one hand, our proposed method avoids the redundancy of base kernels and ensures the diversity of selected base kernels. On the other hand, it expands the search range of the optimal kernel and improves its representation ability. Thus, the quality of the optimal kernel is enhanced. In addition, the BD property of the indicator matrix is helpful to obtain explicit clustering indicators and achieve one-step clustering, which ensures that the final results of our method are optimal for the original problem. Finally, extensive experiments on twelve data sets and comparisons with seven clustering methods show that ONKC-ALK-BD is effective.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability statement

The data sets used in the experiment are openly available, and the corresponding websites can be seen in the footnote of the main text in Sect. 5.1.

Notes

  1. http://featureselection.asu.edu/data sets.php.

  2. http://www.cs.nyu.edu/ roweis/data.html.

  3. https://archive-beta.ics.uci.edu/ml/datasets.

  4. https://jundongl.github.io/scikit-feature/datasets.html.

  5. https://www.cs.cmu.edu/~webkb/

References

  1. Peng L, Mo YJ, Xu J et al (2022) GRLC: graph representation learning with constraints. IEEE Trans Neural Netw Learn Syst Early Access. https://doi.org/10.1109/TNNLS.2022.3230979

    Article  Google Scholar 

  2. Zhu YH, Ma JB, Yuan CA et al (2022) Interpretable learning based dynamic graph convolutional networks for Alzheimer’s disease analysis. Inf Fusion 77:53–61

    Article  Google Scholar 

  3. Liu C, Wu S, Li R et al (2023) Self-supervised graph completion for incomplete multi-view clustering. IEEE Trans Knowl Data Eng Early Access. https://doi.org/10.1109/TKDE.2023.3238416

    Article  Google Scholar 

  4. Wang SW, Liu XW, Liu L et al (2022) Highly-efficient incomplete large-scale multi-view clustering with consensus bipartite graph. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, pp 9766–9775. https://doi.org/10.1109/CVPR52688.2022.00955

  5. Georgieva O (2022) Clustering for gene expression analysis. In: Proceedings of the 15th International Conference Education and Research in the Information Society, Plovdiv, Bulgaria, ERIS. pp 17–24. https://ceur-ws.org/Vol-3372/paper02.pdf

  6. Guo L, Zhang XQ, Liu ZG et al (2022) Correntropy metric-based robust low-rank subspace clustering for motion segmentation. Int J Mach Learn Cybern 13(5):1425–1440

    Article  Google Scholar 

  7. He XN, Kan MY, Xie PC et al (2014) Comment-based multi-view clustering of web 2.0 items. In: 23rd International World Wide Web Conference (WWW), Seoul, Republic of Korea, pp 771–782. https://doi.org/10.1145/2566486.2567975

  8. Jain AK, Murty MN, Flynn PJ (1999) Data clustering: a review. ACM Comput Surv 31(3):264–323

    Article  Google Scholar 

  9. Wagstaff K, Cardie C, Rogers S et al (2001) Constrained k-means clustering with background knowledge. In: Proceedings of the Eighteenth International Conference on Machine Learning (ICML), Williams College, Williamstown, MA, USA, pp 577–584. https://api.semanticscholar.org/CorpusID:13491515

  10. Tao WB, Jin H, Zhang YM (2007) Color image segmentation based on mean shift and normalized cuts. IEEE Trans Syst Man Cybern 37(5):1382–1389

    Article  Google Scholar 

  11. Belongie S, Carson C, Greenspan H et al (1998) Color- and texture-based image segmentation using EM and its application to content-based image retrieval. In: Sixth International Conference on Computer Vision ( ICCV), Bombay, India, pp 675–682. https://doi.org/10.1109/ICCV.1998.710790

  12. Uncu O, Gruver WA, Kotak DB et al (2007) Gridbscan: grid density-based spatial clustering of applications with noise. In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Taipei, Taiwan, pp 2976–2981. https://doi.org/10.1109/ICSMC.2006.384571

  13. Gan JZ, Hu RY, Mo YJ et al (2022) Multigraph fusion for dynamic graph convolutional network. IEEE Trans Neural Netw Learn Syst Early Access. https://doi.org/10.1109/TNNLS.2022.3172588

    Article  Google Scholar 

  14. Yuan CA, Zhong Z, Lei C et al (2021) Adaptive reverse graph learning for robust subspace learning. Inf Process Manag 58(6):102733

    Article  Google Scholar 

  15. Cai JY, Fan JC, Guo WZ et al (2022) Efficient deep embedded subspace clustering. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, pp 21–30. https://doi.org/10.1109/CVPR52688.2022.00012

  16. Girolami MA (2002) Mercer kernel-based clustering in feature space. IEEE Trans Neural Netw 13(3):780–784

    Article  Google Scholar 

  17. Huang HC, Chuang YY, Chen CS (2012) Multiple kernel fuzzy clustering. IEEE Trans Fuzzy Syst 20(1):120–134

    Article  Google Scholar 

  18. Du L, Zhou P, Shi L et al (2015) Robust multiple kernel k-means using \(\ell _{21}\)-norm. In: IJCAI. pp 3476–3482

  19. Kang Z, Lu X, Yi JF et al (2018) Self-weighted multiple kernel learning for graph-based clustering and semi-supervised classification. In: IJCAI. pp 2312–2318

  20. Liu XW, Zhu E, Liu JY et al (2020) Simplemkkm: simple multiple kernel k-means. CoRR. arXiv:2005.04975

  21. Zhu XF, Zhang SC, Li YG et al (2019) Low-rank sparse subspace for spectral clustering. IEEE Trans Knowl Data Eng 33(8):1532–1543

    Article  Google Scholar 

  22. Liu XW, Zhou SH, Wang YQ et al (2017) Optimal neighborhood kernel clustering with multiple kernels. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI), San Francisco, California, USA, pp 2266–2272. http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14761

  23. Ou QY, Gao L, Zhu E (2021) Multiple kernel k-means with low-rank neighborhood kernel. IEEE Access 9:3291–3300

    Article  Google Scholar 

  24. Zhou SH, Liu XW, Li MM et al (2020) Multiple kernel clustering with neighbor-kernel subspace segmentation. IEEE Trans Neural Netw Learn Syst 31(4):1351–1362

    Article  MathSciNet  Google Scholar 

  25. Liu XW, Zhou SH, Liu L et al (2021) Localized simple multiple kernel k-means. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, pp 9273–9281. https://doi.org/10.1109/ICCV48922.2021.00916

  26. Yao Y, Li Y, Jiang B, Chen H (2021) Multiple kernel k-means clustering by selecting representative kernels. IEEE Trans Neural Netw Learn Syst 32(11):4983–4996. https://doi.org/10.1109/TNNLS.2020.3026532

    Article  MathSciNet  Google Scholar 

  27. Liu XW, Dou Y, Yin JP et al (2016) Multiple kernel k-means clustering with matrix-induced regularization. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI), Phoenix, Arizona, USA, pp 1888–1894. http://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/12115

  28. Li L, Wang SW, Liu XW et al (2022) Local sample-weighted multiple kernel clustering with consensus discriminative graph. IEEE Trans Neural Netw Learn Syst Early Access. https://doi.org/10.1109/TNNLS.2022.3184970

    Article  Google Scholar 

  29. Wang R, Lu JT, Liu YH et al (2021) Discrete multiple kernel k-means. In: IJCAI. pp 3111–3117

  30. Feng JS, Lin ZC, Xu H et al (2014) Robust subspace segmentation with block-diagonal prior. In: CVPR. pp 3818–3825

  31. Ram A, Sharma A, Jalal AS et al (2009) An enhanced density based spatial clustering of applications with noise. In: 2009 IEEE International Advance Computing Conference, Patiala, India, pp 1475–1478. https://doi.org/10.1109/IADCC.2009.4809235

  32. Sander J (2011) Density-based clustering. New York

  33. Zhang T, Ramakrishnan R, Livny M (1996) BIRCH: an efficient data clustering method for very large databases. In: Proceedings of the 1996 ACM International Conference on Management of Data (SIGMOD), Montreal, Quebec, Canada, pp 103–114. https://doi.org/10.1145/233269.233324

  34. Wang W, Yang J, Muntz RR (1997) STING: a statistical information grid approach to spatial data mining. In: VLDB. pp 186–195

  35. Valizadegan H, Jin R (2006) Generalized maximum margin clustering and unsupervised kernel learning. In: Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, pp 1417–1424. https://proceedings.neurips.cc/paper/2006/hash/be767243ca8f574c740fb4c26cc6dceb-Abstract.html

  36. Zeng H, Cheung YM (2011) Feature selection and kernel learning for local learning-based clustering. IEEE Trans Pattern Anal Mach Intell 33(8):1532–1547

    Article  Google Scholar 

  37. Cortes C, Mohri M, Rostamizadeh A (2009) \(\ell _{2}\) regularization for learning kernels. In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, Montreal, QC, Canada, pp 109–116. https://www.auai.org/uai2009/papers/UAI2009%5C_0132%5C_16d9521907263bf91a07477a87253260.pdf

  38. Zhao B, Kwok JT, Zhang CS (2009) Multiple kernel clustering. In: Proceedings of the SIAM International Conference on Data Mining (SDM), Sparks, Nevada, USA, pp 638–649. https://doi.org/10.1137/1.9781611972795.55

  39. Kloft M, Brefeld U, Laskov P et al (2009) Efficient and accurate \(\ell _{p}\)-norm multiple kernel learning. In: 23rd Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, pp 997–1005. https://proceedings.neurips.cc/paper/2009/hash/d516b13671a4179d9b7b458a6ebdeb92-Abstract.html

  40. Xu ZL, Jin R, Yang HQ et al (2010) Simple and efficient multiple kernel learning by group lasso. In: Proceedings of the 27th International Conference on Machine Learning (ICML), Haifa, Israel, pp 1175–1182. https://icml.cc/Conferences/2010/papers/540.pdf

  41. Kang Z, Peng C, Cheng Q et al (2018) Unified spectral clustering with optimal graph. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI), pp 3366–3373. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16201

  42. Kang Z, Wen LJ, Chen WY et al (2019) Low-rank kernel learning for graph-based clustering. Knowl Based Syst 163:510–517

    Article  Google Scholar 

  43. Zhou SH, Zhu E, Liu XW et al (2020) Subspace segmentation-based robust multiple kernel clustering. Inf Fusion 53:145–154

    Article  Google Scholar 

  44. Liu JY, Liu XW, Xiong J et al (2022) Optimal neighborhood multiple kernel clustering with adaptive local kernels. IEEE Trans Knowl Data Eng 34(6):2872–2885

    Google Scholar 

  45. Afkanpour A, Szepesvári C, Bowling M (2013) Alignment based kernel learning with a continuous set of base kernels. Mach Learn 91(3):305–324

    Article  MathSciNet  MATH  Google Scholar 

  46. Wang TH, Tian SF, Huang HK et al (2009) Learning by local kernel polarization. Neurocomputing 72(13):3077–3084

    Article  Google Scholar 

  47. Wang L (2008) Feature selection with kernel class separability. IEEE Trans Pattern Anal Mach Intell 30(9):1534–1546

    Article  Google Scholar 

  48. Lu YT, Wang LT, Lu JF et al (2014) Multiple kernel clustering based on centered kernel alignment. Pattern Recognit 47(11):3656–3664

    Article  MATH  Google Scholar 

  49. Li MM, Liu XW, Wang L et al (2016) Multiple kernel clustering with local kernel alignment maximization. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI), New York, NY, USA, pp 1704–1710. http://www.ijcai.org/Abstract/16/244

  50. Wang CL, Zhu E, Liu XW et al (2018) Multiple kernel clustering with global and local structure alignment. IEEE Access 6:77911–77920

    Article  Google Scholar 

  51. Lu CY, Feng JS, Lin ZC et al (2019) Subspace clustering by block diagonal representation. IEEE Trans Pattern Anal Mach Intell 41(2):487–501

    Article  MathSciNet  Google Scholar 

  52. Dattorro J (2016) Convex optimization and Euclidean distance geometry. http://meboo.convexoptimization.com/Meboo.html. Accessed 1 Apr 2016

  53. Nie FP, Zhang R, Li XL (2017) A generalized power iteration method for solving quadratic problem on the stiefel manifold. Sci China Inf Sci 60(11):1121011–11210110

    Article  MathSciNet  Google Scholar 

  54. Gogebakan M (2021) A novel approach for Gaussian mixture model clustering based on soft computing method. IEEE Access 9:159987–160003

    Article  Google Scholar 

  55. Schölkopf B, Smola AJ, Müller KR (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5):1299–1319

    Article  Google Scholar 

  56. Xu L, Jordan MI (1996) On convergence properties of the EM algorithm for gaussian mixtures. Neural Comput 8(1):129–151

    Article  Google Scholar 

  57. Zhou SH, Liu XW, Li MM et al (2020) Multiple kernel clustering with neighbor-kernel subspace segmentation. IEEE Trans Neural Netw Learn Syst 31(4):1351–1362

    Article  MathSciNet  Google Scholar 

  58. Zhan K, Nie FP, Wang J et al (2019) Multiview consensus graph clustering. IEEE Trans Image Process 28(3):1261–1270

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work is partially supported by the Research fund of Guangxi Key Lab of Multi-source Information Mining and Security (Grant no: MIMS22-03, MIMS21-M-01), the National Natural Science Foundation of China (Grant no: 61862009).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Guangxi Normal University, Guilin, 541004, Guangxi, People’s Republic of China

    Cuiling Chen, Jian Wei & Zhi Li

Authors
  1. Cuiling Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhi Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest or personal relationships related to the work in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, C., Wei, J. & Li, Z. Optimal neighborhood kernel clustering with adaptive local kernels and block diagonal property. Neural Comput & Applic 35, 22297–22312 (2023). https://doi.org/10.1007/s00521-023-08885-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08885-3

Keywords

Search

Navigation