Ramp-based twin support vector clustering

  • Zhen Wang
  • Xu Chen
  • Yuan-Hai ShaoEmail author
  • Chun-Na Li
Original Article


Traditional plane-based clustering methods measure the within-cluster or between-cluster scatter by linear, quadratic or some other unbounded functions, which are sensitive to the samples far from the cluster center. This paper introduces the ramp functions into plane-based clustering and proposes a ramp-based twin support vector clustering (RampTWSVC). RampTWSVC is very robust to the samples far from the cluster center, because its within-cluster and between-cluster scatters are measured by the bounded ramp functions. Thus, it is easier to find the intrinsic clusters than other plane-based clustering methods. The nonconvex programming problem in RampTWSVC is solved efficiently through an alternating iteration algorithm, and its local solution can be obtained in a finite number of iterations theoretically. In addition, its nonlinear manifold clustering formation is also proposed via a kernel trick. Experimental results on several benchmark datasets show the better performance of our RampTWSVC compared with other plane-based clustering methods.


Clustering Plane-based clustering Ramp function Twin support vector clustering Nonconvex programming 



This work is supported in part by National Natural Science Foundation of China (Nos. 61966024, 11501310, 61866010, 11871183 and 61703370), in part by Program for Young Talents of Science and Technology in Universities of Inner Mongolia Autonomous Region (No. NJYT-19-B01), in part by Natural Science Foundation of Hainan Province (No. 118QN181), in part by Scientific Research Foundation of Hainan University (No. kyqd(sk)1804) and in part by Zhejiang Provincial Natural Science Foundation of China (No. LQ17F030003).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. 1.
    Jain A, Murty M, Flynn P (1999) Data clustering: a review. ACM Comput Surv 31(3):264–323CrossRefGoogle Scholar
  2. 2.
    Mehmood R, Zhang G, Bie R, Dawood H, Ahmad H (2016) Clustering by fast search and find of density peaks via heat diffusion. Neurocomputing 208:210–217CrossRefGoogle Scholar
  3. 3.
    Zhou J, Chen L, Chen C, Zhang Y, Li H (2016) Fuzzy clustering with the entropy of attribute weights. Neurocomputing 198:125–134CrossRefGoogle Scholar
  4. 4.
    Zhou Z (2017) A brief introduction to weakly supervised learning. Natl Sci Rev 5(1):44–53CrossRefGoogle Scholar
  5. 5.
    Berry M (2004) Survey of text mining I: clustering, classification, and retrieval, vol 1. Springer, BerlinCrossRefGoogle Scholar
  6. 6.
    Ilin R (2012) Unsupervised learning of categorical data with competing models. IEEE Trans Neural Netw Learn Syst 23(11):1726–1737CrossRefGoogle Scholar
  7. 7.
    Zhang H, Chow T, Wu Q (2015) Organizing books and authors by multilayer som. IEEE Trans Neural Netw Learn Syst 27(12):2537–2550CrossRefGoogle Scholar
  8. 8.
    Tan P, Steinbach M, Kumar V (2005) Introduction to data mining, 1st edn. Addison-Wesley Longman Publishing Co. Inc, BostonGoogle Scholar
  9. 9.
    Huang X, Ye Y, Zhang H (2014) Extensions of kmeans-type algorithms: a new clustering framework by integrating intracluster compactness and intercluster separation. IEEE Trans Neural Netw Learn Syst 25(8):1433–1446CrossRefGoogle Scholar
  10. 10.
    Jain A, Dubes R (1988) Algorithms for clustering data. Prentice-Hall Inc, Upper Saddle RiverzbMATHGoogle Scholar
  11. 11.
    Mangasarian O, Wild E (2006) Multisurface proximal support vector classification via generalize eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74CrossRefGoogle Scholar
  12. 12.
    Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910CrossRefGoogle Scholar
  13. 13.
    Wang Z, Shao Y, Bai L, Li C, Liu L, Deng N (2018) Insensitive stochastic gradient twin support vector machines for large scale problems. Inf Sci 462:114–131MathSciNetCrossRefGoogle Scholar
  14. 14.
    Bradley P, Mangasarian O (2000) k-plane clustering. J Global Optim 16(1):23–32MathSciNetCrossRefGoogle Scholar
  15. 15.
    Shao Y, Bai L, Wang Z, Hua X, Deng N (2013) Proximal plane clustering via eigenvalues. Proc Comput Sci 17:41–47CrossRefGoogle Scholar
  16. 16.
    Shao Y, Guo Y, Wang Z, Yang Z, Deng N (2017) k-proximal plane clustering. Int J Mach Learn Cybernet 8(5):1537–1554CrossRefGoogle Scholar
  17. 17.
    Wang Z, Shao Y, Bai L, Deng N (2015) Twin support vector machine for clustering. IEEE Trans Neural Netw Learn Syst 26(10):2583–2588MathSciNetCrossRefGoogle Scholar
  18. 18.
    Ye Q, Zhao H, Li Z, Yang X, Gao S, Yin T, Ye N (2018) L1-norm distance minimization-based fast robust twin support vector k-plane clustering. IEEE Trans Neural Netw Learn Syst 29(9):4494–4503CrossRefGoogle Scholar
  19. 19.
    Khemchandani R, Pal A, Chandra S (2018) Fuzzy least squares twin support vector clustering. Neural Comput Appl 29(2):553–563CrossRefGoogle Scholar
  20. 20.
    Bai L, Shao Y, Wang Z, Li C (2019) Clustering by twin support vector machine and least square twin support vector classifier with uniform output coding. Knowl-Based Syst 163:227–240CrossRefGoogle Scholar
  21. 21.
    Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968CrossRefGoogle Scholar
  22. 22.
    Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMS. J Mach Learn Res 7:1687–1712MathSciNetzbMATHGoogle Scholar
  23. 23.
    Cevikalp H (2017) Best fitting hyperplanes for classification. IEEE Trans Pattern Anal Mach Intell 39(6):1076–1088CrossRefGoogle Scholar
  24. 24.
    Tian Y, Mirzabagheri M, Bamakan S, Wang H, Qu Q (2018) Ramp loss one-class support vector machine: a robust and effective approach to anomaly detection problems. Neurocomputing 310(8):223–235CrossRefGoogle Scholar
  25. 25.
    Liu D, Shi Y, Tian Y, Huang X (2016) Ramp loss least squares support vector machine. J Comput Sci 14:61–68MathSciNetCrossRefGoogle Scholar
  26. 26.
    Souvenir R, Pless R (2005) Manifold clustering. In: Tenth IEEE international conference on computer vision, ICCV 2005, vol 1, IEEE, pp 648–653Google Scholar
  27. 27.
    Cao W, Haralick R (2006) Nonlinear manifold clustering by dimensionality. In: 18th international conference on pattern recognition. ICPR 2006, vol 1. IEEE, pp 920–924Google Scholar
  28. 28.
    Khemchandani R, Chandra S (2009) Optimal kernel selection in twin support vector machines. Optim Lett 3:77–88MathSciNetCrossRefGoogle Scholar
  29. 29.
    Platt J (1999) Fast training of support vector machines using sequential minimal optimization. Advances in kernel methods-support vector learning. MIT Press, Cambridge, pp 185–208Google Scholar
  30. 30.
    Wang Z, Shao Y, Wu T (2013) A ga-based model selection for smooth twin parametric-margin support vector machine. Pattern Recogn 46(8):2267–2277CrossRefGoogle Scholar
  31. 31.
    Blake C, Merz C (1998) UCI Repository for Machine Learning Databases,
  32. 32.
    Li C, Shao Y, Yin W, Liu M (2019) Robust and sparse linear discriminant analysis via an alternating direction method of multipliers. IEEE Trans Neural Netw Learn Syst (in press) Google Scholar
  33. 33.
    Liu H, Fu Y (2015) Clustering with partition level side information. In: 2015 IEEE international conference on data mining (ICDM). IEEE, pp 877–882Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Mathematical SciencesInner Mongolia UniversityHohhotPeople’s Republic of China
  2. 2.School of ManagementHainan UniversityHaikouPeople’s Republic of China

Personalised recommendations