All-in-one multicategory Ramp loss maximum margin of twin spheres support vector machine

  • Sijie Lu
  • Huiru Wang
  • Zhijian ZhouEmail author


Maximum margin of twin spheres support vector machine (MMTSSVM) is effective to deal with imbalanced data classification problems. However, it is sensitive to outliers because of the use of the Hinge loss function. To enhance the stability of MMTSSVM, we propose a Ramp loss maximum margin of twin spheres support vector machine (Ramp-MMTSSVM) in this paper. In terms of the Ramp loss function, the outliers can be given fixed loss values, which reduces the negative effect of outliers on constructing models. Since Ramp-MMTSSVM is a non-differentiable non-convex optimization problem, we adopt Concave-Convex Procedure (CCCP) approach to solve it. We also analyze the properties of parameters and verify them by one artificial experiment. Besides, we use Rest-vs.-One(RVO) strategy to extend Ramp-MMTSSVM to multi-class classification problems. The experimental results on twenty benchmark datasets indicate that no matter in binary or multi-class classification cases, our approaches both can obtain better experimental performance than the compared algorithms.


MMTSSVM Ramp loss CCCP Multi-class 



This work was supported in part by National Natural Science Foundation of China (No.11671010).


  1. 1.
    Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  2. 2.
    Meza J, Espitia H, Montenegro C, Crespo RG (2016) Statistical analysis of a multi-objective optimization algorithm based on a model of particles with vorticity behavior. Soft Comput 20(9):3521–3536CrossRefGoogle Scholar
  3. 3.
    Vapnik V (1995) The nature of statistical learning theory. Springer, New YorkCrossRefGoogle Scholar
  4. 4.
    Zhang W, Yoshida T, Tang X (2008) Text classification based on multi-word with support vector machine. Knowl-Based Syst 21(8):879–886CrossRefGoogle Scholar
  5. 5.
    Ghosh S, Mondal S, Ghosh B (2014) A comparative study of breast cancer detection based on SVM and MLP BPN classifier. In: First international conference on automation, control, energy & systems (ACES-14), pp 87–90Google Scholar
  6. 6.
    Gohariyan E, Esmaeilpour M, Shirmohammadi MM (2017) The combination of mammography and MRI for diagnosing breast cancer using fuzzy NN and SVM. Int J Interact Multimed Artif Intell 4(5):20–24Google Scholar
  7. 7.
    Naz S, Ziauddin S, Shahid AR (2018) Driver fatigue detection using mean intensity, SVM, and SIFT. Int J Interact Multimed Artif Intell 5(IP):1Google Scholar
  8. 8.
    Pang Y, Zhang K, Yuan Y, Wang K (2014) Distributed object detection with linear SVMs. IEEE T Cybern 44(11):2122–2133CrossRefGoogle Scholar
  9. 9.
    Jayadeva Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE T Pattern Anal 29(5):905–910CrossRefGoogle Scholar
  10. 10.
    Peng X (2010) A ν-twin support vector machine (ν-TSVM) classifier and its geometric algorithms. Inform Sciences 180(20):3863–3875MathSciNetCrossRefGoogle Scholar
  11. 11.
    Xie X, Sun S (2014) Multi-view Laplacian twin support vector machines. Appl Intell 41(4):1059–1068CrossRefGoogle Scholar
  12. 12.
    Xu Y, Guo R (2014) An improved ν-twin support vector machine. Appl Intell 41(1):42–54CrossRefGoogle Scholar
  13. 13.
    Wang H, Zhou Z (2017) An improved rough margin-based ν-twin bounded support vector machine. Knowl-Based Syst 128:125–138CrossRefGoogle Scholar
  14. 14.
    Wang H, Zhou Z, Xu Y (2018) An improved ν-twin bounded support vector machine. Appl Intell 48(4):1041–1053CrossRefGoogle Scholar
  15. 15.
    Xu Y, Wang L, Zhong P (2012) A rough margin-based ν-twin support vector machine. Neural Comput Appl 21:1307–1317CrossRefGoogle Scholar
  16. 16.
    Xu Y, Yu J, Zhang Y (2014) KNN-Based weighted rough ν-twin support vector machine. Knowel-Based Syst 71:303–313CrossRefGoogle Scholar
  17. 17.
    Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543CrossRefGoogle Scholar
  18. 18.
    Peng X, Xu D (2013) A twin-hypersphere support vector machine classifier and the fast learning algorithm. Inform Sci 221:12–27MathSciNetCrossRefGoogle Scholar
  19. 19.
    Xu Y (2016) Maximum margin of twin spheres support vector machine for imbalanced data classification. IEEE T Cybern 47(6):1540–1550CrossRefGoogle Scholar
  20. 20.
    Huang X, Shi L, Suykens JAK (2014) Ramp loss linear programming support vector machine. J Mach Learn Res 15:2185–2211MathSciNetzbMATHGoogle Scholar
  21. 21.
    Yuille A, Rangarajan A (2003) The concave-convex procedure. Neural Comput 15:915–936CrossRefGoogle Scholar
  22. 22.
    Liu D, Shi Y, Huang X (2016) Ramp loss least squares support vector machine. J Comput Sci-Neth 14:61–68MathSciNetCrossRefGoogle Scholar
  23. 23.
    Zhang Z, Krawczyk B, Garcìa S, Rosales- Pérez A, Herrera F (2016) Empowering one-vs-one decomposition with ensemble learning for multi-class imbalanced data. Knowl-Based Syst 106:251–263CrossRefGoogle Scholar
  24. 24.
    Zhou L, Wang Q, Fujita H (2017) One versus one multi-class classification fusion using optimizing decision directed acyclic graph for predicting listing status of companies. Inform Fusion 36:80–89CrossRefGoogle Scholar
  25. 25.
    Yang X, Yu Q, Guo T (2013) The one-against-all partition based binary tree support vector machine algorithms for multi-class classification. Neurocomputing 113:1–7CrossRefGoogle Scholar
  26. 26.
    Tomar D, Agarwal S (2015) A comparison on multi-class classification methods based on least squares twin support vector machine. Knowl-Based Syst 81:131–147CrossRefGoogle Scholar
  27. 27.
    Xu Y, Guo R (2014) A twin hyper-sphere multi-class classification support vector machine. J Intell Fuzzy Syst 27(4):1783–1790MathSciNetzbMATHGoogle Scholar
  28. 28.
    Kumar D, Thakur M (2018) All-in-one multicategory least squares nonparallel hyperplanes support vector machine. Pattern Recogn Lett 105:165–174CrossRefGoogle Scholar
  29. 29.
    Angulo C, Parra X, Català A (2003) K-SVCR. a support vector machine for multi-class classification. Neurocomputing 55(1-2):57–77CrossRefGoogle Scholar
  30. 30.
    Xu Y (2016) K-nearest neighbor-based weighted multi-class twin support vector machine. Neurocomputing 205:430–438CrossRefGoogle Scholar
  31. 31.
    Xu Y, Guo R, Wang L (2013) A twin multi-class classification support vector machine. Cogn Comput 5(4):580–588CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of ScienceChina Agricultural UniversityHaidianChina

Personalised recommendations