Skip to main content
SpringerLink
Log in

A spheres-based support vector machine for pattern classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper proposes a new spheres-based support vector machine (SSVM) for binary data classification. The proposed SSVM is formulated by clustering the training points according to the similarity between classes, i.e., it constructs two spheres simultaneously by solving a single optimization programming problem, in which each point is as far as possible away from the sphere center of opposite class and its projection value on the directed line segment between the two centers is as far as possible not larger than the corresponding radius. This SSVM has a perfect geometric interpretation for its dual problem. By considering the characteristics of the dual optimization problem of SSVM, an efficient learning algorithm for SSVM, which can be easily extended to other SVM-type classifiers, based on the gradient descent and the clipping strategy is further presented. Computational results on several synthetic as well as benchmark datasets indicate the significant advantages of the SSVM classifier in the computational cost and test accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Here, we only list the simplified dual problems by discarding the constant items and multiplying the objective functions by 1 − νk, k = 1, 2.

  2. Let \(\mathcal {X}\) be a set with n different points and \(\frac {1}{n}\leq \mu <1\), the RCH of \(\mathcal {X}\), denoted as \(\mathcal {C}_{\mu }(\mathcal {X})\), is defined as

    $$\mathcal{C}_{\mu}(\mathcal{X})=\left\{\boldsymbol{x}:~\boldsymbol{x}=\sum\limits_{i}\alpha_{i}\boldsymbol{x}_{i}, \boldsymbol{x}_{i}\in\mathcal{X},\sum\limits_{i}\alpha_{i}=1, 0\leq\alpha_{i}\leq\mu\right\}. $$

  3. Available at: http://www.mathworks.com.

References

  1. Boser B, Guyon L, Vapnik VN (1992) A training algorithm for optimal margin classifiers Proceedings of the 5th annual workshop on computational learning theory. ACM Press, Pittsburgh, pp 144–152

    Google Scholar 

  2. Vapnik VN (1995) The natural of statistical learning theory. Springer, New York

    Book  MATH  Google Scholar 

  3. Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2 (2):121–167

    Article  Google Scholar 

  4. Vapnik VN (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  5. Osuna E, Freund R, Girosi F (1997) Training support vector machines: an application to face detection Proceedings of IEEE computer vision and pattern recognition, San Juan, Puerto Rico, pp 130–136

    Google Scholar 

  6. Joachims T, Ndellec C, Rouveriol C (1998) Text categorization with support vector machines: learning with many relevant features European conference on machine learning No. 10, Chemnitz, Germany, pp 137–142

    Google Scholar 

  7. Schölkopf B, Tsuda K, Vert J-P (2004) Kernel methods in computational biology. MIT Press, Cambridge

    Book  Google Scholar 

  8. Lal TN, Schroder M, Hinterberger T, Weston J, Bogdan M, Birbaumer N, Schölkopf B (2004) Support vector channel selection in BCI. IEEE Trans Biomed Eng 51(6):1003–1010

    Article  Google Scholar 

  9. Smola A, Schölkopf B (2000) Sparse greedy matrix approximation for machine learning Proceedings of the 17th international conference on machine learning, Stanford, USA , pp 911–918

    Google Scholar 

  10. Achlioptas D, McSherry F, Schölkopf B (2002) Sampling techniques for kernel methods Advances in neural information processing systems, vol 14. MIT Press, Cambridge

  11. Fine S, Scheinberg K (2002) Efficient SVM training using low-rank kernel representations. J Mach Learn Res 2(2):243–264

    MATH  Google Scholar 

  12. Cortes C, Vapnik VN (1995) Support vector networks. Mach Learn 20:273–297

    MATH  Google Scholar 

  13. Osuna E, Freund R, Girosi F (1997) An improved training algorithm for support vector machines Proceedings of the IEEE workshop on neural networks for signal processing, Amelia Island, FL, USA, pp 276–285

    Google Scholar 

  14. Platt J (1999) Fast training of support vector machines using sequential minimal optimization Advances in kernel methods–support vector learning. MIT Press, Cambridge, pp 185–208

    Google Scholar 

  15. Joachims T (1999) Making large-scale SVM learning practical Advances in kernel methods: support vector machine. MIT Press, Cambridge, MA, pp 169–184

    Google Scholar 

  16. Tsang IW, Kwok JT, Cheung P-M (2005) Core vector machines: fast SVM training on very large data sets. J Mach Learn Res 6:363–392

    MathSciNet  MATH  Google Scholar 

  17. Tsang IW, Kocsor A, Kwok JT (2007) Simpler core vector machines with enclosing balls Proceedings of the 24th international conference on machine learning, Corvallis, OR

    Google Scholar 

  18. Fung G, Mangasarian OL (2003) Finite Newton method for Lagrangian support vector machine classification. Neurocomputing 55(1–2):39–55

    Article  Google Scholar 

  19. Pavlov D, Mao J, Dom B (2000) Scaling-up support vector machines using boosting algorithm Proceedings of the international conference on pattern recognition, vol 2. Barcelona, Spain, pp 2219–2222

  20. Collobert R, Bengio S, Bengio Y (2002) A parallel mixture of SVMs for very large scale problems. Neural Comput 14(5):1105–1114

    Article  MATH  Google Scholar 

  21. Lee YJ, Mangasarian OL (2001) RSVM: reduced support vector machines Proceedings of the first SIAM international conference on data mining

    Google Scholar 

  22. Mangasarian OL, Wild EW (2006) Multisurface proximal support vector classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74

    Article  Google Scholar 

  23. Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  MATH  Google Scholar 

  24. Ghorai S, Mukherjee A, Dutta PK (2009) Nonparallel plane proximal classifier. Signal Process 89 (4):510–522

    Article  MATH  Google Scholar 

  25. Chen X, Yang J, Ye Q, Liang J (2011) Recursive projection twin support vector machine via within-class variance minimization. Pattern Recogn 44(10–11):2643–2655

    Article  MATH  Google Scholar 

  26. Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968

    Article  Google Scholar 

  27. Peng X (2011) TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recogn 44(10-11):2678–2692

    Article  MATH  Google Scholar 

  28. Peng X, Xu D (2013) A twin-hypersphere support vector machine classifier and the fast learning algorithm. Inf Sci 221(1):12–27

    Article  MathSciNet  MATH  Google Scholar 

  29. Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245

    Article  Google Scholar 

  30. Crisp DJ, Burges CJC (1999) A geometric interpretation of ν-SVM classifiers Advance neural information processing system, vol 12, pp 244–250

  31. Mavroforakis M, Theodoridis S (2006) A geometric approach to support vector machine (SVM) classification. IEEE Trans Neural Netw 17(3):671–682

    Article  Google Scholar 

  32. López J., Barbero Á, Dorronsoro JR (2011) Clipping algorithms for solving the nearest point problem over reduced convex hulls. Pattern Recogn 44(3):607–614

    Article  MATH  Google Scholar 

  33. Mercer J (1909) Functions of positive and negative type and the connection with the theory of integal equations. Philos Trans R Soc Lond Ser A 209:415–446

    Article  MATH  Google Scholar 

  34. Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13:415–425

    Article  Google Scholar 

  35. Cevikalp H, Triggs B, Yavuz HS, Kücük Y, Kücük M, Barkana A (2010) Large margin classifiers based on affine hulls. Neurocomputing 73(16–18):3160–3168

    Article  Google Scholar 

  36. Hao P-Y (2010) New support vector algorithms with parameteric insensitive/margin model. Neural Netw 23 (1):60–73

    Article  MATH  Google Scholar 

  37. Peng X, Xu D (2013) Geometric algorithms for parametric-margin ν-support vector machine. Neurocomputing 99:197–205

    Article  Google Scholar 

  38. Shao Y, Wang Z, Chen W, Deng N (2013) A regularization for the projection twin support vector machine. Knowl-Based Syst 37:203–210

    Article  Google Scholar 

  39. Peng X, Chen D, Kong L (2014) A clipping dual coordinate descent algorithm for solving support vector machines submitted

  40. Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  41. Rätsch G (2000) Benchmark repository, datasets available at http://ida.first.fhg.de/projects/bench/benchmarks.htm

  42. Peng X, Xu D (2012) Twin Mahalanobis distance-based support vector machines for pattern recognition. Inf Sci 200:22–37

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Shanghai Normal University, Shanghai, 200234, People’s Republic of China

    Xinjun Peng

Authors
  1. Xinjun Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinjun Peng.

Ethics declarations

Conflict of interests

We declare that we have no any conflict with other people or organizations that can inappropriately influence our work.

Additional information

This work is supported partly by the National Natural Science Foundation of China (61202156).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, X. A spheres-based support vector machine for pattern classification. Neural Comput & Applic 31 (Suppl 1), 379–396 (2019). https://doi.org/10.1007/s00521-017-3004-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-017-3004-x

Keywords

Search

Navigation