# A spheres-based support vector machine for pattern classification

- 82 Downloads

## Abstract

This paper proposes a new spheres-based support vector machine (SSVM) for binary data classification. The proposed SSVM is formulated by clustering the training points according to the similarity between classes, i.e., it constructs two spheres simultaneously by solving a single optimization programming problem, in which each point is as far as possible away from the sphere center of opposite class and its projection value on the directed line segment between the two centers is as far as possible not larger than the corresponding radius. This SSVM has a perfect geometric interpretation for its dual problem. By considering the characteristics of the dual optimization problem of SSVM, an efficient learning algorithm for SSVM, which can be easily extended to other SVM-type classifiers, based on the gradient descent and the clipping strategy is further presented. Computational results on several synthetic as well as benchmark datasets indicate the significant advantages of the SSVM classifier in the computational cost and test accuracy.

### Keywords

Support vector machine Spheres Projection Geometric interpretation Learning algorithm## Notes

### Compliance with Ethical Standards

### Conflict of interests

We declare that we have no any conflict with other people or organizations that can inappropriately influence our work.

### References

- 1.Boser B, Guyon L, Vapnik VN (1992) A training algorithm for optimal margin classifiers Proceedings of the 5th annual workshop on computational learning theory. ACM Press, Pittsburgh, pp 144–152Google Scholar
- 2.Vapnik VN (1995) The natural of statistical learning theory. Springer, New YorkCrossRefMATHGoogle Scholar
- 3.Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2 (2):121–167CrossRefGoogle Scholar
- 4.Vapnik VN (1998) Statistical learning theory. Wiley, New YorkMATHGoogle Scholar
- 5.Osuna E, Freund R, Girosi F (1997) Training support vector machines: an application to face detection Proceedings of IEEE computer vision and pattern recognition, San Juan, Puerto Rico, pp 130–136Google Scholar
- 6.Joachims T, Ndellec C, Rouveriol C (1998) Text categorization with support vector machines: learning with many relevant features European conference on machine learning No. 10, Chemnitz, Germany, pp 137–142Google Scholar
- 7.Schölkopf B, Tsuda K, Vert J-P (2004) Kernel methods in computational biology. MIT Press, CambridgeGoogle Scholar
- 8.Lal TN, Schroder M, Hinterberger T, Weston J, Bogdan M, Birbaumer N, Schölkopf B (2004) Support vector channel selection in BCI. IEEE Trans Biomed Eng 51(6):1003–1010CrossRefGoogle Scholar
- 9.Smola A, Schölkopf B (2000) Sparse greedy matrix approximation for machine learning Proceedings of the 17th international conference on machine learning, Stanford, USA , pp 911–918Google Scholar
- 10.Achlioptas D, McSherry F, Schölkopf B (2002) Sampling techniques for kernel methods Advances in neural information processing systems, vol 14. MIT Press, CambridgeGoogle Scholar
- 11.Fine S, Scheinberg K (2002) Efficient SVM training using low-rank kernel representations. J Mach Learn Res 2(2):243–264MATHGoogle Scholar
- 12.Cortes C, Vapnik VN (1995) Support vector networks. Mach Learn 20:273–297MATHGoogle Scholar
- 13.Osuna E, Freund R, Girosi F (1997) An improved training algorithm for support vector machines Proceedings of the IEEE workshop on neural networks for signal processing, Amelia Island, FL, USA, pp 276–285Google Scholar
- 14.Platt J (1999) Fast training of support vector machines using sequential minimal optimization Advances in kernel methods–support vector learning. MIT Press, Cambridge, pp 185–208Google Scholar
- 15.Joachims T (1999) Making large-scale SVM learning practical Advances in kernel methods: support vector machine. MIT Press, Cambridge, MA, pp 169–184Google Scholar
- 16.Tsang IW, Kwok JT, Cheung P-M (2005) Core vector machines: fast SVM training on very large data sets. J Mach Learn Res 6:363–392MathSciNetMATHGoogle Scholar
- 17.Tsang IW, Kocsor A, Kwok JT (2007) Simpler core vector machines with enclosing balls Proceedings of the 24th international conference on machine learning, Corvallis, ORGoogle Scholar
- 18.Fung G, Mangasarian OL (2003) Finite Newton method for Lagrangian support vector machine classification. Neurocomputing 55(1–2):39–55CrossRefGoogle Scholar
- 19.Pavlov D, Mao J, Dom B (2000) Scaling-up support vector machines using boosting algorithm Proceedings of the international conference on pattern recognition, vol 2. Barcelona, Spain, pp 2219–2222Google Scholar
- 20.Collobert R, Bengio S, Bengio Y (2002) A parallel mixture of SVMs for very large scale problems. Neural Comput 14(5):1105–1114CrossRefMATHGoogle Scholar
- 21.Lee YJ, Mangasarian OL (2001) RSVM: reduced support vector machines Proceedings of the first SIAM international conference on data miningGoogle Scholar
- 22.Mangasarian OL, Wild EW (2006) Multisurface proximal support vector classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74CrossRefGoogle Scholar
- 23.Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910CrossRefMATHGoogle Scholar
- 24.Ghorai S, Mukherjee A, Dutta PK (2009) Nonparallel plane proximal classifier. Signal Process 89 (4):510–522CrossRefMATHGoogle Scholar
- 25.Chen X, Yang J, Ye Q, Liang J (2011) Recursive projection twin support vector machine via within-class variance minimization. Pattern Recogn 44(10–11):2643–2655CrossRefMATHGoogle Scholar
- 26.Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968CrossRefGoogle Scholar
- 27.Peng X (2011) TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recogn 44(10-11):2678–2692CrossRefMATHGoogle Scholar
- 28.Peng X, Xu D (2013) A twin-hypersphere support vector machine classifier and the fast learning algorithm. Inf Sci 221(1):12–27MathSciNetCrossRefMATHGoogle Scholar
- 29.Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245CrossRefGoogle Scholar
- 30.Crisp DJ, Burges CJC (1999) A geometric interpretation of
*ν*-SVM classifiers Advance neural information processing system, vol 12, pp 244–250Google Scholar - 31.Mavroforakis M, Theodoridis S (2006) A geometric approach to support vector machine (SVM) classification. IEEE Trans Neural Netw 17(3):671–682CrossRefGoogle Scholar
- 32.López J., Barbero Á, Dorronsoro JR (2011) Clipping algorithms for solving the nearest point problem over reduced convex hulls. Pattern Recogn 44(3):607–614CrossRefMATHGoogle Scholar
- 33.Mercer J (1909) Functions of positive and negative type and the connection with the theory of integal equations. Philos Trans R Soc Lond Ser A 209:415–446CrossRefMATHGoogle Scholar
- 34.Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13:415–425CrossRefGoogle Scholar
- 35.Cevikalp H, Triggs B, Yavuz HS, Kücük Y, Kücük M, Barkana A (2010) Large margin classifiers based on affine hulls. Neurocomputing 73(16–18):3160–3168CrossRefGoogle Scholar
- 36.Hao P-Y (2010) New support vector algorithms with parameteric insensitive/margin model. Neural Netw 23 (1):60–73CrossRefGoogle Scholar
- 37.Peng X, Xu D (2013) Geometric algorithms for parametric-margin
*ν*-support vector machine. Neurocomputing 99:197–205CrossRefGoogle Scholar - 38.Shao Y, Wang Z, Chen W, Deng N (2013) A regularization for the projection twin support vector machine. Knowl-Based Syst 37:203–210CrossRefGoogle Scholar
- 39.Peng X, Chen D, Kong L (2014) A clipping dual coordinate descent algorithm for solving support vector machines submittedGoogle Scholar
- 40.Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, CambridgeCrossRefMATHGoogle Scholar
- 41.Rätsch G (2000) Benchmark repository, datasets available at http://ida.first.fhg.de/projects/bench/benchmarks.htm
- 42.Peng X, Xu D (2012) Twin Mahalanobis distance-based support vector machines for pattern recognition. Inf Sci 200:22–37MathSciNetCrossRefMATHGoogle Scholar