Advertisement

A spheres-based support vector machine for pattern classification

  • Xinjun Peng
Original Article
  • 90 Downloads

Abstract

This paper proposes a new spheres-based support vector machine (SSVM) for binary data classification. The proposed SSVM is formulated by clustering the training points according to the similarity between classes, i.e., it constructs two spheres simultaneously by solving a single optimization programming problem, in which each point is as far as possible away from the sphere center of opposite class and its projection value on the directed line segment between the two centers is as far as possible not larger than the corresponding radius. This SSVM has a perfect geometric interpretation for its dual problem. By considering the characteristics of the dual optimization problem of SSVM, an efficient learning algorithm for SSVM, which can be easily extended to other SVM-type classifiers, based on the gradient descent and the clipping strategy is further presented. Computational results on several synthetic as well as benchmark datasets indicate the significant advantages of the SSVM classifier in the computational cost and test accuracy.

Keywords

Support vector machine Spheres Projection Geometric interpretation Learning algorithm 

Notes

Compliance with Ethical Standards

Conflict of interests

We declare that we have no any conflict with other people or organizations that can inappropriately influence our work.

References

  1. 1.
    Boser B, Guyon L, Vapnik VN (1992) A training algorithm for optimal margin classifiers Proceedings of the 5th annual workshop on computational learning theory. ACM Press, Pittsburgh, pp 144–152Google Scholar
  2. 2.
    Vapnik VN (1995) The natural of statistical learning theory. Springer, New YorkCrossRefMATHGoogle Scholar
  3. 3.
    Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2 (2):121–167CrossRefGoogle Scholar
  4. 4.
    Vapnik VN (1998) Statistical learning theory. Wiley, New YorkMATHGoogle Scholar
  5. 5.
    Osuna E, Freund R, Girosi F (1997) Training support vector machines: an application to face detection Proceedings of IEEE computer vision and pattern recognition, San Juan, Puerto Rico, pp 130–136Google Scholar
  6. 6.
    Joachims T, Ndellec C, Rouveriol C (1998) Text categorization with support vector machines: learning with many relevant features European conference on machine learning No. 10, Chemnitz, Germany, pp 137–142Google Scholar
  7. 7.
    Schölkopf B, Tsuda K, Vert J-P (2004) Kernel methods in computational biology. MIT Press, CambridgeGoogle Scholar
  8. 8.
    Lal TN, Schroder M, Hinterberger T, Weston J, Bogdan M, Birbaumer N, Schölkopf B (2004) Support vector channel selection in BCI. IEEE Trans Biomed Eng 51(6):1003–1010CrossRefGoogle Scholar
  9. 9.
    Smola A, Schölkopf B (2000) Sparse greedy matrix approximation for machine learning Proceedings of the 17th international conference on machine learning, Stanford, USA , pp 911–918Google Scholar
  10. 10.
    Achlioptas D, McSherry F, Schölkopf B (2002) Sampling techniques for kernel methods Advances in neural information processing systems, vol 14. MIT Press, CambridgeGoogle Scholar
  11. 11.
    Fine S, Scheinberg K (2002) Efficient SVM training using low-rank kernel representations. J Mach Learn Res 2(2):243–264MATHGoogle Scholar
  12. 12.
    Cortes C, Vapnik VN (1995) Support vector networks. Mach Learn 20:273–297MATHGoogle Scholar
  13. 13.
    Osuna E, Freund R, Girosi F (1997) An improved training algorithm for support vector machines Proceedings of the IEEE workshop on neural networks for signal processing, Amelia Island, FL, USA, pp 276–285Google Scholar
  14. 14.
    Platt J (1999) Fast training of support vector machines using sequential minimal optimization Advances in kernel methods–support vector learning. MIT Press, Cambridge, pp 185–208Google Scholar
  15. 15.
    Joachims T (1999) Making large-scale SVM learning practical Advances in kernel methods: support vector machine. MIT Press, Cambridge, MA, pp 169–184Google Scholar
  16. 16.
    Tsang IW, Kwok JT, Cheung P-M (2005) Core vector machines: fast SVM training on very large data sets. J Mach Learn Res 6:363–392MathSciNetMATHGoogle Scholar
  17. 17.
    Tsang IW, Kocsor A, Kwok JT (2007) Simpler core vector machines with enclosing balls Proceedings of the 24th international conference on machine learning, Corvallis, ORGoogle Scholar
  18. 18.
    Fung G, Mangasarian OL (2003) Finite Newton method for Lagrangian support vector machine classification. Neurocomputing 55(1–2):39–55CrossRefGoogle Scholar
  19. 19.
    Pavlov D, Mao J, Dom B (2000) Scaling-up support vector machines using boosting algorithm Proceedings of the international conference on pattern recognition, vol 2. Barcelona, Spain, pp 2219–2222Google Scholar
  20. 20.
    Collobert R, Bengio S, Bengio Y (2002) A parallel mixture of SVMs for very large scale problems. Neural Comput 14(5):1105–1114CrossRefMATHGoogle Scholar
  21. 21.
    Lee YJ, Mangasarian OL (2001) RSVM: reduced support vector machines Proceedings of the first SIAM international conference on data miningGoogle Scholar
  22. 22.
    Mangasarian OL, Wild EW (2006) Multisurface proximal support vector classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74CrossRefGoogle Scholar
  23. 23.
    Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910CrossRefMATHGoogle Scholar
  24. 24.
    Ghorai S, Mukherjee A, Dutta PK (2009) Nonparallel plane proximal classifier. Signal Process 89 (4):510–522CrossRefMATHGoogle Scholar
  25. 25.
    Chen X, Yang J, Ye Q, Liang J (2011) Recursive projection twin support vector machine via within-class variance minimization. Pattern Recogn 44(10–11):2643–2655CrossRefMATHGoogle Scholar
  26. 26.
    Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968CrossRefGoogle Scholar
  27. 27.
    Peng X (2011) TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recogn 44(10-11):2678–2692CrossRefMATHGoogle Scholar
  28. 28.
    Peng X, Xu D (2013) A twin-hypersphere support vector machine classifier and the fast learning algorithm. Inf Sci 221(1):12–27MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245CrossRefGoogle Scholar
  30. 30.
    Crisp DJ, Burges CJC (1999) A geometric interpretation of ν-SVM classifiers Advance neural information processing system, vol 12, pp 244–250Google Scholar
  31. 31.
    Mavroforakis M, Theodoridis S (2006) A geometric approach to support vector machine (SVM) classification. IEEE Trans Neural Netw 17(3):671–682CrossRefGoogle Scholar
  32. 32.
    López J., Barbero Á, Dorronsoro JR (2011) Clipping algorithms for solving the nearest point problem over reduced convex hulls. Pattern Recogn 44(3):607–614CrossRefMATHGoogle Scholar
  33. 33.
    Mercer J (1909) Functions of positive and negative type and the connection with the theory of integal equations. Philos Trans R Soc Lond Ser A 209:415–446CrossRefMATHGoogle Scholar
  34. 34.
    Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13:415–425CrossRefGoogle Scholar
  35. 35.
    Cevikalp H, Triggs B, Yavuz HS, Kücük Y, Kücük M, Barkana A (2010) Large margin classifiers based on affine hulls. Neurocomputing 73(16–18):3160–3168CrossRefGoogle Scholar
  36. 36.
    Hao P-Y (2010) New support vector algorithms with parameteric insensitive/margin model. Neural Netw 23 (1):60–73CrossRefGoogle Scholar
  37. 37.
    Peng X, Xu D (2013) Geometric algorithms for parametric-margin ν-support vector machine. Neurocomputing 99:197–205CrossRefGoogle Scholar
  38. 38.
    Shao Y, Wang Z, Chen W, Deng N (2013) A regularization for the projection twin support vector machine. Knowl-Based Syst 37:203–210CrossRefGoogle Scholar
  39. 39.
    Peng X, Chen D, Kong L (2014) A clipping dual coordinate descent algorithm for solving support vector machines submittedGoogle Scholar
  40. 40.
    Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, CambridgeCrossRefMATHGoogle Scholar
  41. 41.
    Rätsch G (2000) Benchmark repository, datasets available at http://ida.first.fhg.de/projects/bench/benchmarks.htm
  42. 42.
    Peng X, Xu D (2012) Twin Mahalanobis distance-based support vector machines for pattern recognition. Inf Sci 200:22–37MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2017

Authors and Affiliations

  1. 1.Department of MathematicsShanghai Normal UniversityShanghaiPeople’s Republic of China

Personalised recommendations