Neural Computing and Applications

, Volume 22, Issue 7–8, pp 1627–1635

A novel margin-based twin support vector machine with unity norm hyperplanes

Original Article

Abstract

For classification problems, twin support vector machine (TWSVM) determines two nonparallel hyperplanes by solving two related SVM-type problems. TWSVM classifies binary patterns by the proximity of it to one of the two nonparallel hyperplanes. Thus, to calculate the distance of a pattern from the hyperplane, we need the unity norm of the normal vector of the hyperplane. But in the formulation of TWSVM, these equality constraints were not considered. In this paper, we consider unity norm constraints by using Euclidean norm and add a regularization term with the idea of maximizing some margin in TWSVM and propose a novel margin-based twin support vector machines with unity norm hyperplanes (UNH-MTSVM). We solved UNH-MTSVM by Newton’s method, and the solution is updated by conjugate gradient method. The performance of both the linear and nonlinear UNH-MTSVM is verified experimentally on several bench mark and synthetic datasets. Experimental results show the effectiveness of our methods in both computation time and classification accuracy.

Keywords

Classification Support vector machines Twin support vector machines Maximum margin Unity norm hyperplanes 

References

  1. 1.
    Cortes C, Vapnik VN (1995) Support vector networks. Mach Learn 20:273–297MATHGoogle Scholar
  2. 2.
    Burges C (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2:121–167CrossRefGoogle Scholar
  3. 3.
    Vapnik VN (1998) Statistical learning theory. Wiley, New YorkMATHGoogle Scholar
  4. 4.
    Deng NY, Tian YJ, Zhang CH (2012) Support vector machines: theory, algorithms, and extensions. CRC Press, Boca RatonGoogle Scholar
  5. 5.
    Noble WS (2004) Kernel methods in computational biology. In: Support vector machine applications in computational biology, MIT Press, Cambridge, pp 71–92Google Scholar
  6. 6.
    Lee S, Verri A (2002) Pattern recognition with support vector machines. In: First international workshop, Springer, Niagara Falls, CanadaGoogle Scholar
  7. 7.
    Ince H, Trafalis TB (2002) Support vector machine for regression and applications to financial forecasting, In: International joint conference on neural networks, Como, Italy, IEEE-INNS-ENNSGoogle Scholar
  8. 8.
    Bradley PS, Mangasarian OL (2000) K-plane clustering. J Glob Optim 16(1):23–32MathSciNetMATHCrossRefGoogle Scholar
  9. 9.
    Mangasarian OL, Wild EW (2006) Multisurface proximal support vector classification via generalize deigenvalues. IEEE Trans Pattern Anal Machine Intell 28(1):69–74CrossRefGoogle Scholar
  10. 10.
    Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910CrossRefGoogle Scholar
  11. 11.
    Shao YH, Zhang CH, Wang XB, Deng NY (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968CrossRefGoogle Scholar
  12. 12.
    Shao YH, Deng NY (2012) A coordinate descent margin based-twin support vector machine for classification. Neural Netw 25:114–121MATHCrossRefGoogle Scholar
  13. 13.
    Kumar MA, Gopal M (2008) Application of smoothing technique on twin support vector machines. Pattern Recogn Lett 29(13):1842–1848CrossRefGoogle Scholar
  14. 14.
    Khemchandani R, Jayadeva, Chandra S (2009) Optimal kernel selection in twin support vector machines. Optim Lett 3:77–88MathSciNetMATHCrossRefGoogle Scholar
  15. 15.
    Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543CrossRefGoogle Scholar
  16. 16.
    Ghorai S, Mukherjee A, Dutta PK (2009) Nonparallel plane proximal classifier. Signal Process 89(4):510–522MATHCrossRefGoogle Scholar
  17. 17.
    Peng X (2010) A ν-twin support vector machine (ν-TSVM) classifier and its geometric algorithms. Inf Sci 180:3863–3875MATHCrossRefGoogle Scholar
  18. 18.
    Ghorai S, Hossian SJ, Mukherjee A, Dutta PK (2010) Unity norm twin support vector machine classifier. In: India Conference (INDICON), India, pp 1–4Google Scholar
  19. 19.
    Ghorai S, Hossain SJ, Mukherjee A, Dutta PK (2010) Newton’s method for nonparallel plane proximal classifier. Signal Process 90:93–104MATHCrossRefGoogle Scholar
  20. 20.
    Peng X (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23(3):365–372CrossRefGoogle Scholar
  21. 21.
    Zhong P, Xu Y, Zhao Y (2011) Training twin support vector regression via linear programming. Neural Comput Appl. doi:10.1007/s 00521-011-0526-6
  22. 22.
    Chen X, Yang J, Liang J, Ye Q (2011) Smooth twin support vector regression. Neural Comput Appl. doi:10.1007/s00521-010-0454-9
  23. 23.
    Nocedal J, Wright S (2006) Numerical optimization, (Chapter16). 2nd edn. Springer, BerlinGoogle Scholar
  24. 24.
    Bertsekas DP (1999) Nonlinear programming. 2nd edn. Athena Scientific, BelmontMATHGoogle Scholar
  25. 25.
    Lee Y, Mangasarian OL (2001) SSVM: a smooth support vector machine for classification. Comput Optim Appl 20:5–22MathSciNetMATHCrossRefGoogle Scholar
  26. 26.
    Mangasarian OL (1994) Nonlinear programming. SIAM Publishers, PhiladelphiaMATHCrossRefGoogle Scholar
  27. 27.
    Bertsekas DP (1999) Nonlinear programming. 2nd edn. Athena Scientific, BelmontMATHGoogle Scholar
  28. 28.
    Lee YJ, Hsieh WF, Huang CM (2005) ε-SSVR: a smooth support vector machine for ε-insensitive regression. IEEE Trans Knowl Data Eng 17(5):678–685CrossRefGoogle Scholar
  29. 29.
    Burges C (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2:1–43CrossRefGoogle Scholar
  30. 30.
    Schölkopf B, Smola A (2002) Learning with kernels. MIT Press, CambridgeGoogle Scholar
  31. 31.
    Lee Y, Mangasarian OL (2000) RSVM: reduced support vector machines. Data Mining Institute, Computer Science Department, University of Wisconsin, Madison, WI, Technical Report 00-07Google Scholar
  32. 32.
    Lin KM, Lin CJ (2003) A study on reduced support vector machines. IEEE Trans Neural Netw 14(6):1449–1459CrossRefGoogle Scholar
  33. 33.
    Lee Y, Huang SY (2007) Reduced support vector machines: a statistical theory. IEEE Trans Neural Netw 18(1):1–13CrossRefGoogle Scholar
  34. 34.
    MathWorks (2007) Matlab. http://www.mathworks.com
  35. 35.
    Blake CL, Merz CJ (1998) UCI repository for machine learning databases. Dept. of Information and Computer Sciences, Univ. of California, Irvine, http://www.ics.uci.edu/mlearn/MLRepository.html
  36. 36.
    Duda RO, Hart PE, Stork DG (2001) Pattern classification, 2nd edn. Wiley, New YorkGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2012

Authors and Affiliations

  1. 1.Zhijiang CollegeZhejiang University of TechnologyHangzhouPeople’s Republic of China
  2. 2.College of ScienceChina Agricultural UniversityBeijingPeople’s Republic of China

Personalised recommendations