Applied Intelligence

, Volume 39, Issue 3, pp 451–464 | Cite as

Least squares twin parametric-margin support vector machine for classification

  • Yuan-Hai Shao
  • Zhen Wang
  • Wei-Jie Chen
  • Nai-Yang Deng


In this paper, we propose a novel least squares twin parametric-margin support vector machine (TPMSVM) for binary classification, called LSTPMSVM for short. LSTPMSVM attempts to solve two modified primal problems of TPMSVM, instead of two dual problems usually solved. The solution of the two modified primal problems reduces to solving just two systems of linear equations as opposed to solving two quadratic programming problems along with two systems of linear equations in TPMSVM, which leads to extremely simple and fast algorithm. Classification using nonlinear kernel with reduced technique also leads to systems of linear equations. Therefore our LSTPMSVM is able to solve large datasets accurately without any external optimizers. Further, a particle swarm optimization (PSO) algorithm is introduced to do the parameter selection. Our experiments on synthetic as well as on several benchmark data sets indicate that our LSTPMSVM has comparable classification accuracy to that of TPMSVM but with remarkably less computational time.


Pattern classification Support vector machines Twin support vector machines Least squares Particle swarm optimization 


  1. 1.
    Cortes C, Vapnik VN (1995) Support vector networks. Mach Learn 20:273–297 MATHGoogle Scholar
  2. 2.
    Vapnik VN (1998) Statistical learning theory. Wiley, New York MATHGoogle Scholar
  3. 3.
    Deng NY, Tian YJ, Zhang CH (2013) Support vector machines: optimization based theory, algorithms, and extensions. CRC Press, Boca Raton Google Scholar
  4. 4.
    Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2:121–167 CrossRefGoogle Scholar
  5. 5.
    Smola A, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14:199–222 MathSciNetCrossRefGoogle Scholar
  6. 6.
    Hao PY, Chiang JH, Lin YH (2009) A new maximal-margin spherical-structured multi-class support vector machine. Appl Intell 30(2):98–111 CrossRefGoogle Scholar
  7. 7.
    Lee LH, Wan CH, Rajkumar R, Isa D (2012) An enhanced support vector machine classification framework by using Euclidean distance function for text document categorization. Appl Intell 37(1):80–99 CrossRefMATHGoogle Scholar
  8. 8.
    Lee LH, Rajkumar R, Isa D (2012) Automatic folder allocation system using Bayesian-support vector machines hybrid classification approach. Appl Intell 36(2):295–307 CrossRefGoogle Scholar
  9. 9.
    Li C, Liu K, Wang H (2011) The incremental learning algorithm with support vector machine based on hyperplane-distance. Appl Intell 34(1):19–27 CrossRefMATHGoogle Scholar
  10. 10.
    Platt J (1999) Fast training of support vector machines using sequential minimal optimization. In: Advances in kernel methods-support vector learning. MIT Press, Cambridge Google Scholar
  11. 11.
    Joachims T (1998) Making large-scale support vector machine learning practical. In: Advances in kernel methods-support vector learning. MIT Press, Cambridge Google Scholar
  12. 12.
    Chang C, Lin C (2001) LIBSVM: a library for support vector machines. Technical report, Department of computer science and information engineering, National Taiwan University Google Scholar
  13. 13.
    Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ (2008) LIBLINEAR: a library for large linear classification. J Mach Learn Res 9(6):1871–1874 MATHGoogle Scholar
  14. 14.
    Suykens JAK, Lukas L, VanDooren P, DeMoor B, Vandewalle J (1999) Least squares support vector machine classifiers: a large scale algorithm. In: European conference of circuit theory design, pp 839–842 Google Scholar
  15. 15.
    Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9:293–300 MathSciNetCrossRefGoogle Scholar
  16. 16.
    Tian YJ, Shi Y, Liu XH (2012) Recent advances on support vector machines research. Technol Econ Dev Econ 18(1):5–33 CrossRefGoogle Scholar
  17. 17.
    Mangasarian O, Wild E (2006) Multisurface proximal support vector classification via generalize deigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74 CrossRefGoogle Scholar
  18. 18.
    Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910 CrossRefGoogle Scholar
  19. 19.
    Shao YH, Zhang CH, Wang XB, Deng NY (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968 CrossRefGoogle Scholar
  20. 20.
    Shao YH, Zhang CH, Yang ZM, Jing L, Deng NY (2012) An ε-twin support vector machine for regression. Neural Comput Appl. doi:10.1007/s00521-012-0924-3 Google Scholar
  21. 21.
    Shao YH, Deng NY (2012) A novel margin based twin support vector machine with unity norm hyperplanes. Neural Comput Appl. doi:10.1007/s00521-012-0894-5 Google Scholar
  22. 22.
    Peng XJ (2011) TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recognit 44(10–11):2678–2692 CrossRefMATHGoogle Scholar
  23. 23.
    Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543 CrossRefGoogle Scholar
  24. 24.
    Shao YH, Deng NY, Yang ZM (2012) Least squares recursive projection twin support vector machine for classification. Pattern Recognit 45(6):2299–2307 CrossRefMATHGoogle Scholar
  25. 25.
    Wang Z, Shao YH, Wu TR (2012) Proximal parametric-margin support vector classifier and its applications. Neural Comput Appl. doi:10.1007/s00521-012-1278-6 Google Scholar
  26. 26.
    Lee YJ, Mangasarian OL (2001) RSVM: reduced support vector machines. In: First SIAM international conference on data mining, pp 5–7 Google Scholar
  27. 27.
    Schölkopf B, Smola A (2002) Learning with kernels. MIT Press, Cambridge Google Scholar
  28. 28.
    Qi ZQ, Tian YJ, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 2012(35):46–53 CrossRefGoogle Scholar
  29. 29.
    Qi ZQ, Tian YJ, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recognit 46(1):305–316 CrossRefMATHGoogle Scholar
  30. 30.
    Shao YH, Wang Z, Chen WJ, Deng NY (2013) A regularization for the projection twin support vector machine. Knowl-Based Syst 37:203–210 CrossRefGoogle Scholar
  31. 31.
    Shao YH, Deng NY, Yang ZM, Chen WJ, Wang Z (2012) Probabilistic outputs for twin support vector machines. Knowl-Based Syst 33:145–151 CrossRefGoogle Scholar
  32. 32.
    Yang ZX, Shao YH, Zhang XS (2012) Multiple birth support vector machine for multi-class classification. Neural Comput Appl. doi:10.1007/s00521-012-1108-x Google Scholar
  33. 33.
    Golub GH, Van Loan CF (1996) Matrix computations, 3rd edn. The John Hopkins University Press, Baltimore, p 50 MATHGoogle Scholar
  34. 34.
    Shao YH, Deng NY (2012) A coordinate descent margin based-twin support vector machine for classification. Neural Netw 25:114–121 CrossRefMATHGoogle Scholar
  35. 35.
    Lee YJ, Huang SY (2007) Reduced support vector machines: a statistical theory. IEEE Trans Neural Netw 18(1):1–13 CrossRefGoogle Scholar
  36. 36.
    Kennedy J, Eberhart RC (1995) Particle swarm optimization. In: Proceedings of IEEE conference on neural network, vol 4, pp 1942–1948 Google Scholar
  37. 37.
    Shi Y, Eberhart RC (1998) A modified particle swarm optimizer. In: Proceeding of the IEEE congress on evolutionary computation, pp 69–73 Google Scholar
  38. 38.
    Lin S-W, Lee Z-J, Chen S-C, et al (2008) Parameter determination of support vectormachines and feature selection using simulated annealing approach. Appl Soft Comput 8:1505–1512 CrossRefGoogle Scholar
  39. 39.
    Lin S-W Ying K-C Chen S-C, et al (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35:1817–1824 CrossRefGoogle Scholar
  40. 40.
    García-Nieto J, Alba E (2012) Parallel multi-swarm optimizer for gene selection in DNA microarrays. Appl Intell 37(2):255–266 CrossRefGoogle Scholar
  41. 41.
    Ahn CW, Ramakrishna RS (2010) A diversity preserving selection in multiobjective evolutionary algorithms. Appl Intell 32(3):231–248 CrossRefGoogle Scholar
  42. 42.
    Blake CI, Merz CJ (1998) UCI repository of machine learning databases.
  43. 43.
    Musicant DR (1998) NDC: normally distributed clustered datasets.
  44. 44.
    MathWorks (2007). MATLAB.
  45. 45.
    Duda RO, Hart PE, Stork DG (2001) Pattern classification 2nd edn. Wiley, New York MATHGoogle Scholar
  46. 46.
    Hao PY (2010) New support vector algorithms with parametric insensitive/margin model. Neural Netw 23(1):293–300 CrossRefGoogle Scholar
  47. 47.
    Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, Cambridge MATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Yuan-Hai Shao
    • 1
  • Zhen Wang
    • 2
  • Wei-Jie Chen
    • 1
  • Nai-Yang Deng
    • 3
  1. 1.Zhijiang CollegeZhejiang University of TechnologyHangzhouP.R. China
  2. 2.Mathematics Colledge of Jilin UniversityChangchunP.R. China
  3. 3.College of Science China Agricultural UniversityBeijingP.R. China

Personalised recommendations