Applied Intelligence

, Volume 41, Issue 4, pp 1097–1107 | Cite as

Sparse least square twin support vector machine with adaptive norm

  • Zhiqiang Zhang
  • Ling Zhen
  • Naiyang Deng
  • Junyan Tan


By promoting the parallel hyperplanes to non-parallel ones in SVM, twin support vector machines (TWSVM) have attracted more attention. There are many modifications of them. However, most of the modifications minimize the loss function subject to the I2-norm or I1-norm penalty. These methods are non-adaptive since their penalty forms are fixed and pre-determined for any types of data. To overcome the above shortcoming, we propose lp norm least square twin support vector machine (lpLSTSVM). Our new model is an adaptive learning procedure with lp-norm (0<p<1), where p is viewed as an adjustable parameter and can be automatically chosen by data. By adjusting the parameter p, lpLSTSVM can not only select relevant features but also improve the classification accuracy. The solutions of the optimization problems in lpLSTSVM are obtained by solving a series systems of linear equations (LEs) and the lower bounds of the solution is established which is extremely helpful for feature selection. Experiments carried out on several standard UCI data sets and synthetic data sets show the feasibility and effectiveness of the proposed method.


Least square twin support vector machine Twin support vector machine lp-norm Sparsity Feature selection 


  1. 1.
    Mangasarian O, Wild E (2006) Multisurface proximal support vector classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74CrossRefGoogle Scholar
  2. 2.
    Jayadeva R, Khemchandani S, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910Google Scholar
  3. 3.
    Kumar MA, Gopal M (2008) Application of smoothing technique on twin support vector machines. Pattern Recog Lett 29(13):1842–1848Google Scholar
  4. 4.
    Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543CrossRefGoogle Scholar
  5. 5.
    Jayadeva R, Khemchandani S, Chandra S (2009) Optimal kernel selection in twin support vector machines. Optim Lett 3(1):77–88Google Scholar
  6. 6.
    Ghorai S, Mukherjee A, Dutta PK (2009) Nonparallel plane proximal classifier. Signal Process 89(4):510–522CrossRefMATHGoogle Scholar
  7. 7.
    Shao YH, Zhang CH, Wang XB, Deng NY (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968CrossRefGoogle Scholar
  8. 8.
    Shao YH, Wang Z, Chen WJ, Deng NY (2013) Least squares twin parametric-margin support vector machines for classification. Appl Intell 39(3):451–464CrossRefGoogle Scholar
  9. 9.
    Chen W-J, Shao YH, Xu DK, Fu YF (2014) Manifold proximal support vector machine for semi-supervised classification. Appl Intell 40(4):623–638CrossRefGoogle Scholar
  10. 10.
    Xu YT, Guo R (2014) An improved ν-twin support vector machine. Appl Intell 41:42–54CrossRefGoogle Scholar
  11. 11.
    Xu YT, Wang LS (2014) K-nearest neighbor-based weighted twin support vector regression. Appl Intell 41:299–309CrossRefGoogle Scholar
  12. 12.
    Gao SB, Ye QL, Ye N (2011) 1-norm least squares twin support vector machine. Neurocomputing 74:3590–3597CrossRefGoogle Scholar
  13. 13.
    Chen W J, Tian Y J (2010) L p-norm proximal support vector machine and its applications. Procedia Comput Sci ICCS 1(1):2417–2423CrossRefGoogle Scholar
  14. 14.
    Tian YJ, Yu J, Chen WJ (2010) l p-norm support vector machine with CCCP. In: Proc. the 7th FSKD, pp 1560-1564Google Scholar
  15. 15.
    Tan J Y, Zhang C H, Deng N Y (2010) Cancer related gene identification via p-norm support vector machine. In: The 4th international conference on computational systems biology, pp 101–108Google Scholar
  16. 16.
    Tan J-Y, Zhang Z.-Q, Zhen L., Zhang C-H, Deng N-Y (2013) Adaptive feature selection via a new version of support vector machine. Neural Comput Appl 23:937–945CrossRefGoogle Scholar
  17. 17.
    Zhang C-H, Shao Y-H, Tan J-Y, Deng N-Y (2013) A mixed-norm linear support vector machine. Neural Comput Appl 23:2159–2166CrossRefGoogle Scholar
  18. 18.
    Chen XJ, Xu FM, Ye YY (2009) Lower bound theory of nonzero entries in solutions of l 2- l p minimization.
  19. 19.
    Bruckstein A M, Donoho D L, Elad M (2009) From sparse sulutions of systems of equations to sparse modeling of signals and images. SIAM Rev 51:34–81CrossRefMATHMathSciNetGoogle Scholar
  20. 20.
    Fan J, Li R (2001) Varible selection via nonconcave penalized likelihood and its oracle properties. J Amer Statis Assoc 96:1348–1360CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    Xu Z, Zhang H, Wang Y, Chang X (2009) \(L_{\frac {1}{2}}\) regularizer. Sci Chin Series F-InfSci 52:1–9Google Scholar
  22. 22.
    Saad Y (2003) Iterative methods for sparse linear systems. SIAM Press, PhiladelphiaCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Zhiqiang Zhang
    • 1
  • Ling Zhen
    • 2
  • Naiyang Deng
    • 2
  • Junyan Tan
    • 2
  1. 1.School of Mechanical Engineering, Beijing Institute of TechnologyBeijingChina
  2. 2.College of ScienceChina Agricultural UniversityBeijingChina

Personalised recommendations