Laplacian smooth twin support vector machine for semi-supervised classification

Original Article


Laplacian twin support vector machine (Lap-TSVM) is a state-of-the-art nonparallel-planes semi-supervised classifier. It tries to exploit the geometrical information embedded in unlabeled data to boost its generalization ability. However, Lap-TSVM may endure heavy burden in training procedure since it needs to solve two quadratic programming problems (QPPs) with the matrix “inversion” operation. In order to enhance the performance of Lap-TSVM, this paper presents a new formulation of Lap-TSVM, termed as Lap-STSVM. Rather than solving two QPPs in dual space, firstly, we convert the primal constrained QPPs of Lap-TSVM into unconstrained minimization problems (UMPs). Afterwards, a smooth technique is introduced to make these UMPs twice differentiable. At last, a fast Newton–Armijo algorithm is designed to solve the UMPs in Lap-STSVM. Experimental evaluation on both artificial and real-world datasets demonstrate the benefits of the proposed approach.


Semi-supervised classification Manifold regularization Twin support vector machine Smooth technique 


  1. 1.
    Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Patt Anal Machine Intell 29(5):905-910Google Scholar
  2. 2.
    Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans on Neural Netw 22(6):962–968CrossRefGoogle Scholar
  3. 3.
    Vapnik VN (1998) Statistical learning theory. Wiley Press, New YorkGoogle Scholar
  4. 4.
    Huang G, Wang D, Lan Y (2011) Extreme learning machines: a survey. Int J Machine Learn Cybern 2(2):107–122CrossRefGoogle Scholar
  5. 5.
    Deng N, Tian Y, Zhang C (2012) Support vector machines: theory, algorithms and extensions. CRC Press, PhiladelphiaGoogle Scholar
  6. 6.
    Chen W, Shao Y, Bao W (2012) A novel ensemble TBSVM classifier for imbalanced data classification. J Comput Inf Syst 8(19):8223–8230Google Scholar
  7. 7.
    Peng X (2011) Building sparse twin support vector machine classifiers in primal space. Inf Sci 181(18):3967–3980CrossRefGoogle Scholar
  8. 8.
    Shao Y, Deng N, Yang Z (2012) Least squares recursive projection twin support vector machine for classification. Pattern Recogn 45(6):2299–2307CrossRefMATHGoogle Scholar
  9. 9.
    Yang Z, Shao Y, Zhang X (2013) Multiple birth support vector machine for multi-class classification. Neural Comput Appl 22(1):153–161CrossRefGoogle Scholar
  10. 10.
    Peng X (2011) TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recogn 44(10-11):2678–2692CrossRefMATHGoogle Scholar
  11. 11.
    Khemchandani R, Karpatne A, Chandra S (2013) Twin support vector regression for the simultaneous learning of a function and its derivatives. Int J Machine Learn Cybern 4(1):51–63Google Scholar
  12. 12.
    Guzella TS, Caminhas WM (2009) A review of machine learning approaches to spam filtering. Exp Syst Appl 36(7):10206–10222CrossRefGoogle Scholar
  13. 13.
    Zhang T, Liu S, Xu C, Lu H (2011) Boosted multi-class semi-supervised learning for human action recognition. Pattern Recogn 44(10–11):2334–2342CrossRefMATHGoogle Scholar
  14. 14.
    Maulik U, Chakraborty D (2012) A novel semisupervised SVM for pixel classification of remote sensing imagery. Int J Machine Learn Cybern 3(3):247–258CrossRefGoogle Scholar
  15. 15.
    Nguyen T, Ho T (2012) Detecting disease genes based on semi-supervised learning and protein–protein interaction networks. Artif Intell Med 54(1):63–71Google Scholar
  16. 16.
    Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J Machine Learn Res 7:2399–2434MATHMathSciNetGoogle Scholar
  17. 17.
    Melacci S, Belkin M (2011) Laplacian support vector machines trained in the primal. J Machine Learn Res 12:1149–1184MATHMathSciNetGoogle Scholar
  18. 18.
    Chapelle O, Schölkopf B, Zien A (2010) Semi-supervised learning. MIT Press, MassachusettsGoogle Scholar
  19. 19.
    Wang Y, Chen S, Zhou Z (2012) New semi-supervised classification method based on modified cluster assumption. IEEE Trans Neural Netw Learn Syst 23(5):689–702CrossRefMathSciNetGoogle Scholar
  20. 20.
    Zhang S, Lei Y, Wu Y (2011) Semi-supervised locally discriminant projection for classification and recognition. Knowl-Based Syst 24(2):341–346CrossRefGoogle Scholar
  21. 21.
    Xue H, Chen S, Yang Q (2009) Discriminatively regularized least-squares classification. Pattern Recogn 42(1):93–104MATHGoogle Scholar
  22. 22.
    Soares RGF, Chen H, Yao X (2012) Semisupervised classification with cluster regularization. IEEE Trans Neural Netw Learn Syst 23(11):1779–1792CrossRefGoogle Scholar
  23. 23.
    Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53CrossRefMATHGoogle Scholar
  24. 24.
    Chen W, Shao Y, Ye Y (2013) Improving Lap-TSVM with successive overrelaxation and differential evolution. Procedia Comput Sci 17:33–40.Google Scholar
  25. 25.
    Lee Y, Mangasarian OL (2001) SSVM: A smooth support vector machine for classification. Comput Optim Appl 20(1):5–22CrossRefMATHMathSciNetGoogle Scholar
  26. 26.
    Wang Z, Shao Y, Wu T (2013) A GA-based model selection for smooth twin parametric-margin support vector machine. Pattern Recogn 46:2267–2277.Google Scholar
  27. 27.
    Kumar MA, Gopal M (2008) Application of smoothing technique on twin support vector machines. Pattern Recogn Lett 29(13):1842–1848CrossRefGoogle Scholar
  28. 28.
    Chen X, Yang J, Liang J, Ye Q (2012) Smooth twin support vector regression. Neural Comput Appl 21(3):505–513CrossRefGoogle Scholar
  29. 29.
    Mangasarian OL, Musicant DR (2001) Lagrangian support vector machines. J Machine Learn Res 1:161–177MATHMathSciNetGoogle Scholar
  30. 30.
    Joachims T (2002) Learning to classify text using support vector machines: methods, theory and algorithms. In: The Kluwer international series in engineering and computer science. Springer, New YorkGoogle Scholar
  31. 31.
    Gan H, Sang N, Huang R, Tong X, Dan Z (2013) Using clustering analysis to improve semi-supervised classification. Neurocomputing 101:290–298CrossRefGoogle Scholar
  32. 32.
    Yang Z, Fang K, Kotz S (2007) On the student’s t-distribution and the t-statistic. J Multivar Anal 98(6):1293–1304CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Zhijiang CollegeZhejiang University of TechnologyHangzhouPeople’s Republic of China

Personalised recommendations