Laplacian smooth twin support vector machine for semi-supervised classification

  • Wei-Jie ChenEmail author
  • Yuan-Hai Shao
  • Ning Hong
Original Article


Laplacian twin support vector machine (Lap-TSVM) is a state-of-the-art nonparallel-planes semi-supervised classifier. It tries to exploit the geometrical information embedded in unlabeled data to boost its generalization ability. However, Lap-TSVM may endure heavy burden in training procedure since it needs to solve two quadratic programming problems (QPPs) with the matrix “inversion” operation. In order to enhance the performance of Lap-TSVM, this paper presents a new formulation of Lap-TSVM, termed as Lap-STSVM. Rather than solving two QPPs in dual space, firstly, we convert the primal constrained QPPs of Lap-TSVM into unconstrained minimization problems (UMPs). Afterwards, a smooth technique is introduced to make these UMPs twice differentiable. At last, a fast Newton–Armijo algorithm is designed to solve the UMPs in Lap-STSVM. Experimental evaluation on both artificial and real-world datasets demonstrate the benefits of the proposed approach.


Semi-supervised classification Manifold regularization Twin support vector machine Smooth technique 



The authors would like to thank the editors and the anonymous reviewers, whose invaluable comments helped improve the presentation of this paper substantially. This work is supported by the National Natural Science Foundation of China (11201426, 61203133, 10971223 and 11071252), the Zhejiang Provincial Natural Science Foundation of China (LQ12A01020, LQ13F030010) and the Science and Technology Foundation of Department of Education of Zhejiang Province (Y201225179).


  1. 1.
    Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Patt Anal Machine Intell 29(5):905-910Google Scholar
  2. 2.
    Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans on Neural Netw 22(6):962–968CrossRefGoogle Scholar
  3. 3.
    Vapnik VN (1998) Statistical learning theory. Wiley Press, New YorkGoogle Scholar
  4. 4.
    Huang G, Wang D, Lan Y (2011) Extreme learning machines: a survey. Int J Machine Learn Cybern 2(2):107–122CrossRefGoogle Scholar
  5. 5.
    Deng N, Tian Y, Zhang C (2012) Support vector machines: theory, algorithms and extensions. CRC Press, PhiladelphiaGoogle Scholar
  6. 6.
    Chen W, Shao Y, Bao W (2012) A novel ensemble TBSVM classifier for imbalanced data classification. J Comput Inf Syst 8(19):8223–8230Google Scholar
  7. 7.
    Peng X (2011) Building sparse twin support vector machine classifiers in primal space. Inf Sci 181(18):3967–3980CrossRefGoogle Scholar
  8. 8.
    Shao Y, Deng N, Yang Z (2012) Least squares recursive projection twin support vector machine for classification. Pattern Recogn 45(6):2299–2307CrossRefzbMATHGoogle Scholar
  9. 9.
    Yang Z, Shao Y, Zhang X (2013) Multiple birth support vector machine for multi-class classification. Neural Comput Appl 22(1):153–161CrossRefGoogle Scholar
  10. 10.
    Peng X (2011) TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recogn 44(10-11):2678–2692CrossRefzbMATHGoogle Scholar
  11. 11.
    Khemchandani R, Karpatne A, Chandra S (2013) Twin support vector regression for the simultaneous learning of a function and its derivatives. Int J Machine Learn Cybern 4(1):51–63Google Scholar
  12. 12.
    Guzella TS, Caminhas WM (2009) A review of machine learning approaches to spam filtering. Exp Syst Appl 36(7):10206–10222CrossRefGoogle Scholar
  13. 13.
    Zhang T, Liu S, Xu C, Lu H (2011) Boosted multi-class semi-supervised learning for human action recognition. Pattern Recogn 44(10–11):2334–2342CrossRefzbMATHGoogle Scholar
  14. 14.
    Maulik U, Chakraborty D (2012) A novel semisupervised SVM for pixel classification of remote sensing imagery. Int J Machine Learn Cybern 3(3):247–258CrossRefGoogle Scholar
  15. 15.
    Nguyen T, Ho T (2012) Detecting disease genes based on semi-supervised learning and protein–protein interaction networks. Artif Intell Med 54(1):63–71Google Scholar
  16. 16.
    Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. J Machine Learn Res 7:2399–2434zbMATHMathSciNetGoogle Scholar
  17. 17.
    Melacci S, Belkin M (2011) Laplacian support vector machines trained in the primal. J Machine Learn Res 12:1149–1184zbMATHMathSciNetGoogle Scholar
  18. 18.
    Chapelle O, Schölkopf B, Zien A (2010) Semi-supervised learning. MIT Press, MassachusettsGoogle Scholar
  19. 19.
    Wang Y, Chen S, Zhou Z (2012) New semi-supervised classification method based on modified cluster assumption. IEEE Trans Neural Netw Learn Syst 23(5):689–702CrossRefMathSciNetGoogle Scholar
  20. 20.
    Zhang S, Lei Y, Wu Y (2011) Semi-supervised locally discriminant projection for classification and recognition. Knowl-Based Syst 24(2):341–346CrossRefGoogle Scholar
  21. 21.
    Xue H, Chen S, Yang Q (2009) Discriminatively regularized least-squares classification. Pattern Recogn 42(1):93–104zbMATHGoogle Scholar
  22. 22.
    Soares RGF, Chen H, Yao X (2012) Semisupervised classification with cluster regularization. IEEE Trans Neural Netw Learn Syst 23(11):1779–1792CrossRefGoogle Scholar
  23. 23.
    Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53CrossRefzbMATHGoogle Scholar
  24. 24.
    Chen W, Shao Y, Ye Y (2013) Improving Lap-TSVM with successive overrelaxation and differential evolution. Procedia Comput Sci 17:33–40.Google Scholar
  25. 25.
    Lee Y, Mangasarian OL (2001) SSVM: A smooth support vector machine for classification. Comput Optim Appl 20(1):5–22CrossRefzbMATHMathSciNetGoogle Scholar
  26. 26.
    Wang Z, Shao Y, Wu T (2013) A GA-based model selection for smooth twin parametric-margin support vector machine. Pattern Recogn 46:2267–2277.Google Scholar
  27. 27.
    Kumar MA, Gopal M (2008) Application of smoothing technique on twin support vector machines. Pattern Recogn Lett 29(13):1842–1848CrossRefGoogle Scholar
  28. 28.
    Chen X, Yang J, Liang J, Ye Q (2012) Smooth twin support vector regression. Neural Comput Appl 21(3):505–513CrossRefGoogle Scholar
  29. 29.
    Mangasarian OL, Musicant DR (2001) Lagrangian support vector machines. J Machine Learn Res 1:161–177zbMATHMathSciNetGoogle Scholar
  30. 30.
    Joachims T (2002) Learning to classify text using support vector machines: methods, theory and algorithms. In: The Kluwer international series in engineering and computer science. Springer, New YorkGoogle Scholar
  31. 31.
    Gan H, Sang N, Huang R, Tong X, Dan Z (2013) Using clustering analysis to improve semi-supervised classification. Neurocomputing 101:290–298CrossRefGoogle Scholar
  32. 32.
    Yang Z, Fang K, Kotz S (2007) On the student’s t-distribution and the t-statistic. J Multivar Anal 98(6):1293–1304CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Zhijiang CollegeZhejiang University of TechnologyHangzhouPeople’s Republic of China

Personalised recommendations