Applied Intelligence

, Volume 40, Issue 4, pp 623–638

Manifold proximal support vector machine for semi-supervised classification

  • Wei-Jie Chen
  • Yuan-Hai Shao
  • Deng-Ke Xu
  • Yong-Feng Fu


Recently, semi-supervised learning (SSL) has attracted a great deal of attention in the machine learning community. Under SSL, large amounts of unlabeled data are used to assist the learning procedure to construct a more reasonable classifier. In this paper, we propose a novel manifold proximal support vector machine (MPSVM) for semi-supervised classification. By introducing discriminant information in the manifold regularization (MR), MPSVM not only introduces MR terms to capture as much geometric information as possible from inside the data, but also utilizes the maximum distance criterion to characterize the discrepancy between different classes, leading to the solution of a pair of eigenvalue problems. In addition, an efficient particle swarm optimization (PSO)-based model selection approach is suggested for MPSVM. Experimental results on several artificial as well as real-world datasets demonstrate that MPSVM obtains significantly better performance than supervised GEPSVM, and achieves comparable or better performance than LapSVM and LapTSVM, with better learning efficiency.


Semi-supervised classification Manifold regularization Support vector machine Nonparallel hyperplanes Particle swarm optimization 


  1. 1.
    Vapnik VN (1998) Statistical learning theory. Wiley, New York MATHGoogle Scholar
  2. 2.
    Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2):121–167 CrossRefGoogle Scholar
  3. 3.
    Deng N, Tian Y, Zhang C (2013) Support vector machines: theory, algorithms and extensions. CRC Press, Philadelphia Google Scholar
  4. 4.
    Hao P, Chiang J, Lin Y (2009) A new maximal-margin spherical-structured multi-class support vector machine. Appl Intell 30(2):98–111 CrossRefGoogle Scholar
  5. 5.
    Zhang HH, Ahn J, Lin XD, Park C (2006) Gene selection using support vector machines with non-convex penalty. Bioinformatics 22(1):88–95 CrossRefGoogle Scholar
  6. 6.
    Lee L, Wan C, Rajkumar R, Isa D (2012) An enhanced support vector machine classification framework by using Euclidean distance function for text document categorization. Appl Intell 37(1):80–99 CrossRefGoogle Scholar
  7. 7.
    Lee L, Rajkumar R, Isa D (2012) Automatic folder allocation system using Bayesian-support vector machines hybrid classification approach. Appl Intell 36(2):295–307 CrossRefGoogle Scholar
  8. 8.
    Wang C, You W (2013) Boosting-SVM: effective learning with reduced data dimension. Appl Intell 39(3):465–474 CrossRefGoogle Scholar
  9. 9.
    Mangasarian OL, Wild EW (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74 CrossRefGoogle Scholar
  10. 10.
    Shao Y, Deng N, Chen W, Zhen W (2013) Improved generalized eigenvalue proximal support vector machine. IEEE Signal Process Lett 20(3):213–216 CrossRefGoogle Scholar
  11. 11.
    Ye Q, Zhao C, Zhang H, Ye N (2011) Distance difference and linear programming nonparallel plane classifier. Expert Syst Appl 38(8):9425–9433 CrossRefGoogle Scholar
  12. 12.
    Jayadeva KR, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910 CrossRefGoogle Scholar
  13. 13.
    Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968 CrossRefGoogle Scholar
  14. 14.
    Peng X (2011) TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recognit 44(10–11):2678–2692 CrossRefMATHGoogle Scholar
  15. 15.
    Qi Z, Tian Y, Shi Y (2013) Structural twin support vector machine for classification. Knowl-Based Syst 43:74–81 CrossRefGoogle Scholar
  16. 16.
    Shao Y, Deng N, Yang Z, Chen W, Wang Z (2012) Probabilistic outputs for twin support vector machines. Knowl-Based Syst 33:145–151 CrossRefGoogle Scholar
  17. 17.
    Shao Y, Deng N, Yang Z (2012) Least squares recursive projection twin support vector machine for classification. Pattern Recognit 45(6):2299–2307 CrossRefMATHGoogle Scholar
  18. 18.
    Qi Z, Tian Y, Shi Y (2012) Twin support vector machine with universum data. Neural Netw 36:112–119 CrossRefMATHGoogle Scholar
  19. 19.
    Qi Z, Tian Y, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recognit 46(1):305–316 CrossRefMATHGoogle Scholar
  20. 20.
    Ding S, Yu J, Qi B, Huang H (2013) An overview on twin support vector machines. Artif Intell Rev. doi:10.1007/s10462-012-9336-0 Google Scholar
  21. 21.
    Chapelle O, Schölkopf B, Zien A (2010) Semi-supervised learning. MIT Press, Massachusetts Google Scholar
  22. 22.
    Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. Morgan & Claypool, San Rafael MATHGoogle Scholar
  23. 23.
    Tur G, Hakkani D, Schapire RE (2005) Combining active and semi-supervised learning for spoken language understanding. Speech Commun 45(2):171–186 CrossRefGoogle Scholar
  24. 24.
    Guzella TS, Caminhas WM (2009) A review of machine learning approaches to spam filtering. Expert Syst Appl 36(7):10206–10222 CrossRefGoogle Scholar
  25. 25.
    Zhang T, Liu S, Xu C, Lu H (2011) Boosted multi-class semi-supervised learning for human action recognition. Pattern Recognit 44(10–11):2334–2342 CrossRefMATHGoogle Scholar
  26. 26.
    Nguyen T, Ho T (2012) Detecting disease genes based on semi-supervised learning and protein protein interaction networks. Artif Intell Med 54(1):63–71 CrossRefGoogle Scholar
  27. 27.
    Soares RGF, Chen H, Yao X (2012) Semisupervised classification with cluster regularization. IEEE Trans Neural Netw Learn Syst 23(11):1779–1792 CrossRefGoogle Scholar
  28. 28.
    Fan M, Gu N, Qiao H, Zhang B (2011) Sparse regularization for semi-supervised classification. Pattern Recognit 44(8):1777–1784 CrossRefMATHGoogle Scholar
  29. 29.
    Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434 MATHMathSciNetGoogle Scholar
  30. 30.
    Melacci S, Belkin M (2011) Laplacian support vector machines trained in the primal. J Mach Learn Res 12:1149–1184 MATHMathSciNetGoogle Scholar
  31. 31.
    Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53 CrossRefMATHGoogle Scholar
  32. 32.
    Chen W, Shao Y, Ye Y (2013) Improving Lap-TSVM with successive overrelaxation and differential evolution. Proc Comput Sci 17:33–40 CrossRefGoogle Scholar
  33. 33.
    Chen W, Shao Y, Hong N (2013) Laplacian smooth twin support vector machine for semi-supervised classification. Int J Mach Learn Res Cybern. doi:10.1007/s13042-013-0183-3 Google Scholar
  34. 34.
    Tikhonov AN, Arsenin VY (1979) Methods for solving ill-posed problems. Nauka, Moscow Google Scholar
  35. 35.
    Parlett B (1998) The symmetric eigenvalue problem. SIAM, Philadelphia CrossRefMATHGoogle Scholar
  36. 36.
    Lin SW, Ying KC, Chen SC, Lee ZJ (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35(4):1817–1824 CrossRefGoogle Scholar
  37. 37.
    Shao Y, Wang Z, Chen W, Deng N (2013) Least squares twin parametric-margin support vector machine for classification. Appl Intell 39(3):451–464 CrossRefGoogle Scholar
  38. 38.
    Huang CL, Dun JF (2008) A distributed pso-svm hybrid system with feature selection and parameter optimization. Appl Soft Comput 8(4):1381–1391 CrossRefGoogle Scholar
  39. 39.
    Das S, Suganthan PN (2011) Differential evolution: a survey of the state-of-the-art. IEEE Trans Evol Comput 15(1):4–31 CrossRefGoogle Scholar
  40. 40.
    Kennedy J, Eberhart R (1995) Particle swarm optimization. In: IEEE international conference on neural networks, vol 4, pp 1942–1948 Google Scholar
  41. 41.
    Poli R, Kennedy J, Blackwell T (2007) Particle swarm optimization. Swarm Intell 1(1):33–57 CrossRefGoogle Scholar
  42. 42.
    Gan H, Sang N, Huang R, Tong X, Dan Z (2013) Using clustering analysis to improve semi-supervised classification. Neurocomputing 101:290–298 CrossRefGoogle Scholar
  43. 43.
    Yang Z, Fang K, Kotz S (2007) On the student’s t-distribution and the t-statistic. J Multivar Anal 98(6):1293–1304 CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Wei-Jie Chen
    • 1
  • Yuan-Hai Shao
    • 1
  • Deng-Ke Xu
    • 2
  • Yong-Feng Fu
    • 1
  1. 1.Zhijiang CollegeZhejiang University of TechnologyHangzhouP.R. China
  2. 2.Department of StatisticsZhejiang Agriculture and Forest UniversityLin’anP.R. China

Personalised recommendations