Skip to main content
SpringerLink
Log in

An overview on semi-supervised support vector machine

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Support vector machine (SVM) is a machine learning method based on statistical learning theory. It has a lot of advantages, such as solid theoretical foundation, global optimization, the sparsity of the solution, nonlinear and generalization. The standard form of SVM only applies to supervised learning. Large amount of data generated in real life is unlabeled, and the standard form of SVM cannot make good use of these data to improve its learning ability. However, semi-supervised support vector machine (S3VM) is a good solution to this problem. This paper reviews the recent progress in semi-supervised support vector machine. First, the basic theory of S3VM is expounded and discussed in detail; then, the mainstream model of S3VM is presented, including transductive support vector machine, Laplacian support vector machine, S3VM training via the label mean, S3VM based on cluster kernel; finally, we give the conclusions and look ahead to the research on S3VM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Altun Y, Belkin M, Mcallester DA (2005) Maximum margin semi-supervised learning for structured variables. In: Proceedings of the 2005 annual conference on neural information processing systems, pp 33–40

  2. Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. Synth Lect Artif Intell Mach Learn 3(1):1–130

    Article  MATH  Google Scholar 

  3. Hady MFA, Schwenker F (2013) Semi-supervised learning. Handbook on neural information processing. Springer, Berlin, pp 215–239

    Book  Google Scholar 

  4. Ding SF, Qi BJ, Tan YH (2011) An overview on theory and algorithm of support vector machines. J Univ Electron Sci Technol China 40(1):2–10

    Google Scholar 

  5. Gu YX, Ding SF (2011) Advances of support vector machines. J Comput Sci Technol 38(2):14–17

    Google Scholar 

  6. Schölkopf B, Smola AJ (2001) Learning with kernels: SUPPORT vector machines, regularization, optimization, and beyond. MIT Press, Cambridge

    Google Scholar 

  7. Ding S (2011) Incremental learning algorithm for support vector data description. J Softw 6(7):1166–1173

    Article  Google Scholar 

  8. Liu XL, Ding SF (2010) Appropriateness in applying SVMs to text classification. Comput Eng Sci 32(6):106–108

    MathSciNet  Google Scholar 

  9. Bennett K, Demiriz A (1999) Semi-supervised support vector machines. In: Advances in neural information processing systems, vol 11, pp 368–374

  10. Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning. Springer, New York

    Book  MATH  Google Scholar 

  11. Steinwart I, Christmann A (2008) Support vector machines. Springer, New York

    MATH  Google Scholar 

  12. Jordan MI, Jacobs RI (2014) Supervised learning and divide-and-conquer: a statistical approach. In: Proceedings of the tenth international conference on machine learning, pp 159–166

  13. Shipp MA, Ross KN, Tamayo P et al (2002) Diffuse large B-cell lymphoma outcome prediction by gene-expression profiling and supervised machine learning. Nat Med 8(1):68–74

    Article  Google Scholar 

  14. Zhang CG, Zhang Y (2013) Semi-supervised learning. China Agricultural Science and Technology Press, Beijing

    Google Scholar 

  15. Miller DJ, Uyar HS (1997) A mixture of experts classifier with learning based on both labelled and unlabelled data. In: Advances in neural information processing systems. MIT Press, Cambridge, pp 571–577

  16. Zhang T, Oles F (2000) The value of unlabeled data for classification problems. In: Proceedings of 17th international conference on machine learning, pp 1191–1198

  17. Chapelle O, Schölkopf B, Zien A (2006) Semi-supervised learning. MIT Press, Cambridge

    Book  Google Scholar 

  18. Triguero I, Sáez JA, Luengo J et al (2014) On the characterization of noise filters for self-training semi-supervised in nearest neighbor classification. Neurocomputing 132:30–41

    Article  Google Scholar 

  19. Dópido I, Li J, Marpu PR et al (2013) Semi-supervised self-learning for hyperspectral image classification. IEEE Trans Geosci Remote Sens 51(7):4032–4044

    Article  Google Scholar 

  20. Li Y, Li H, Guan C, et al (2007) A self-training semi-supervised support vector machine algorithm and its applications in brain computer interface. In: International conference on acoustics, speech, and signal processing, pp 385–387

  21. McLachlan GJ, Krishnan T (1997) The EM algorithm and extensions. Wiley-Interscience, New York

    MATH  Google Scholar 

  22. Dong C, Yin Y, Guo X et al (2008) On co-training style algorithms. Int Conf Nat Comput 18(20):196–201

    Google Scholar 

  23. Feger F, Koprinska I (2006) Co-training using RBF nets and different feature splits. In: International joint conference on neural networks, pp 1878–1885

  24. Yu S, Krishnapuram B, Rosales R et al (2011) Bayesian co-training. J Mach Learn Res 12:2649–2680

    MathSciNet  MATH  Google Scholar 

  25. Zhu X (2005) Semi-supervised learning with graphs, Carnegie Mellon University, language technologies institute, school of computer science

  26. Tao D, Li X, Wu X et al (2007) Supervised tensor learning. Knowl Inf Syst 13(1):1–42

    Article  Google Scholar 

  27. Chapelle O, Sindhwani V, Keerthi SS (2008) Optimization techniques for semi-supervised support vector machines. J Mach Learn Res 9:203–233

    MATH  Google Scholar 

  28. Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, pp 57–64

  29. Gieseke F, Airola A, Pahikkala T et al (2014) Fast and simple gradient-based optimization for semi-supervised support vector machines. Neurocomputing 123:23–32

    Article  Google Scholar 

  30. Collobert R, Sinz F, Weston J et al (2006) Large scale transductive SVMs. J Mach Learn Res 7:1687–1712

    MathSciNet  MATH  Google Scholar 

  31. Sindhwani V, Keerthi SS, Chapelle O (2006) Deterministic annealing for semi-supervised kernel machines. In: Proceedings of the 23rd international conference on machine learning, pp 841–848

  32. Chapelle O, Chi M, Zien A (2006) A continuation method for semi-supervised SVMs. In: Proceedings of the 23rd international conference on machine learning, pp 185–192

  33. De Bie T, Cristianini N (2006) Semi-supervised learning using semi-definite programming. MIT press, Cambridge

    Google Scholar 

  34. Le HM, Le Thi HA, Nguyen MC (2015) Sparse semi-supervised support vector machines by DC programming and DCA. Neurocomputing 153:62–76

    Article  Google Scholar 

  35. Chapelle O, Sindhwani V, Keerthi S (2007) Branch and bound for semi-supervised support vector machines. In: 20th Annual conference on neural information processing systems, pp 217–224

  36. Adankon MM, Cheriet M (2010) Genetic algorithm–based training for semi-supervised SVM. Neural Comput Appl 19(8):1197–1206

    Article  Google Scholar 

  37. Fung G, Mangasarian OL (2002) Semi-supervised support vector machines for unlabeled data classification. Optim Methods Softw 15:29–44

    Article  MATH  Google Scholar 

  38. Li Y, Zhou Z (2015) Towards making unlabeled data never hurt. IEEE Trans Pattern Anal Mach Intell 37(1):175–188

    Article  Google Scholar 

  39. Hu QH, Ding LX, He JR (2013) Lp norm constraint multi-kernel learning method for semi-supervised support vector machine. J Softw 24(11):2522–2534

    Article  MATH  Google Scholar 

  40. Yu J, Vishwanathan SVN, Günter S et al (2010) A quasi-Newton approach to nonsmooth convex optimization problems in machine learning. J Mach Learn Res 11(5):1145–1200

    MathSciNet  MATH  Google Scholar 

  41. Reddy IS, Shevade S, Murty MN (2011) A fast quasi-Newton method for semi-supervised SVM. Pattern Recogn 44(10):2305–2313

    Article  MATH  Google Scholar 

  42. Gieseke F, Airola A, Pahikkala T, et al (2012) Sparse quasi-newton optimization for semi-supervised support vector machines. In: Proceedings of the 1st international conference on pattern recognition applications and methods, pp 45–54

  43. Jiang W, Yao L, Jiang X et al (2015) A new classification method based on semi-supervised support vector machine. In: Human-centred computing. First international conference, pp 633–645

  44. Joachims T (1999) Transductive inference for text classification using support vector machines. In: Proceedings of the sixteenth international conference, vol 99, pp 200–209

  45. Guillaumin M, Verbeek J, Schmid C (2010) Multimodal semi-supervised learning for image classification. In: 2010 IEEE Computer society conference on computer vision and pattern recognition, pp 902–909

  46. Li M, Wang R, Tang K (2013) Combining Semi-Supervised and active learning for hyperspectral image classification. In: Proceedings of the 2013 IEEE symposium on computational intelligence and data mining, pp 89–94

  47. Xie L, Pan P, Lu Y (2014) Markov random field based fusion for supervised and semi-supervised multi-modal image classification. Multimed Tools Appl 74(2):613–634

    Article  Google Scholar 

  48. Yang L, Su Q, Yang B et al (2014) A new semi-supervised support vector machine classifier based on wavelet transform and its application in the iris image recognition. Int J Appl Math Stat 52(5):86–93

    MATH  Google Scholar 

  49. Lu K, He X, Zhao J (2006) Semi-supervised support vector learning for face recognition. Advances in neural networks-ISNN 2006. Springer, Berlin, pp 104–109

    Chapter  Google Scholar 

  50. Liang P, Xueming Y (2012) Explore semi-supervised support vector machine algorithm for the application of physical education effect evaluation. Int J Adv Comput Technol 4(9):266–271

    Google Scholar 

  51. Jun CA, Habibollah H, Haza NAH (2015) Semi-supervised SVM-based feature selection for cancer classification using microarray gene expression data. Springer International Publishing, Switzerland, pp 468–477

    Google Scholar 

  52. Zhou Y, Liu T, Li J (2015) Rapid identification between edible oil and swill-cooked dirty oil by using a semi-supervised support vector machine based on graph and near-infrared spectroscopy. Chemometr Intell Lab Syst 143:1–6

    Article  Google Scholar 

  53. Chen YS, Wang GP, Dong SH (2003) A progressive transductive inference algorithm based on support vector machine. J Softw 14(3):451–460

    Google Scholar 

  54. Wang Y, Huang S (2005) Training TSVM with the proper number of positive samples. Pattern Recogn Lett 26(14):2187–2194

    Article  Google Scholar 

  55. Zhang R, Wang W, Ma Y et al (2009) Least square transduction support vector machine. Neural Process Lett 29(2):133–142

    Article  Google Scholar 

  56. Yu X, Yang J, Zhang J (2012) A transductive support vector machine algorithm based on spectral clustering. AASRI Procedia 1:384–388

    Article  Google Scholar 

  57. Tian X, Gasso G, Canu S (2012) A multiple kernel framework for inductive semi-supervised SVM learning. Neurocomputing 90:46–58

    Article  Google Scholar 

  58. Zhou B, Hu C, Chen B, et al. (2014) A Transductive Support Vector Machine with adjustable quasi-linear kernel for semi-supervised data classification. In: Proceedings of the 2014 international joint conference on neural networks, pp 1409–1415

  59. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434

    MathSciNet  MATH  Google Scholar 

  60. Gómez-Chova L, Camps-Valls G, Munoz-Mari J et al (2008) Semisupervised image classification with Laplacian support vector machines. IEEE Geosci Remote Sens Lett 5(3):336–340

    Article  Google Scholar 

  61. Geng B, Tao D, Xu C et al (2012) Ensemble manifold regularization. IEEE Trans Pattern Anal Mach Intell 34(6):1227–1233

    Article  Google Scholar 

  62. Luo Y, Tao D, Xu C et al (2013) Multiview vector-valued manifold regularization for multilabel image classification. IEEE Trans Neural Netw Learn Syst 24(5):709–722

    Article  Google Scholar 

  63. Luo Y, Tao D, Geng B et al (2013) Manifold regularized multitask learning for semi-supervised multilabel image classification. IEEE Trans Image Process 22(2):523–536

    Article  MathSciNet  Google Scholar 

  64. Melacci S, Belkin M (2011) Laplacian support vector machines trained in the primal. J Mach Learn Res 12:1149–1184

    MathSciNet  MATH  Google Scholar 

  65. Qi Z, Tian Y, Shi Y et al (2013) Cost-sensitive support vector machine for semi-supervised learning. Procedia Comput Sci 18:1684–1689

    Article  Google Scholar 

  66. Tan J, Zhen L, Deng N et al (2014) Laplacian p-norm proximal support vector machine for semi-supervised classification. Neurocomputing 144:151–158

    Article  Google Scholar 

  67. Yang L, Yang S, Jin P et al (2014) Semi-supervised hyperspectral image classification using spatio-spectral Laplacian support vector machine. IEEE Geosci Remote Sens Lett 11(3):651–655

    Article  Google Scholar 

  68. Qi Z, Tian Y, Shi Y (2015) Successive overrelaxation for Laplacian support vector machine. IEEE Trans Neural Netw Learn Syst 26(4):674–683

    Article  MathSciNet  Google Scholar 

  69. Li Y, Kwok JT, Zhou Z (2009) Semi-supervised learning using label mean. In: Proceedings of the 26th international conference on machine learning, pp 633–640

  70. Li Y, Kwok J, Zhou Z (2010) Cost-sensitive semi-supervised support vector machine. In: Proceedings of the 24th AAAI conference on artificial intelligences, pp 500–505

  71. Chapelle O, Weston J, Scholkopf B (2002) Cluster kernels for semi-supervised learning. In: Proceedings of 16th annual neural information processing systems conference, pp 585–592

  72. Ng AY, Jordan MI, Weiss Y (2002) On spectral clustering: analysis and an algorithm. Adv Neural Inf Process Syst 2:849–856

    Google Scholar 

  73. Szummer M, Jaakkola T (2002) Partially labeled classification with Markov random walks. Adv Neural Inf Process Syst 14:945–952

    Google Scholar 

  74. Tuia D, Camps-Vall G (2009) Semi-supervised remote sensing image classification with cluster kernels. IEEE Geosci Remote Sens Lett 6(1):224–228

    Article  Google Scholar 

  75. Li T, Wang XL (2013) Semi-supervised SVM classification method based on cluster kernel. Appl Res Comput 30(1):42–45

    Google Scholar 

  76. Gao HZ, Wang JW, Xu K et al (2011) Semisupervised classification of hyperspectral image based on clustering kernel and LS-SVM. J Signal Process 27(2):76–81

    Google Scholar 

  77. Tao XM, Cao PD, Song SY et al (2013) The SVM classification algorithm based on semi-supervised gauss mixture model kernel. Inf Control 42(1):18–26

    Google Scholar 

  78. Li J, Allinson N, Tao D et al (2006) Multitraining support vector machine for image retrieval. IEEE Trans Image Process 15(11):3597–3601

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (No. 61379101).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Shifei Ding, Zhibin Zhu & Xiekai Zhang

  2. Jiangsu Key Laboratory of Mine Mechanical and Electrical Equipment, China University of Mining and Technology, Xuzhou, 221116, China

    Shifei Ding

Authors
  1. Shifei Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhibin Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiekai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shifei Ding.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, S., Zhu, Z. & Zhang, X. An overview on semi-supervised support vector machine. Neural Comput & Applic 28, 969–978 (2017). https://doi.org/10.1007/s00521-015-2113-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-015-2113-7

Keywords

Search

Navigation