Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

An overview on semi-supervised support vector machine

  • 1524 Accesses

  • 15 Citations

Abstract

Support vector machine (SVM) is a machine learning method based on statistical learning theory. It has a lot of advantages, such as solid theoretical foundation, global optimization, the sparsity of the solution, nonlinear and generalization. The standard form of SVM only applies to supervised learning. Large amount of data generated in real life is unlabeled, and the standard form of SVM cannot make good use of these data to improve its learning ability. However, semi-supervised support vector machine (S3VM) is a good solution to this problem. This paper reviews the recent progress in semi-supervised support vector machine. First, the basic theory of S3VM is expounded and discussed in detail; then, the mainstream model of S3VM is presented, including transductive support vector machine, Laplacian support vector machine, S3VM training via the label mean, S3VM based on cluster kernel; finally, we give the conclusions and look ahead to the research on S3VM.

This is a preview of subscription content, log in to check access.

References

  1. 1.

    Altun Y, Belkin M, Mcallester DA (2005) Maximum margin semi-supervised learning for structured variables. In: Proceedings of the 2005 annual conference on neural information processing systems, pp 33–40

  2. 2.

    Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. Synth Lect Artif Intell Mach Learn 3(1):1–130

  3. 3.

    Hady MFA, Schwenker F (2013) Semi-supervised learning. Handbook on neural information processing. Springer, Berlin, pp 215–239

  4. 4.

    Ding SF, Qi BJ, Tan YH (2011) An overview on theory and algorithm of support vector machines. J Univ Electron Sci Technol China 40(1):2–10

  5. 5.

    Gu YX, Ding SF (2011) Advances of support vector machines. J Comput Sci Technol 38(2):14–17

  6. 6.

    Schölkopf B, Smola AJ (2001) Learning with kernels: SUPPORT vector machines, regularization, optimization, and beyond. MIT Press, Cambridge

  7. 7.

    Ding S (2011) Incremental learning algorithm for support vector data description. J Softw 6(7):1166–1173

  8. 8.

    Liu XL, Ding SF (2010) Appropriateness in applying SVMs to text classification. Comput Eng Sci 32(6):106–108

  9. 9.

    Bennett K, Demiriz A (1999) Semi-supervised support vector machines. In: Advances in neural information processing systems, vol 11, pp 368–374

  10. 10.

    Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning. Springer, New York

  11. 11.

    Steinwart I, Christmann A (2008) Support vector machines. Springer, New York

  12. 12.

    Jordan MI, Jacobs RI (2014) Supervised learning and divide-and-conquer: a statistical approach. In: Proceedings of the tenth international conference on machine learning, pp 159–166

  13. 13.

    Shipp MA, Ross KN, Tamayo P et al (2002) Diffuse large B-cell lymphoma outcome prediction by gene-expression profiling and supervised machine learning. Nat Med 8(1):68–74

  14. 14.

    Zhang CG, Zhang Y (2013) Semi-supervised learning. China Agricultural Science and Technology Press, Beijing

  15. 15.

    Miller DJ, Uyar HS (1997) A mixture of experts classifier with learning based on both labelled and unlabelled data. In: Advances in neural information processing systems. MIT Press, Cambridge, pp 571–577

  16. 16.

    Zhang T, Oles F (2000) The value of unlabeled data for classification problems. In: Proceedings of 17th international conference on machine learning, pp 1191–1198

  17. 17.

    Chapelle O, Schölkopf B, Zien A (2006) Semi-supervised learning. MIT Press, Cambridge

  18. 18.

    Triguero I, Sáez JA, Luengo J et al (2014) On the characterization of noise filters for self-training semi-supervised in nearest neighbor classification. Neurocomputing 132:30–41

  19. 19.

    Dópido I, Li J, Marpu PR et al (2013) Semi-supervised self-learning for hyperspectral image classification. IEEE Trans Geosci Remote Sens 51(7):4032–4044

  20. 20.

    Li Y, Li H, Guan C, et al (2007) A self-training semi-supervised support vector machine algorithm and its applications in brain computer interface. In: International conference on acoustics, speech, and signal processing, pp 385–387

  21. 21.

    McLachlan GJ, Krishnan T (1997) The EM algorithm and extensions. Wiley-Interscience, New York

  22. 22.

    Dong C, Yin Y, Guo X et al (2008) On co-training style algorithms. Int Conf Nat Comput 18(20):196–201

  23. 23.

    Feger F, Koprinska I (2006) Co-training using RBF nets and different feature splits. In: International joint conference on neural networks, pp 1878–1885

  24. 24.

    Yu S, Krishnapuram B, Rosales R et al (2011) Bayesian co-training. J Mach Learn Res 12:2649–2680

  25. 25.

    Zhu X (2005) Semi-supervised learning with graphs, Carnegie Mellon University, language technologies institute, school of computer science

  26. 26.

    Tao D, Li X, Wu X et al (2007) Supervised tensor learning. Knowl Inf Syst 13(1):1–42

  27. 27.

    Chapelle O, Sindhwani V, Keerthi SS (2008) Optimization techniques for semi-supervised support vector machines. J Mach Learn Res 9:203–233

  28. 28.

    Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, pp 57–64

  29. 29.

    Gieseke F, Airola A, Pahikkala T et al (2014) Fast and simple gradient-based optimization for semi-supervised support vector machines. Neurocomputing 123:23–32

  30. 30.

    Collobert R, Sinz F, Weston J et al (2006) Large scale transductive SVMs. J Mach Learn Res 7:1687–1712

  31. 31.

    Sindhwani V, Keerthi SS, Chapelle O (2006) Deterministic annealing for semi-supervised kernel machines. In: Proceedings of the 23rd international conference on machine learning, pp 841–848

  32. 32.

    Chapelle O, Chi M, Zien A (2006) A continuation method for semi-supervised SVMs. In: Proceedings of the 23rd international conference on machine learning, pp 185–192

  33. 33.

    De Bie T, Cristianini N (2006) Semi-supervised learning using semi-definite programming. MIT press, Cambridge

  34. 34.

    Le HM, Le Thi HA, Nguyen MC (2015) Sparse semi-supervised support vector machines by DC programming and DCA. Neurocomputing 153:62–76

  35. 35.

    Chapelle O, Sindhwani V, Keerthi S (2007) Branch and bound for semi-supervised support vector machines. In: 20th Annual conference on neural information processing systems, pp 217–224

  36. 36.

    Adankon MM, Cheriet M (2010) Genetic algorithm–based training for semi-supervised SVM. Neural Comput Appl 19(8):1197–1206

  37. 37.

    Fung G, Mangasarian OL (2002) Semi-supervised support vector machines for unlabeled data classification. Optim Methods Softw 15:29–44

  38. 38.

    Li Y, Zhou Z (2015) Towards making unlabeled data never hurt. IEEE Trans Pattern Anal Mach Intell 37(1):175–188

  39. 39.

    Hu QH, Ding LX, He JR (2013) Lp norm constraint multi-kernel learning method for semi-supervised support vector machine. J Softw 24(11):2522–2534

  40. 40.

    Yu J, Vishwanathan SVN, Günter S et al (2010) A quasi-Newton approach to nonsmooth convex optimization problems in machine learning. J Mach Learn Res 11(5):1145–1200

  41. 41.

    Reddy IS, Shevade S, Murty MN (2011) A fast quasi-Newton method for semi-supervised SVM. Pattern Recogn 44(10):2305–2313

  42. 42.

    Gieseke F, Airola A, Pahikkala T, et al (2012) Sparse quasi-newton optimization for semi-supervised support vector machines. In: Proceedings of the 1st international conference on pattern recognition applications and methods, pp 45–54

  43. 43.

    Jiang W, Yao L, Jiang X et al (2015) A new classification method based on semi-supervised support vector machine. In: Human-centred computing. First international conference, pp 633–645

  44. 44.

    Joachims T (1999) Transductive inference for text classification using support vector machines. In: Proceedings of the sixteenth international conference, vol 99, pp 200–209

  45. 45.

    Guillaumin M, Verbeek J, Schmid C (2010) Multimodal semi-supervised learning for image classification. In: 2010 IEEE Computer society conference on computer vision and pattern recognition, pp 902–909

  46. 46.

    Li M, Wang R, Tang K (2013) Combining Semi-Supervised and active learning for hyperspectral image classification. In: Proceedings of the 2013 IEEE symposium on computational intelligence and data mining, pp 89–94

  47. 47.

    Xie L, Pan P, Lu Y (2014) Markov random field based fusion for supervised and semi-supervised multi-modal image classification. Multimed Tools Appl 74(2):613–634

  48. 48.

    Yang L, Su Q, Yang B et al (2014) A new semi-supervised support vector machine classifier based on wavelet transform and its application in the iris image recognition. Int J Appl Math Stat 52(5):86–93

  49. 49.

    Lu K, He X, Zhao J (2006) Semi-supervised support vector learning for face recognition. Advances in neural networks-ISNN 2006. Springer, Berlin, pp 104–109

  50. 50.

    Liang P, Xueming Y (2012) Explore semi-supervised support vector machine algorithm for the application of physical education effect evaluation. Int J Adv Comput Technol 4(9):266–271

  51. 51.

    Jun CA, Habibollah H, Haza NAH (2015) Semi-supervised SVM-based feature selection for cancer classification using microarray gene expression data. Springer International Publishing, Switzerland, pp 468–477

  52. 52.

    Zhou Y, Liu T, Li J (2015) Rapid identification between edible oil and swill-cooked dirty oil by using a semi-supervised support vector machine based on graph and near-infrared spectroscopy. Chemometr Intell Lab Syst 143:1–6

  53. 53.

    Chen YS, Wang GP, Dong SH (2003) A progressive transductive inference algorithm based on support vector machine. J Softw 14(3):451–460

  54. 54.

    Wang Y, Huang S (2005) Training TSVM with the proper number of positive samples. Pattern Recogn Lett 26(14):2187–2194

  55. 55.

    Zhang R, Wang W, Ma Y et al (2009) Least square transduction support vector machine. Neural Process Lett 29(2):133–142

  56. 56.

    Yu X, Yang J, Zhang J (2012) A transductive support vector machine algorithm based on spectral clustering. AASRI Procedia 1:384–388

  57. 57.

    Tian X, Gasso G, Canu S (2012) A multiple kernel framework for inductive semi-supervised SVM learning. Neurocomputing 90:46–58

  58. 58.

    Zhou B, Hu C, Chen B, et al. (2014) A Transductive Support Vector Machine with adjustable quasi-linear kernel for semi-supervised data classification. In: Proceedings of the 2014 international joint conference on neural networks, pp 1409–1415

  59. 59.

    Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434

  60. 60.

    Gómez-Chova L, Camps-Valls G, Munoz-Mari J et al (2008) Semisupervised image classification with Laplacian support vector machines. IEEE Geosci Remote Sens Lett 5(3):336–340

  61. 61.

    Geng B, Tao D, Xu C et al (2012) Ensemble manifold regularization. IEEE Trans Pattern Anal Mach Intell 34(6):1227–1233

  62. 62.

    Luo Y, Tao D, Xu C et al (2013) Multiview vector-valued manifold regularization for multilabel image classification. IEEE Trans Neural Netw Learn Syst 24(5):709–722

  63. 63.

    Luo Y, Tao D, Geng B et al (2013) Manifold regularized multitask learning for semi-supervised multilabel image classification. IEEE Trans Image Process 22(2):523–536

  64. 64.

    Melacci S, Belkin M (2011) Laplacian support vector machines trained in the primal. J Mach Learn Res 12:1149–1184

  65. 65.

    Qi Z, Tian Y, Shi Y et al (2013) Cost-sensitive support vector machine for semi-supervised learning. Procedia Comput Sci 18:1684–1689

  66. 66.

    Tan J, Zhen L, Deng N et al (2014) Laplacian p-norm proximal support vector machine for semi-supervised classification. Neurocomputing 144:151–158

  67. 67.

    Yang L, Yang S, Jin P et al (2014) Semi-supervised hyperspectral image classification using spatio-spectral Laplacian support vector machine. IEEE Geosci Remote Sens Lett 11(3):651–655

  68. 68.

    Qi Z, Tian Y, Shi Y (2015) Successive overrelaxation for Laplacian support vector machine. IEEE Trans Neural Netw Learn Syst 26(4):674–683

  69. 69.

    Li Y, Kwok JT, Zhou Z (2009) Semi-supervised learning using label mean. In: Proceedings of the 26th international conference on machine learning, pp 633–640

  70. 70.

    Li Y, Kwok J, Zhou Z (2010) Cost-sensitive semi-supervised support vector machine. In: Proceedings of the 24th AAAI conference on artificial intelligences, pp 500–505

  71. 71.

    Chapelle O, Weston J, Scholkopf B (2002) Cluster kernels for semi-supervised learning. In: Proceedings of 16th annual neural information processing systems conference, pp 585–592

  72. 72.

    Ng AY, Jordan MI, Weiss Y (2002) On spectral clustering: analysis and an algorithm. Adv Neural Inf Process Syst 2:849–856

  73. 73.

    Szummer M, Jaakkola T (2002) Partially labeled classification with Markov random walks. Adv Neural Inf Process Syst 14:945–952

  74. 74.

    Tuia D, Camps-Vall G (2009) Semi-supervised remote sensing image classification with cluster kernels. IEEE Geosci Remote Sens Lett 6(1):224–228

  75. 75.

    Li T, Wang XL (2013) Semi-supervised SVM classification method based on cluster kernel. Appl Res Comput 30(1):42–45

  76. 76.

    Gao HZ, Wang JW, Xu K et al (2011) Semisupervised classification of hyperspectral image based on clustering kernel and LS-SVM. J Signal Process 27(2):76–81

  77. 77.

    Tao XM, Cao PD, Song SY et al (2013) The SVM classification algorithm based on semi-supervised gauss mixture model kernel. Inf Control 42(1):18–26

  78. 78.

    Li J, Allinson N, Tao D et al (2006) Multitraining support vector machine for image retrieval. IEEE Trans Image Process 15(11):3597–3601

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (No. 61379101).

Author information

Correspondence to Shifei Ding.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ding, S., Zhu, Z. & Zhang, X. An overview on semi-supervised support vector machine. Neural Comput & Applic 28, 969–978 (2017). https://doi.org/10.1007/s00521-015-2113-7

Download citation

Keywords

  • Semi-supervised
  • Support vector machine
  • Semi-supervised support vector machine