Advertisement

Advances in Data Analysis and Classification

, Volume 7, Issue 4, pp 417–433 | Cite as

A class of semi-supervised support vector machines by DC programming

  • Liming YangEmail author
  • Laisheng Wang
Regular Article

Abstract

This paper investigate a class of semi-supervised support vector machines (\(\text{ S }^3\mathrm{VMs}\)) with arbitrary norm. A general framework for the \(\text{ S }^3\mathrm{VMs}\) was first constructed based on a robust DC (Difference of Convex functions) program. With different DC decompositions, DC optimization formulations for the linear and nonlinear \(\text{ S }^3\mathrm{VMs}\) are investigated. The resulting DC optimization algorithms (DCA) only require solving simple linear program or convex quadratic program at each iteration, and converge to a critical point after a finite number of iterations. The effectiveness of proposed algorithms are demonstrated on some UCI databases and licorice seed near-infrared spectroscopy data. Moreover, numerical results show that the proposed algorithms offer competitive performances to the existing \(\text{ S }^3\mathrm{VM}\) methods.

Keywords

Nonconvex optimization DC programming Semi-supervised support vector machine 

Mathematics Subject Classification (2000)

90C26 90C90 

Notes

Acknowledgments

This work is supported by National Nature Science Foundation of China (11271367) and Research Projects of Teachers (2010JS052). Moreover, the authors thank the referees and the editors for their constructive comments. Their suggestions improved the paper significantly.

References

  1. Alamoa T, Bravob JM et al (2008) A set-membership state estimation algorithm based on DC programming. Automatica 44:216–224CrossRefGoogle Scholar
  2. Astorino A, Fuduli A (2007) Nonsmooth optimization techniques for semi-supervised classification. IEEE Trans Pattern Anal Machine Intell 9(12):2135–2142CrossRefGoogle Scholar
  3. Astorino A, Fuduli A, Gaudioso M (2010) DC models for spherical separation. J Global Optim 48(4): 657–669MathSciNetCrossRefzbMATHGoogle Scholar
  4. Bennett K, Demiriz A (1998) Semi-supervised support vector machines. In: Advances in neural information processing systems, vol 12. MIT Press, Cambridge, pp 368–374Google Scholar
  5. Chapelle O, Sindhwani V, Keerthi S (2006) Branch and bound for semi-supervised support vector machines. In: Advances in neural information processing systems, vol 17. MIT Press, Cambridge pp 217–224Google Scholar
  6. Chapelle O, Sindhwani V, Keerthi S (2008) Optimization techniques for semi-supervised support vector machines. J Mach Learn 9:203–233zbMATHGoogle Scholar
  7. Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of 10th International Workshop on artificial intelligence and statistics, Barbados, pp 57–64Google Scholar
  8. Collobert R, Sinz F et al (2006) Large scale transductive SVMs. J Mach Learn 7:1687–1712MathSciNetzbMATHGoogle Scholar
  9. Fawcett T (2005) An introduction to ROC analysis. Pattern Recogn Lett 27:861–874CrossRefGoogle Scholar
  10. Fung G, Mangasarian O (2001) Semi-supervised support vector machines for unlabeled data classification. Optim Methods Softw 15:29–44CrossRefzbMATHGoogle Scholar
  11. Horst R, Thoai N (1999) DC programming: overview. J Optim Theory Appl 103(1):1–43MathSciNetCrossRefGoogle Scholar
  12. Joachims T (1999) Transductive inference for text classification using support vector machines. In: 16th International Conference on machine learning, San Francisco, USA, pp 200–209Google Scholar
  13. Kimeldorf G, Wahba G (1971) Some results on Tchebycheffian spline functions. J Math Anal 33:82–95MathSciNetCrossRefzbMATHGoogle Scholar
  14. Le Thi HA, Pham Dinh T (2005) The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann Oper Res 133:23–46MathSciNetCrossRefzbMATHGoogle Scholar
  15. Le Thi HA et al (2008) A DC programming approach for feature selection in support vector machines learning. Adv Data Anal Classif 2:259–278MathSciNetCrossRefGoogle Scholar
  16. Le Thi HA, Pham Dinh T (2011) On solving linear complementarity problems by DC programming and DCA. Comput Optim Appl 50(3):507–524MathSciNetCrossRefzbMATHGoogle Scholar
  17. Le Thi HA et al (2012) Binary classification via spherical separator by DC programming and DCA. J Global Optim. doi: 10.1007/s10898-012-9859-6
  18. Reddy IS, Shevade S, Murty MN (2011) A fast quasi-Newton method for semi-supervised SVM. Pattern Recogn 44:2305–2313CrossRefzbMATHGoogle Scholar
  19. Toland J (1978) Duality in nonconvex optimization. J Math Anal Appl 66:399–415MathSciNetCrossRefzbMATHGoogle Scholar
  20. Vapnik VN (1998) Statistical learning theory. Wiley, New YorkzbMATHGoogle Scholar
  21. Wang L, Shen X, Pan W (2007) On transductive support vector machines. In: Verducci J, Shen X, Lafferty J (eds) Prediction and Discovery. American Mathematical Society. http://www.stat.umn.edu/xshen/paper/tsvm.pdf
  22. Wang G et al (2013) A novel DPSO-SVM system for variable interval selection of endometrial tissue sections by near infrared spectroscopy. Talanta 112(15):136–142CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.College of ScienceChina Agricultural UniversityBeijingChina

Personalised recommendations