Neural Computing and Applications

, Volume 23, Issue 7–8, pp 2159–2166 | Cite as

Mixed-norm linear support vector machine

  • Chunhua Zhang
  • Yuanhai Shao
  • Junyan Tan
  • Naiyang Deng
Original Article

Abstract

This paper presents a new version of support vector machine (SVM) named l2 − lp SVM (0 < p < 1) which introduces the lp-norm (0 < p < 1) of the normal vector of the decision plane in the standard linear SVM. To solve the nonconvex optimization problem in our model, an efficient algorithm is proposed using the constrained concave–convex procedure. Experiments with artificial data and real data demonstrate that our method is more effective than some popular methods in selecting relevant features and improving classification accuracy.

Keywords

Support vector machine Optimization Norm Feature selection Constrained concave–convex procedure 

References

  1. 1.
    Vapnik VN (1998) Statistical learning theory. Wiley, New YorkMATHGoogle Scholar
  2. 2.
    Deng NY, Tian YJ, Zhang CH (2012) Suppport vector machines: optimization based theory, algorithms and applications. CRC press, Boca Raton (in press)Google Scholar
  3. 3.
    Guyon I, Weston J, Barnhill S, Vapnik VN (2002) Gene selection for cancer classification using support vector machines. Mach Learn 46:389–422CrossRefMATHGoogle Scholar
  4. 4.
    Mangasarian OL (1997) Minimum-support solutions of polyhedral concave programs, Technical Report Tr-1997-05, Mathematical Programming, University of WisconsinGoogle Scholar
  5. 5.
    Zhu J, Rosset S, Hastie T, Tibshirani R (2003) 1-norm support vector machines. Adv Neural Inf Process Syst 16:49-56Google Scholar
  6. 6.
    Bradley PS, Mangasarian OL (1998) Feature selection via concave minimization and support vector machines. In: Proceedings of the 13th ICML, pp 82–90Google Scholar
  7. 7.
    Mangasarian OL, Wild EW (2007) Feature selection for nonlinear kernel support vector machines. In: IEEE seventh international conference on data mining, pp 231–236Google Scholar
  8. 8.
    Chen XJ, Xu FM, Ye YY (2009) Lower bound theory of nonzero entries in solutions of l 2-l p minimization. http://www.standardford.edu/yyye/
  9. 9.
    Bruckstein AM, Donoho DL, Elad M (2009) From sparse sulutions of systems of equations to sparse modeling of signals and images. SIAM Rev 51:34–81MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Fan J, Li R (2001) Varible selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 96:1348–1360MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Xu Z, Zhang H, Wang Y, Chang X (2009) L1/2 regularizer. Sci Chin Ser F-Inf Sci 52:1–9CrossRefGoogle Scholar
  12. 12.
    Tan JY, Zhang CH, etc. (2010) Cancer Related Gene Identification via p-norm support vector machine. In: The international conference on computational systems biology, pp 101–108Google Scholar
  13. 13.
    Zhang CH, Tan JY et al (2010) Feature Selection in multi-instance learning. In: The international symposium on operations research and its Applications: 462-469Google Scholar
  14. 14.
    Tan JY, Zhang ZQ, Zhen L, Zhang CH, Deng NY (2012) Adaptive feature selection via a new version of support vector machine. Neural Comput Appl. doi:10.1007/s00521-012-1018-y
  15. 15.
    Chen WJ, Tian YJ (2010) lp-norm proximal support vector machine and its application. Proc Comput Sci ICCS 1(1):2411-2417Google Scholar
  16. 16.
    Yuille AL, Rangarijan (2003) The concave-convex procedure. Neural Comput 15:915–936Google Scholar
  17. 17.
    Smola AJ, Vishwanathan SVN, Hofman T (2005) Kernel methods for missing variables. In: Proceedings of the tenth international workshop on artificial intelligence and statistics, BarbodosGoogle Scholar
  18. 18.
    Singh D, Febbo P, Ross K, Jackson D, Manola J, Ladd C, Tamayo P, Renshaw A, D’Amico A, Richie J, Lander E, Loda M, Kantoff P, Golub T, Sellers W (2002) Gene expression correlates of clinical prostate cancer behavior. Cancer Cell 1:203–209CrossRefGoogle Scholar
  19. 19.
    Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, Coller H, Loh ML, Downing JR, Caligiuri MA, Bloomfield CD, Lander ES (1999) Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286:531–537CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2012

Authors and Affiliations

  • Chunhua Zhang
    • 1
  • Yuanhai Shao
    • 2
  • Junyan Tan
    • 3
  • Naiyang Deng
    • 3
  1. 1.Department of Mathematics, Information SchoolRenmin University of ChinaBeijingChina
  2. 2.Zhijiang CollegeZhejiang University of TechnologyHangzhouChina
  3. 3.College of ScienceChina Agricultural UniversityBeijingChina

Personalised recommendations