Skip to main content
SpringerLink
Log in

Multiple birth support vector machine for multi-class classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

For multi-class classification problem, a novel algorithm, called as multiple birth support vector machine (MBSVM), is proposed, which can be considered as an extension of twin support vector machine. Our MBSVM has been compared with the several typical support vector machines. From theoretical point of view, it has been shown that its computational complexity is remarkably low, especially when the class number K is large. Based on our MBSVM, the dual problems of MBSVM are equivalent to symmetric mixed linear complementarity problems to which successive overrelaxation (SOR) can be directly applied. We establish our SOR algorithm for MBSVM. The SOR algorithm handles one data point at a time, so it can process large dataset that need no reside in memory. From practical point of view, its accuracy has been validated by the preliminary numerical experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Cortes C, Vapnik V (1995) Support-vector network. Mach Learn 20:273–297

    MATH  Google Scholar 

  2. Vapnik V (1998) The nature of statistical learning, 2nd edn. Springer, New York

    Google Scholar 

  3. Jayadeva R, Khemchandani R, Chandra S (2007) Twin support vector machine for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  Google Scholar 

  4. Mangasarian OL, Wild EW (2006) Multisurface proximal support vector classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74

    Article  Google Scholar 

  5. Balasundaram S, Kapil N (2010) Application of lagrangian twin support vector machines for classification. Second international conference on machine learning and computing, pp 193–397

  6. Ghorai S, Mukherjee A, Dutta PK (2009) Nonparallel plane proximal classifier. Signal Process 89:510–522

    Article  MATH  Google Scholar 

  7. Jayadeva R, Khemchandani R, Chandra S (2009) Optimal kernel selection in twin support vector machines. Optim Lett 3:77–88

    Article  MathSciNet  MATH  Google Scholar 

  8. Kumar MA, Gopal M (2008) Application of smoothing technique on twin support vector machines. Pattern Recognit Lett 29:1842–1848

    Article  Google Scholar 

  9. Kumar MA, Khemchandani R, Gopal M, Chandra S (2010) Knowledge based least squares twin support vector machines. Inf Sci 180:4606–4618

    Article  MATH  Google Scholar 

  10. Peng XJ (2010) Least squares twin support vector hypersphere (LS-TSVH) for pattern recognition. Expert Syst Appl 37:8371–8378

    Article  Google Scholar 

  11. Peng XJ (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23:365–372

    Article  Google Scholar 

  12. Peng XJ (2010) Primal twin support vector regression and its sparse approximation. Neurocomputing 73(16–18):2846–2858

    Article  Google Scholar 

  13. Peng XJ (2010) A ν-twin support vector machine (ν-TSVM) classifier and its geometric algorithms. Inf Sci 180(20):3863–3875

    Article  MATH  Google Scholar 

  14. Shao YH, Zhang CH, Wang XB, Deng NY (2011) Improvoments on twin support vector machine. IEEE Trans Neural Netw 22(6):962–968

    Article  Google Scholar 

  15. Ye QL, Zhao CX, Ye N (2010) Least squares twin support vector machine classification via maximum one-class within class variance. Optimization methods and software, 18 August

  16. Ye QL, Zhao CX, Ye N, Zheng H, Chen XB (2010) A feature selection method for nonparallel plane support vector machine classification. Optimization methods and software, 29 November

  17. Ye QL, Zhao CX, Ye N, Chen XB (2011) Localized twin SVM via convex minimization. Neurocomputing 74:580–587

    Article  Google Scholar 

  18. Nemmour H, Chibani Y (2006) Multi-class SVMs based on fuzzy integral mixture for handwritten digit recognition. Geometric modeling and imaging—new trends, pp 145–149

  19. Rogers S, Girolami M, Krebs R, Mischak H (2005) Disease classification from capillary electrophoresis: mass spectrometry. Lect Notes Comput Sci 3686:183–191

    Article  Google Scholar 

  20. Ding CH, Dubchak I (2001) Multi-class protein fold recognition using support vector machines and neural networks. Bioinform Biol Insights 17:349–358

    Article  Google Scholar 

  21. Allwein EL, Schapire RE, Singer Y (2001) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141

    MathSciNet  MATH  Google Scholar 

  22. Bennett KP (1999) Combining support vector and mathematical programming methods for classification. In: Scholkopf B, Burges CJC, Smola AJ (eds) Advances in Kernel methods: support vector learning. MIT Press, Cambridge, MA, pp 307–326

    Google Scholar 

  23. Crammer K, Singer Y (2002) On the learnability and design of output codes for multiclass problems. Mach Learn 47:201–233

    Article  MATH  Google Scholar 

  24. Dietterich TG, Bakiri G (1995) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286

    MATH  Google Scholar 

  25. Hastie TJ, Ribshirani RJ (1998) Classification by pairwise coupling. In: Jordan MI, Kearns MJ, Solla SA (eds) Advances in neural information processing systems 10. MIT Press, Cambridge, MA, pp 507–513

    Google Scholar 

  26. Kreβ U (1999) Pairwise classification and support vector machines. In: Scholkopf B, Burges CJC, Smola AJ (eds) Advances in Kernel methods: support vector learning. MIT Press, Cambridge, MA, pp 255–268

    Google Scholar 

  27. Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. Adv Neural Inf Process Syst 12:547–553

    Google Scholar 

  28. Crammer K, Singer Y (2002) On the algorithmic implementation of multiclass kernel-based vector machines. J Mach Learn Res 2:265–292

    MATH  Google Scholar 

  29. Lee Y, Lin Y, Wahba G (2001) Multicategory support vector machines. Comput Sci Stat 33:498–512

    Google Scholar 

  30. Weston J, Watkins C (1998) Multi-class support vector machines. CSD-TR-98-04 royal holloway. University of London, Egham, UK

    Google Scholar 

  31. Mangasarian OL, Musicant DR (1999) Successive overrelaxation for support vector machines. IEEE Trans Neural Netw 10(5):1032–1037

    Article  Google Scholar 

  32. Bottou L, Cortes C, Denker JS, Drucher H, Guyon I, Jackel LD, LeCun Y, M\(\ddot{u}\)ller UA, Sackinger E, Simard P, Vapnik V (1994) Comparison of classifier methods: a case study in handwriting digit recognition. In: IAPR (eds) Proceedings of the international conference on pattern recognition. IEEE Computer Society Press, pp 77–82

  33. Moreira M, Mayoraz E (1998) Improved pairwise coupling classification with correcting classifiers. In: Nédellec C, Rouveirol C (eds) Proceedings of the ECML-98. Chemnitz, Germany, pp 160–171

    Google Scholar 

  34. Platt JC (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Smola AJ, Bartlett P, Schölkopf B, Schuurmans D (eds) Advances in large margin classifiers. MIT Press, Cambridge, MA

    Google Scholar 

  35. Golub GH, Van Loan CF (1996) Matrix computations, 3rd edn. The John Hopkins University Press, Baltimore

    Google Scholar 

  36. Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: 7th International proceedings on knowledge discovery and data mining, pp 77–86

  37. Luo ZQ, Tseng P (1993) Error bounds and convergence analysis of feasible descent methods: a general approach. Ann Oper Res 46:157–178

    Article  MathSciNet  Google Scholar 

  38. Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425

    Article  Google Scholar 

  39. Blake CL, Merz CJ (1998) UCI repository of machine learning databases. Technical Report University of California, Department of Information and Computer Science, Irvine, CA. Available at: http://www.ics.uci.edu/mlearn/MLRepository.html

Download references

Author information

Authors and Affiliations

  1. College of Mathematics and Systems Science, Xinjiang University, Urumqi, 830046, People’s Republic of China

    Zhi-Xia Yang

  2. Zhijiang College, Zhejiang University of Technology, Hangzhou, 310024, People’s Republic of China

    Yuan-Hai Shao

  3. Academy of Mathematics and Systems Science, China Academy of Sciences, Beijing, 100190, People’s Republic of China

    Xiang-Sun Zhang

Authors
  1. Zhi-Xia Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuan-Hai Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiang-Sun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhi-Xia Yang.

Additional information

This work is supported by the National Natural Science Foundation of China (No.11161045) and Zhejiang Provincial Natural Science Foundation of China (No. LQ12A01020).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yang, ZX., Shao, YH. & Zhang, XS. Multiple birth support vector machine for multi-class classification. Neural Comput & Applic 22 (Suppl 1), 153–161 (2013). https://doi.org/10.1007/s00521-012-1108-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-012-1108-x

Keywords

Search

Navigation