Neural Computing and Applications

, Volume 22, Supplement 1, pp 153–161 | Cite as

Multiple birth support vector machine for multi-class classification

  • Zhi-Xia Yang
  • Yuan-Hai Shao
  • Xiang-Sun Zhang
Original Article


For multi-class classification problem, a novel algorithm, called as multiple birth support vector machine (MBSVM), is proposed, which can be considered as an extension of twin support vector machine. Our MBSVM has been compared with the several typical support vector machines. From theoretical point of view, it has been shown that its computational complexity is remarkably low, especially when the class number K is large. Based on our MBSVM, the dual problems of MBSVM are equivalent to symmetric mixed linear complementarity problems to which successive overrelaxation (SOR) can be directly applied. We establish our SOR algorithm for MBSVM. The SOR algorithm handles one data point at a time, so it can process large dataset that need no reside in memory. From practical point of view, its accuracy has been validated by the preliminary numerical experiments.


Multi-class classification Support vector machine Quadratic programming Multiple birth support vector machine 


  1. 1.
    Cortes C, Vapnik V (1995) Support-vector network. Mach Learn 20:273–297zbMATHGoogle Scholar
  2. 2.
    Vapnik V (1998) The nature of statistical learning, 2nd edn. Springer, New YorkGoogle Scholar
  3. 3.
    Jayadeva R, Khemchandani R, Chandra S (2007) Twin support vector machine for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910CrossRefGoogle Scholar
  4. 4.
    Mangasarian OL, Wild EW (2006) Multisurface proximal support vector classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74CrossRefGoogle Scholar
  5. 5.
    Balasundaram S, Kapil N (2010) Application of lagrangian twin support vector machines for classification. Second international conference on machine learning and computing, pp 193–397Google Scholar
  6. 6.
    Ghorai S, Mukherjee A, Dutta PK (2009) Nonparallel plane proximal classifier. Signal Process 89:510–522zbMATHCrossRefGoogle Scholar
  7. 7.
    Jayadeva R, Khemchandani R, Chandra S (2009) Optimal kernel selection in twin support vector machines. Optim Lett 3:77–88MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Kumar MA, Gopal M (2008) Application of smoothing technique on twin support vector machines. Pattern Recognit Lett 29:1842–1848CrossRefGoogle Scholar
  9. 9.
    Kumar MA, Khemchandani R, Gopal M, Chandra S (2010) Knowledge based least squares twin support vector machines. Inf Sci 180:4606–4618zbMATHCrossRefGoogle Scholar
  10. 10.
    Peng XJ (2010) Least squares twin support vector hypersphere (LS-TSVH) for pattern recognition. Expert Syst Appl 37:8371–8378CrossRefGoogle Scholar
  11. 11.
    Peng XJ (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23:365–372CrossRefGoogle Scholar
  12. 12.
    Peng XJ (2010) Primal twin support vector regression and its sparse approximation. Neurocomputing 73(16–18):2846–2858CrossRefGoogle Scholar
  13. 13.
    Peng XJ (2010) A ν-twin support vector machine (ν-TSVM) classifier and its geometric algorithms. Inf Sci 180(20):3863–3875zbMATHCrossRefGoogle Scholar
  14. 14.
    Shao YH, Zhang CH, Wang XB, Deng NY (2011) Improvoments on twin support vector machine. IEEE Trans Neural Netw 22(6):962–968CrossRefGoogle Scholar
  15. 15.
    Ye QL, Zhao CX, Ye N (2010) Least squares twin support vector machine classification via maximum one-class within class variance. Optimization methods and software, 18 AugustGoogle Scholar
  16. 16.
    Ye QL, Zhao CX, Ye N, Zheng H, Chen XB (2010) A feature selection method for nonparallel plane support vector machine classification. Optimization methods and software, 29 NovemberGoogle Scholar
  17. 17.
    Ye QL, Zhao CX, Ye N, Chen XB (2011) Localized twin SVM via convex minimization. Neurocomputing 74:580–587CrossRefGoogle Scholar
  18. 18.
    Nemmour H, Chibani Y (2006) Multi-class SVMs based on fuzzy integral mixture for handwritten digit recognition. Geometric modeling and imaging—new trends, pp 145–149Google Scholar
  19. 19.
    Rogers S, Girolami M, Krebs R, Mischak H (2005) Disease classification from capillary electrophoresis: mass spectrometry. Lect Notes Comput Sci 3686:183–191CrossRefGoogle Scholar
  20. 20.
    Ding CH, Dubchak I (2001) Multi-class protein fold recognition using support vector machines and neural networks. Bioinform Biol Insights 17:349–358CrossRefGoogle Scholar
  21. 21.
    Allwein EL, Schapire RE, Singer Y (2001) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141MathSciNetzbMATHGoogle Scholar
  22. 22.
    Bennett KP (1999) Combining support vector and mathematical programming methods for classification. In: Scholkopf B, Burges CJC, Smola AJ (eds) Advances in Kernel methods: support vector learning. MIT Press, Cambridge, MA, pp 307–326Google Scholar
  23. 23.
    Crammer K, Singer Y (2002) On the learnability and design of output codes for multiclass problems. Mach Learn 47:201–233zbMATHCrossRefGoogle Scholar
  24. 24.
    Dietterich TG, Bakiri G (1995) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286zbMATHGoogle Scholar
  25. 25.
    Hastie TJ, Ribshirani RJ (1998) Classification by pairwise coupling. In: Jordan MI, Kearns MJ, Solla SA (eds) Advances in neural information processing systems 10. MIT Press, Cambridge, MA, pp 507–513Google Scholar
  26. 26.
    Kreβ U (1999) Pairwise classification and support vector machines. In: Scholkopf B, Burges CJC, Smola AJ (eds) Advances in Kernel methods: support vector learning. MIT Press, Cambridge, MA, pp 255–268Google Scholar
  27. 27.
    Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. Adv Neural Inf Process Syst 12:547–553Google Scholar
  28. 28.
    Crammer K, Singer Y (2002) On the algorithmic implementation of multiclass kernel-based vector machines. J Mach Learn Res 2:265–292zbMATHGoogle Scholar
  29. 29.
    Lee Y, Lin Y, Wahba G (2001) Multicategory support vector machines. Comput Sci Stat 33:498–512Google Scholar
  30. 30.
    Weston J, Watkins C (1998) Multi-class support vector machines. CSD-TR-98-04 royal holloway. University of London, Egham, UKGoogle Scholar
  31. 31.
    Mangasarian OL, Musicant DR (1999) Successive overrelaxation for support vector machines. IEEE Trans Neural Netw 10(5):1032–1037CrossRefGoogle Scholar
  32. 32.
    Bottou L, Cortes C, Denker JS, Drucher H, Guyon I, Jackel LD, LeCun Y, M\(\ddot{u}\)ller UA, Sackinger E, Simard P, Vapnik V (1994) Comparison of classifier methods: a case study in handwriting digit recognition. In: IAPR (eds) Proceedings of the international conference on pattern recognition. IEEE Computer Society Press, pp 77–82Google Scholar
  33. 33.
    Moreira M, Mayoraz E (1998) Improved pairwise coupling classification with correcting classifiers. In: Nédellec C, Rouveirol C (eds) Proceedings of the ECML-98. Chemnitz, Germany, pp 160–171Google Scholar
  34. 34.
    Platt JC (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Smola AJ, Bartlett P, Schölkopf B, Schuurmans D (eds) Advances in large margin classifiers. MIT Press, Cambridge, MAGoogle Scholar
  35. 35.
    Golub GH, Van Loan CF (1996) Matrix computations, 3rd edn. The John Hopkins University Press, BaltimoreGoogle Scholar
  36. 36.
    Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: 7th International proceedings on knowledge discovery and data mining, pp 77–86Google Scholar
  37. 37.
    Luo ZQ, Tseng P (1993) Error bounds and convergence analysis of feasible descent methods: a general approach. Ann Oper Res 46:157–178MathSciNetCrossRefGoogle Scholar
  38. 38.
    Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425CrossRefGoogle Scholar
  39. 39.
    Blake CL, Merz CJ (1998) UCI repository of machine learning databases. Technical Report University of California, Department of Information and Computer Science, Irvine, CA. Available at:

Copyright information

© Springer-Verlag London Limited 2012

Authors and Affiliations

  1. 1.College of Mathematics and Systems ScienceXinjiang UniversityUrumqiPeople’s Republic of China
  2. 2.Zhijiang CollegeZhejiang University of TechnologyHangzhouPeople’s Republic of China
  3. 3.Academy of Mathematics and Systems ScienceChina Academy of SciencesBeijingPeople’s Republic of China

Personalised recommendations