Advertisement

Artificial Intelligence Review

, Volume 52, Issue 2, pp 775–801 | Cite as

A review on multi-class TWSVM

  • Shifei DingEmail author
  • Xingyu Zhao
  • Jian Zhang
  • Xiekai Zhang
  • Yu Xue
Article

Abstract

Twin support vector machines (TWSVM), a novel machine learning algorithm developing from traditional support vector machines (SVM), is one of the typical nonparallel support vector machines. Since the TWSVM has superiorities of the simple model, the high training speed and the good performance, it has drawn extensive attention. The initial TWSVM can only handle binary classification, however, the multi-class classification problems are also common in practice. How to extend TWSVM from binary classification to multi-class classification is an interesting issue. Many researchers have devoted to the study of multi-class TWSVM. Although the study of multi-class TWSVM has made great progress, there is little literature on the comparisons and summaries of different multi-class classifiers based on TWSVM, which not only makes it difficult for novices to understand the essential differences, but also leads to the problem that how to choose the suitable multi-class TWSVM for a practical multi-class classification problem. This paper aims to review the development of multi-class TWSVM in recent years. We group multi-classTWSVM reasonably and analyze them with the respect to the basic theories and geometric meaning. According to the structures of the multi-class TWSVM, we divide them to the following groups: “one-versus-rest” strategy based multi-classTWSVM, “one-versus-one” strategy based multi-class TWSVM, binary tree structure based multi-class TWSVM, “one-versus-one-versus-rest” strategy based multi-class TWSVM and “all-versus-one” strategy based multi-class TWSVM. Although the training processes of direct acyclic graph based multi-class TWSVM are much similar to that of “one-versus-one” multi-class TWSVM, the decision processes of direct acyclic graph based multi-class TWSVM have their own characteristics and disadvantages, so we divide them to a separate group. This paper analyzes and summarizes the basic thoughts, theories, applicability and complexities of different multi-class TWSVM of different groups and presents experimental results to compare the performances.

Keywords

Support vector machines (SVM) Twin support vector machines (TWSVM) Multi-class TWSVM Multiple birth support vector machine (MBSVM) 

Notes

Acknowledgements

This work was supported by the Fundamental Research Funds for the Central Universities (No. 2017XKQY076).

References

  1. Al-Dhaifallah M (2015) Twin support vector machine method for identification of Wiener models. Math Probl Eng 2015(2):1–7MathSciNetzbMATHCrossRefGoogle Scholar
  2. Angulo C, Parra X, Catala A (2003a) K-SVCR: a support vector machine for multi-class classification. Neurocomputing 55(1):57–77Google Scholar
  3. Angulo C, Parra X, Català A (2003b) K-SVCR. A support vector machine for multi-class classification. Neurocomputing 55(1–2):57–77Google Scholar
  4. Chen J, Ji G (2010) Multi-class LSTSVM classifier based on optimal directed acyclic graph. In: International conference on computer and automation engineering. IEEE, pp 100–104Google Scholar
  5. Chen S, Xu J (2015) Least squares twin support vector machine for multi-class classification. Int J Database Theory Appl 8(5):65–76CrossRefGoogle Scholar
  6. Cheng F, Wang W, Guo H (2014) Dynamic granular support vector machine learning algorithm. Pattern Recogn Artif Intell 27(4):372–376Google Scholar
  7. Chu M, Gong R, Wang A (2014a) Strip steel surface defect classification method based on enhanced twin support vector machine. Trans Iron Steel Inst Jpn 54(1):119–124CrossRefGoogle Scholar
  8. Chu M, Wang A, Gong R, Sha M (2014b) Multi-class classification methods of enhanced LS-TWSVM for strip steel surface defects. Int J Iron Steel Res 21(2):174–180CrossRefGoogle Scholar
  9. Cong H, Yang C, Pu X (2008) Efficient speaker recognition based on multi-class twin support vector machines and GMMs. In: IEEE conference on robotics, automation and mechatronics, Ram, Chengdu, China, pp 348–352Google Scholar
  10. Cortes C, Vapnik VN (1995) Support vector networks. Mach Learn 20(2):273–297zbMATHGoogle Scholar
  11. Ding SF, Han YZ, Yu J et al (2013) A fast fuzzy support vector machine based on information granulation. Neural Comput Appl 23:139–144CrossRefGoogle Scholar
  12. Ding SF, Yu JZ, Qi BJ, Huang H (2014a) An overview on twin support vector machines. Artif Intell Rev 42(2):245–252CrossRefGoogle Scholar
  13. Ding S, Huang H, Xu X, Wang J (2014b) Polynomial smooth twin support vector machines. Appl Math Inf Sci 8(4):2063–2071MathSciNetCrossRefGoogle Scholar
  14. Ding SF, Huang HJ, Yu JZ et al (2015a) Research on the hybrid models of granular computing and support vector machine. Artif Intell Rev 43(4):565–577CrossRefGoogle Scholar
  15. Ding SF, Huang HJ, Shi ZZ (2015b) Smooth CHKS twin support vector regression. J Comput Res Dev 52(3):561–568Google Scholar
  16. Ding SF, Zhang XK, Yu JZ (2016) Twin support vector machines based on fruit fly optimization algorithm. Int J Mach Learn Cybern 7(2):193–203CrossRefGoogle Scholar
  17. Feng Y, Palomar DP (2015) Normalization of linear support vector machines. IEEE Trans Signal Process 63(7):4673–4688MathSciNetzbMATHCrossRefGoogle Scholar
  18. Gu B, Sheng VS (2016) A robust regularization path algorithm for \(\nu \)-support vector classification. IEEE Trans Neural Netw Learn Syst 99:1–8Google Scholar
  19. Gu H, Niu B, Gao Z (2014) A directed acyclic graph algorithm for multi-class classification based on twin support vector machine. J Inf Comput Sci 11(18):6529–6536CrossRefGoogle Scholar
  20. Gu B, Sheng VS, Tay KY, Romano W, Li S (2015) Incremental support vector learning for ordinal regression. IEEE Trans Neural Netw Learn Syst 26(7):1403–1416MathSciNetCrossRefGoogle Scholar
  21. Guerbai Y, Chibani Y, Hadjadji B (2015) The effective use of the one-class SVM classifier for handwritten signature verification based on writer-independent parameters. Pattern Recogn 48(1):103–113CrossRefGoogle Scholar
  22. Hou M, Zhang XX, Fan LY (2014) Comparison of performance for four kinds of multi-class classifiers based on support vector machines. J Liaocheng Univ 4:1Google Scholar
  23. Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425CrossRefGoogle Scholar
  24. Hu QH, Zhang SG, Xie ZX, Mi JS, Wan J (2014) Noise model based v-support vector regression with its application to short-term wind speed forecasting. Neural Netw. 57:1–11zbMATHCrossRefGoogle Scholar
  25. Hua XP, Ding S (2015) Weighted least squares projection twin support vector machines with local information. Neurocomputing 160:228–237CrossRefGoogle Scholar
  26. Huang HJ, Ding SF, Shi ZZ (2014) Weighted smooth CHKS twin support vector machines. J Softw 24(11):2548–2557MathSciNetCrossRefGoogle Scholar
  27. Iosifidis A, Gabbouj M (2016) Multi-class support vector machine classifiers using intrinsic and penalty graphs. Pattern Recogn 55:231–246zbMATHCrossRefGoogle Scholar
  28. Jayadeva R, Khemchandni R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910CrossRefGoogle Scholar
  29. Joachims T, Finley T, Yu CNJ (2009) Cutting-plane training of structural SVMs. Kluwer Academic Publishers, DordrechtzbMATHCrossRefGoogle Scholar
  30. Ju X, Tian Y, Liu D, Qi Z (2015) Nonparallel hyperplanes support vector machine for multi-class classification. Proc Comput Sci 51(1):1574–1582CrossRefGoogle Scholar
  31. Kan LI, Huang WX, Huang ZH (2013) Multi-sensor detected object classification method based on support vector machine. J Zhejiang Univ (Eng Sci Ed) 47(1):15–22Google Scholar
  32. Khemchandani R, Pal A (2016a) Multi-category Laplacian least squares twin support vector machine. Appl Intell.  https://doi.org/10.1007/s10489-016-0770-6 CrossRefGoogle Scholar
  33. Khemchandani R, Pal A (2016b) Tree based multi-category Laplacian TWSVM for content based image retrieval. Int J Mach Learn Cybern 8:1–14Google Scholar
  34. Khemchandani R, Saigal P (2015a) Color image classification and retrieval through ternary decision structure based multi-category TWSVM. Neurocomputing 165(C):444–455CrossRefGoogle Scholar
  35. Khemchandani R, Saigal P (2015b) Color image classification and retrieval through ternary decision structure based multi-category TWSVM. Neurocomputing 165:444–455CrossRefGoogle Scholar
  36. Khemchandani R, Sharma S (2016) Robust least squares twin support vector machine for human activity recognition. Appl Soft Comput 47:33–46CrossRefGoogle Scholar
  37. Khemchandani R, Saigal P, Chandra S (2016a) Improvements on \(\nu \)-twin support vector machine. Neural Netw 79:97–107CrossRefGoogle Scholar
  38. Khemchandani R, Pal A, Chandra S (2016b) Fuzzy least squares twin support vector clustering. Neural Comput Appl 1:1–11Google Scholar
  39. Kumar MA, Gopal M (2008) Application of smoothing technique on twin support vector machines. Pattern Recogn Lett 29(13):1842–1848CrossRefGoogle Scholar
  40. Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543CrossRefGoogle Scholar
  41. Li ST, Chen CC (2015) A regularized monotonic fuzzy support vector machine model for data mining with prior knowledge. IEEE Trans Fuzzy Syst 23(5):1713–1727CrossRefGoogle Scholar
  42. Li D, Tian Y (2014) Twin support vector machine in linear programs. Proc Comput Sci 29:1770–1778CrossRefGoogle Scholar
  43. Li CN, Huang YF, Wu HJ, Shao YH, Yang ZM (2014) Multiple recursive projection twin support vector machine for multi-class classification. Int J Mach Learn Cybern.  https://doi.org/10.1007/s13042-014-0289-2 CrossRefGoogle Scholar
  44. Li W, Huang Z, Lang R, Qin H, Zhou K, Cao Y (2016) A real-time interference monitoring technique for GNSS based on a twin support vector machine method. Sensors 16(3):329CrossRefGoogle Scholar
  45. Lingras P, Butz C (2007) Rough set based 1-v-1 and 1-v-r approaches to support vector machine multi-classification. Inf Sci 177(18):3782–3798CrossRefGoogle Scholar
  46. López J, Maldonado S, Carrasco M (2016) A novel multi-class SVM model using second-order cone constraints. Appl Intell 44(2):457–469CrossRefGoogle Scholar
  47. Mangasarian OL, Wild E (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74CrossRefGoogle Scholar
  48. Mehrkanoon S, Huang X, Suykens JAK (2014) Non-parallel support vector classifiers with different loss functions. Neurocomputing 143:294–301CrossRefGoogle Scholar
  49. Nasiri JA, Charkari NM, Mozafari K (2014) Energy-based model of least squares twin support vector machines for human action recognition. Sig Process 104(6):248–257CrossRefGoogle Scholar
  50. Nasiri JA, Charkari NM, Jalili S (2015) Least squares twin multi-class classification support vector machine. Pattern Recogn 48(3):984–992zbMATHCrossRefGoogle Scholar
  51. Nie P, Li Z, Liu L (2013) Application of multi-class classification algorithm based on twin support vector machine in intrusion detection. J Comput Appl 33(2):426–429Google Scholar
  52. Pal A, Pal A (2016) Weighted linear loss twin support vector clustering. In: Ikdd conference on data science. ACM, p 18Google Scholar
  53. Peng XJ, Xu D (2012) Twin Mahalanobis distance-based support vector machines for pattern recognition. Inf Sci 200:22–37MathSciNetzbMATHCrossRefGoogle Scholar
  54. Peng X, Xu D (2013) A twin-hypersphere support vector machine classifier and the fast learning algorithm. Inf Sci 221:12–27MathSciNetzbMATHCrossRefGoogle Scholar
  55. Peng X, Kong L, Chen D (2015) Improvements on twin parametric-margin support vector machine. Neurocomputing 151(2):857–863CrossRefGoogle Scholar
  56. Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. Adv Neural Inf Process Syst 12(3):547–553Google Scholar
  57. Pouladzadeh P, Shirmohammadi S, Bakirov A et al (2015) Cloud-based SVM for food categorization. Multimed Tools Appl 74(14):5243–5260CrossRefGoogle Scholar
  58. Rastogi R, Saigal P (2017) Tree-based localized fuzzy twin support vector clustering with square loss function. Appl Intell 47:1–18CrossRefGoogle Scholar
  59. Saigal P, Khemchandani R (2015) Nonparallel hyperplane classifiers for multi-category classification. In: IEEE workshop on computational intelligence: theories, applications and future directions. IEEE, vol 2015, pp 1–6Google Scholar
  60. Shao YH, Wang Z, Chen WJ, Deng NY (2013a) A regularization for the projection twin support vector machine. Knowl Based Syst 37:203–210CrossRefGoogle Scholar
  61. Shao YH, Chen WJ, Huang WB, Yang ZM, Deng NY (2013b) The best separating decision tree twin support vector machine for multi-class classification. Proc Comput Sci 17:1032–1038CrossRefGoogle Scholar
  62. Shao YH, Chen WJ, Zhang JJ, Wang Z, Deng NY (2014) An efficient weighted Lagrangian twin support vector machine for imbalanced data classification. Pattern Recogn 47(9):3158–3167zbMATHCrossRefGoogle Scholar
  63. Shao YH, Chen WJ, Wang Z, Li CN, Deng NY (2015) Weighted linear loss twin support vector machine for large-scale classification. Knowl Based Syst 73:276–288CrossRefGoogle Scholar
  64. Song X, Wyrwicz AM (2009) Unsupervised spatiotemporal fMRI data analysis using support vector machines. NeuroImage 47(1):204–212CrossRefGoogle Scholar
  65. Suykens J, Vandewalle J (1999) Least square support vector machine classifiers. Neural Process Lett 9(3):293–300CrossRefGoogle Scholar
  66. Tanveer M (2015) Robust and sparse linear programming twin support vector machines. Cogn Comput 7(1):137–149CrossRefGoogle Scholar
  67. Tanveer M, Shubham K, Aldhaifallah M, Ho SS (2016) An efficient regularized K-nearest neighbor based weighted twin support vector regression. Knowl Based Syst 94:70–87CrossRefGoogle Scholar
  68. Tomar D, Agarwal S (2015a) A comparison on multi-class classification methods based on least squares twin support vector machine. Knowl Based Syst 81:131–147CrossRefGoogle Scholar
  69. Tomar D, Agarwal S (2015b) Twin support vector machine: a review from 2007 to 2014. Egypt Inf J 20(1):55–69CrossRefGoogle Scholar
  70. Tomar D, Agarwal S (2015c) Multiclass least squares twin support vector machine for pattern classification. Int J Database Theory Appl 8(6):285–302CrossRefGoogle Scholar
  71. Tomar D, Agarwal S (2015d) An effective weighted multi-class least squares twin support vector machine for imbalanced data classification. Int J Computat Intell Syst 8(4):761–778CrossRefGoogle Scholar
  72. Tomar D, Agarwal S (2015e) Direct acyclic graph based multi-class twin support vector machine for pattern classification. In: ACM Ikdd conference, pp 80–85Google Scholar
  73. Tomar D, Agarwal S (2016) Multi-class twin support vector machine for pattern classification. In: Proceedings of 3rd international conference on advanced computing, networking and informatics. Springer, vol 43, pp 97–110Google Scholar
  74. Vapnik VN (1998) The nature of statistical learning theory. Springer, New YorkzbMATHGoogle Scholar
  75. Wang X, Lu S, Zhai J (2008) Fast fuzzy multi-category SVM based on support vector domain description. Int J Pattern Recogn Artif Intell 22(1):109–120CrossRefGoogle Scholar
  76. Wang Z, Shao YH, Wu TR (2013) A GA-based model selection for smooth twin parametric-margin support vector machine. Pattern Recogn 46(8):2267–2277zbMATHCrossRefGoogle Scholar
  77. Wang Z, Chen J, Qin M (2010) Non-parallel planes support vector machine for multi-class classification. In: 2010 International conference on logistics systems and intelligent management, vol 1, pp 581–585Google Scholar
  78. Wu J (2012) Efficient HIK SVM learning for image classification. IEEE Trans Image Process 21(10):4442–4453MathSciNetzbMATHCrossRefGoogle Scholar
  79. Xie X, Sun S (2015) Multitask centroid twin support vector machines. Neurocomputing 149(2):1085–1091CrossRefGoogle Scholar
  80. Xie JY, Zhang BQ, Wang WZ (2011) A partial binary tree algorithm for multiclass classification based on twin support vector machines. J Nanjing Univ 47(4):354–363MathSciNetGoogle Scholar
  81. Xie J, Hone K, Xie W, Gao X, Shi Y, Liu X (2013) Extending twin support vector machine classifier for multi-category classification problems. Intell Data Anal 17(4):649–664CrossRefGoogle Scholar
  82. Xu Y, Guo R (2014) A twin hyper-sphere multi-class classification support vector machine. J Intell Fuzzy Syst 27(4):1783–1790MathSciNetzbMATHGoogle Scholar
  83. Xu Y, Guo R, Wang L (2013) A twin multi-class classification support vector machine. Cogn Comput 5(4):580–588CrossRefGoogle Scholar
  84. Yang ZX, Shao YH, Zhang XS (2013) Multiple birth support vector machine for multi-class classification. Neural Comput Appl 22(Suppl 1):S153–S161CrossRefGoogle Scholar
  85. Yang ZM, Wu HJ, Li CN, Shao YH (2016) Least squares recursive projection twin support vector machine for multi-class classification. Int J Mach Learn Cybern 7(3):411–426CrossRefGoogle Scholar
  86. Yin HT, Qiao JQ, Fu P, Xia XY (2014) Face feature selection with binary particle swarm optimization and support vector machine. Biol Reprod 5(4):731–739Google Scholar
  87. Zhang L, Zhou WD (2016) Fisher-regularized support vector machine. Inf Sci 343:79–93MathSciNetzbMATHCrossRefGoogle Scholar
  88. Zhang S, Zhao S, Sui Y, Zhang L (2015) Single Object Tracking with Fuzzy Least Squares Support Vector Machine. IEEE Trans Image Process 24(12):5723–5738MathSciNetzbMATHCrossRefGoogle Scholar
  89. Zhang C, Guo C, Liu F, Lou B (2016a) Hyperspectral imaging analysis for ripeness evaluation of strawberry with support vector machine. J Food Eng 179:11–18CrossRefGoogle Scholar
  90. Zhang XK, Ding S, Sun T (2016b) Multi-class LSTMSVM based on optimal directed acyclic graph and shuffled frog leaping algorithm. Int J Mach Learn Cybern 7(2):241–251CrossRefGoogle Scholar
  91. Zhang S, Sui Y, Zhao S et al (2017) Graph regularized structured support vector machine for object tracking. IEEE Trans Circuits Syst Video Technol 27(6):1249–1262CrossRefGoogle Scholar
  92. Zhu C, Zhao H (2016) Least square support vector machine for structural reliability analysis. Int J Comput Appl Technol 53(1):51–61CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  • Shifei Ding
    • 1
    Email author
  • Xingyu Zhao
    • 1
  • Jian Zhang
    • 1
  • Xiekai Zhang
    • 1
  • Yu Xue
    • 2
  1. 1.School of Computer Science and TechnologyChina University of Mining and TechnologyXuzhouChina
  2. 2.School of Computer and SoftwareNanjing University of Information Science and TechnologyNanjingChina

Personalised recommendations