Skip to main content
SpringerLink
Log in

Multi-Class Support Vector Machine

  • Chapter
  • First Online:
Support Vector Machines Applications

Abstract

Support vector machine (SVM) was initially designed for binary classification. To extend SVM to the multi-class scenario, a number of classification models were proposed such as the one by Crammer and Singer (J Mach Learn Res 2:265–292, 2001). However, the number of variables in Crammer and Singer’s dual problem is the product of the number of samples (l) by the number of classes (k), which produces a large computational complexity. This chapter sorts the existing classical techniques for multi-class SVM into the indirect and direct ones and further gives the comparison for them in terms of theory and experiments. Especially, this chapter exhibits a new Simplified Multi-class SVM (SimMSVM) that reduces the size of the resulting dual problem from l × k to l by introducing a relaxed classification error bound. The experimental discussion demonstrates that the SimMSVM approach can greatly speed up the training process, while maintaining a competitive classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Asuncion, A., Newman, D.: UCI machine learning repository. http://archive.ics.uci.edu/ml/datasets.html (2007)

  2. Baldi, P., Pollastri, G.: A machine-learning strategy for protein analysis. IEEE Intell. Syst. 17(2), 28–35 (2002)

    Google Scholar 

  3. Bartlett, P., Jordan, M., McAuliffe, J.: Convexity, classification, and risk bounds. J. Am. Stat. Assoc. 101, 138–156 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  4. Bredensteiner, E., Bennett, K.: Multicategory classification by support vector machines. Comput. Optim. Appl. 12, 53–79 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  5. Chawla, N.V., Japkowicz, N., Kolcz, A.: Editorial: special issue on learning from imbalanced data sets. ACM SIGKDD Explor. 6(1), 1–6 (2004)

    Article  Google Scholar 

  6. Cortes, C., Vapnik, V.: Support vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  7. Crammer, K., Singer, Y.: On the algorithmic implementation of multiclass kernel-based vector machines. J. Mach. Learn. Res. 2, 265–292 (2001)

    Google Scholar 

  8. Dietterich, T., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artif. Intell. Res. 2, 263–286 (1995)

    MATH  Google Scholar 

  9. Fung, G., Mangasarian, O.: Proximal support vector machine classifiers. In: Provost, F., Srikant, R. (eds.) Proceedings KDD-2001: Knowledge Discovery and Data Mining, August 26–29, 2001, San Francisco, CA, pp. 77–86. Asscociation for Computing Machinery, New York (2001)

    Google Scholar 

  10. Fung, G., Mangasarian, O.: Multicategory proximal support vector machine classifiers. Mach. Learn. 59(1–2), 77–97 (2005)

    Article  MATH  Google Scholar 

  11. Ganapathiraju, A., Hamaker, J., Picone, J.: Applications of support vector machines to speech recognition. IEEE Trans. Signal Process. 52(8), 2348–2355 (2004)

    Article  Google Scholar 

  12. Guermeur, Y.: Combining discriminant models with new multi-class svms. Pattern Anal. Appl. 5(2), 168–179 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  13. He, X., Wang, Z., Jin, C., Zheng, Y., Xue, X.Y.: A simplified multi-class support vector machine with reduced dual optimization. Pattern Recognit. Lett. 33, 71–82 (2012)

    Article  Google Scholar 

  14. Hsu, C., Lin, C.: A comparison of methods for multi-class support vector machines. IEEE Trans. Neural Netw. 13, 415–425 (2002)

    Article  Google Scholar 

  15. Hsu, C., Lin, C.: A simple decomposition method for support vector machines. Mach. Learn. 46, 291–314 (2002)

    Article  MATH  Google Scholar 

  16. Hsu, C., Lin, C.: Bsvm. http://mloss.org/software/view/62/ (2008)

  17. Hull, J.: A database for handwritten text recognition research. IEEE Trans. Pattern Anal. Mach. Intell. 16(5), 550–554 (1994)

    Article  Google Scholar 

  18. Khan, L., Awad, M., Thuraisingham, B.: A new intrusion detection system using support vector machines and hierarchical clustering. VLDB J. 16(4), 507–521 (2007)

    Article  Google Scholar 

  19. King, R., Feng, C., Sutherland, A.: Statlog: comparison of classification algorithms on large real-world problems. Appl. Artif. Intell. 9(3), 289–333 (1995)

    Article  Google Scholar 

  20. Knerr, S., Personnaz., L., Dreyfus, G.: Single-layer learning revisited: a stepwise procedure for building and training neural network. In: Fogelman, J. (ed.) Neurocomputing: Algorithms, Architectures and Applications. Springer, Berlin (1990)

    Google Scholar 

  21. Kreßel, U.: Pairwise classification and support vector machines. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods: Support Vector Learning, pp. 255–268. MIT Press, Cambridge (1999)

    Google Scholar 

  22. Lee, Y., Lin, Y., Wahba, G.: Multicategory support vector machines. In: Wegman, E., Braverman, A., Goodman, A., Smyth, P. (eds.) Computing Science and Statistics, vol. 33, pp. 498–512. Interface Foundation of North America, Inc., Fairfax Station, VA, USA (2002)

    Google Scholar 

  23. Lee, Y., Lin, Y., Wahba, G.: Multicategory support vector machines: theory and application to the classification of microarray data and satellite radiance data. J. Am. Stat. Assoc. 99, 67–81 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  24. Liu, Y.: Fisher consistency of multicategory support vector machines. In: Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS-07) (2007)

    Google Scholar 

  25. Mangasarian, O., Musicant, D.: Successive overrelaxation for support vector machines. IEEE Trans. Neural Netw. 10(5), 1032–1037 (1999)

    Article  Google Scholar 

  26. Mangasarian, O., Musicant, D.: Lagrangian support vector machines. J. Mach. Learn. Res. 1, 161–177 (2001)

    MATH  MathSciNet  Google Scholar 

  27. Mori, S., Suen, C., Yamamoto, K.: Historical review of OCR research and development, pp. 244–273. IEEE Computer Society Press, Los Alamitos (1995)

    Google Scholar 

  28. Platt, J., Cristianini, N., Shawe-Taylor, J.: Large margin dags for multiclass classification. In: Advances in Neural Information Processing Systems, vol. 12, pp. 547–553. MIT Press, Cambridge (2000)

    Google Scholar 

  29. Rifkin, R., Klautau, A.: In defense of one-vs-all classification. J. Mach. Learn. Res. 5, 101–141 (2004)

    MATH  MathSciNet  Google Scholar 

  30. Schölkopf, B., Platt, J., Shawe-Taylor, J., Smola, A., Williamson, R.: Estimating the support of a high-dimensional distribution. Neural Comput. 13(7), 1443–1471 (2001)

    Article  MATH  Google Scholar 

  31. Suykens, J., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)

    Article  MathSciNet  Google Scholar 

  32. Suykens, J., Vandewalle, J.: Multiclass least squares support vector machines. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN99). World Scientific, Washington, DC (1999)

    Google Scholar 

  33. Szedmak, S., Shawe-Taylor, J., Saunders, C., Hardoon, D.: Multiclass classification by l1 norm support vector machine. In: Pattern Recognition and Machine Learning in Computer Vision Workshop (2004)

    Google Scholar 

  34. Tang, Y., Zhang, Y., Chawla, N., Krasser, S.: Svms modeling for highly imbalanced classification. IEEE Trans. Syst. Man Cybern. Part B 39(1), 281–288 (2009)

    Article  Google Scholar 

  35. Tax, D., Duin, R.: Data domain description using support vectors. In: Proceedings of the European Symposium on Artificial Neural Networks, pp. 251–256 (1999)

    Google Scholar 

  36. Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)

    MATH  Google Scholar 

  37. Wang, L., Shen, X.: On l1-norm multiclass support vector machines: methodology and theory. J. Am. Stat. Assoc. 102, 583–594 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  38. Weston, J., Watkins, C.: Multi-class support vector machines. In: Proceedings of ESANN99 (1999)

    Google Scholar 

  39. Xia, X., Li, K.: A sparse multi-class least-squares support vector machine. In: IEEE International Symposium on Industrial Electronics, 2008 (ISIE 2008), pp. 1230–1235 (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai, 200237, China

    Zhe Wang

  2. School of Computer Science, Fudan University, Shanghai, 200433, China

    Xiangyang Xue

Authors
  1. Zhe Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiangyang Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhe Wang .

Editor information

Editors and Affiliations

  1. Honeywell, Golden Valley, Minnesota, USA

    Yunqian Ma

  2. West Virginia University, Morgantown, West Virginia, USA

    Guodong Guo

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Wang, Z., Xue, X. (2014). Multi-Class Support Vector Machine. In: Ma, Y., Guo, G. (eds) Support Vector Machines Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-02300-7_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-02300-7_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-02299-4

  • Online ISBN: 978-3-319-02300-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation