Abstract
Machine learning is a very interesting and important branch of artificial intelligence. Among many learning models, the support vector machine is a popular model with high classification ability which can be trained by mathematical programming methods. Since the model was originally formulated for binary classification, various kinds of extensions have been investigated for multi-class classification. In this paper, we review some existing models, and introduce new models which we recently proposed. The models are derived from the viewpoint of multi-objective maximization of geometric margins for a discriminant function, and each model can be trained by solving a second-order cone programming problem. We show that discriminant functions with high generalization ability can be obtained by these models through some numerical experiments.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abe S (2010) Support vector machines for pattern classification. Springer, New York
Abe S, Inoue T (2002) Fuzzy support vector machines for multiclass problems. In: Proceedings of European symposium on artificial neural networks, pp 113–118
Andersen ED, Roos C, Terlaky T (2003) On implementing a primal-dual interior-point method for conic quadratic optimization. Math Program Ser B 95:249–277
Alizadeh F, Goldfarb D (2003) Second-order cone programming. Math Program Ser B 95:3–51
Allwein EL, Schapire RE, Singer Y (2000) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141
Bottou L, Cortes C, Denker J, Drucker H, Guyon I, Jackel L, LeCun Y, Muller U, Sackinger E, Simard P, Vapnik V (1994) Comparison of classifier methods: a case study in handwriting digit recognition. In: Proceedings of international conference on pattern recognition, pp 77–87
Bredensteiner EJ, Bennett KP (1999) Multicategory classification by support vector machines. Comput Optim Appl 12:53–79
Chang CC, Lin C-J (2001) LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/cjlin/libsvm
Crammer K, Singer Y (2002) On the learnability and design of output codes for multiclass problems. Mach Learn 47:201–233
Ehrgott M (2005) Multicriteria optimization, 2nd edn. Springer, Berlin
Guermeur Y (2002) Combining discriminant models with new multi-class SVMs. Pattern Anal Appl 5:168–179
Hastie T, Tibshirani R (1998) Classification by pairwise coupling. Ann Stat 26(2):451–471
Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):181–201
Hsu CW, Lin CJ (2002) A simple decomposition method for support vector machines. Mach Learn 46:291–314
Kijsirikul B, Ussivakul N (2002) Multiclass support vector machines using adaptive directed acyclic graph. In: Proceedings of international joint conference on neural network, pp 980–985
Kressel U (1999) Pairwise classification and support vector machines. In: Schölkopf B, Burges C, Smola AJ (eds) Advances in kernel methods–support vector learning. MIT Press, Cambridge, pp 255–268
Lee Y, Lin Y, Wahba G (2004) Multicategory support vector machines, theory, and application to the classification of microarray data and satellite radiance data. J Am Stat Assoc 99:67–81
Lei H, Govindaraju V (2005) Half-against-half multi-class support vector machines. Multiple classifier systems, pp 156–164
Mika S, Rätsch G, Weston J, Shölkopf B, Müller KR (1999) Fisher discriminant analysis with kernels, neural networks for signal processing. In: Proceedings of the IEEE workshop, pp 41–48
Mittelmann HD (2003) An independent benchmarking of SDP and SOCP solvers. Math Program Ser B 95:407–430
Platt JC, Cristianini N, Shawe-Taylor J (2002) Large margin DAG’s for multiclass classification. Advances in neural information processing systems, vol 12. MIT Press, Cambridge, pp 547–553
Rifkin R, Klautau A (2004) In defense of one-vs-all classification. J Mach Learn Res 5:101–141
Schölkopf B, Herbrich R, Smola AJ (2001) A generalized representer theorem. Comput Learn Theory Lecture Notes Comput Sci 2111:416–426
Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge
Tatsumi K, Kawachi R, Hayashida K, Tanino T (2009) Multiobjective multiclass soft-margin support vector machine maximizing pair-wise interclass margins. In: Proceedings 15th international conference on neural information processing of the Asia-Pacific neural network assembly 2008, Part I, LNCS, vol 5506, pp 967–974
Tatsumi K, Kawachi R, Hayashida K and Tanino T (2009) Multiobjective multiclass soft-margin support vector machine and its solving technique based on Benson’s method. In: Proceedings of the 6th international conference on modeling decisions for artificial intelligence, LNAI, vol 5861, pp 360–371
Tatsumi K, Kawachi R, Hayashida K, Tanino T (2010) Multiobjective multiclass support vector machines maximizing geometric margins. Pac J Optim 6(1):115–141
Tatsumi K, Kawachi R, Tanino T (2010) Nonlinear extension of multiobjective multiclass support vector machine. In: Proceedings in IEEE SMC, pp 1338–1343
Tatsumi K, Tai M, Tanino T (2010) Multiobjective multiclass support vector machine based on the one-against-all method. In: Proceedings in 2010 IEEE world congress on computational intelligence, pp 2844–2850
Tatsumi K, Tai M, Tanino T (2011) Nonlinear extension of multi-objective multiclass support vector machine maximizing margins based on one-against-all method. In: Proceedings of the international joint conference on neural networks 2011, pp 1571–1576
Tatsumi K, Akao M, Kawachi R, Tanino T (2011) Performance evaluation of multiobjective multiclass support vector machines maximizing geometric margins. Numer Algebra Control Optim 1:151–169
Tatsumi K, Akao M, Tanino T (2012) A soft-margin multiobjective multiclass support vector machine with unified objective functions for maximizing geometric margins and minimizing slack variables. In: Proceedings of 15th Czech-Japan seminar on data analysis and decision making under uncertainty, pp 127–134
UCI benchmark repository of artificial and real data sets, University of California Irvine. http://www.ics.uci.edu/~mlearn/databases/
Vapnik VN (1998) Statistical learning theory. A Wiley-Interscience Publication, New York
Weston J, Watkins C (1998) Multi-class support vector machines. Technical report CSD-TR-98-04, Univ. London, Royal Holloway
Author information
Authors and Affiliations
Corresponding author
Additional information
This invited paper is discussed in the comments available at doi:10.1007/s11750-014-0339-7, doi:10.1007/s11750-014-0340-1, doi:10.1007/s11750-014-0341-0.
Rights and permissions
About this article
Cite this article
Tatsumi, K., Tanino, T. Support vector machines maximizing geometric margins for multi-class classification. TOP 22, 815–840 (2014). https://doi.org/10.1007/s11750-014-0338-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11750-014-0338-8
Keywords
- Multi-class classification
- Support vector machine
- Machine learning
- Multiobjective optimization
- Second-order cone programming problem