Pattern Analysis and Applications

, Volume 7, Issue 2, pp 164–175 | Cite as

A decision based one-against-one method for multi-class support vector machine

Theoretical Advances
First Online:
Received:
Accepted:

Abstract

The support vector machine (SVM) has a high generalisation ability to solve binary classification problems, but its extension to multi-class problems is still an ongoing research issue. Among the existing multi-class SVM methods, the one-against-one method is one of the most suitable methods for practical use. This paper presents a new multi-class SVM method that can reduce the number of hyperplanes of the one-against-one method and thus it returns fewer support vectors. The proposed algorithm works as follows. While producing the boundary of a class, no more hyperplanes are constructed if the discriminating hyperplanes of neighbouring classes happen to separate the rest of the classes. We present a large number of experiments that show that the training time of the proposed method is the least among the existing multi-class SVM methods. The experimental results also show that the testing time of the proposed method is less than that of the one-against-one method because of the reduction of hyperplanes and support vectors. The proposed method can resolve unclassifiable regions and alleviate the over-fitting problem in a much better way than the one-against-one method by reducing the number of hyperplanes. We also present a direct acyclic graph SVM (DAGSVM) based testing methodology that improves the testing time of the DAGSVM method.

Keywords

Direct acyclic graph support vector machine (DAGSVM) One-against-all One-against-one Support vector machine (SVM) 
This is a preview of subscription content, log in to check access.

References

  1. 1.
    Cristanni N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge, UKGoogle Scholar
  2. 2.
    Vapnik VN (1998) Statistical learning theory. Wiley, New YorkGoogle Scholar
  3. 3.
    Kreßel UH-G (1998) Pairwise classification and support vector machines. In: Schölkopf B, Burges C, Somla A (eds) Advances in kernel methods: support vector machine. MIT Press, Cambridge, MA, pp 255–268Google Scholar
  4. 4.
    Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. In: Advances in neural information processing systems, vol 12, MIT Press, Cambridge, MA, pp 547–553Google Scholar
  5. 5.
    Hsu C-W, Lin C-J (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Networks, 13:415–425Google Scholar
  6. 6.
    Weston J, Watkins C (1998) Multi-class support vector machines. Technical Report CSD-TR-98–04, Royal Holloway, University of LondonGoogle Scholar
  7. 7.
    Crammer K, Singer Y (2000) On the learnability and design of output codes for multiclass problems. In: Proceedings of the 13th Annual Conference on Computational learning theory, 28 June–1 July 2000, pp 35–46Google Scholar
  8. 8.
    Lee Y, Lin Y, Wahba G (2001) Multicategory support vector machines. Technical Report No. 1043, University of WisconsinGoogle Scholar
  9. 9.
    Hastie T, Tibshirani R (1998) Classification by pairwise coupling. In: Jordan MI, Kearnsa MJ, Solla SA (eds) Advances in neural information processing systems, vol 10. MIT Press, Cambridge, MAGoogle Scholar
  10. 10.
    Debnath R, Takahide N, Takahashi H (2003) A fast learning decision-based SVM for multi-class problems. In: Proceedings of the International Conference on Machine learning and applications, Los Angeles, CA, USA, 23–24 June 2003, pp 128–134Google Scholar
  11. 11.
    Shawe-Taylor J, Cristianini N (1998) Data-dependent structral risk minimization for perceptron decision trees. In: Jordan MI, Kearnsa MJ, Solla SA (eds) Advances in neural information processing systems, vol 10. MIT Press, Cambridge, MA, pp 336–342Google Scholar
  12. 12.
    Freund Y, Iyer R, Schapire RE, Singer Y (1998) An efficient boosting algorithm for combining preferences. In: Proceedings of the 15th International Conference on Machine learning, 24–26 July 1998, Madison, WIGoogle Scholar
  13. 13.
    Opper M, Urbanczik R (2001) Universal learning curves of support vector machines. Phys Rev Lett 86:4410–4413CrossRefGoogle Scholar
  14. 14.
    Blake CL, Merz CJ (1998) UCI repository of machine learning databases. University of California Department Information and Computer Science, Irvine, CA. Online: available at http://www.ics.uci.edu/~mlearn/MLRepository.htmlGoogle Scholar
  15. 15.
    Tsujinishi D, Abe S (2003) Fuzzy least squares support vector machines for multiclass problems. Neural Networks 16:785–792CrossRefGoogle Scholar
  16. 16.
    Platt J (1998) Fast training of support vector machines using sequential minimal optimization. In: Schölkopf B, Burges C, Somla A (eds) Advances in kernel methods: support vector machine. MIT Press, Cambridge, MA, pp 185–208Google Scholar

Copyright information

© Springer-Verlag London Limited 2004

Authors and Affiliations

  1. 1.Department of Information and Communication EngineeringThe University of Electro-CommunicationsTokyoJapan

Personalised recommendations