Multi-Class SVM Classifier Based on Pairwise Coupling

  • Zeyu Li
  • Shiwei Tang
  • Shuicheng Yan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2388)


In this paper, a novel structure is proposed to extend standard support vector classifier to multi-class cases. For a K-class classification task, an array of K optimal pairwise coupling classifier (O-PWC) is constructed, each of which is the most reliable and optimal for the corresponding class in the sense of cross entropy or square error. The final decision will be produced through combining the results of these K O-PWCs. The accuracy rate is improved while the computational cost will not increase too much. Our approach is applied to two applications: handwritten digital recognition on MNIST database and face recognition on Cambridge ORL face database, experimental results reveal that our method is effective and efficient.


Support Vector Machine Face Recognition Face Image Probability Vector Probabilistic Output 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    B. Scholkopf, C. Burges, and V. Vapnik, ”Extracting support data for a given task”, In Proceedings, First International Conference on Knowledge Discovery and Data Mining. AAAI Press, Menlo Park, CA, 1995.Google Scholar
  2. 2.
    B. Scholkopf, K. Sung, C. Burges,etc, ”Comparing Support Vector Machines with Gaussian Kernels to Radial Bias Function Classifiers”, A. I. Memo No.1599 C. B. C. L. Paper No.142.Google Scholar
  3. 3.
    Christopher J. C. Burges, ”A Tutorial On Support Vector Machines for Pattern Recognition”, Data Mining and Knowledge Discovery, 2, 121–167(1998).CrossRefGoogle Scholar
  4. 4.
    Chih-Jen Lin, web site,
  5. 5.
    C.-W. Hsu and C.-J. Lin. ”A comparison of methods for multi-class support vector machines”, April 2001. To appear in IEEE Transactions on Neural Networks.Google Scholar
  6. 6.
    Erin L. Allwein, Singer. Y, ”Reducing multiclass to binary: A unifying approach to margin classifiers”, Journal of Machine Research, 1:113–141.Google Scholar
  7. 7.
    Fukunaga L., ”Introduction to Statistical Pattern Reognition”, Academic Press, INC, 2nded, 1990.Google Scholar
  8. 8.
    James Tin-Yau Kwok, ”The Evidence Framework Applied to Support Vector Machines”, IEEE Trans. On Neural Network, Vol.?, No.?, 2000.Google Scholar
  9. 9.
    K. Goh, E. Chang and T. Cheng, ”Support Vector Machine Pairwise Classifiers with Error Reduction for Image Classification”, Proceedings of ACM International Conference on Multimedia (MIR Workshop), Ottawa, October 2001.Google Scholar
  10. 10.
    M. Moreira and E. Mayoraz, ”Inproving pairwise coupling classification with error correcting classifiers”, Proceeding of Tenth Eruopean Conference on Machine Learning, April 1998.Google Scholar
  11. 11.
    Platt J., ”Probabilistic outputs for SVMs and comparisons to regularized likehood methods”, In Advances in Large Margin Classifiers. MIT Press, 1999.Google Scholar
  12. 12.
    Platt J, Cristianini N, Shawe-Taylor J. ”Large margin DAGs for multiclass classification”, In: Advances in Neural Information Processing Systems, Vol. 12, pp547–553. MIT Press, 2000.Google Scholar
  13. 13.
    T. Dietterich and G. Bakiri. ”Solving multiclass learning problems via error-correcting output codes”, Journal of Artifical Intelligence Research, 2, 1995.Google Scholar
  14. 14.
    Trevir Hastie, Robert Tibshirani, ”Classification by pairwise coupling”, Technical report, stanford Univ and Univ. of Toronto, 1996. in Proceeding of NIPS*97.Google Scholar
  15. 15.
    U. Krebel. ”Pairwise classification and support vector machines”, In ”Advance in Kernel Methods — Support Vector Learning”, pages 255–268, Cambridge, MA, 1999. MIT Press.Google Scholar
  16. 16.
    Vapnik V., ”The Nature of Statistical Learning Theory”, New York: Springer-Verlag, 1995.zbMATHGoogle Scholar
  17. 17.
    Vapnik V., ”Statistical Learning Theory”, Wiley, 1998.Google Scholar
  18. 18.
    Volker Rothm, Koji Tsuda, ”Pairwise Coupling for Machine Recognition of Hand-Printed Japanese Characters”, Accepted for CVPR01.Google Scholar
  19. 19.
    Weston, J. and Watkins, C., Multi-class support vector machines. Technical Report CSD-TR-98-04, Dept. of Computer Science, Royal Holloway, University of London, Egham, Surrey TW20 0EX, England, 1998.Google Scholar
  20. 20.

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Zeyu Li
    • 1
  • Shiwei Tang
    • 1
  • Shuicheng Yan
    • 2
  1. 1.National Laboratory on Machine Perception and Center for Information SciencePeking UniversityBeijingP.R. China
  2. 2.Dept. of Info. science, School of Math. SciencesPeking UniversityBeijing

Personalised recommendations