Efficient Pairwise Classification Using Local Cross Off Strategy

  • Mohammad Ali Bagheri
  • Qigang Gao
  • Sergio Escalera
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7310)


The pairwise classification approach tends to perform better than other well-known approaches when dealing with multiclass classification problems. In the pairwise approach, however, the nuisance votes of many irrelevant classifiers may result in a wrong prediction class. To overcome this problem, a novel method, Local Crossing Off (LCO), is presented and evaluated in this paper. The proposed LCO system takes advantage of nearest neighbor classification algorithm because of its simplicity and speed, as well as the strength of other two powerful binary classifiers to discriminate between two classes. This paper provides a set of experimental results on 20 datasets using two base learners: Neural Networks and Support Vector Machines. The results show that the proposed technique not only achieves better classification accuracy, but also is computationally more efficient for tackling classification problems which have a relatively large number of target classes.


Multiclass Pairwise classification Neural Networks Support Vector Machines 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: a unifying approach for margin classifiers. Journal of Machine Learning Research 1, 113–141 (2001)MathSciNetMATHGoogle Scholar
  2. 2.
    Anand, R., Mehrotra, K., Mohan, C.K., Ranka, S.: Efficient classification for multiclass problems using modular neural networks. IEEE Transactions on Neural Networks 6(1), 117–124 (1995)CrossRefGoogle Scholar
  3. 3.
    Blake, C., Merz, C.: Uci repository of machine learning databases, department of information and computer sciences, university of california, irvine (1998)Google Scholar
  4. 4.
    Chang, C.C., Lin, C.J.: Libsvm: A library for support vector machines (2001)Google Scholar
  5. 5.
    Chang, C.C., Chien, L.J., Lee, Y.J.: A novel framework for multi-class classification via ternary smooth support vector machine. Pattern Recognition 44(6), 1235–1244 (2011)MATHCrossRefGoogle Scholar
  6. 6.
    Clark, P., Boswell, R.: Rule Induction with CN2: Some Recent Improvements. In: Kodratoff, Y. (ed.) EWSL 1991. LNCS, vol. 482, pp. 151–163. Springer, Heidelberg (1991)CrossRefGoogle Scholar
  7. 7.
    Demsar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)MathSciNetMATHGoogle Scholar
  8. 8.
    Dietterich, T., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)MATHGoogle Scholar
  9. 9.
    Escalera, S., Pujol, O., Radeva, P.: Re-coding ecocs without re-training. Pattern Recognition Letters 31, 555–562 (2010)CrossRefGoogle Scholar
  10. 10.
    Fei, B., Liu, J.: Binary tree of svm: A new fast multiclass training and cassification algorithm. IEEE Transactions on Neural Networks 17(696-704) (2006)Google Scholar
  11. 11.
    Fürnkranz, J.: Round robin classification. Journal of Machine Learning Research 2, 721–747 (2002)MATHGoogle Scholar
  12. 12.
    Garcia-Pedrajas, N., Ortiz-Boyer, D.: Improving multiclass pattern recognition by the combination of two strategies. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(6), 1001–1006 (2006)CrossRefGoogle Scholar
  13. 13.
    Garcia-Pedrajas, N., Ortiz-Boyer, D.: An empirical study of binary classifier fusion methods for multiclass classification. Information Fusion 12(2), 111–130 (2011)CrossRefGoogle Scholar
  14. 14.
    Hastie, T., Tibshirani, R.: Classification by pairwise coupling. In: Proceedings of the 1997 Conference on Advances in Neural Information Processing Systems 10, NIPS 1997, pp. 507–513. MIT Press, Cambridge (1998)Google Scholar
  15. 15.
    Hsu, C.W., Lin, C.J.: A comparison of methods for multiclass support vector machines. IEEE Transactions on Neural Networks 13(2), 415–425 (2002)CrossRefGoogle Scholar
  16. 16.
    Hullermeier, E., Vanderlooy, S.: Combining predictions in pairwise classication: An optimal adaptive voting strategy and its relation to weighted voting. Pattern Recognition 43(1), 128–142 (2010)CrossRefGoogle Scholar
  17. 17.
    Iman, R., Davenport, J.: Approximations of the critical regions of the friedman statistic. Communications in Statistics 6, 571–595 (1980)Google Scholar
  18. 18.
    Knerr, S., Personnaz, L., Dreyfus, G.: Single-layer learning revisited: a stepwise procedure for building and training a neural network. In: Fogelman, J. (ed.) Neurocomputing: Algorithms, Architectures and Applications. Springer (1990)Google Scholar
  19. 19.
    Ko, J., Byun, H.: Binary Classifier Fusion Based on the Basic Decomposition Methods. In: Windeatt, T., Roli, F. (eds.) MCS 2003. LNCS, vol. 2709, pp. 146–155. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  20. 20.
    Meyer, D., Leisch, F., Hornik, K.: The support vector machine under test. Neurocomputing 55(1-2), 169–186 (2003)CrossRefGoogle Scholar
  21. 21.
    Moreira, M., Mayoraz, E.: Improved Pairwise Coupling Classification with Correcting Classifiers. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 160–171. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  22. 22.
    Park, S.-H., Fürnkranz, J.: Efficient Pairwise Classification. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 658–665. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  23. 23.
    Platt, J.C., Cristianini, N., Shawe-taylor, J.: Large margin dags for multiclass classification. Advances in Neural Information Processing Systems 12, 547–553 (2000)Google Scholar
  24. 24.
    Wang, S.-J., Mathew, A., Chen, Y., Xi, L.-F., Ma, L., Lee, J.: Empirical analysis of support vector machine ensemble classifiers. Expert Systems with Applications 36(3, Part 2), 6466–6476 (2009)CrossRefGoogle Scholar
  25. 25.
    Windeatt, T., Ghaderi, R.: Binary labelling and decision-level fusion. Information Fusion 2(2), 103–112 (2001)CrossRefGoogle Scholar
  26. 26.
    Wu, T.F., Lin, C.J., Weng, R.C.: Probability estimates for multi-class classification by pairwise coupling. J. Mach. Learn. Res. 5, 975–1005 (2004)MathSciNetMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Mohammad Ali Bagheri
    • 1
  • Qigang Gao
    • 1
  • Sergio Escalera
    • 2
    • 3
  1. 1.Faculty of Computer ScienceDalhousie UniversityHalifaxCanada
  2. 2.Computer Vision CenterBellaterraSpain
  3. 3.Dept. Matemtica Aplicada i AnlisiUniversitat de BarcelonaBarcelonaSpain

Personalised recommendations