Advertisement

A Game-Theoretic Approach to Weighted Majority Voting for Combining SVM Classifiers

  • Harris Georgiou
  • Michael Mavroforakis
  • Sergios Theodoridis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4131)

Abstract

A new approach from the game-theoretic point of view is proposed for the problem of optimally combining classifiers in dichotomous choice situations. The analysis of weighted majority voting under the viewpoint of coalition gaming, leads to the existence of analytical solutions to optimal weights for the classifiers based on their prior competencies. The general framework of weighted majority rules (WMR) is tested against common rank-based and simple majority models, as well as two soft-output averaging rules. Experimental results with combined support vector machine (SVM) classifiers on benchmark classification tasks have proven that WMR, employing the theoretically optimal solution for combination weights proposed in this work, outperformed all the other rank-based, simple majority and soft-output averaging methods. It also provides a very generic and theoretically well-defined framework for all hard-output (voting) combination schemes between any type of classifier architecture.

Keywords

Support Vector Machine Support Vector Machine Classifier Combination Rule Simple Majority Rule Optimal Decision Rule 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    de, C.M.: An Essay on the Application of Probability Theory to Plurality Decision Making: An Election Between Three Candidates. In: Sommerlad and Mclean, pp. 66-80 (1989)Google Scholar
  2. 2.
    Owen, G.: Game Theory, 3rd edn. Academic Press, San Diego (1995)Google Scholar
  3. 3.
    Nitzan, S., Paroush, J.: Optimal Decision Rules in Uncertain Dichotomous Choice Situations. Int. Econ. Rev. 23, 289–297 (1982)MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Shapley, L., Grofman, B.: Optimizing Group Judgemental Accuracy in the Presence of Independence. Public Choice 43, 329–343 (1984)CrossRefGoogle Scholar
  5. 5.
    Littlestone, N., Warmuth, M.K.: The Weighted Majority Algorithm. Information and Computation 108, 212–261 (1994)MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Ben-Yashar, R., Nitzan, S.: The Optimal Decision Rule for Fixed-Size Committees in Dichotomous Choice Situations: The General Result. International Economic Review 38, 175–186 (1997)MATHCrossRefGoogle Scholar
  7. 7.
    Karotkin, D.: The Network of Weighted Majority Rules and Weighted Majority Games. Games and Econ. Beh. 22, 299–315 (1998)MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Kuncheva, L.I.: Combining Pattern Classifiers. John Wiley and Sons, New Jersey (2004)MATHCrossRefGoogle Scholar
  9. 9.
    Mavroforakis, M., Theodoridis, S.: Support Vector Machine Classification through Geometry. In: Proc. XII Eur. Sig. Proc. Conf (EUSIPCO 2005), Antalya, Turkey (September 2005)Google Scholar
  10. 10.
    Mavroforakis, M., Theodoridis, S.: A Geometric Approach to Support Vector Machine (SVM) Classification. IEEE Trans. NN 2006 (in press)Google Scholar
  11. 11.
    Evgeniou, T., Pontil, M., Elisseeff, A.: Leave One Out Error, Stability, and Generalization of Voting Combinations of Classifiers. Machine Learning 55, 71–97 (2004)MATHCrossRefGoogle Scholar
  12. 12.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press, UK (2000)Google Scholar
  13. 13.
    Jain, A., Duin, R., Mao, J.: Statistical pattern recognition: a review. IEEE Trans. PAMI 22, 4–37 (2000)Google Scholar
  14. 14.
    Mavroforakis, M., Sdralis, M., Theodoridis, S.: A novel SVM Geometric Algorithm based on Reduced Convex Hulls. In: 18th International Conference on Pattern Recognition (ICPR 2006), Hong Kong (August 2006)Google Scholar
  15. 15.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 4th edn. Academic Press, San Diego (2006)MATHGoogle Scholar
  16. 16.
    Cooley, W., Lohnes, P.: Multivariate data analysis. John Willey & Sons, New York (1971)MATHGoogle Scholar
  17. 17.
    Parker, J.R.: Rank and Response Combination from Confusion Matrix Data. Information Fusion 2, 113–120 (2001)CrossRefGoogle Scholar
  18. 18.
    Forsythe, G., Malcom, M., Moler, C.: Computer Methods for Mathematical Computations. Prentice-Hall, New Jersey (1977)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Harris Georgiou
    • 1
  • Michael Mavroforakis
    • 1
  • Sergios Theodoridis
    • 1
  1. 1.Dept. of Informatics and Telecommunications, Division of Communications and, Signal ProcessingUniversity of Athens, GreecePanepistimioupolis, Ilissia, AthensGreece

Personalised recommendations