A Game-Theoretic Approach to Weighted Majority Voting for Combining SVM Classifiers
A new approach from the game-theoretic point of view is proposed for the problem of optimally combining classifiers in dichotomous choice situations. The analysis of weighted majority voting under the viewpoint of coalition gaming, leads to the existence of analytical solutions to optimal weights for the classifiers based on their prior competencies. The general framework of weighted majority rules (WMR) is tested against common rank-based and simple majority models, as well as two soft-output averaging rules. Experimental results with combined support vector machine (SVM) classifiers on benchmark classification tasks have proven that WMR, employing the theoretically optimal solution for combination weights proposed in this work, outperformed all the other rank-based, simple majority and soft-output averaging methods. It also provides a very generic and theoretically well-defined framework for all hard-output (voting) combination schemes between any type of classifier architecture.
KeywordsSupport Vector Machine Support Vector Machine Classifier Combination Rule Simple Majority Rule Optimal Decision Rule
Unable to display preview. Download preview PDF.
- 1.de, C.M.: An Essay on the Application of Probability Theory to Plurality Decision Making: An Election Between Three Candidates. In: Sommerlad and Mclean, pp. 66-80 (1989)Google Scholar
- 2.Owen, G.: Game Theory, 3rd edn. Academic Press, San Diego (1995)Google Scholar
- 9.Mavroforakis, M., Theodoridis, S.: Support Vector Machine Classification through Geometry. In: Proc. XII Eur. Sig. Proc. Conf (EUSIPCO 2005), Antalya, Turkey (September 2005)Google Scholar
- 10.Mavroforakis, M., Theodoridis, S.: A Geometric Approach to Support Vector Machine (SVM) Classification. IEEE Trans. NN 2006 (in press)Google Scholar
- 12.Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press, UK (2000)Google Scholar
- 13.Jain, A., Duin, R., Mao, J.: Statistical pattern recognition: a review. IEEE Trans. PAMI 22, 4–37 (2000)Google Scholar
- 14.Mavroforakis, M., Sdralis, M., Theodoridis, S.: A novel SVM Geometric Algorithm based on Reduced Convex Hulls. In: 18th International Conference on Pattern Recognition (ICPR 2006), Hong Kong (August 2006)Google Scholar