Ensembles and Multiple Classifiers: A Game-Theoretic View
The study of multiple classifier systems is a fundamental topic in modern machine learning. However, early work on aggregation of predictors can be traced back to the Fifties, in the area of game theory. At that time, the pioneering work of James Hannan  and David Blackwell  laid down the foundations of repeated game theory. In a nutshell, a repeated game is the game-theoretic interpretation of learning. In games played once, lacking any information about the opponent, the best a player can do is to play the minimax strategy (the best strategy against the worst possible opponent). In repeated games, by examining the history of past opponent moves, the player acquires information about the opponent’s behavior and can adapt to it, in order to achieve a better payoff than that guaranteed by the minimax strategy.
Unable to display preview. Download preview PDF.
- 6.Cesa-Bianchi, N., Long, P.M., Warmuth, M.K.: Worst-case quadratic loss bounds for a generalization of the Widrow-Hoff rule. In: Proceedings of the 6th Annual ACM Workshop on Computational Learning Theory, pp. 429–438. ACM Press, New York (1993)Google Scholar
- 7.Cover, T.: Behaviour of sequential predictors of binary sequences. In: Proceedings of the 4th Prague Conference on Information Theory, Statistical Decision Functions and Random Processes, pp. 263–272. Publishing house of the Czechoslovak Academy of Sciences (1965)Google Scholar
- 9.Freund, Y., Schapire, R.: Game theory, on-line prediction and boosting. In: Proceedings of the 9th Annual Conference on Computational Learning Theory. ACM Press, New York (1996)Google Scholar
- 12.Jie, L., Orabona, F., Fornoni, M., Caputo, B., Cesa-Bianchi, N.: OM-2: An online multi-class multi-kernel learning algorithm. In: Proceedings of the 4th IEEE Online Learning for Computer Vision Workshop. IEEE Press, Los Alamitos (2007)Google Scholar
- 13.Kakade, S., Shalev-Shwartz, S., Tewari, A.: On the duality of strong convexity and strong smoothness: Learning applications and matrix regularization (2009)Google Scholar
- 14.Littlestone, N.: Learning quickly when irrelevant attributes abound: a new linear-threshold algorithm. Machine Learning 2(4), 285–318 (1988)Google Scholar
- 16.Novikoff, A.B.J.: On convergence proofs of Perceptrons. In: Proceedings of the Symposium on the Mathematical Theory of Automata vol. XII, pp. 615–622 (1962)Google Scholar
- 18.Vovk, V.G.: Aggregating strategies. In: Proceedings of the 3rd Annual Workshop on Computational Learning Theory, pp. 372–383 (1990)Google Scholar