An asymptotic analysis of AdaBoost in the binary classification case
Recent work has shown that combining multiple versions of weak classifiers such as decision trees or neural networks results in reduced test set error. To study this in greater detail, we analyze the asymptotic behavior of AdaBoost type algorithms. The theoretical analysis establishes the relation between the distribution of margins of the training examples and the generated voting classification rule. The paper shows asymptotic experimental results for the binary classification case underlining the theoretical findings. Finally, the relation between the model complexity and noise in the training data, and how to improve AdaBoost type algorithms in practice are discussed.
Unable to display preview. Download preview PDF.
- C.M. Bishop. Neural Networks for Pattern Recognition. Clarendon, 1995.Google Scholar
- L. Breiman. Prediction games and arcing algorithms. Technical Report 504, Statistics Department, University of Berkeley, December 1997.Google Scholar
- Y. LeCun et al. Learning algorithms for classification: A comparism on handwritten digit recognistion. Neural Networks, pages 261–276, 1995.Google Scholar
- R. Schapire et al. Boosting the margin: A new explanation for the effectiveness of voting methods. Machine Learning, pages 148–156, 1998.Google Scholar
- H. Schwenk, Y. Bengio. Adaboosting neural networks: Application to online character recongnition. In ICANN’97, LNCS 1327, 967–972, 1997.Google Scholar
- V. N. Vapnik. The Nature of Statistical Learning Theory. Springer, 1995.Google Scholar