Abstract
We prove new margin type bounds on the generalization error of voting classifiers that take into account the sparsity of weights and certain measures of clustering of weak classifiers in the convex combination. We also present experimental results to illustrate the behavior of the parameters of interest for several data sets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Breiman, L.: Prediction games and arcing algorithms. Neural Computation 11(7), 1493–1517 (1999)
Dudley, R.M.: Uniform Central Limit Theorems. Cambridge University Press, Cambridge (1999)
Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting. Ann. Statist. 28, 337–374 (2000)
Haussler, D.: Sphere packing numbers for subsets of the boolean n−cube with bounded Vapnik-Chervonenkis dimension. J. Combin. Theory Ser. A 69, 217–232 (1995)
Antos, A., Kégl, B., Linder, T., Lugosi, G.: Data-dependent marginbased generalization bounds for classification. Journal of Machine Learning Research (2002)
Koltchinskii, V.: Bounds on margin distributions in learning problems, Annales de l’Institut Henri Poincaré (2002) (to appear)
Koltchinskii, V., Panchenko, D.: Empirical margin distributions and bounding the generalization error of combined classifiers. Ann. Statist. 30(1) (2002)
Koltchinskii, V., Panchenko, D., Lozano, F.: Bounding the generalization error of convex combinations of classifiers: balancing the dimensionality and the margins. Ann. Appl. Probab. 13(1) (2003)
Panchenko, D.: Some extensions of an inequality of Vapnik and Chervonenkis. Elect. Comm. in Probab. 7, 55–65 (2002)
Panchenko, D.: On the generalization ability of voting classifiers (2002) (submitted)
Schapire, R., Freund, Y., Bartlett, P., Lee, W.S.: Boosting the margin: A new explanation of effectiveness of voting methods. Ann. Statist. 26, 1651–1687 (1998)
van der Vaart, A.W., Wellner, J.A.: Weak convergence and Empirical Processes. With Applications to Statistics. Springer, New York (1996)
Vapnik, V.N., Chervonenkis, A.Y.: On the uniform convergence of relative frequencies of event to their probabilities. Soviet Math. Dokl. 9, 915–918 (1968)
Vapnik, V.: Statistical Learning Theory. John Wiley & Sons, New York (1998)
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Koltchinskii, V., Panchenko, D., Andonova, S. (2003). Generalization Bounds for Voting Classifiers Based on Sparsity and Clustering. In: Schölkopf, B., Warmuth, M.K. (eds) Learning Theory and Kernel Machines. Lecture Notes in Computer Science(), vol 2777. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45167-9_36
Download citation
DOI: https://doi.org/10.1007/978-3-540-45167-9_36
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40720-1
Online ISBN: 978-3-540-45167-9
eBook Packages: Springer Book Archive