Abstract
Ensemble learning is a strong tool to strengthen weak classifiers. A large amount of diversity among those weak classifiers is a key to accelerate the effectiveness. Therefore, many diversity measures on a given training sample set have been proposed so far. However, they are almost all based on the oracle output that is one if the class predicted by the classifier is correct, zero otherwise. We point out such an oracle output scheme is not appropriate for the problems of more than two classes, and extend one of the most popular diversity measures, disagreement measure, to multi-class cases. On the other hand, the concept of margin has been recognized as an analytic tool to measure the generalization performance of a given classifier. Therefore, we analyze when some criteria for maximizing margins of an ensemble classifier over training samples are maximized under the assumption that the average accuracy of the base classifiers is constant. We also reveal the relationship between those criteria and the extended disagreement measure. As a result, it turns out that diversity is necessary not only over samples but also over predicted classes, if we want to extract the highest potential of ensemble.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The margin in (4) is connected to this \(m_i\) by \(margin(f, x_i, y_i) = q_{iy_i}-q_{it_i} = m_i\).
References
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Ho, T.: The random subspace method for constructing decision forests. IEEE Pattern Anal. Mach. Intell. 20(8), 832–844 (1998)
Kuncheva, L.I., Whitaker, C.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51(2), 181–207 (2003)
Schapire, R.E., Freund, Y., Bartlett, P.L., Lee, W.S.: Boosting the margin: a new explanation for the effectiveness of voting methods. Ann. Stat. 26(5), 1651–1686 (1998)
Tang, E., Suganthan, P., Yao, X.: An analysis of diversity measures. Mach. Learn. 65(1), 247–271 (2006)
Acknowledgment
This work was partly supported by JSPS KAKENHI Grant Numbers 25280079 and 15H02719.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Mikami, A., Kudo, M., Nakamura, A. (2015). Diversity Measures and Margin Criteria in Multi-class Majority Vote Ensemble. In: Schwenker, F., Roli, F., Kittler, J. (eds) Multiple Classifier Systems. MCS 2015. Lecture Notes in Computer Science(), vol 9132. Springer, Cham. https://doi.org/10.1007/978-3-319-20248-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-20248-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-20247-1
Online ISBN: 978-3-319-20248-8
eBook Packages: Computer ScienceComputer Science (R0)