Margin-based Diversity Measures for Ensemble Classifiers
The classifier ensembles have been used successfully in many applications. Their superiority over single classifiers depends on the diversity of the classifiers forming the ensemble. Till now, most of the ensemble diversity measures were derived basing on the binary classification information. In this paper we propose a new group of methods, which use the margins of individual classifiers from the ensemble. These methods process the margins with a bipolar sigmoid function, as the most important information is contained in margins of low magnitude. The proposed diversity measures are evaluated for three types of ensembles of linear classifiers. The tests show that these measures are better at predicting recognition accuracy than established diversity measures, such as Q or disagreement measures, or entropy.
KeywordsDiversity Measure Feature Subset Decision Boundary Classifier Ensemble Weak Classifier
Unable to display preview. Download preview PDF.
- 3.Ho TK (1995) Random decision forests. In: Proc. of the 3rd Int’l Conference on Document Analysis and Recognition:278–282Google Scholar
- 5.Schapire RE, Freund Y, Bartlett P, Lee WS (1997) Boosting the margin: a new explanation for the effectiveness of voting methods. In: Proc. 14th International Conference on Machine Learning:322–330, Morgan KaufmannGoogle Scholar
- 8.Kuncheva L (2003) That elusive diversity in classifier ensembles. In: Proc. First Iberian Conference on Pattern Recognition and Image Analysis:1126–1138Google Scholar
- 10.Arodz T (2005) Boosting the Fisher Linear Discriminant with random feature subsets. To appear in: IV International Conference on Computer Recognition Systems, CORES 2005, Advances in Soft Computing, SpringerGoogle Scholar