A simple method for combining estimates to improve the overall error rates in classification

Abstract

We present a new and easy-to-implement procedure for combining $J\ge 2$ different classifiers in order to develop more effective classification rules. The method works by finding nonparametric estimates of the class conditional expectation of a new observation (that has to be classified), conditional on the vector of $J$ predicted values corresponding to the $J$ individual classifiers. Here, we propose a data-splitting method to carry out the estimation of various class conditional expectations. It turns out that, under rather minimal assumptions, the proposed combined classifier is optimal in the sense that its overall misclassification error rate is asymptotically less than (or equal to) that of any one of the individual classifiers. Simulation studies are also carried out to evaluate the proposed method. Furthermore, to make the numerical results more challenging, we also consider stable distributions (Cauchy) with rather high dimensions.