Ensemble Learning

Introduction

The subject of this chapter is ensemble learning in which our system is characterized by an ensemble of M models. The models may share the same common representational format or each model may have its own distinct common representational format. To make our discussion more concrete we shall concentrate on the (supervised) classification of an object O using a multiple classifier system (MCS). Given an unknown object O, our goal is to optimally assign it to one of K classes, c _k ,k ∈ {1,2,…,K}, using an ensemble of M (supervised) classifiers, S _m ,m ∈ {1,2,…,M}. The theory of multiple classifier systems suggests that if the pattern of errors made by one classifier, S _m , is different from the pattern of errors made by another classifier, S _n , then we may exploit this difference to give a more accurate and more reliable classification of O. If the error rates of the classifiers are less than $\frac{1}{2}$, then the MCS error rate, $E_{\text{MCS}}$, should decrease with the number of classifiers, M, and with the mean diversity, $\bar{\sigma}$, between the classifiers.