Abstract
In Dynamic Ensemble Selection (DES), only the most competent classifiers are selected to classify a given query sample. A crucial issue faced in DES is the definition of a criterion for measuring the level of competence of each base classifier. To that end, a criterion commonly used is the estimation of the competence of a base classifier using its local accuracy in small regions of the feature space surrounding the query instance. However, such a criterion cannot achieve results close to the performance of the Oracle, which is the upper limit performance of any DES technique. In this paper, we conduct a dissimilarity analysis between various DES techniques in order to better understand the relationship between them and as well as the behavior of the Oracle. In our experimental study, we evaluate seven DES techniques and the Oracle using eleven public datasets. One of the seven DES techniques was proposed by the authors and uses meta-learning to define the competence of base classifiers based on different criteria. In the dissimilarity analysis, this proposed technique appears closer to the Oracle when compared to others, which would seem to indicate that using different bits of information on the behavior of base classifiers is important for improving the precision of DES techniques. Furthermore, DES techniques, such as LCA, OLA, and MLA, which use similar criteria to define the level of competence of base classifiers, are more likely to produce similar results.
Chapter PDF
Similar content being viewed by others
References
Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 226–239 (1998)
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley-Interscience (2004)
Cruz, R.M.O., Cavalcanti, G.D.C., Ren, T.I.: Handwritten digit recognition using multiple feature extraction techniques and classifier ensemble. In: 17th International Conference on Systems, Signals and Image Processing, pp. 215–218 (2010)
de Souza Britto Jr., A., Sabourin, R., Oliveira, L.: Dynamic selection of classifiers a comprehensive review. Pattern Recognition (in press, 2014), http://dx.doi.org/10.1016/j.patcog.2014.05.003
Ko, A.H.R., Sabourin, R., Britto Jr., A.S.: From dynamic classifier selection to dynamic ensemble selection. Pattern Recognition 41, 1735–1748 (2008)
Woods, K., Kegelmeyer Jr., W.P., Bowyer, K.: Combination of multiple classifiers using local accuracy estimates. IEEE Transactions on Pattern Analysis Machine Intelligence 19, 405–410 (1997)
Cruz, R.M.O., Cavalcanti, G.D.C., Ren, T.I.: A method for dynamic ensemble selection based on a filter and an adaptive distance to improve the quality of the regions of competence. In: International Joint Conference on Neural Networks, pp. 1126–1133 (2011)
Smits, P.C.: Multiple classifier systems for supervised remote sensing image classification based on dynamic classifier selection. IEEE Transactions on Geoscience and Remote Sensing 40(4), 801–813 (2002)
Cruz, R.M.O., Sabourin, R., Cavalcanti, G.D.: On meta-learning for dynamic ensemble selection. In: International Conference on Pattern Recognition (in press, 2014)
Pękalska, E., Duin, R.P.W., Skurichina, M.: A discussion on the classifier projection space for classifier combining. In: Roli, F., Kittler, J. (eds.) MCS 2002. LNCS, vol. 2364, pp. 137–148. Springer, Heidelberg (2002)
Cavalin, P.R., Sabourin, R., Suen, C.Y.: Dynamic selection approaches for multiple classifier systems. Neural Computing and Applications 22(3-4), 673–688 (2013)
Giacinto, G., Roli, F.: Dynamic classifier selection based on multiple classifier behaviour. Pattern Recognition 34, 1879–1881 (2001)
Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)
Cox, T.F., Cox, M.A.A.: Multidimensional Scaling, 2nd edn. Chapman and Hall (2000)
Cruz, R.M.O., Cavalcanti, G.D., Tsang, I.R., Sabourin, R.: Feature representation selection based on classifier projection space and oracle analysis. Expert Systems with Applications 40(9), 3813–3827 (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Cruz, R.M.O., Sabourin, R., Cavalcanti, G.D.C. (2014). Analyzing Dynamic Ensemble Selection Techniques Using Dissimilarity Analysis. In: El Gayar, N., Schwenker, F., Suen, C. (eds) Artificial Neural Networks in Pattern Recognition. ANNPR 2014. Lecture Notes in Computer Science(), vol 8774. Springer, Cham. https://doi.org/10.1007/978-3-319-11656-3_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-11656-3_6
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11655-6
Online ISBN: 978-3-319-11656-3
eBook Packages: Computer ScienceComputer Science (R0)