Probabilistic Approach to the Dynamic Ensemble Selection Using Measures of Competence and Diversity of Base Classifiers

  • Rafal Lysiak
  • Marek Kurzynski
  • Tomasz Woloszynski
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6679)


In the paper measures of classifier competence and diversity using a probabilistic model are proposed. The multiple classifier system (MCS) based on dynamic ensemble selection scheme was constructed using both measures developed. The performance of proposed MCS was compared against three multiple classifier systems using six databases taken from the UCI Machine Learning Repository and the StatLib statistical dataset. The experimental results clearly show the effectiveness of the proposed dynamic selection methods regardless of the ensemble type used (homogeneous or heterogeneous).


Dynamic ensemble selection Classifier competence Diversity measure 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aksela, M.: Comparison of classifier selection methods for improving committee performance. In: Windeatt, T., Roli, F. (eds.) MCS 2003. LNCS, vol. 2709, pp. 84–93. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  2. 2.
    Aksela, M., Laaksonen, J.: Using diversity of errors for selecting members of a committee classifier. Pattern Recognition 39, 608–623 (2006)CrossRefzbMATHGoogle Scholar
  3. 3.
    Berger, J.O.: Statistical Decision Theory and Bayesian Analysis. Springer, New York (1987)Google Scholar
  4. 4.
    Brown, G., Wyatt, J., Harris, R., Yao, X.: Diversity creation methods: a survey and categorisation. Information Fusion 6, 5–20 (2005)CrossRefGoogle Scholar
  5. 5.
    Canuto, A., Abreu, M., et al.: Investigating the influence of the choice of the ensemble members in accuracy and diversity of selection-based and fusion-based methods for ensemble. Pattern Recognition Letters 28, 472–486 (2007)CrossRefGoogle Scholar
  6. 6.
    Didaci, L., Giacinto, G., Roli, F., Marcialis, G.: A study on the performances of dynamic classifier selection based on local accuracy estimation. Pattern Recognition 38, 2188–2191 (2005)CrossRefzbMATHGoogle Scholar
  7. 7.
    Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation 10, 1895–1923 (1998)CrossRefGoogle Scholar
  8. 8.
    Duda, R., Hart, P., Stork, G.: Pattern Classification. John Wiley and Sons, New York (2000)zbMATHGoogle Scholar
  9. 9.
    Duin, R., Juszczak, P., Paclik, P., et al.: PRTools4. In: A Matlab Toolbox for Pattern Recognition, Delft University of Technology (2007)Google Scholar
  10. 10.
    Eulanda, M., Santos, D., Sabourin, R., Maupin, P.: A dynamic overproduce-and-choose strategy for the selection of classifier ensembles. Pattern Recognition 41, 2993–3009 (2008)CrossRefzbMATHGoogle Scholar
  11. 11.
    Giacinto, G., Roli, F.: Dynamic classifier selection based on multiple classifier behaviour. Pattern Recognition 34, 1879–1881 (2001)CrossRefzbMATHGoogle Scholar
  12. 12.
    Huenupan, F., Yoma, N., et al.: Confidence based multiple classifier fusion in speaker verification. Pattern Recognition Letters 29, 957–966 (2008)CrossRefGoogle Scholar
  13. 13.
    Kanal, L.: Patterns in pattern recognition. IEEE Trans. Information Theory 20, 697–722 (1974)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Ko, A., Sabourin, R., Britto, A.: From dynamic classifier selection to dynamic ensemble selection. Pattern Recognition 41, 1718–1733 (2008)CrossRefzbMATHGoogle Scholar
  15. 15.
    Kuncheva, I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley Interscience, Hoboken (2004)CrossRefzbMATHGoogle Scholar
  16. 16.
    Smits, P.: Multiple classifier systems for supervised remote sensing image classification based on dynamic classifier selection. IEEE Trans. on Geoscience and Remote Sensing 40, 717–725 (2002)CrossRefGoogle Scholar
  17. 17.
    Woloszynski, M., Kurzynski, M.: A measure of competence based on randomized reference classifier for dynamic ensemble selection. In: 20th Int. Conf. on Pattern Recognition, pp. 4194–4197. IEEE Computer Press, Istanbul (2010)Google Scholar
  18. 18.
  19. 19.
    Woods, K., Kegelmeyer, W., Bowyer, W.: Combination of multiple classifiers using local accuracy estimates. IEEE Trans. on Pattern Analysis and Machine Learning 19, 405–410 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Rafal Lysiak
    • 1
  • Marek Kurzynski
    • 1
  • Tomasz Woloszynski
    • 1
  1. 1.Dept. of Systems and Computer NetworksWroclaw University of TechnologyWroclawPoland

Personalised recommendations