On a New Competence Measure Applied to the Dynamic Selection of Classifiers Ensemble

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10558)


In this paper a new method for calculating the classifier competence in the dynamic mode is developed. In the method, first decision profile of the classified object is calculated using K nearest objects from the validation set. Next, the decision profile is compared with the support vector produced by the classifier. The competence measure reflects the outcome of this comparison and rates the classifier with respect to the similarity of its support vector and decision profile of the test object in a continuous manner. Three different procedures for calculating decision profile and three different measures for comparing decision profile and support vector are proposed, which leads to nine methods of competence calculation. Two multiclassifier systems (MC) with homogeneous and heterogeneous pool of base classifiers and with dynamic ensemble selection scheme (DES) were constructed using the methods developed. The performance of constructed MC systems was compared against seven state-of-the-art MC systems using 15 benchmark data sets taken from the UCI Machine Learning Repository. The experimental investigations clearly show the effectiveness of the combined multiclassifier system in dynamic fashion with the use of the proposed measures of competence regardless of the ensemble type used.


Multiclassifier system Dynamic ensemble selection Measure of competence 



This work was supported by the statutory funds of the Department of Systems and Computer Networks, Wroclaw University of Technology.


  1. 1.
    Lichman, M.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine (2013).
  2. 2.
    Britto, A., Sabourin, R., de Oliveira, L.: Dynamic selection of classifiers - a comprehensive review. Pattern Recogn. 47(11), 3665–3680 (2014)CrossRefGoogle Scholar
  3. 3.
    Cavalin, P., Sabourin, R., Suen, C.: Dynamic selection approaches for multiple classifier systems. Neural Comput. Appl. 22(3–4), 673–688 (2013)CrossRefGoogle Scholar
  4. 4.
    Cruz, R., Sabourin, R., et al.: META-DES: a dynamic ensemble selection framework using meta-learning. Pattern Recogn. 48, 1925–1935 (2015)CrossRefGoogle Scholar
  5. 5.
    Demsar, J.: Statistical comparison of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Didaci, L., Giacinto, G., Roli, F., Marcialis, G.: A study of the performance of dynamic classifier selection based on local accuracy estimation. Pattern Recogn. 38, 2188–2191 (2005)CrossRefzbMATHGoogle Scholar
  7. 7.
    Duda, R., Hart, P., Stork, D.: Pattern Classification. Wiley-Interscience, New York (2001)zbMATHGoogle Scholar
  8. 8.
    Duin, R., Juszczak, P., et al.: PR-Tools 4.1, A Matlab Toolbox for Pattern Recognition. Delft University of Technology (2007).
  9. 9.
    Giacinto, G., Roli, F.: Methods for dynamic classifier selection. In: Proceedings of the 10th International Conference on Image Analysis and Processing, pp. 659–664 (1999)Google Scholar
  10. 10.
    Giacinto, G., Roli, F.: Dynamic classifier selection based on multiple classifier behaviour. Pattern Recogn. 34, 1879–1881 (2001)CrossRefzbMATHGoogle Scholar
  11. 11.
    Kittler, J., Hatef, M., Duin, R., Matas, J.: On combining classifier. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–20, 226–239 (1998)CrossRefGoogle Scholar
  12. 12.
    Ko, A., Sabourin, R., Britto, A.: From dynamic classifier selection to dynamic ensemble selection. Pattern Recogn. 41(5), 1718–1731 (2008)CrossRefzbMATHGoogle Scholar
  13. 13.
    Kuncheva, L.: Combining Pattern Classifiers: Methods and Algorithms. Wiley-Interscience, Hoboken (2004)CrossRefzbMATHGoogle Scholar
  14. 14.
    Kurzynski, M.: On a new competence measure applied to the combining multiclassifier system. Int. J. Sig. Process. Syst. 4(3), 185–191 (2016)MathSciNetGoogle Scholar
  15. 15.
    Kurzynski, M., Krysmann, M., Trajdos, P., Wolczowski, A.: Multiclassifier system with hybrid learning applied to control of bioprosthetic hand. Comput. Biol. Med. 69, 286–297 (2016)CrossRefGoogle Scholar
  16. 16.
    Meisel, W.: Potential functions in mathematical pattern recognition. IEEE Trans. Comput. C–18, 911–918 (1969)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Sabourin, M., Mitiche, A., Thomas, D., Nagy, G.: Classifier combination for hand-printed digit recognition. In: Proceedings of the 2nd International Conference on Document Analysis and Recognition, pp. 163–166 (1993)Google Scholar
  18. 18.
    dos Santos, E., Sabourin, R., Maupin, P.: A dynamic over produce-and-choose strategy for the selection of classifier ensembles. Pattern Recogn. 41(10), 2993–3009 (2008)CrossRefzbMATHGoogle Scholar
  19. 19.
    Smits, P.: Multiple classifier systems for supervised remote sensing image classification based on dynamics classifier selection. IEEE Trans. Geosci. Remote Sens. 40, 801–813 (2002)CrossRefGoogle Scholar
  20. 20.
    Woloszynski, T., Kurzynski, M.: On a new measure of classifier competence applied to the design of multiclassifier systems. In: Foggia, P., Sansone, C., Vento, M. (eds.) ICIAP 2009. LNCS, vol. 5716, pp. 995–1004. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-04146-4_106 CrossRefGoogle Scholar
  21. 21.
    Woloszynski, T., Kurzynski, M.: A probabilistic model of classifier competence for dynamic ensemble selection. Pattern Recogn. 44, 2656–2668 (2011)CrossRefzbMATHGoogle Scholar
  22. 22.
    Woloszynski, T., Kurzynski, M., et al.: A measure of competence based on random classification for dynamic ensemble selection. Inf. Fusion 13, 207–213 (2012)CrossRefzbMATHGoogle Scholar
  23. 23.
    Woods, K., Kegelmeyer, W., Bowyer, K.: Combination of multiple classifiers using local accuracy estimates. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–19, 405–410 (1997)CrossRefGoogle Scholar
  24. 24.
    Wolpert, D.: Stacked generalization. Neural Netw. 5, 214–259 (1992)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Systems and Computer NetworksWroclaw University of Science and TechnologyWroclawPoland

Personalised recommendations