Set Analysis of Coincident Errors and Its Applications for Combining Classifiers

  • Dymitr Ruta
  • Bogdan Gabrys
Part of the Combinatorial Optimization book series (COOP, volume 13)


There is a common agreement in many recent publications related to pattern recognition that dependency among classifier outputs plays a key role in combining classifiers [1]–[7]. Diversity, independence, disagreement and most recently negative dependency are the terms often used to express a desirable relation among classifiers to ensure the maximum improvement of the fusion system [4]–[7]. In this variety of concepts the idea is the same: how to measure relationship among classifiers from their outputs so that it is possible to say something about the combined classifier performance? Recent investigations indicate that error coincidences seem to be the most valuable information in this pursuit [7]–[9].


Majority Vote Venn Diagram Binary Matrix Linear Classifier Correlation Curve 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A.J.C. Sharkey, Combining Artificial Neural Nets: Ensemble and Modular Multi-net Systems, (Springer-Verlag, Berlin Heidelberg New York, 1999 ).CrossRefMATHGoogle Scholar
  2. [2]
    A.J.C. Sharkey, N.E. Sharkey, Combining Diverse Neural Nets. The Knowledge Engineering ReviewVol. 12 No. 3 (1997) pp. 231–247.CrossRefGoogle Scholar
  3. [3]
    G. Rogova, Combining the results of several neural network classifiers. Neural NetworksVol. 5 No. 5 (1994) pp. 777–781.CrossRefGoogle Scholar
  4. [4]
    D. Partridge, N. Griffith. Strategies for improving neural net generalization. Neural Computing & ApplicationsVol. 3 (1995) pp. 27–37.CrossRefMATHGoogle Scholar
  5. [5]
    L.I. Kuncheva, C.J. Whitaker, Ten Measures of Diversity in Classifier Ensembles: Limits for Two Classifiers. IEE Workshop on Intelligent Sensor Processing, Birmingham (2001), 10/1–10/6.Google Scholar
  6. [6]
    C.A. Shipp, L.I. Kuncheva, Relationships between combination methods and measures of diversity in combining classifiers. Information Fusion, accepted.Google Scholar
  7. [7]
    D. Ruta, B. Gabrys, Analysis of the Correlation Between Majority Voting Errors and the Diversity Measures in Multiple Classifier Systems. In Proceedings of the SOCO/ISFI2001 Conference, ISBN: 3–906454–27–4, Abstract page 50, Paper No. #1824–025, Paisley, UK, 2001.Google Scholar
  8. [8]
    D. Petridge, W.J. Krzanowski, Software diversity: practical statistics for its measurements and exploitation. Information & Software TechnologyVol. 39 (1997) pp. 707–717.CrossRefGoogle Scholar
  9. [9]
    B. Littlewood, D.R. Miller, Conceptual modeling of coincident failures in multiversion software. IEEE Transactions on Software Engineering Vol. 15 No. 12 (1989) pp. 1596–1614.CrossRefMathSciNetGoogle Scholar
  10. [10]
    H.-J. Kang, K. Kim, J.H. Kim, Optimal approximation of discrete probability distribution with kth-order dependency and its application to combining multiple classifiers. Pattern Recognition Letters Vol. 18 (1997) pp. 515–523.CrossRefGoogle Scholar
  11. [11]
    K. Devlin, Joy of sets: fundamentals of contemporary set theory, 2nd eds., (Springer-Verlag New York 1993 ).MATHGoogle Scholar
  12. [12]
    F. Ruskey, A Survey of Venn Diagrams. The Electronic Journal of Combinatorics eds. March 2001, (the electronic version available at: Scholar
  13. [13]
    G. Giacinto, F. Roli, An approach to the automatic design of multiple classifier systems. Pattern Recognition LettersVol. 22 (2001) pp. 25–33.CrossRefMATHGoogle Scholar
  14. [14]
    L. Lam, C.Y. Suen, Application of majority voting to pattern recognition: an analysis of its behavior and performance. IEEE Transactions on Systems, Man, and Cybernetics Vo. 27 No. 5 (1997) pp. 553–568.CrossRefGoogle Scholar
  15. [15]
    D. Ruta, B. Gabrys. A theoretical analysis of the limits of majority voting errors for multiple classifier systems. To appear in Pattern Analysis and Applications.Google Scholar
  16. [16]
    R. Battiti, A.M. Colla, Democracy in neural nets: voting schemes for classification. Neural NetworksVol. 7 No. 4 (1994) pp. 691–707.CrossRefGoogle Scholar
  17. [17]
    L.I. Kuncheva, C.J. Whitaker, C.A. Shipp, R.P.W Duin, Limits on the majority vote accuracy in classifier fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, accepted.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Dymitr Ruta
    • 1
  • Bogdan Gabrys
    • 1
  1. 1.Applied Computational Intelligence Research Unit Division of Computing and Information SystemsUniversity of PaisleyPaisleyUK

Personalised recommendations