Generating Classifier Outputs with Fixed Diversity for Evaluating Voting Methods

  • Héla Zouari
  • Laurent Heutte
  • Yves Lecourtier
  • Adel Alimi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3138)


Recently, it has been shown that for majority voting combination methods, (negatively) dependent classifiers may provide better performance compared to that obtained with independent classifiers. The aim of this paper is to analyse the performance of plurality voting according to classifier diversity (agreement). This analysis is conducted in parallel with majority voting in order to show which method is more efficient with dependent classifiers. For this purpose, we develop a new method for the artificial generation of classifier outputs with fixed individual accuracies and pair-wise agreement. A diversity measure is applied for building the classifier teams. The experimental results show that the plurality voting is less sensitive to the correlation between classifiers than majority voting. It is also more efficient in achieving the trade- off between the recognition rate and rejection rate than the majority voting.


  1. 1.
    Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)zbMATHMathSciNetGoogle Scholar
  2. 2.
    Duin, R.P.: The combining classifier: to train or not to train? In: Kasturi, R., Laurendeau, D., Suen, C. (eds.) ICPR16, Proceedings 16th International Conference on Pattern Recognition, Canada, vol. II, pp. 765–770. IEEE Computer Society Press, Los Alamitos (2002)Google Scholar
  3. 3.
    Fleiss, J.: Statistical methods for rates and proportions. John Wily & sons, West Sussex (1981)zbMATHGoogle Scholar
  4. 4.
    Giacinto, G., Roli, F.: Design of effective neural network ensembles for image classification processes. Image Vision and Computing Journal 19(9/10), 699–707 (2001)CrossRefGoogle Scholar
  5. 5.
    Hansen, L., Salamon, P.: Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 993–1001 (1990)CrossRefGoogle Scholar
  6. 6.
    Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3) (1998)Google Scholar
  7. 7.
    Kuncheva, L.I., Whitaker, C.J., Shipp, C.A., Duin, R.P.W.: Is independence good for combining classifiers? In: ICPRI 5, Proceedings 15th International Conference on Pattern Recognition, vol. 2, pp. 168–171 (2000)Google Scholar
  8. 8.
    Kuncheva, L.I., Kountchev, R.K.: Generating classifier outputs of fixed accuracy and diversity. Pattern Recognition Letters 23, 593–600 (2002)zbMATHCrossRefGoogle Scholar
  9. 9.
    Kuncheva, L.I., Whitaker, C.J.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning 51, 181–207 (2003)zbMATHCrossRefGoogle Scholar
  10. 10.
    Kuncheva, L.I.: That elusive diversity in classifier ensembles. In: Perales, F.J., Campilho, A.C., Pérez, N., Sanfeliu, A. (eds.) IbPRIA 2003. LNCS, vol. 2652, pp. 1126–1138. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  11. 11.
    Kuncheva, L.I., Whitaker, C.J., Duin, R.P.W.: Limits on the majority vote accuracy in classifier fusion. Pattern Analysis and Applications 6, 22–31 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Lam, L., Suen, C.Y.: Application of majority voting to pattern recognition: an analysis of its behavior and performance. IEEE Transactions on System, Man, and Cybernetics 27(5) (1997)Google Scholar
  13. 13.
    Lecce, V.D., Dimauro, G., Guerrierro, A., Impedovo, S., Pirlo, G., Salzo, A.: Classifier combination: the role of a-priori knowledge. In: Proceedings of the 7th International Workshop on Frontiers in Handwriting Recognition, pp. 143–152 (2000)Google Scholar
  14. 14.
    Lin, X., Yacoub, S., Burns, J., Simske, S.: Performance analysis of pattern combination by plurality voting. Pattern Recognition Letters 24, 1959–1969 (2003)CrossRefGoogle Scholar
  15. 15.
    Ruta, D., Gabrys, B.: New measure of classifier dependency in multiple classifier systems. In: Roli, F., Kittler, J. (eds.) MCS 2002. LNCS, vol. 2364, pp. 127–136. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  16. 16.
    Tax, D.M.J., Breukelen, M.V., Duin, R.P.W., Kittler, J.: Combining multiple classifiers by averaging or by multiplying? Pattern Recognition 33, 1475–1485 (2000)CrossRefGoogle Scholar
  17. 17.
    VanErp, M.: An overview and comparison of voting methods for pattern recognition. In: International Workshop Frontiers Handwritten Recognition, pp. 195–200 (2002)Google Scholar
  18. 18.
    Zouari, H., Heutte, L., Lecourtier, Y., Alimi, A.: Simulating classifier outputs for evaluating parallel combination method. In: Windeatt, T., Roli, F. (eds.) MCS 2003. LNCS, vol. 2709, pp. 296–305. Springer, Heidelberg (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Héla Zouari
    • 1
  • Laurent Heutte
    • 1
  • Yves Lecourtier
    • 1
  • Adel Alimi
    • 2
  1. 1.Laboratoire Perception, Systèmes, Information (PSI)Université de RouenMont-Saint-Aignan, CEDEXFrance
  2. 2.Groupe de Recherche sur les Machines Intelligentes (REGIM)Université de Sfax, Ecole National des ingénieurs, BP WSfaxTunisie

Personalised recommendations