Application of the Evolutionary Algorithms for Classifier Selection in Multiple Classifier Systems with Majority Voting

  • Dymitr Ruta
  • Bogdan Gabrys
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2096)

Abstract

In many pattern recognition tasks, an approach based on combining classifiers has shown a significant potential gain in comparison to the performance of an individual best classifier. This improvement turned out to be subject to a sufficient level of diversity exhibited among classifiers, which in general can be assumed as a selective property of classifier subsets. Given a large number of classifiers, an intelligent classifier selection process becomes a crucial issue of multiple classifier system design. In this paper, we have investigated three evolutionary optimization methods for the classifier selection task. Based on our previous studies of various diversity measures and their correlation with majority voting error we have adopted majority voting performance computed for the validation set directly as a fitness function guiding the search. To prevent from training data overfitting we extracted a population of best unique classifier combinations, and used them for second stage majority voting. In this work we intend to show empirically, that using efficient evolutionary-based selection leads to the results comparable to absolutely best, found exhaustively. Moreover, as we showed for selected datasets, introducing a second stage combining by majority voting has the potential for both, further improvement of the recognition rate and increase of the reliability of combined outputs.

References

  1. 1.
    Bezdek J.C.: Fuzzy Models and Algorithms for Pattern Recognition and Image Processing. Kluwer Academic Boston (1999)MATHGoogle Scholar
  2. 2.
    Sharkey A.J.C.: Combining Artificial Neural Nets: Ensemble and Modular Multi-net Systems. Springer-Verlag, Berlin Heidelberg New York (1999)MATHGoogle Scholar
  3. 3.
    Zhilkin P.A., Somorjai R.L.: Application of Several Methods of Classification Fusion to Magnetic Resonance Spectra. Connection Science 8(3,4) (1996) 427–442CrossRefGoogle Scholar
  4. 4.
    Rogova G.: Combining the Results of Several Neural Network Classifiers. Neural Networks 7(5) (1994) 777–781CrossRefGoogle Scholar
  5. 5.
    Xu L., Krzyzak A.: Methods of Combining Multiple Classifiers and Their Applications to Handwriting Recognition. IEEE Transactions on Systems, Man, and Cybernetics 23(8) (1992) 418–434CrossRefGoogle Scholar
  6. 6.
    Partridge D., Griffith N.: Strategies for Improving Neural Net Generalization. Neural Computing and Applications 3 (1995) 27–37MATHCrossRefGoogle Scholar
  7. 7.
    Sharkey A.J.C., Sharkey N.E.: Combining Diverse Neural Nets. The Knowledge Engineering Review 12(3) (1997) 231–247CrossRefGoogle Scholar
  8. 8.
    Kuncheva L.I., Whitaker C.J., Shipp C.A., Duin R.P.W.: Limits on the Majority Vote Accuracy in Classifier Fusion. Submitted to IEEE Transactions on Pattern Analysis and Machine IntelligenceGoogle Scholar
  9. 9.
    Ruta D., Gabrys B.: A Theoretical Analysis of the Limits of Majority Voting in Multiple Classifier Systems. Technical Report No. 11. University of Paisley (2000)Google Scholar
  10. 10.
    Kuncheva L.I., Whitaker C.J.: Measures of Diversity in Classifier Ensembles. Submitted to Machine LearningGoogle Scholar
  11. 11.
    Ruta D., Gabrys B.: Analysis of the Correlation Between Majority Voting Errors and the Diversity Measures in Multiple Classifier Systems. Accepted for the International Symposium on Soft Computing SOCO’2001Google Scholar
  12. 12.
    Kuncheva L., Jain L.C.: Designing Classifier Fusion Systems by Genetic Algorithms. To appear in IEEE Transactions on Evolutionary ComputationGoogle Scholar
  13. 13.
    Cho S.B.: Pattern Recognition With Neural Networks Combined by Genetic Algorithms. Fuzzy Sets and Systems 103 (1999) 339–347CrossRefGoogle Scholar
  14. 14.
    Cho S.B., Kim J.H.: Combining Multiple Neural Networks by Fuzzy Integral for Robust Classification. IEEE Trans. on Systems, Man, and Cybernetics 25(2) (1995) 380–384CrossRefGoogle Scholar
  15. 15.
    Kuncheva L.I., Bezdek J.C.: On Combining Classifiers by Fuzzy Templates. Proc. NAFIPS’98, Pensacola, FL (1998) 193–197Google Scholar
  16. 16.
    Davis L.: Handbook of Genetic Algorithms. Van Nostrand Reinhold New York (1991)Google Scholar
  17. 17.
    Glover F., Laguna M.: Tabu Search. Kluver Academic Publishers Boston (1997)MATHGoogle Scholar
  18. 18.
    Baluja S.: Population-Based Incremental Learning: A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning. Technical Report No. 163. Carnegie Melon University, Pittsburgh PA (1994)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Dymitr Ruta
    • 1
  • Bogdan Gabrys
    • 1
  1. 1.Applied Computational Intelligence Research Unit, Division of Computing and Information SystemsUniversity of PaisleyPaisleyUK

Personalised recommendations