Information Analysis of Multiple Classifier Fusion?

  • Jiří Grim
  • Josef Kittler
  • Pavel Pudil
  • Petr Somol
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2096)


We consider a general scheme of parallel classifier combinations in the framework of statistical pattern recognition. Each statistical classifier defines a set of output variables in terms of a posteriori probabilities, i.e. it is used as a feature extractor. Unlike usual combining schemes the output vectors of classifiers are combined in parallel. The statistical Shannon information is used as a criterion to compare different combining schemes from the point of view of the theoretically available decision information. By means of relatively simple arguments we derive a theoretical hierarchy between different schemes of classifier fusion in terms of information inequalities.


Information Loss Information Analysis Probabilistic Neural Network Posteriori Probability Statistical Pattern Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fukunaga K., Ando S.: The optimum nonlinear features for a scatter criterion and discriminant analysis. In Proc. ICPR 1976, pp 50–54.Google Scholar
  2. 2.
    Grim J.: Design of multilayer neural networks by information preserving transforms. In E. Pessa M.P. Penna A. Montesanto (Eds.), Proceedings of the Third European Congress on System Science, Roma: Edizzioni Kappa, 1996, pp 977–982.Google Scholar
  3. 3.
    Grim J., Kittler J., Pudil P., Somol P.: Combining multiple classifiers in probabilistic neural networks. In Multiple Classifier Systems, Kittler J., Roli F., (Eds.), Springer, 2000, pp 157–166.Google Scholar
  4. 4.
    Grim J., Pudil P.: On virtually binary nature of probabilistic neural networks. In: Advances in Pattern Recognition, Sydney, August 11–13, 1998), A. Amin, D. Dori, P. Pudil, H. Freeman (Eds.), pp. 765–774, Springer: New York, Berlin, 1998.CrossRefGoogle Scholar
  5. 5.
    Grim J., Pudil P., Somol P.: Recognition of handwritten numerals by structural probabilistic neural networks. In Proceedings of the Second ICSC Symposium on Neural Computation, Berlin, 2000. (Bothe H., Rojas R. eds.). ICSC, Wetaskiwin, 2000, pp 528–534.Google Scholar
  6. 6.
    Hansen L.K., Salamon P.: Neural network ensembles, IEEE Transactions on Pattern Analysis and Machine Intelligence 1990; 12(10):993–1001.CrossRefGoogle Scholar
  7. 7.
    Huang Y.S., Suen C.Y.: A method of combining multiple classifiers–a neural network approach. In Proc. 12th IAPR Int. Conf. on Pattern Recognition and Computer Vision, Los Alamitos: IEEE Comp. Soc. Press., 1994, pp 473–475.Google Scholar
  8. 8.
    Huang Y.S., Suen C.Y.: Combination of multiple experts for the recognition of unconstrained handwritten numerals. IEEE Transactions on Pattern Analysis and Machine Intelligence 1995; 17(1):90–94.CrossRefGoogle Scholar
  9. 9.
    Cho S.B., Kim J.H.: Combining multiple neural networks by fuzzy integral for robust classification, IEEE Transactions on Systems Man and Cybernetics 1995; 25(2):380–384.CrossRefGoogle Scholar
  10. 10.
    Cho S.B., Kim J.H.: Multiple network fusion using fuzzy logic, IEEE Transactions on Neural Networks 1995; 6(2):497–501.CrossRefGoogle Scholar
  11. 11.
    Chuanyi Ji, Sheng Ma.: Combinations of weak classifiers. IEEE Transactions on Neural Networks, 8(1): 32–42.Google Scholar
  12. 12.
    Kittler J.: Combining classifiers: A theoretical framework. Pattern Analysis and Applications 1998; 1:18–27.CrossRefGoogle Scholar
  13. 13.
    Kittler J., Duin R.P.W., Hatef M., Matas J.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998; 20(3):226–239.CrossRefGoogle Scholar
  14. 14.
    Lam L., Suen C.Y.: A theoretical analysis of the application of majority voting to pattern recognition. In Proc. 12th IAPR Int. Conf. on Pattern Recognition and Computer Vision, Los Alamitos: IEEE Comp. Soc. Press., 1994, pp 418–420.Google Scholar
  15. 15.
    Rogova G.: Combining the results of several neural network classifiers, Neural Networks 1994; 7(5):777–781.CrossRefGoogle Scholar
  16. 16.
    Soulie F.F., Vinnet E., Lamy B.: Multi-Modular Neural Network Architectures: Applications in Optical Character and Human Face Recognition, Int. Journal of Pattern Recognition and Artificial Intelligence, Vol. 5,No. 4, pp 721–755, 1993.CrossRefGoogle Scholar
  17. 17.
    Vajda I., Grim J.: About the maximum information and maximum likelihood principles in neural networks. Kybernetika 1998; 34(4):485–494.Google Scholar
  18. 18.
    Wolpert D.H.: Stacked generalization. Neural Networks, Vol. 5,No. 2, pp 241–260, 1992.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Jiří Grim
    • 1
  • Josef Kittler
    • 2
  • Pavel Pudil
    • 1
  • Petr Somol
    • 1
  1. 1.Institute of Information Theory and AutomationPrague 8Czech Republic
  2. 2.School of Electronic Engineering, Information Technology and MathematicsUniversity of SurreyGuildfordUK

Personalised recommendations