Combined Classifier Optimisation via Feature Selection

  • David Windridge
  • Josef Kittler
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1876)


We propose a new method for performance-constraining the feature selection process as it relates to combined classifiers, and assert that the resulting technique provides an alternative to the more familiar optimisation methodology of weight adjustment. The procedure then broadly involves the prior selection of features via performan-ceconstrained sequential forward selection applied to the classifiers individually, with a subsequent forward selection process applied to the classifiers acting in combination, the selection criterion in the latter case deriving from the combined classification performance. We also provide a number of parallel investigations to indicate the performance enhancement expected of the technique, including an exhaustive weight optimisation procedure of the customary type, as well as an alternative backward selection technique applied to the individually optimised feature sets.


Feature Selection Synthetic Data Investigation Number Pattern Space Pattern Recognition Letter 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    R A Jacobs, Methods for combining experts’ probability assessments, Neural Computation, 3, pp 79–87, 1991CrossRefGoogle Scholar
  2. 2.
    J. Kittler, M. Hatef, R.P.W. Duin, and J. Matas, On combining classifiers, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 3, 1998, 226–239CrossRefGoogle Scholar
  3. 3.
    L. Lam and C.Y. Suen, Optimal combinations of pattern classifiers, Pattern Recognition Letters, vol. 16, no. 9, 1995, 945–954.CrossRefGoogle Scholar
  4. 4.
    A F R Rahman and M C Fairhurst, An evaluation of multi-expert configurations for the recognition of handwritten numerals, Pattern Recognition Letters, 31, pp 1255–1273, 1998Google Scholar
  5. 5.
    A F R Rahman and M C Fairhurst, A new hybrid approach in combining multiple experts to recognise handwritten numerals, Pattern Recognition Letters, 18, pp 781–790, 1997CrossRefGoogle Scholar
  6. 6.
    K Woods, W P Kegelmeyer and K Bowyer, Combination of multiple classifiers using local accuracy estimates, IEEE Trans. Pattern Analysis and Machine Intelligence, 19, pp 405–410, 1997CrossRefGoogle Scholar
  7. 7.
    A. Hojjatoleslami and J. Kittler, Strategies for weighted combination of classifiers employing shared and distinct representations, International Conference on Pattern Recognition, pp 338–340, 1998Google Scholar
  8. 8.
    P. A. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach, 1982, Prentice-Hall International, Inc. ISBN 0-13-654236-0.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • David Windridge
    • 1
  • Josef Kittler
    • 1
  1. 1.Centre for Vision, Speech and Signal Processing Dept. of Electronic & Electrical EngineeringUniversity of SurreyGuildfordUK

Personalised recommendations