Pattern Analysis & Applications

, Volume 5, Issue 2, pp 189–200

Robust Combining of Disparate Classifiers through Order Statistics

  • Kagan Tumer
  • Joydeep Ghosh

DOI: 10.1007/s100440200017

Cite this article as:
Tumer, K. & Ghosh, J. Pattern Anal Appl (2002) 5: 189. doi:10.1007/s100440200017


Integrating the outputs of multiple classifiers via combiners or meta-learners has led to substantial improvements in several difficult pattern recognition problems. In this article, we investigate a family of combiners based on order statistics, for robust handling of situations where there are large discrepancies in performance of individual classifiers. Based on a mathematical modelling of how the decision boundaries are affected by order statistic combiners, we derive expressions for the reductions in error expected when simple output combination methods based on the median, the maximum and in general, the ith order statistic, are used. Furthermore, we analyse the trim and spread combiners, both based on linear combinations of the ordered classifier outputs, and show that in the presence of uneven classifier performance, they often provide substantial gains over both linear and simple order statistics combiners. Experimental results on both real world data and standard public domain data sets corroborate these findings.

Key words: Classification error; Ensembles; Order statistics; Robust statistics; Trimmed means

Copyright information

© Springer-Verlag London Limited 2002

Authors and Affiliations

  • Kagan Tumer
    • 1
  • Joydeep Ghosh
    • 2
  1. 1.Computational Sciences Division, NASA Ames Research Center, Moffett Field, CA, USAUS
  2. 2.Department of Electrical and Computer Engineering, The University of Texas, Austin, TX, USAUS