Constraints in Weighted Averaging

  • Amber Tomas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5519)


Weighted averaging of classifier outputs is used in many MCSs, yet is still not well understood. Several empirical studies have investigated the effect that non-negativity and sum-one constraints have on the error rate of weighted averaging rules, but there is little theory available to understand the results.

In this paper we study how constraints on the weights affect the location of the decision boundary of a MCS using weighted averaging. This allows us to explain many of the empirical findings, and suggest guidelines for when the application of constraints may or may not be appropriate. We also consider how these results relate to the analytical framework first proposed by Tumer and Ghosh [5].


Optimal Weight Decision Boundary Simple Average Lower Error Rate Negative Weight 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Breiman, L.: Stacked Regressions. Machine Learning 24, 49–64 (1996)zbMATHGoogle Scholar
  2. 2.
    Fumera, G., Roli, F.: Performance Analysis and Comparison of Linear Combiners for Classifier Fusion. In: Caelli, T.M., Amin, A., Duin, R.P.W., Kamel, M.S., de Ridder, D. (eds.) SPR 2002 and SSPR 2002. LNCS, vol. 2396, pp. 424–432. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  3. 3.
    Fumera, G., Roli, F.: A Theoretical and Experimental Analysis of Linear Combiners for Multiple Classifier Systems. IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 942–956 (2005)CrossRefGoogle Scholar
  4. 4.
    Le Blanc, M., Tibshirani, R.: Combining Estimates in Regression and Classification. Journal of the American Statistical Association 91, 1641–1650 (1996)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Tumer, K., Ghosh, J.: Analysis of Decision Boundaries in Linearly Combined Neural Classifiers. Pattern Recognition 29, 341–348 (1996)CrossRefGoogle Scholar
  6. 6.
    Ting, K.M., Witten, I.H.: Issues in Stacked Generalization. Journal of Artificial Intelligence Research 10, 271–289 (1999)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Amber Tomas
    • 1
  1. 1.Department of StatisticsThe University of OxfordOxfordUnited Kingdom

Personalised recommendations