Abstract
Weighted averaging of classifier outputs is used in many MCSs, yet is still not well understood. Several empirical studies have investigated the effect that non-negativity and sum-one constraints have on the error rate of weighted averaging rules, but there is little theory available to understand the results.
In this paper we study how constraints on the weights affect the location of the decision boundary of a MCS using weighted averaging. This allows us to explain many of the empirical findings, and suggest guidelines for when the application of constraints may or may not be appropriate. We also consider how these results relate to the analytical framework first proposed by Tumer and Ghosh [5].
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Breiman, L.: Stacked Regressions. Machine Learning 24, 49–64 (1996)
Fumera, G., Roli, F.: Performance Analysis and Comparison of Linear Combiners for Classifier Fusion. In: Caelli, T.M., Amin, A., Duin, R.P.W., Kamel, M.S., de Ridder, D. (eds.) SPR 2002 and SSPR 2002. LNCS, vol. 2396, pp. 424–432. Springer, Heidelberg (2002)
Fumera, G., Roli, F.: A Theoretical and Experimental Analysis of Linear Combiners for Multiple Classifier Systems. IEEE Trans. Pattern Anal. Mach. Intell. 27(6), 942–956 (2005)
Le Blanc, M., Tibshirani, R.: Combining Estimates in Regression and Classification. Journal of the American Statistical Association 91, 1641–1650 (1996)
Tumer, K., Ghosh, J.: Analysis of Decision Boundaries in Linearly Combined Neural Classifiers. Pattern Recognition 29, 341–348 (1996)
Ting, K.M., Witten, I.H.: Issues in Stacked Generalization. Journal of Artificial Intelligence Research 10, 271–289 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tomas, A. (2009). Constraints in Weighted Averaging. In: Benediktsson, J.A., Kittler, J., Roli, F. (eds) Multiple Classifier Systems. MCS 2009. Lecture Notes in Computer Science, vol 5519. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02326-2_36
Download citation
DOI: https://doi.org/10.1007/978-3-642-02326-2_36
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02325-5
Online ISBN: 978-3-642-02326-2
eBook Packages: Computer ScienceComputer Science (R0)