Averaging Weak Classifiers

  • Dechang Chen
  • Jian Liu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2096)


We present a learning algorithm for two-class pattern recognition. It is based on combining a large number of weak classifiers. The weak classifiers are produced independently with diversity. And they are combined through a weighted average, weighted exponentially with respect to their apparent errors on the training data. Experimental results are also given.


Feature Vector Test Error Rectangular Region Weak Classifier Test Error Rate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), pp. 123–140.zbMATHMathSciNetGoogle Scholar
  2. 2.
    Chen, D. and Cheng, X. (2000, a). A simple implementation of the stochastic discrimination for pattern recognition. In Proc. Joint IAPR International Workshops SSPR 2000 and SPR 2000 (Francesc J. Ferri, José M. Iñesta, et al., eds.), Alicante, Spain, Springer LNCS-1876, pp. 882–887.Google Scholar
  3. 3.
    Chen, D. and Cheng, X. (2000, b). Majority vote based on weak classifiers. 2000 5th International Conference on Signal Processing Proceedings, Beijing, China, vol. 3, pp. 1560–1562.Google Scholar
  4. 4.
    Freund, Y. and Schapire, R. E. (1996). Experiments with a new boosting algorithm. Proc. 13th Int’l Conf. Machine Learning, pp. 148–156.Google Scholar
  5. 5.
    Freund, Y., Mansour, Y., Schapire, R. E. (2001). Why averaging classifiers can protect against overfitting. In Proceedings of the Eighth International Workshop on Artificial Intelligence and Statistics.Google Scholar
  6. 6.
    Ho, T. K. (1998). The random subspace method for constructing decision forests. IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20,no. 8, pp. 832–844.CrossRefGoogle Scholar
  7. 7.
    Ji, C. and Ma, S. (1997). Combinations of weak classifiers. IEEE Trans. Neural Networks, vol. 8,no. 1, pp. 32–42.CrossRefGoogle Scholar
  8. 8.
    Kleinberg, E. M. (1990). Stochastic discrimination. Annals of Math. and Artificial Intelligence, 1, pp. 207–239.zbMATHCrossRefGoogle Scholar
  9. 9.
    Kleinberg, E. M. (1996). An overtraining-resistant stochastic modeling method for pattern recognition. Annals of Statistics, 24, pp. 2319–2349.zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Kleinberg, E. M. (2000, a). On the algorithmic implementation of stochastic discrimination. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22,No. 5, pp. 473–490.CrossRefGoogle Scholar
  11. 11.
    Kleinberg, E. M. (2000, b). A mathematically rigorous foundation for supervised learning. Proc. First Int’l Workshop on Multiple Classifier Systems, Cagliari, Italy, pp. 67–76.Google Scholar
  12. 12.
    MacKay, D. J. C. (1991). Bayesian Methods for Adaptive Models, Ph.D. thesis, California Institute of Technology.Google Scholar
  13. 13.
    Smith, J., Everhart, J., Dickson, W., Knowler, W., & Johannes, R. (1988). Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In Proceedings of the Symposium on Computer Applications and Medical Care, IEEE Computer Society Press, pp. 261–265.Google Scholar
  14. 14.
    Wolberg, W. H. & Mangasarian, O. L. (1990). Multisurface method of pattern separation for medical diagnosis applied to breast cytology. Proceedings of the National Academy of Sciences, U.S.A., Volume 87, pp. 9193–9196.zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Dechang Chen
    • 1
  • Jian Liu
    • 2
  1. 1.University of WisconsinGreen BayUSA
  2. 2.University of MinnesotaMinneapolisUSA

Personalised recommendations