We propose a novel algorithm to improve the ensemble performance of AdaBoost. The main contribution in our algorithm includes two aspects: (1) we aim to generate a distribution at each step that has less correlation with the previous classifiers so as to enhance the searching efficiency for new classifiers; (2) the classifiers weights can be iteratively modified along with the training process. In the proposed algorithm, the distribution is required to be corrective to some previous classifiers or some previous classifiers’ linear combinations. Experiments on UCI Repository have validated the new algorithm’s effectiveness.


Cost Function Descent Direction Training Error Subset Selection Variable Selection Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Freund, Y., Schapire, R.E.: A Decision-Theoretic Generalization of On-line Learning and an Application to Boosting. In: Vitányi, P.M.B. (ed.) EuroCOLT 1995. LNCS, vol. 904, pp. 23–37. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  2. 2.
    Mason, L., Baxter, J., Bartlett, P.L., et al.: Boosting Algorithms as Gradient Descent. In: Advances in Neural Information Processing Systems (NIPS 1999), pp. 512–518 (1999)Google Scholar
  3. 3.
    Friedman, J.H., Hastie, T., Tibshirani, R.: Additive Logistic Regression: A Statistical View of Boosting. Annals of Statistics 28(2), 337–407 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Li, L., Abu-Mostafa, Y.S., Pratap, A.: CGBoost: Conjugate Gradient in Function Space. Technical Report CaltechCSTR: 2003.007, Learning Systems Group, California Institute of Technology (August 2003)Google Scholar
  5. 5.
    Kivinen, J., Warmuth, M.K.: Boosting as Entropy Projection. In: 12th Annual Conference on Computational Learning Theory (COLT 1999), pp. 134–144 (1999)Google Scholar
  6. 6.
    Bühlmann, P., Yu, B.: Boosting with the L 2-Loss: Regression and Classification. Journal of American Statistical Association 98, 324–339 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Kuncheva, L.I., Whitaker, C.J.: Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy. Machine Learning 51(2), 181–207 (2003)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Yan Jiang
    • 1
  • Xiaoqing Ding
    • 1
  1. 1.Department of Electronic EngineeringTsinghua UniversityBeijingChina

Personalised recommendations