Cost-Sensitive Boosting: Fitting an Additive Asymmetric Logistic Regression Model

  • Qiu-jie Li
  • Yao-bin Mao
  • Zhi-quan Wang
  • Wen-bo Xiang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5828)


Conventional machine learning algorithms like boosting tend to equally treat misclassification errors that are not adequate to process certain cost-sensitive classification problems such as object detection. Although many cost-sensitive extensions of boosting by directly modifying the weighting strategy of correspond original algorithms have been proposed and reported, they are heuristic in nature and only proved effective by empirical results but lack sound theoretical analysis. This paper develops a framework from a statistical insight that can embody almost all existing cost-sensitive boosting algorithms: fitting an additive asymmetric logistic regression model by stage-wise optimization of certain criterions. Four cost-sensitive versions of boosting algorithms are derived, namely CSDA, CSRA, CSGA and CSLB which respectively correspond to Discrete AdaBoost, Real AdaBoost, Gentle AdaBoost and LogitBoost. Experimental results on the application of face detection have shown the effectiveness of the proposed learning framework in the reduction of the cumulative misclassification cost.


Boosting Cost-Sensitive Boosting Logistic Regression 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Freund, Y., Schapire, R.E.: A Decision-Theoretic Generalization of on-Line Learning and an Application on Boosting. Journal of Computer and System Sciences 55(1), 119–139 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Schapire, R.E., Singer, Y.: Improved Boosting Algorithms Using Confidence-Rated Predictions. Journal of Machine Learning 37(3), 297–336 (1999)zbMATHCrossRefGoogle Scholar
  3. 3.
    Freund, Y., Schapire, R.E.: Game Theory, on-Line Prediction and Boosting. In: Proceedings of the Ninth Annual Conference on Computational Learning Theory, pp. 325–332 (1996)Google Scholar
  4. 4.
    Schapire, R.E., Freund, Y., Bartlett, P., Lee, W.: Boosting the Margin: a New Explanation for the Effectiveness of Voting Methods. The Annals of Statistics 26(5), 1651–1686 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Friedman, J., Hastie, T., Tibshirani, R.: Additive Logistic Regression: a Statistical View of Boosting. The Annals of Statistics 28(2), 337–407 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Fan, W., Stolfo, S.J., Zhang, J., Chan, P.K.: Adacost: Misclassification Cost-Sensitive Boosting. In: ICML, pp. 97–105 (1999)Google Scholar
  7. 7.
    Ting, K.M.: A Comparative Study of Cost-Sensitive Boosting Algorithms. In: ICML, pp. 983–990 (2000)Google Scholar
  8. 8.
    Sun, Y., Kamel, M.S., Wong, A.K.C., Wang, Y.: Cost-Sensitive Boosting for Classification of Imbalanced Data. Pattern Recognition 40(12), 3358–3378 (2007)zbMATHCrossRefGoogle Scholar
  9. 9.
    Masnadi-Shirazi, H., Vasconcelos, N.: Asymmetric Boosting. In: ICML, pp. 609–619 (2007)Google Scholar
  10. 10.
    Viola, P., Jones, M.: Fast and Robust Classification Using Asymmetric AdaBoost and a Detector Cascade. In: Neural Information Processing Systems, pp. 1311–1318 (2001)Google Scholar
  11. 11.
    Lienhart, R., Kuranov, A., Pisarevsky, V.: Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection. In: Michaelis, B., Krell, G. (eds.) DAGM 2003. LNCS, vol. 2781, pp. 297–304. Springer, Heidelberg (2003)Google Scholar
  12. 12.
    Tuzel, O., Porikli, F., Meer, P.: Human Detection via Classification on Riemannian Manifolds. In: CVPR, pp. 1–8 (2007)Google Scholar
  13. 13.
    Viola, P., Jones, M.: Rapid Object Detection Using a Boosted Cascade of Simple Features. In: CVPR (2001)Google Scholar
  14. 14.
    Pham, M.T., Hoang, V.D., Cham, T.J.: Detection with Multi-Exit Asymmetric Boosting. In: CVPR (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Qiu-jie Li
    • 1
  • Yao-bin Mao
    • 1
  • Zhi-quan Wang
    • 1
  • Wen-bo Xiang
    • 1
  1. 1.School of AutomationNanjing University of Sci. & Tech.NanjingP.R. China

Personalised recommendations