Bootstrap bias corrections for ensemble methods


DOI: 10.1007/s11222-016-9717-3

Cite this article as:
Hooker, G. & Mentch, L. Stat Comput (2016). doi:10.1007/s11222-016-9717-3


This paper examines the use of a residual bootstrap for bias correction in machine learning regression methods. Accounting for bias is an important obstacle in recent efforts to develop statistical inference for machine learning. We demonstrate empirically that the proposed bootstrap bias correction can lead to substantial improvements in both bias and predictive accuracy. In the context of ensembles of trees, we show that this correction can be approximated at only double the cost of training the original ensemble. Our method is shown to improve test set accuracy over random forests by up to 70% on example problems from the UCI repository.


Bagging Ensemble methods Bias correction Bootstrap 

Supplementary material

11222_2016_9717_MOESM1_ESM.r (6 kb)
Supplementary material 1 (R 5 KB)
11222_2016_9717_MOESM2_ESM.r (6 kb)
Supplementary material 2 (R 5 KB)
11222_2016_9717_MOESM3_ESM.r (6 kb)
Supplementary material 3 (R 5 KB)
11222_2016_9717_MOESM4_ESM.r (5 kb)
Supplementary material 4 (r 5 KB)
11222_2016_9717_MOESM5_ESM.r (6 kb)
Supplementary material 5 (R 5 KB)
11222_2016_9717_MOESM6_ESM.r (5 kb)
Supplementary material 6 (R 5 KB)
11222_2016_9717_MOESM8_ESM.r (10 kb)
Supplementary material 7 (R 9 KB)

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Cornell UniversityIthacaUSA
  2. 2.University of PittsburghPittsburghUSA

Personalised recommendations