Bootstrap bias corrections for ensemble methods

Article

DOI: 10.1007/s11222-016-9717-3

Cite this article as:
Hooker, G. & Mentch, L. Stat Comput (2016). doi:10.1007/s11222-016-9717-3
  • 68 Downloads

Abstract

This paper examines the use of a residual bootstrap for bias correction in machine learning regression methods. Accounting for bias is an important obstacle in recent efforts to develop statistical inference for machine learning. We demonstrate empirically that the proposed bootstrap bias correction can lead to substantial improvements in both bias and predictive accuracy. In the context of ensembles of trees, we show that this correction can be approximated at only double the cost of training the original ensemble. Our method is shown to improve test set accuracy over random forests by up to 70% on example problems from the UCI repository.

Keywords

Bagging Ensemble methods Bias correction Bootstrap 

Supplementary material

11222_2016_9717_MOESM1_ESM.r (6 kb)
Supplementary material 1 (R 5 KB)
11222_2016_9717_MOESM2_ESM.r (6 kb)
Supplementary material 2 (R 5 KB)
11222_2016_9717_MOESM3_ESM.r (6 kb)
Supplementary material 3 (R 5 KB)
11222_2016_9717_MOESM4_ESM.r (5 kb)
Supplementary material 4 (r 5 KB)
11222_2016_9717_MOESM5_ESM.r (6 kb)
Supplementary material 5 (R 5 KB)
11222_2016_9717_MOESM6_ESM.r (5 kb)
Supplementary material 6 (R 5 KB)
11222_2016_9717_MOESM8_ESM.r (10 kb)
Supplementary material 7 (R 9 KB)

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Cornell UniversityIthacaUSA
  2. 2.University of PittsburghPittsburghUSA

Personalised recommendations