Wavelet-based gradient boosting
A new data science tool named wavelet-based gradient boosting is proposed and tested. The approach is special case of componentwise linear least squares gradient boosting, and involves wavelet functions of the original predictors. Wavelet-based gradient boosting takes advantages of the approximate \(\ell _1\) penalization induced by gradient boosting to give appropriate penalized additive fits. The method is readily implemented in R and produces parsimonious and interpretable regression fits and classifiers.
KeywordsClassification Data science Generalized additive models Nonparametric regression
We are grateful to Andrew Chernih for his provision of the Sydney residential property price data and to Peter Green for his comments on aspects of this research. Partial support was provided by Australian Research Council Discovery Project DP0877055. Assistance from the University of Technology, Sydney’s Distinguished Visitor programme is gratefully acknowledged.
- Hothorn, T., Bühlmann, P., Kneib, T., Schmid, M. & Hofner, B.: mboost 2.2. Model-based boosting. R package.(2011) http://cran.r-project.org
- Hyndman, R.J.: hdrcde 2.15. Highest density regions and conditional density estimation. R package. (2010) http://cran.r-project.org
- Nason, G.P.: wavethresh 4.5. Wavelets statistics and transforms. R package. (2010) http://cran.r-project.org
- R Development Core Team R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, (2012) http://www.R-project.org
- Ridgeway G.: gbm 1.6. Generalized boosted regression models. R package. (2012) http://cran.r-project.org