Improving the Weighted Distribution Estimation for AdaBoost Using a Novel Concurrent Approach
AdaBoost is one of the most known Ensemble approaches used in the Machine Learning literature. Several AdaBoost approaches that use Parallel processing, in order to speed up the computation in Large datasets, have been recently proposed. These approaches try to approximate the classic AdaBoost, thus sacrificing its generalization ability. In this work, we use Concurrent Computing in order to improve the Distribution Weight estimation, hence obtaining improvements in the capacity of generalization. We train in parallel in each round several weak hypotheses, and using a weighted ensemble we update the distribution weights of the following boosting rounds. Our results show that in most cases the performance of AdaBoost is improved and that the algorithm converges rapidly. We validate our proposal with 4 well-known real data sets.
This work was supported by the following research grants: Fondecyt 1110854 and DGIP-UTFSM. The work of C. Moraga was partially supported by the Foundation for the Advancement of Soft Computing, Mieres, Spain.
- 3.Fercoq, O.: Parallel Coordinate Descent for the Adaboost Problem. In: Proceedings of the 12th International Conference on Machine Learning and Applications (ICMLA), vol. 1, no. 1, pp. 354-358, 4-7 December 2013Google Scholar
- 5.Freund, Y., Shapire, R.E.: A short introduction to boosting. J. Jpn. Soc. Artif. Int. 14(5), 771–780 (1999)Google Scholar
- 8.Lichman, M.: UCI Machine Learning Repository. University of California, Irvine. School of Information and Computer Science (2013). http://archive.ics.uci.edu/ml
- 11.Palit, I., Reddy, C.K.: Parallelized Boosting with Map-Reduce. In: IEEE International Conference on Data Mining Workshops (ICDMW), vol. 1, no. 1, pp. 1346–1353, 13 December 2010Google Scholar
- 13.Seiffert, C., Khoshgoftaar, T.M., Van Hulse, J., Napolitano, A.: Resampling or reweighting: a comparison of boosting implementations. In Proceedings of the 20th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 08), vol. 1, no. 1, pp. 445–451, November 2008Google Scholar