Bagging Decision Multi-trees
Ensemble methods improve accuracy by combining the predictions of a set of different hypotheses. A well-known method for generating hypothesis ensembles is Bagging. One of the main drawbacks of ensemble methods in general, and Bagging in particular, is the huge amount of computational resources required to learn, store, and apply the set of models. Another problem is that even using the bootstrap technique, many simple models are similar, so limiting the ensemble diversity. In this work, we investigate an optimization technique based on sharing the common parts of the models from an ensemble formed by decision trees in order to minimize both problems. Concretely, we employ a structure called decision multi-tree which can contain simultaneously a set of decision trees and hence consider just once the ”repeated” parts. A thorough experimental evaluation is included to show that the proposed optimisation technique pays off in practice.
KeywordsEnsemble Methods Decision Trees Bagging
Unable to display preview. Download preview PDF.
- 1.Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar
- 3.Dietterich, T.G.: Ensemble methods in machine learning. In: First International Workshop on Multiple Classifier Systems, pp. 1–15 (2000)Google Scholar
- 8.Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Proc. 13th International Conference on Machine Learning, pp. 146–148. Morgan Kaufmann, San Francisco (1996)Google Scholar
- 10.Kohavi, R., Kunz, C.: Option decision trees with majority votes. In: Proc. 14th International Conference on Machine Learning, pp. 161–169. Morgan Kaufmann, San Francisco (1997)Google Scholar
- 13.Parmanto, B., Munro, P.W., Doyle, H.R.: Improving committee diagnosis with resampling techniques. In: Advances in Neural Information Processing Systems, vol. 8, pp. 882–888. The MIT Press, Cambridge (1996)Google Scholar
- 14.Pearl, J.: Heuristics: Intelligence search strategies for computer problem solving. Addison Wesley, Reading (1985)Google Scholar
- 15.Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)Google Scholar
- 16.Quinlan, J.R.: Bagging, Boosting, and C4.5. In: Proc. of the 30th Nat. Conf. on A.I. and the 8th Innovative Applications of A.I. Conf., pp. 725–730. AAAI Press/MIT Press (1996)Google Scholar