Abstract
Decision tree learning is a machine learning technique that allows accurate and comprehensible models to be generated. Accuracy can be improved by ensemble methods which combine the predictions of a set of different trees. However, a large amount of resources is necessary to generate the ensemble. In this paper, we introduce a new ensemble method that minimises the usage of resources by sharing the common parts of the components of the ensemble. For this purpose, we learn a decision multi-tree instead of a decision tree. We call this newapproac h shared ensembles. The use of a multi-tree produces an exponential number of hypotheses to be combined, which provides better results than boosting/bagging. We performed several experiments, showing that the technique allows us to obtain accurate models and improves the use of resources with respect to classical ensemble methods.
This work has been partially supported by CICYT under grant TIC2001-2705-C03- 01, Generalitat Valenciana under grant GV00-092-14, and Acción Integrada Hispano- Austriaca HU2001-19.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Leo Breiman. Bagging predictors. Machine Learning, 24(2):123–140, 1996.
Leo Breiman. Random forests. Machine Learning, 45(1):5–32, 2001.
T. G Dietterich. Ensemble methods in machine learning. In First International Workshop on Multiple Classifier Systems, pages 1–15, 2000.
Thomas G. Dietterich. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, Boosting, and Randomization. Machine Learning, 40(2):139–157, 2000.
C. Ferri, J. Hernández, and M. J. Ramírez. Induction of Decision Multi-trees using Levin Search. In Int. Conf. on Computational Science, ICCS’02, LNCS, 2002.
C. Ferri, J. Hernández, and M. J. Ramírez. Learning multiple and different hypotheses. Technical report, D.S.I.C., Universitat Politécnica de Valéncia, 2002.
C. Ferri, J. Hernández, and M. J. Ramírez. SMILES system, a multi-purpose learning system. http://www.dsic.upv.es/~flip/smiles/, 2002.
Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. In the 13th Int. Conf. on Machine Learning (ICML’1996), pages 148–156, 1996.
Tim Kam Ho. Random decision forests. In Proc. of the 3rd International Conference on Document Analysis and Recognition, pages 278–282, 1995.
Tim Kam Ho. C4.5 decision forests. In Proc. of 14th Intl. Conf. on Pattern Recognition, Brisbane, Australia, pages 545–549, 1998.
Ludmila I. Kuncheva. A Theoretical Study on Six Classifier Fusion Strategies. IEEE Trans. on Pattern Analysis and Machine Intelligence, 24(2):281–286, 2002.
Dragos D. Margineantu and Thomas G. Dietterich. Pruning adaptive boosting. In 14th Int. Conf. on Machine Learning, pages 211–218. Morgan Kaufmann, 1997.
N. J. Nilsson. Artificial Intelligence: a new synthesis. Morgan Kaufmann, 1998.
University of California. UCI Machine Learning Repository Content Summary. http://www.ics.uci.edu/~mlearn/MLSummary.html.
J. Pearl. Heuristics: Intelligence search strategies for computer problem solving. Addison Wesley, 1985.
J. R. Quinlan. Induction of Decision Trees. In Read. in Machine Learning. M. Kaufmann, 1990.
J. R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.
J. R. Quinlan. Bagging, Boosting, and C4.5. In Proc. of the 13th Nat. Conf. on A.I. and the 8th Innovative Applications of A.I. Conf., pages 725–730. AAAI/MIT Press, 1996.
David H. Wolpert. Stacked generalization. Neural Networks, 5(2):241–259, 1992.
Zijian Zheng and Geoffrey I. Webb. Stochastic attribute selection committees. In Australian Joint Conference on Artificial Intelligence, pages 321–332, 1998.
Zijian Zheng, Geoffrey I. Webb, and K. M. Ting. Integrating boosting and stochastic attribute selection committees for further improving the performance of decision tree learning. In Proc. of 10th Int. Conf. on Tools with Artificial Intelligence (ICTAI-98), IEEE Computer Society Press, pages 216–223, 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Estruch, V., Ferri, C., Hernández-Orallo, J., Ramírez-Quintana, M.J. (2002). Shared Ensemble Learning Using Multi-trees. In: Garijo, F.J., Riquelme, J.C., Toro, M. (eds) Advances in Artificial Intelligence — IBERAMIA 2002. IBERAMIA 2002. Lecture Notes in Computer Science(), vol 2527. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36131-6_21
Download citation
DOI: https://doi.org/10.1007/3-540-36131-6_21
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-00131-7
Online ISBN: 978-3-540-36131-2
eBook Packages: Springer Book Archive